From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B517BCCA483 for ; Mon, 13 Jun 2022 22:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344302AbiFMW62 (ORCPT ); Mon, 13 Jun 2022 18:58:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232490AbiFMW5g (ORCPT ); Mon, 13 Jun 2022 18:57:36 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60C2CFFB for ; Mon, 13 Jun 2022 15:57:28 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id g14-20020a17090a128e00b001e882d66615so3237879pja.9 for ; Mon, 13 Jun 2022 15:57:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=dqOTd+rrEbe0Ho1KL+nrXG1TMVjZBcGq3Scgu3Hlczc=; b=G2vOugzu0HbRynHAIdfDvh7H6N8NAGp6OeQXZpZEPcoDoGxCjUfJtyrMb3Sup/4kFm UNEVjfAQsq1t27BPYIgf8RCWCi+Kl0Guq5Kaqyvj5Mp3Z5yiTWq1cFX3kk0m0Yo5MGZM OLllczkwvkBu2yDMqgQWndoHXH8PHXWQEKtSreWMXQc4uBTLX2YAOnUxIKr+f03kcCQw uFOQ6XzWTYeVn2eIKI8Gor8qkHvlybXEyCdJlgV5yjSekW3oBiCWRRwvh81U/BlPNR8R RrrZh7vfuqRkfsThIb8YazidnNzNDqp6XwpoE/MARqV0PlZdygplQfsvxUQZ7BVh7aRn VluQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=dqOTd+rrEbe0Ho1KL+nrXG1TMVjZBcGq3Scgu3Hlczc=; b=sthPyDIUj+UfDS8GexorbkW4GgWl8oQszbE0IcrYJNqGBSqUFMjjV1L9j9+D7tgS9O aEzuzz0YeDg/VSMdPr4XAewBjVnLcjl+xbaOgUWVg3tF16nGckh4Wjx3+2ik4nnE3p3d LSfQYu5avLvXrHX/VA12VMZUAuBHsGflO+t9b3mUQwKtt+E7NB/E95yzwQx+Ubs9+MZq wuLaMxYoAfZLN8Y1pJi9B44SnCoNzz4toRynZPdtAxOD9bzVOB9JAvXU2VZFi86HGcE0 V5UkBVUgPmMsDLu7QMlEAoQB23LzNdQpeocTL2fpCQq5BbRks/Fxzst0DB/v6ojyBigs b8gw== X-Gm-Message-State: AJIora8hgP8Ue7PdItC5cyqgjaO0WOosnHm86yVsjngcfMoliMk2s9Uc y2djtxLqyfKsnCenNC7+YxBFNZZSXXA= X-Google-Smtp-Source: AGRyM1s2pBEr39PeNzqKK5HLiCeXP22OWsE3v+YL7CLPBqUac5YpEvSaxDH40rsOJG+z5l0HV6tHa3Sk/h4= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:902:b909:b0:167:8c44:9bc1 with SMTP id bf9-20020a170902b90900b001678c449bc1mr1584462plb.47.1655161047715; Mon, 13 Jun 2022 15:57:27 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:16 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-2-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 1/8] KVM: x86/mmu: Drop unused CMPXCHG macro from paging_tmpl.h From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the CMPXCHG macro from paging_tmpl.h, it's no longer used now that KVM uses a common uaccess helper to do 8-byte CMPXCHG. Fixes: f122dfe44768 ("KVM: x86: Use __try_cmpxchg_user() to update guest PT= E A/D bits") Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/paging_tmpl.h | 6 ------ 1 file changed, 6 deletions(-) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index fe35d8fd3276..f595c4b8657f 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -34,7 +34,6 @@ #define PT_HAVE_ACCESSED_DIRTY(mmu) true #ifdef CONFIG_X86_64 #define PT_MAX_FULL_LEVELS PT64_ROOT_MAX_LEVEL - #define CMPXCHG "cmpxchgq" #else #define PT_MAX_FULL_LEVELS 2 #endif @@ -51,7 +50,6 @@ #define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT #define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT #define PT_HAVE_ACCESSED_DIRTY(mmu) true - #define CMPXCHG "cmpxchgl" #elif PTTYPE =3D=3D PTTYPE_EPT #define pt_element_t u64 #define guest_walker guest_walkerEPT @@ -64,9 +62,6 @@ #define PT_GUEST_DIRTY_SHIFT 9 #define PT_GUEST_ACCESSED_SHIFT 8 #define PT_HAVE_ACCESSED_DIRTY(mmu) (!(mmu)->cpu_role.base.ad_disabled) - #ifdef CONFIG_X86_64 - #define CMPXCHG "cmpxchgq" - #endif #define PT_MAX_FULL_LEVELS PT64_ROOT_MAX_LEVEL #else #error Invalid PTTYPE value @@ -1100,7 +1095,6 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, st= ruct kvm_mmu_page *sp) #undef PT_MAX_FULL_LEVELS #undef gpte_to_gfn #undef gpte_to_gfn_lvl -#undef CMPXCHG #undef PT_GUEST_ACCESSED_MASK #undef PT_GUEST_DIRTY_MASK #undef PT_GUEST_DIRTY_SHIFT --=20 2.36.1.476.g0c4daa206d-goog From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C591CCA481 for ; Mon, 13 Jun 2022 22:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245724AbiFMW60 (ORCPT ); Mon, 13 Jun 2022 18:58:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235127AbiFMW5g (ORCPT ); Mon, 13 Jun 2022 18:57:36 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60F1921BD for ; Mon, 13 Jun 2022 15:57:30 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id z67-20020a254c46000000b0065cd3d2e67eso6149273yba.7 for ; Mon, 13 Jun 2022 15:57:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=66YOE3D++YjiCQYAOPbfPb7xNbNJtfXwRmXra4+Vga0=; b=OGQQT/kEl+HJEIxg1pFOPK9rS28ovUya8ri7Z+Zs4qy6oaWb7o7khwI7JVXjbdrJXe DLQxDiuUabALcAyEtJUjYxq2JOljiWGRFUIj7pSxS7hoG/Zy2Bv/zq2FowrMCezO3plx X1ECBFm5XlSTdkMkilClv/2pAGu+Cq4sz1O2+6Cyo9AJdmdxqvjUlHg1v/bxfGMWajBQ gPBXlGgq38ijIWB767GbrfGwhU4V3hw0R4AdHzTldbQWT1tIWyhGKf1Fln1qYgTg6daa 0Bm/KBbPyObSermiYbLLx6MB+OJvj+oL+15400uar6hxXbM5fdgTHTVkDhWKGJynREYW r4vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=66YOE3D++YjiCQYAOPbfPb7xNbNJtfXwRmXra4+Vga0=; b=finDOoWwDWisodwhvdUV7XDIHOZB1Zy+FRNAwpKj5ZwhJUPrc7XSQJGMLzTXRrkcR3 6B2Gsnwjj8adTcpvS7ta3OkvbxLTOfD2YmsuJFYtdK3vQe4lvOP071KBdVSLk5wrw3Hy SUxWSOIas0GDoT4hOXWyMrPduy0t4aRdA66uJbkMDNwSBecXZko5VlN3koGwBgEVG2+B /NSc8hwwo3Cwa2jdJVwyJjNhRfDaz3jOhIdKUtmW2Bws+uY5fSYGBJdNh9Rf6MWCtnuo dpwJkdfnjB+1WvYTb6z1V5sub9Iukhn+rGqwPVtcpp6a5jIde5VnMmT2Odz+llEzR53m TgWw== X-Gm-Message-State: AJIora8y7SnUy8GFLevCYnHwFiJZZ9zWRZ3fg0qz5zJCZngznc8UcBSo xx9KoNHSkN1pa9g0nA5czeCLtGOt2hE= X-Google-Smtp-Source: AGRyM1vFi9XO4bi7ATHD/tcJRWZuWdYliTFf3K/ZUTGXuMM7BsmCw/F9cNYih8H8aUm6iETOgytGKbTPvGk= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a5b:101:0:b0:64d:ae10:3d26 with SMTP id 1-20020a5b0101000000b0064dae103d26mr1933397ybx.103.1655161049615; Mon, 13 Jun 2022 15:57:29 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:17 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-3-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 2/8] KVM: VMX: Refactor 32-bit PSE PT creation to avoid using MMU macro From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Compute the number of PTEs to be filled for the 32-bit PSE page tables using the page size and the size of each entry. While using the MMU's PT32_ENT_PER_PAGE macro is arguably better in isolation, removing VMX's usage will allow a future namespacing cleanup to move the guest page table macros into paging_tmpl.h, out of the reach of code that isn't directly related to shadow paging. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 5e14e4c40007..b774f8c1b952 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3704,7 +3704,7 @@ static int init_rmode_identity_map(struct kvm *kvm) } =20 /* Set up identity-mapping pagetable for EPT in real mode */ - for (i =3D 0; i < PT32_ENT_PER_PAGE; i++) { + for (i =3D 0; i < (PAGE_SIZE / sizeof(tmp)); i++) { tmp =3D (i << 22) + (_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE); if (__copy_to_user(uaddr + i * sizeof(tmp), &tmp, sizeof(tmp))) { --=20 2.36.1.476.g0c4daa206d-goog From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90F71C433EF for ; Mon, 13 Jun 2022 23:01:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245142AbiFMW6V (ORCPT ); Mon, 13 Jun 2022 18:58:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235230AbiFMW5g (ORCPT ); Mon, 13 Jun 2022 18:57:36 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D42FDF56 for ; Mon, 13 Jun 2022 15:57:31 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id w70-20020a638249000000b00406e420acfdso2252222pgd.2 for ; Mon, 13 Jun 2022 15:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=v4qCXjacc6DdzOgMgc8SdnwLNJ9D7HWzM5sk7a+r0QU=; b=tKbylohEMB3Jl6tpTMwQ6d8evAHk5Dz/0QN9wd9DmbHW4KeZ6M8jdnKU50lHcBNgDO z2GGrj64iIF78JHH1W4PJGeb1jnfnzpCv077HHYlpbJAxdUslC67ezPo0J76TrNk5NNA St3KeWrfmvgSpjMUGHivfOtU2H8Yjn53K//iZgUMaBp5OmnIZKbnZUzZccXhoWAN8GPq +XQsqANyvpke9U67fkPrTV+aRULg1yZHcHjcQKdzWNqcdQek4u+1yxVBqk98rx73dbYq 0x3oL4RGTiUCD6foo02yQOQqggQXNuhl4KfdidThn7+p6PBv/T5/pRZa0GHDO396q7j1 S3wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=v4qCXjacc6DdzOgMgc8SdnwLNJ9D7HWzM5sk7a+r0QU=; b=unMKSGEcf2/jknNOzZ6fbGhYHaQvGZpB4klk+gP3qz+hXIcx8+IHkMvQOVSoCF+AG2 WCSAfziEPTHXbJMfeuPlSKA0tXGoipX0JhXD2chTaR4zt0lr1dwCuuRc5Scstn1OjymE VXip7dhVjic18PWLRd9uXIKuAOtOIlcyWDGoOYrkUvDkFrmJdhYNYoIknnUMgl+1jlFK k8OvdhpADiPiahgtnnxZz71FZeYO8JsAp49ebhf3FBvObH/MitA45OvtLpOhjXoX23sS 9cQcedH9O/oSDiwuDbeciCMHn7ZrHBJAKm7sM7fQioKxte3a92yVPWoBCI4xdrqmykHp bCJg== X-Gm-Message-State: AOAM530CAqHn/l45j2jaY+2oISekCYc0F70pfNixLgR12Ota7osJl8r7 QFG+iw149m7GqEFWOO0B7TRPf5Lkplk= X-Google-Smtp-Source: ABdhPJzmxrTOiQI7sxpaSIbvdsIoS7RfkibMVQwFP2P08oMwufrTOwRKrpZkn+gUpDEhaIBvYQ0wu2dxWCc= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a05:6a00:164c:b0:50a:472a:6b0a with SMTP id m12-20020a056a00164c00b0050a472a6b0amr1466395pfc.77.1655161051343; Mon, 13 Jun 2022 15:57:31 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:18 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-4-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 3/8] KVM: x86/mmu: Bury 32-bit PSE paging helpers in paging_tmpl.h From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move a handful of one-off macros and helpers for 32-bit PSE paging into paging_tmpl.h and hide them behind "PTTYPE =3D=3D 32". Under no circumstan= ce should anything but 32-bit shadow paging care about PSE paging. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu.h | 5 ----- arch/x86/kvm/mmu/mmu.c | 12 ------------ arch/x86/kvm/mmu/paging_tmpl.h | 19 ++++++++++++++++++- 3 files changed, 18 insertions(+), 18 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index f8192864b496..d1021e34ac15 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -34,11 +34,6 @@ #define PT_DIR_PAT_SHIFT 12 #define PT_DIR_PAT_MASK (1ULL << PT_DIR_PAT_SHIFT) =20 -#define PT32_DIR_PSE36_SIZE 4 -#define PT32_DIR_PSE36_SHIFT 13 -#define PT32_DIR_PSE36_MASK \ - (((1ULL << PT32_DIR_PSE36_SIZE) - 1) << PT32_DIR_PSE36_SHIFT) - #define PT64_ROOT_5LEVEL 5 #define PT64_ROOT_4LEVEL 4 #define PT32_ROOT_LEVEL 2 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 17252f39bd7c..f1961fe3fe67 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -321,18 +321,6 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64= spte) return likely(kvm_gen =3D=3D spte_gen); } =20 -static int is_cpuid_PSE36(void) -{ - return 1; -} - -static gfn_t pse36_gfn_delta(u32 gpte) -{ - int shift =3D 32 - PT32_DIR_PSE36_SHIFT - PAGE_SHIFT; - - return (gpte & PT32_DIR_PSE36_MASK) << shift; -} - #ifdef CONFIG_X86_64 static void __set_spte(u64 *sptep, u64 spte) { diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index f595c4b8657f..ef02e6bb0bcb 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -50,6 +50,12 @@ #define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT #define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT #define PT_HAVE_ACCESSED_DIRTY(mmu) true + + #define is_cpuid_PSE36() true + #define PT32_DIR_PSE36_SIZE 4 + #define PT32_DIR_PSE36_SHIFT 13 + #define PT32_DIR_PSE36_MASK \ + (((1ULL << PT32_DIR_PSE36_SIZE) - 1) << PT32_DIR_PSE36_SHIFT) #elif PTTYPE =3D=3D PTTYPE_EPT #define pt_element_t u64 #define guest_walker guest_walkerEPT @@ -92,6 +98,15 @@ struct guest_walker { struct x86_exception fault; }; =20 +#if PTTYPE =3D=3D 32 +static inline gfn_t pse36_gfn_delta(u32 gpte) +{ + int shift =3D 32 - PT32_DIR_PSE36_SHIFT - PAGE_SHIFT; + + return (gpte & PT32_DIR_PSE36_MASK) << shift; +} +#endif + static gfn_t gpte_to_gfn_lvl(pt_element_t gpte, int lvl) { return (gpte & PT_LVL_ADDR_MASK(lvl)) >> PAGE_SHIFT; @@ -416,8 +431,10 @@ static int FNAME(walk_addr_generic)(struct guest_walke= r *walker, gfn =3D gpte_to_gfn_lvl(pte, walker->level); gfn +=3D (addr & PT_LVL_OFFSET_MASK(walker->level)) >> PAGE_SHIFT; =20 - if (PTTYPE =3D=3D 32 && walker->level > PG_LEVEL_4K && is_cpuid_PSE36()) +#if PTTYPE =3D=3D 32 + if (walker->level > PG_LEVEL_4K && is_cpuid_PSE36()) gfn +=3D pse36_gfn_delta(pte); +#endif =20 real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(gfn), access, &walke= r->fault); if (real_gpa =3D=3D UNMAPPED_GVA) --=20 2.36.1.476.g0c4daa206d-goog From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C8BACCA480 for ; Mon, 13 Jun 2022 22:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243093AbiFMW6Q (ORCPT ); Mon, 13 Jun 2022 18:58:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235357AbiFMW5h (ORCPT ); Mon, 13 Jun 2022 18:57:37 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 597D0270C for ; Mon, 13 Jun 2022 15:57:33 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id j4-20020a170902da8400b00168b0b2341dso3759413plx.5 for ; Mon, 13 Jun 2022 15:57:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=bo31PGP16JR8fO1mwvO3YvvmHEE1iup/pKqJeMLN6sA=; b=e6mdfRC+yH8DEWKDztiUFCOlT+hLyEMdHwuptEyVZvR7QErW1ltJvFMXWlbIAQ4lHk fdDqosiVo/VvRX2iE5aDMrJNpDubEIcXcMdNNA/0okoOvrSjGMRXYe4W1zkuAk5/BZp2 gnhSE7S3N7sIdLVMWGaTn5VGHy3W2iUCuqlT73kH3PdWRutT1JjLcfvgdGaYX68NyyHE DQxB3IZ3F0jx3PdAT4BQ7grTnkyjeShTZCWLd7oRwuGGM/neE1o60pxJeijrzYCK4cxq fPdHCnHZjYU9Bk64kKdbG9A9js+5u65/J4O9KULfONepX4MT4682ciloSt125Ndg/dmU tJmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=bo31PGP16JR8fO1mwvO3YvvmHEE1iup/pKqJeMLN6sA=; b=1FUIB4tTmMAHnLv/DbrjGuGtBHHTFuUdMvZzoacLwtMTRSCV1qlzhZEcwgm1pA85bq 3swDi7ZTwE9BfkeztFV1CKVs3C0hMWPpCoq0Su4b7gD72/f7WsWuAdcDsEzF03PrRUTb LBQpr37dUgxTVEkLKg3NLcaCN7w+YfL+KYMpb28Qfx5H3WJivzLkLzXK82s2YkUPX1zt 72ur10Z5BQ4nbmEl/onzopHwP6fGbfjgVvLHIdb59m5BmndP1mMFyBlnPLIdAu21AkRo 3wLqKxIhWOxOP1wMGugxlawWxLVvoxI9Nug8b1y4XGGujWrSUGwiTs6tY7TMFypcGdwe EOCQ== X-Gm-Message-State: AOAM5301Rg3AeIIReZ0e4ls2JdcwMhxtN9lciOjF//lpj763okIry0sN 9KvFi4xwP3ac4S/Ad52j7qsUhF1FIu0= X-Google-Smtp-Source: ABdhPJwWEaECxcGZhw2o4gKX7Dh5diIt/fzvRU1hWYTIrwPktOsvYDx1piS89RKElwqnExMrOQPTBO35hlY= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:aa7:831d:0:b0:51e:ec7a:82a7 with SMTP id bk29-20020aa7831d000000b0051eec7a82a7mr1583270pfb.51.1655161053132; Mon, 13 Jun 2022 15:57:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:19 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-5-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 4/8] KVM: x86/mmu: Dedup macros for computing various page table masks From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide common helper macros to generate various masks, shifts, etc... for 32-bit vs. 64-bit page tables. Only the inputs differ, the actual calculations are identical. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu.h | 4 ++-- arch/x86/kvm/mmu/mmu.c | 15 ++++++--------- arch/x86/kvm/mmu/mmu_internal.h | 14 ++++++++++++++ arch/x86/kvm/mmu/paging.h | 9 +++++---- arch/x86/kvm/mmu/spte.h | 7 +++---- 5 files changed, 30 insertions(+), 19 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index d1021e34ac15..6efe6bd7fb6e 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -7,9 +7,9 @@ #include "cpuid.h" =20 #define PT64_PT_BITS 9 -#define PT64_ENT_PER_PAGE (1 << PT64_PT_BITS) +#define PT64_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT64_PT_BITS) #define PT32_PT_BITS 10 -#define PT32_ENT_PER_PAGE (1 << PT32_PT_BITS) +#define PT32_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT32_PT_BITS) =20 #define PT_WRITABLE_SHIFT 1 #define PT_USER_SHIFT 2 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f1961fe3fe67..afe3deaa0d95 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -113,23 +113,20 @@ module_param(dbg, bool, 0644); =20 #define PT32_LEVEL_BITS 10 =20 -#define PT32_LEVEL_SHIFT(level) \ - (PAGE_SHIFT + (level - 1) * PT32_LEVEL_BITS) +#define PT32_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, PT32_LEVEL_BITS) =20 #define PT32_LVL_OFFSET_MASK(level) \ - (PT32_BASE_ADDR_MASK & ((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT32_LEVEL_BITS))) - 1)) - -#define PT32_INDEX(address, level)\ - (((address) >> PT32_LEVEL_SHIFT(level)) & ((1 << PT32_LEVEL_BITS) - 1)) + __PT_LVL_OFFSET_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) =20 +#define PT32_INDEX(address, level) __PT_INDEX(address, level, PT32_LEVEL_B= ITS) =20 #define PT32_BASE_ADDR_MASK PAGE_MASK + #define PT32_DIR_BASE_ADDR_MASK \ (PAGE_MASK & ~((1ULL << (PAGE_SHIFT + PT32_LEVEL_BITS)) - 1)) + #define PT32_LVL_ADDR_MASK(level) \ - (PAGE_MASK & ~((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT32_LEVEL_BITS))) - 1)) + __PT_LVL_ADDR_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) =20 #include =20 diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index bd2a26897b97..5e1e3c8f8aaa 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -20,6 +20,20 @@ extern bool dbg; #define MMU_WARN_ON(x) do { } while (0) #endif =20 +/* Page table builder macros common to shadow (host) PTEs and guest PTEs. = */ +#define __PT_LEVEL_SHIFT(level, bits_per_level) \ + (PAGE_SHIFT + ((level) - 1) * (bits_per_level)) +#define __PT_INDEX(address, level, bits_per_level) \ + (((address) >> __PT_LEVEL_SHIFT(level, bits_per_level)) & ((1 << (bits_pe= r_level)) - 1)) + +#define __PT_LVL_ADDR_MASK(base_addr_mask, level, bits_per_level) \ + ((base_addr_mask) & ~((1ULL << (PAGE_SHIFT + (((level) - 1) * (bits_per_l= evel)))) - 1)) + +#define __PT_LVL_OFFSET_MASK(base_addr_mask, level, bits_per_level) \ + ((base_addr_mask) & ((1ULL << (PAGE_SHIFT + (((level) - 1) * (bits_per_le= vel)))) - 1)) + +#define __PT_ENT_PER_PAGE(bits_per_level) (1 << (bits_per_level)) + /* * Unlike regular MMU roots, PAE "roots", a.k.a. PDPTEs/PDPTRs, have a PRE= SENT * bit, and thus are guaranteed to be non-zero when valid. And, when a gu= est diff --git a/arch/x86/kvm/mmu/paging.h b/arch/x86/kvm/mmu/paging.h index de8ab323bb70..23f3f64b8092 100644 --- a/arch/x86/kvm/mmu/paging.h +++ b/arch/x86/kvm/mmu/paging.h @@ -4,11 +4,12 @@ #define __KVM_X86_PAGING_H =20 #define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1= )) + #define PT64_LVL_ADDR_MASK(level) \ - (GUEST_PT64_BASE_ADDR_MASK & ~((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT64_LEVEL_BITS))) - 1)) + __PT_LVL_ADDR_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) + #define PT64_LVL_OFFSET_MASK(level) \ - (GUEST_PT64_BASE_ADDR_MASK & ((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT64_LEVEL_BITS))) - 1)) + __PT_LVL_OFFSET_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) + #endif /* __KVM_X86_PAGING_H */ =20 diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 0127bb6e3c7d..d5a8183b7232 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -55,11 +55,10 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK =3D=3D 0); =20 #define PT64_LEVEL_BITS 9 =20 -#define PT64_LEVEL_SHIFT(level) \ - (PAGE_SHIFT + (level - 1) * PT64_LEVEL_BITS) +#define PT64_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, PT64_LEVEL_BITS) + +#define PT64_INDEX(address, level) __PT_INDEX(address, level, PT64_LEVEL_B= ITS) =20 -#define PT64_INDEX(address, level)\ - (((address) >> PT64_LEVEL_SHIFT(level)) & ((1 << PT64_LEVEL_BITS) - 1)) #define SHADOW_PT_INDEX(addr, level) PT64_INDEX(addr, level) =20 /* --=20 2.36.1.476.g0c4daa206d-goog From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3D0ECCA487 for ; Mon, 13 Jun 2022 22:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347250AbiFMW6l (ORCPT ); Mon, 13 Jun 2022 18:58:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236280AbiFMW5j (ORCPT ); Mon, 13 Jun 2022 18:57:39 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76FA52ACD for ; Mon, 13 Jun 2022 15:57:35 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-2f7dbceab08so10371667b3.10 for ; Mon, 13 Jun 2022 15:57:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=Wfuc2yFn+6O12rprpPrZ8DBVEL/9MJ9/SEFQWLvRY/Y=; b=NLjWdNgMUVtColhZ3Cid0dk+TpgaOrCAwS0HbYCbF9buRnePY2t5abSTPGNIxNWP8G Di9At1B8NTvFF/TPhh+/a06t2KX+kh7xbG0oMj3SBmZ9hZr1/Jqsreyl5WtzHbKRI0tQ NjA8wh/sMIrIQ+zHh4BUnXT+1+QESzCuylWC49r8NDXDxmj5/IUp6EtwBqcRxA87Z+vs jbxh0YaqE3V1hzPXiuLC2gsdZZgKpF90MagFlgytxU5qh97Gs31Gn8rG7YdEF/GkDK7X 3DmQyTyrTXPUQCMJCzIdVDzQ4VFMKwq+XcLuM1XYVOkl16z3lYVKve17GjfiQrJH+38n HG9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=Wfuc2yFn+6O12rprpPrZ8DBVEL/9MJ9/SEFQWLvRY/Y=; b=2wR2LAM+ZozBkpri+6C3fd/HlKlzHTyZjtmcHWrKN05qJAqTyeuMhQpjQbzXa4xrfZ s2P35C9cIKonR7+IYPdyGVWz/rD4IKUnkj6WNUlDCli4VZZDK1Rx7kMRmiS7Kj/xoKCB h/6jcOQtff/ia5kNrJle5/qgQ3wWleNPx56vKXwveK3u5gZJ0Wz25i37gHjuzOUQX46K YXTl+vfd5bv8VYdqUykNY59CMqSPr05nw50n6DM34D+GWEbROclnycJB+GdWCnc4nsWA nsEWZUnOnRzr9HmIlECnpEGtFs2xofO6hAcK2uw8Lm5WxrjDI8qRVhGIBfxDnG4kWdLY f60Q== X-Gm-Message-State: AJIora88sctIGuKgm1FAGxHb8E/2PcuJMiqy+nEY5CW5FIzW6L252SOS 99vKiwttrecpwh4eSIShG8ZCWfkx/iU= X-Google-Smtp-Source: AGRyM1t4TsqDBsBtRrvFjoZMpjvl2A1b6s8M1LkHPgyxRN9zOV2au2u+5wxwy1kehqKiSq5mvhcqr4QblqU= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a0d:d994:0:b0:313:3a26:8336 with SMTP id b142-20020a0dd994000000b003133a268336mr2405485ywe.444.1655161054817; Mon, 13 Jun 2022 15:57:34 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:20 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-6-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 5/8] KVM: x86/mmu: Use separate namespaces for guest PTEs and shadow PTEs From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Separate the macros for KVM's shadow PTEs (SPTE) from guest 64-bit PTEs (PT64). SPTE and PT64 are _mostly_ the same, but the few differences are quite critical, e.g. *_BASE_ADDR_MASK must differentiate between host and guest physical address spaces, and SPTE_PERM_MASK (was PT64_PERM_MASK) is very much specific to SPTEs. Opportunistically (and temporarily) move most guest macros into paging.h to clearly associate them with shadow paging, and to ensure that they're not used as of this commit. A future patch will eliminate them entirely. Sadly, PT32_LEVEL_BITS is left behind in mmu_internal.h because it's needed for the quadrant calculation in kvm_mmu_get_page(). The quadrant calculation is hot enough (when using shadow paging with 32-bit guests) that adding a per-context helper is undesirable, and burying the computation in paging_tmpl.h with a forward declaration isn't exactly an improvement. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu.h | 5 ---- arch/x86/kvm/mmu/mmu.c | 47 +++++++++++---------------------- arch/x86/kvm/mmu/mmu_internal.h | 3 +++ arch/x86/kvm/mmu/paging.h | 20 ++++++++++++++ arch/x86/kvm/mmu/paging_tmpl.h | 4 +-- arch/x86/kvm/mmu/spte.c | 2 +- arch/x86/kvm/mmu/spte.h | 27 +++++++++---------- arch/x86/kvm/mmu/tdp_iter.c | 6 ++--- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++--- 9 files changed, 59 insertions(+), 61 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 6efe6bd7fb6e..a99acec925eb 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -6,11 +6,6 @@ #include "kvm_cache_regs.h" #include "cpuid.h" =20 -#define PT64_PT_BITS 9 -#define PT64_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT64_PT_BITS) -#define PT32_PT_BITS 10 -#define PT32_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT32_PT_BITS) - #define PT_WRITABLE_SHIFT 1 #define PT_USER_SHIFT 2 =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index afe3deaa0d95..aedb8d871030 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -111,23 +111,6 @@ module_param(dbg, bool, 0644); =20 #define PTE_PREFETCH_NUM 8 =20 -#define PT32_LEVEL_BITS 10 - -#define PT32_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, PT32_LEVEL_BITS) - -#define PT32_LVL_OFFSET_MASK(level) \ - __PT_LVL_OFFSET_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) - -#define PT32_INDEX(address, level) __PT_INDEX(address, level, PT32_LEVEL_B= ITS) - -#define PT32_BASE_ADDR_MASK PAGE_MASK - -#define PT32_DIR_BASE_ADDR_MASK \ - (PAGE_MASK & ~((1ULL << (PAGE_SHIFT + PT32_LEVEL_BITS)) - 1)) - -#define PT32_LVL_ADDR_MASK(level) \ - __PT_LVL_ADDR_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) - #include =20 /* make pte_list_desc fit well in cache lines */ @@ -702,7 +685,7 @@ static gfn_t kvm_mmu_page_get_gfn(struct kvm_mmu_page *= sp, int index) if (!sp->role.direct) return sp->gfns[index]; =20 - return sp->gfn + (index << ((sp->role.level - 1) * PT64_LEVEL_BITS)); + return sp->gfn + (index << ((sp->role.level - 1) * SPTE_LEVEL_BITS)); } =20 static void kvm_mmu_page_set_gfn(struct kvm_mmu_page *sp, int index, gfn_t= gfn) @@ -1774,7 +1757,7 @@ static int __mmu_unsync_walk(struct kvm_mmu_page *sp, continue; } =20 - child =3D to_shadow_page(ent & PT64_BASE_ADDR_MASK); + child =3D to_shadow_page(ent & SPTE_BASE_ADDR_MASK); =20 if (child->unsync_children) { if (mmu_pages_add(pvec, child, i)) @@ -2025,8 +2008,8 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct k= vm_vcpu *vcpu, role.direct =3D direct; role.access =3D access; if (role.has_4_byte_gpte) { - quadrant =3D gaddr >> (PAGE_SHIFT + (PT64_PT_BITS * level)); - quadrant &=3D (1 << ((PT32_PT_BITS - PT64_PT_BITS) * level)) - 1; + quadrant =3D gaddr >> (PAGE_SHIFT + (SPTE_LEVEL_BITS * level)); + quadrant &=3D (1 << ((PT32_LEVEL_BITS - SPTE_LEVEL_BITS) * level)) - 1; role.quadrant =3D quadrant; } if (level <=3D vcpu->arch.mmu->cpu_role.base.level) @@ -2130,7 +2113,7 @@ static void shadow_walk_init_using_root(struct kvm_sh= adow_walk_iterator *iterato =20 iterator->shadow_addr =3D vcpu->arch.mmu->pae_root[(addr >> 30) & 3]; - iterator->shadow_addr &=3D PT64_BASE_ADDR_MASK; + iterator->shadow_addr &=3D SPTE_BASE_ADDR_MASK; --iterator->level; if (!iterator->shadow_addr) iterator->level =3D 0; @@ -2149,7 +2132,7 @@ static bool shadow_walk_okay(struct kvm_shadow_walk_i= terator *iterator) if (iterator->level < PG_LEVEL_4K) return false; =20 - iterator->index =3D SHADOW_PT_INDEX(iterator->addr, iterator->level); + iterator->index =3D SPTE_INDEX(iterator->addr, iterator->level); iterator->sptep =3D ((u64 *)__va(iterator->shadow_addr)) + iterator->inde= x; return true; } @@ -2162,7 +2145,7 @@ static void __shadow_walk_next(struct kvm_shadow_walk= _iterator *iterator, return; } =20 - iterator->shadow_addr =3D spte & PT64_BASE_ADDR_MASK; + iterator->shadow_addr =3D spte & SPTE_BASE_ADDR_MASK; --iterator->level; } =20 @@ -2201,7 +2184,7 @@ static void validate_direct_spte(struct kvm_vcpu *vcp= u, u64 *sptep, * so we should update the spte at this point to get * a new sp with the correct access. */ - child =3D to_shadow_page(*sptep & PT64_BASE_ADDR_MASK); + child =3D to_shadow_page(*sptep & SPTE_BASE_ADDR_MASK); if (child->role.access =3D=3D direct_access) return; =20 @@ -2222,7 +2205,7 @@ static int mmu_page_zap_pte(struct kvm *kvm, struct k= vm_mmu_page *sp, if (is_last_spte(pte, sp->role.level)) { drop_spte(kvm, spte); } else { - child =3D to_shadow_page(pte & PT64_BASE_ADDR_MASK); + child =3D to_shadow_page(pte & SPTE_BASE_ADDR_MASK); drop_parent_pte(child, spte); =20 /* @@ -2248,7 +2231,7 @@ static int kvm_mmu_page_unlink_children(struct kvm *k= vm, int zapped =3D 0; unsigned i; =20 - for (i =3D 0; i < PT64_ENT_PER_PAGE; ++i) + for (i =3D 0; i < SPTE_ENT_PER_PAGE; ++i) zapped +=3D mmu_page_zap_pte(kvm, sp, sp->spt + i, invalid_list); =20 return zapped; @@ -2661,7 +2644,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct= kvm_memory_slot *slot, struct kvm_mmu_page *child; u64 pte =3D *sptep; =20 - child =3D to_shadow_page(pte & PT64_BASE_ADDR_MASK); + child =3D to_shadow_page(pte & SPTE_BASE_ADDR_MASK); drop_parent_pte(child, sptep); flush =3D true; } else if (pfn !=3D spte_to_pfn(*sptep)) { @@ -3250,7 +3233,7 @@ static void mmu_free_root_page(struct kvm *kvm, hpa_t= *root_hpa, if (!VALID_PAGE(*root_hpa)) return; =20 - sp =3D to_shadow_page(*root_hpa & PT64_BASE_ADDR_MASK); + sp =3D to_shadow_page(*root_hpa & SPTE_BASE_ADDR_MASK); if (WARN_ON(!sp)) return; =20 @@ -3722,7 +3705,7 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu) hpa_t root =3D vcpu->arch.mmu->pae_root[i]; =20 if (IS_VALID_PAE_ROOT(root)) { - root &=3D PT64_BASE_ADDR_MASK; + root &=3D SPTE_BASE_ADDR_MASK; sp =3D to_shadow_page(root); mmu_sync_children(vcpu, sp, true); } @@ -5184,11 +5167,11 @@ static bool need_remote_flush(u64 old, u64 new) return false; if (!is_shadow_present_pte(new)) return true; - if ((old ^ new) & PT64_BASE_ADDR_MASK) + if ((old ^ new) & SPTE_BASE_ADDR_MASK) return true; old ^=3D shadow_nx_mask; new ^=3D shadow_nx_mask; - return (old & ~new & PT64_PERM_MASK) !=3D 0; + return (old & ~new & SPTE_PERM_MASK) !=3D 0; } =20 static u64 mmu_pte_write_fetch_gpte(struct kvm_vcpu *vcpu, gpa_t *gpa, diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 5e1e3c8f8aaa..cb9d4d358335 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -20,6 +20,9 @@ extern bool dbg; #define MMU_WARN_ON(x) do { } while (0) #endif =20 +/* The number of bits for 32-bit PTEs is to needed compute the quandrant. = */ +#define PT32_LEVEL_BITS 10 + /* Page table builder macros common to shadow (host) PTEs and guest PTEs. = */ #define __PT_LEVEL_SHIFT(level, bits_per_level) \ (PAGE_SHIFT + ((level) - 1) * (bits_per_level)) diff --git a/arch/x86/kvm/mmu/paging.h b/arch/x86/kvm/mmu/paging.h index 23f3f64b8092..3fed2c101de3 100644 --- a/arch/x86/kvm/mmu/paging.h +++ b/arch/x86/kvm/mmu/paging.h @@ -5,11 +5,31 @@ =20 #define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1= )) =20 +#define PT64_LEVEL_BITS 9 + +#define PT64_INDEX(address, level) __PT_INDEX(address, level, PT64_LEVEL_B= ITS) + #define PT64_LVL_ADDR_MASK(level) \ __PT_LVL_ADDR_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) =20 #define PT64_LVL_OFFSET_MASK(level) \ __PT_LVL_OFFSET_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) =20 + +#define PT32_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, PT32_LEVEL_BITS) + +#define PT32_LVL_OFFSET_MASK(level) \ + __PT_LVL_OFFSET_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) + +#define PT32_INDEX(address, level) __PT_INDEX(address, level, PT32_LEVEL_B= ITS) + +#define PT32_BASE_ADDR_MASK PAGE_MASK + +#define PT32_DIR_BASE_ADDR_MASK \ + (PAGE_MASK & ~((1ULL << (PAGE_SHIFT + PT32_LEVEL_BITS)) - 1)) + +#define PT32_LVL_ADDR_MASK(level) \ + __PT_LVL_ADDR_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) + #endif /* __KVM_X86_PAGING_H */ =20 diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index ef02e6bb0bcb..75f6b01edcf8 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -900,7 +900,7 @@ static gpa_t FNAME(get_level1_sp_gpa)(struct kvm_mmu_pa= ge *sp) WARN_ON(sp->role.level !=3D PG_LEVEL_4K); =20 if (PTTYPE =3D=3D 32) - offset =3D sp->role.quadrant << PT64_LEVEL_BITS; + offset =3D sp->role.quadrant << SPTE_LEVEL_BITS; =20 return gfn_to_gpa(sp->gfn) + offset * sizeof(pt_element_t); } @@ -1035,7 +1035,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, st= ruct kvm_mmu_page *sp) =20 first_pte_gpa =3D FNAME(get_level1_sp_gpa)(sp); =20 - for (i =3D 0; i < PT64_ENT_PER_PAGE; i++) { + for (i =3D 0; i < SPTE_ENT_PER_PAGE; i++) { u64 *sptep, spte; struct kvm_memory_slot *slot; unsigned pte_access; diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index cda1851ec155..242e4828d7df 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -301,7 +301,7 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte= , kvm_pfn_t new_pfn) { u64 new_spte; =20 - new_spte =3D old_spte & ~PT64_BASE_ADDR_MASK; + new_spte =3D old_spte & ~SPTE_BASE_ADDR_MASK; new_spte |=3D (u64)new_pfn << PAGE_SHIFT; =20 new_spte &=3D ~PT_WRITABLE_MASK; diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index d5a8183b7232..121c5eaaec77 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -36,12 +36,12 @@ extern bool __read_mostly enable_mmio_caching; static_assert(SPTE_TDP_AD_ENABLED_MASK =3D=3D 0); =20 #ifdef CONFIG_DYNAMIC_PHYSICAL_MASK -#define PT64_BASE_ADDR_MASK (physical_mask & ~(u64)(PAGE_SIZE-1)) +#define SPTE_BASE_ADDR_MASK (physical_mask & ~(u64)(PAGE_SIZE-1)) #else -#define PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) +#define SPTE_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) #endif =20 -#define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_m= ask \ +#define SPTE_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_m= ask \ | shadow_x_mask | shadow_nx_mask | shadow_me_mask) =20 #define ACC_EXEC_MASK 1 @@ -50,16 +50,13 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK =3D=3D 0); #define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK) =20 /* The mask for the R/X bits in EPT PTEs */ -#define PT64_EPT_READABLE_MASK 0x1ull -#define PT64_EPT_EXECUTABLE_MASK 0x4ull +#define SPTE_EPT_READABLE_MASK 0x1ull +#define SPTE_EPT_EXECUTABLE_MASK 0x4ull =20 -#define PT64_LEVEL_BITS 9 - -#define PT64_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, PT64_LEVEL_BITS) - -#define PT64_INDEX(address, level) __PT_INDEX(address, level, PT64_LEVEL_B= ITS) - -#define SHADOW_PT_INDEX(addr, level) PT64_INDEX(addr, level) +#define SPTE_LEVEL_BITS 9 +#define SPTE_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, SPTE_LEVEL_BITS) +#define SPTE_INDEX(address, level) __PT_INDEX(address, level, SPTE_LEVEL_B= ITS) +#define SPTE_ENT_PER_PAGE __PT_ENT_PER_PAGE(SPTE_LEVEL_BITS) =20 /* * The mask/shift to use for saving the original R/X bits when marking the= PTE @@ -68,8 +65,8 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK =3D=3D 0); * restored only when a write is attempted to the page. This mask obvious= ly * must not overlap the A/D type mask. */ -#define SHADOW_ACC_TRACK_SAVED_BITS_MASK (PT64_EPT_READABLE_MASK | \ - PT64_EPT_EXECUTABLE_MASK) +#define SHADOW_ACC_TRACK_SAVED_BITS_MASK (SPTE_EPT_READABLE_MASK | \ + SPTE_EPT_EXECUTABLE_MASK) #define SHADOW_ACC_TRACK_SAVED_BITS_SHIFT 54 #define SHADOW_ACC_TRACK_SAVED_MASK (SHADOW_ACC_TRACK_SAVED_BITS_MASK << \ SHADOW_ACC_TRACK_SAVED_BITS_SHIFT) @@ -281,7 +278,7 @@ static inline bool is_executable_pte(u64 spte) =20 static inline kvm_pfn_t spte_to_pfn(u64 pte) { - return (pte & PT64_BASE_ADDR_MASK) >> PAGE_SHIFT; + return (pte & SPTE_BASE_ADDR_MASK) >> PAGE_SHIFT; } =20 static inline bool is_accessed_spte(u64 spte) diff --git a/arch/x86/kvm/mmu/tdp_iter.c b/arch/x86/kvm/mmu/tdp_iter.c index ee4802d7b36c..9c65a64a56d9 100644 --- a/arch/x86/kvm/mmu/tdp_iter.c +++ b/arch/x86/kvm/mmu/tdp_iter.c @@ -11,7 +11,7 @@ static void tdp_iter_refresh_sptep(struct tdp_iter *iter) { iter->sptep =3D iter->pt_path[iter->level - 1] + - SHADOW_PT_INDEX(iter->gfn << PAGE_SHIFT, iter->level); + SPTE_INDEX(iter->gfn << PAGE_SHIFT, iter->level); iter->old_spte =3D kvm_tdp_mmu_read_spte(iter->sptep); } =20 @@ -116,8 +116,8 @@ static bool try_step_side(struct tdp_iter *iter) * Check if the iterator is already at the end of the current page * table. */ - if (SHADOW_PT_INDEX(iter->gfn << PAGE_SHIFT, iter->level) =3D=3D - (PT64_ENT_PER_PAGE - 1)) + if (SPTE_INDEX(iter->gfn << PAGE_SHIFT, iter->level) =3D=3D + (SPTE_ENT_PER_PAGE - 1)) return false; =20 iter->gfn +=3D KVM_PAGES_PER_HPAGE(iter->level); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 7b9265d67131..26cb9fed2f18 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -425,7 +425,7 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep= _t pt, bool shared) =20 tdp_mmu_unlink_sp(kvm, sp, shared); =20 - for (i =3D 0; i < PT64_ENT_PER_PAGE; i++) { + for (i =3D 0; i < SPTE_ENT_PER_PAGE; i++) { tdp_ptep_t sptep =3D pt + i; gfn_t gfn =3D base_gfn + i * KVM_PAGES_PER_HPAGE(level); u64 old_spte; @@ -1487,7 +1487,7 @@ static int tdp_mmu_split_huge_page(struct kvm *kvm, s= truct tdp_iter *iter, * No need for atomics when writing to sp->spt since the page table has * not been linked in yet and thus is not reachable from any other CPU. */ - for (i =3D 0; i < PT64_ENT_PER_PAGE; i++) + for (i =3D 0; i < SPTE_ENT_PER_PAGE; i++) sp->spt[i] =3D make_huge_page_split_spte(huge_spte, level, i); =20 /* @@ -1507,7 +1507,7 @@ static int tdp_mmu_split_huge_page(struct kvm *kvm, s= truct tdp_iter *iter, * are overwriting from the page stats. But we have to manually update * the page stats with the new present child pages. */ - kvm_update_page_stats(kvm, level - 1, PT64_ENT_PER_PAGE); + kvm_update_page_stats(kvm, level - 1, SPTE_ENT_PER_PAGE); =20 out: trace_kvm_mmu_split_huge_page(iter->gfn, huge_spte, level, ret); --=20 2.36.1.476.g0c4daa206d-goog From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4AEECCA482 for ; Mon, 13 Jun 2022 22:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345169AbiFMW6b (ORCPT ); Mon, 13 Jun 2022 18:58:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235607AbiFMW5j (ORCPT ); Mon, 13 Jun 2022 18:57:39 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E13410E5 for ; Mon, 13 Jun 2022 15:57:37 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id g129-20020a636b87000000b00401b8392ac8so4024982pgc.4 for ; Mon, 13 Jun 2022 15:57:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=XfyrmAuBNG96ESd4ge8XFRdhqAAWlj/bUQmrHUvO0vg=; b=aB+mHJdxCtw86zizW4wBXbT1gFqQGoe0qOMmsjfrKUSWynkwhUxvP0niYP7qIZHIop TEd087yHrArSh02i6aFoQcQezKWHPMlYuE7ifax/CdCzxoL167+1Xbbn2H2J2qPv5bS0 57rClqQ3jgTawmKTUNi2mByyvnqGXucQSedyoI8rD1/bRS8q2JgxT5YFvtO8dqjPE3Eh pjDVeYsv7h50iP0tyG+9IIaNc3srWTjyYt3IrjeAmHhSzaJEEi0AAUTE9O/xAPrbsZ19 Ghf+uSGLF6CHyxIccu/zMK51xLO43hjPFHTqzTh1iZNdvr78E0L9Rpew4+eNhrGAto9r oc6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=XfyrmAuBNG96ESd4ge8XFRdhqAAWlj/bUQmrHUvO0vg=; b=wkbAsEDT2WCEAuS8vn8S8XNoXfcjh1ATs/WSXGptf28zceWUU4kPHz2HxkBqI30eWN Ec1ByFeO9PClMfct0tqIjRt7EnK0NOwwr7rbXIkY2zqRDqKZSlS+oKszBB4TvzpDEZav TYFTB1HgOgV5aqA9EuGL/JGGT/YMCdJgrWh7ygTBbtDxTF7CEJPuW64Y5uEnHUc+9dkz cFsXT2rVaPUGvqrmt4Z1T6RDnoQq4mCsnN1QijrVLAmI0ANbPaKY3TEkwZWt90wqzOoF OgJ81BtkZF9LN5KJiuezO7BfKbbua+iTkQBeAdvUIzGhnrv+TtEmzEOsbppO/5vddJDm RTUg== X-Gm-Message-State: AJIora9hLGjCIi/VLtUSKT178hQ/dtRvmsuYNRZi63MDwb25O9caX9eW MwwXanSdmwPGW11QzdtdTBmJdvR9m6I= X-Google-Smtp-Source: AGRyM1urB2wXKYaVwBFbTVFrOgFVE+BwUDOaPZYhK78OGINuG4bhotXlqzGx0Puv+VHs0tpYMhvfFoKKHxk= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:902:dac7:b0:166:4ce4:7e32 with SMTP id q7-20020a170902dac700b001664ce47e32mr1389307plx.168.1655161056535; Mon, 13 Jun 2022 15:57:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:21 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-7-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 6/8] KVM: x86/mmu: Use common macros to compute 32/64-bit paging masks From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Dedup the code for generating (most of) the per-type PT_* masks in paging_tmpl.h. The relevant macros only vary based on the number of bits per level, and that smidge of info is already provided in a common form as PT_LEVEL_BITS. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/paging.h | 26 -------------------------- arch/x86/kvm/mmu/paging_tmpl.h | 25 +++++++++++-------------- 2 files changed, 11 insertions(+), 40 deletions(-) diff --git a/arch/x86/kvm/mmu/paging.h b/arch/x86/kvm/mmu/paging.h index 3fed2c101de3..9de4976b2d46 100644 --- a/arch/x86/kvm/mmu/paging.h +++ b/arch/x86/kvm/mmu/paging.h @@ -5,31 +5,5 @@ =20 #define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1= )) =20 -#define PT64_LEVEL_BITS 9 - -#define PT64_INDEX(address, level) __PT_INDEX(address, level, PT64_LEVEL_B= ITS) - -#define PT64_LVL_ADDR_MASK(level) \ - __PT_LVL_ADDR_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) - -#define PT64_LVL_OFFSET_MASK(level) \ - __PT_LVL_OFFSET_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) - - -#define PT32_LEVEL_SHIFT(level) __PT_LEVEL_SHIFT(level, PT32_LEVEL_BITS) - -#define PT32_LVL_OFFSET_MASK(level) \ - __PT_LVL_OFFSET_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) - -#define PT32_INDEX(address, level) __PT_INDEX(address, level, PT32_LEVEL_B= ITS) - -#define PT32_BASE_ADDR_MASK PAGE_MASK - -#define PT32_DIR_BASE_ADDR_MASK \ - (PAGE_MASK & ~((1ULL << (PAGE_SHIFT + PT32_LEVEL_BITS)) - 1)) - -#define PT32_LVL_ADDR_MASK(level) \ - __PT_LVL_ADDR_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) - #endif /* __KVM_X86_PAGING_H */ =20 diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 75f6b01edcf8..0bb2a6c97ebb 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -16,8 +16,9 @@ */ =20 /* - * We need the mmu code to access both 32-bit and 64-bit guest ptes, - * so the code in this file is compiled twice, once per pte size. + * The MMU needs to be able to access/walk 32-bit and 64-bit guest page ta= bles, + * as well as guest EPT tables, so the code in this file is compiled thric= e, + * once per guest PTE type. The per-type defines are #undef'd at the end. */ =20 #if PTTYPE =3D=3D 64 @@ -25,10 +26,7 @@ #define guest_walker guest_walker64 #define FNAME(name) paging##64_##name #define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK - #define PT_LVL_ADDR_MASK(lvl) PT64_LVL_ADDR_MASK(lvl) - #define PT_LVL_OFFSET_MASK(lvl) PT64_LVL_OFFSET_MASK(lvl) - #define PT_INDEX(addr, level) PT64_INDEX(addr, level) - #define PT_LEVEL_BITS PT64_LEVEL_BITS + #define PT_LEVEL_BITS 9 #define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT #define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT #define PT_HAVE_ACCESSED_DIRTY(mmu) true @@ -41,10 +39,7 @@ #define pt_element_t u32 #define guest_walker guest_walker32 #define FNAME(name) paging##32_##name - #define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK - #define PT_LVL_ADDR_MASK(lvl) PT32_LVL_ADDR_MASK(lvl) - #define PT_LVL_OFFSET_MASK(lvl) PT32_LVL_OFFSET_MASK(lvl) - #define PT_INDEX(addr, level) PT32_INDEX(addr, level) + #define PT_BASE_ADDR_MASK PAGE_MASK #define PT_LEVEL_BITS PT32_LEVEL_BITS #define PT_MAX_FULL_LEVELS 2 #define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT @@ -61,10 +56,7 @@ #define guest_walker guest_walkerEPT #define FNAME(name) ept_##name #define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK - #define PT_LVL_ADDR_MASK(lvl) PT64_LVL_ADDR_MASK(lvl) - #define PT_LVL_OFFSET_MASK(lvl) PT64_LVL_OFFSET_MASK(lvl) - #define PT_INDEX(addr, level) PT64_INDEX(addr, level) - #define PT_LEVEL_BITS PT64_LEVEL_BITS + #define PT_LEVEL_BITS 9 #define PT_GUEST_DIRTY_SHIFT 9 #define PT_GUEST_ACCESSED_SHIFT 8 #define PT_HAVE_ACCESSED_DIRTY(mmu) (!(mmu)->cpu_role.base.ad_disabled) @@ -73,6 +65,11 @@ #error Invalid PTTYPE value #endif =20 +/* Common logic, but per-type values. These also need to be undefined. */ +#define PT_LVL_ADDR_MASK(lvl) __PT_LVL_ADDR_MASK(PT_BASE_ADDR_MASK, lvl, P= T_LEVEL_BITS) +#define PT_LVL_OFFSET_MASK(lvl) __PT_LVL_OFFSET_MASK(PT_BASE_ADDR_MASK, lv= l, PT_LEVEL_BITS) +#define PT_INDEX(addr, lvl) __PT_INDEX(addr, lvl, PT_LEVEL_BITS) + #define PT_GUEST_DIRTY_MASK (1 << PT_GUEST_DIRTY_SHIFT) #define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT) =20 --=20 2.36.1.476.g0c4daa206d-goog From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4625CCA485 for ; Mon, 13 Jun 2022 22:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346342AbiFMW6h (ORCPT ); Mon, 13 Jun 2022 18:58:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239886AbiFMW5k (ORCPT ); Mon, 13 Jun 2022 18:57:40 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18524F56 for ; Mon, 13 Jun 2022 15:57:39 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id d11-20020a170902cecb00b00163fe890197so3765772plg.1 for ; Mon, 13 Jun 2022 15:57:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=CH5DREzyS2tatBMXpdEXlWqxqYpXIqULDbAajGOOJKo=; b=RIJuXisytih2NpyVfyghweWip1f0Rg+Br76HuU1orwNbLKH1cNyhUeCfnj1YUhfv6g uyRN33fSoGMSawGtzvMYXn7mthUpreQxJJ1ogm/LPUypufIe9Hf8lhXjY1wHGQusChRz bmZPtYtWj0CgR8ynUC9hRcYuBiEWP7/WS8BjJYdRO2B89AbXZ3AWw31O+sSqZ6HCNZ3f OnxRRhM0kqF4xcUwUIgYY7AyNP2BbJR6jsEAR6AVSCJKi42zS4zj1ulaGcqOV/38JklQ UBKB8OK/TxWV/Lvw609IYCcv98GCsLyDHHaFzjmIR+XRkBG765wPAdhEs2OUBEpqxLf3 pl8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=CH5DREzyS2tatBMXpdEXlWqxqYpXIqULDbAajGOOJKo=; b=ElLy5d8XIISGdN+L3WUyxa1bnEokqsvt2BKic8oP3zL7ViPL09ju+g7gqw0M+6PwY+ 2ddHfoCcH9JSMzMUfZwCKZShiHhiyEfAE53o8QVl75v1bHsF49hJKmnCKNOuDYtDp/0X 6t+m95+qB5peRYIqhR82U+7HCS2YgTATP7z/EOi/7rHLYUzGV4x/X/H04Fit8HqoJG14 WChWaC1sDCGBMIGOX3BA93Wx5f936AqSBcGUuVRxOoptC9WKEjX2i6sRySkHt3uScPGr 85qDJLlFqcrBbsxDNcFpiywMlMAbP5pGtCmSOi6V5ZdvOHI8sUVLX5dcOz/RjsNhPqbz 4uZQ== X-Gm-Message-State: AJIora8g24f0C3DTm6Rcdj0KpogNXHDTVuzh1zdUNG4t03/6yqKU9d78 JMBp/blErzqt200h5yLg5q186nHx7uQ= X-Google-Smtp-Source: AGRyM1vzPhUIRhhvS3eRTgxLMPv9LYjTKe77b+/SMc6756A7kV3g3V5r9jXLT2nDC2hVGFG+vf4bXu/wm7M= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:90a:249:b0:1e0:a8a3:3c6c with SMTP id t9-20020a17090a024900b001e0a8a33c6cmr48916pje.0.1655161058035; Mon, 13 Jun 2022 15:57:38 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:22 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-8-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 7/8] KVM: x86/mmu: Truncate paging32's PT_BASE_ADDR_MASK to 32 bits From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Truncate paging32's PT_BASE_ADDR_MASK to a pt_element_t, i.e. to 32 bits. Ignoring PSE huge pages, the mask is only used in conjunction with gPTEs, which are 32 bits, and so the address is limited to bits 31:12. PSE huge pages encoded PA bits 39:32 in PTE bits 20:13, i.e. need custom logic to handle their funky encoding regardless of PT_BASE_ADDR_MASK. Note, PT_LVL_OFFSET_MASK is someone confusing in that it computes the offset of the _gfn_, not of the gpa, i.e. not having bits 63:32 set in PT_BASE_ADDR_MASK is again correct. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 0bb2a6c97ebb..4087e58e2232 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -39,7 +39,7 @@ #define pt_element_t u32 #define guest_walker guest_walker32 #define FNAME(name) paging##32_##name - #define PT_BASE_ADDR_MASK PAGE_MASK + #define PT_BASE_ADDR_MASK ((pt_element_t)PAGE_MASK) #define PT_LEVEL_BITS PT32_LEVEL_BITS #define PT_MAX_FULL_LEVELS 2 #define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT --=20 2.36.1.476.g0c4daa206d-goog From nobody Mon Apr 27 11:42:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4AF9CCA486 for ; Mon, 13 Jun 2022 22:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345816AbiFMW6c (ORCPT ); Mon, 13 Jun 2022 18:58:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240975AbiFMW5l (ORCPT ); Mon, 13 Jun 2022 18:57:41 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6537125CB for ; Mon, 13 Jun 2022 15:57:40 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id c11-20020a17090a4d0b00b001e4e081d525so7040508pjg.7 for ; Mon, 13 Jun 2022 15:57:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=RHYcnFXTT7ac0ob6di8kcCxjr6TYxAsCityKr+RTKLI=; b=Ht+WkXrzNY4aCQJpUyHeoWPWNTBiTlUW3Lagepr4HbSdxEqbjS+ssk0l8Zamu5d1SR jjMxcznKpxbilodFMTbwrpKMcGsDf3SN15/AHVZrOIWYxZNkjMDYM5vYWcYW5iomnR8d ENCPPz0daRuFtW9s37PyTlx1y5/jAYunRaYufcaL3bKBWOyjrRQ6YKofuf8OAdaElQYX 6fyug2OPvXy5nnN8B/AfFA82ZAlHIVmrpOqVheuMEB1MgTqPNrSqJqoWpEvMLy5hICph MwWuJkEmgQzcxqCKlGF9srg9wvaxtk8Pa4iWY14QwGls9+deEB7KJO5apyO8sxQyixMG IxtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=RHYcnFXTT7ac0ob6di8kcCxjr6TYxAsCityKr+RTKLI=; b=d1eTzLE6waFMStwrj2mIRGJ4huxv7MxpDtSVHFlqrU4cm/XGloUe0Xc9E1D96KNgEE S0/i1DbElrl2Tlu2YsZAbyN9xh85bwkdIX4QWzkXqOhM+aN7rARuHfjMaZFw4GlotJAz +XrjBRl0Qruz2poHElA6t20pzWlPfsKm6Yhr0H3gOMPgLXcU8NAYHbcozU00yvEAuXEB TtXKTXV4qQjWuhhOOwO0Mj429ldr5lAUH6TSbA/1R7W/jicjdSPOcqcMf2OKV+cUVDPN Wl4xcaQ1N7aOFCy2AvrAi9ZkaNEN/I/WD0vDr0hiRkN+OapewZ2Sxj4Tlkg0e/gHd2sd yLjw== X-Gm-Message-State: AJIora+x1AJ5GZw4zLq7pIq4HGM+A4bwzUyd47s3FU3l78EYE3Ufg306 iDmoDVB+Aej/ysGtyBn4MNF242paNag= X-Google-Smtp-Source: AGRyM1usL/eeVClnixov12j9knTnRvi5pIAGqLnemUdbfR1Ufy3iDbbtQEYUfhjil6JScKSo4HiKr6s8g5Q= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:90a:66c1:b0:1e8:43ae:f7c0 with SMTP id z1-20020a17090a66c100b001e843aef7c0mr1054309pjl.245.1655161059904; Mon, 13 Jun 2022 15:57:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 13 Jun 2022 22:57:23 +0000 In-Reply-To: <20220613225723.2734132-1-seanjc@google.com> Message-Id: <20220613225723.2734132-9-seanjc@google.com> Mime-Version: 1.0 References: <20220613225723.2734132-1-seanjc@google.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog Subject: [PATCH 8/8] KVM: x86/mmu: Use common logic for computing the 32/64-bit base PA mask From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use common logic for computing PT_BASE_ADDR_MASK for 32-bit, 64-bit, and EPT paging. Both PAGE_MASK and the new-common logic are supsersets of what is actually needed for 32-bit paging. PAGE_MASK sets bits 63:12 and the former GUEST_PT64_BASE_ADDR_MASK sets bits 51:12, so regardless of which value is used, the result will always be bits 31:12. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 -- arch/x86/kvm/mmu/paging.h | 9 --------- arch/x86/kvm/mmu/paging_tmpl.h | 4 +--- 3 files changed, 1 insertion(+), 14 deletions(-) delete mode 100644 arch/x86/kvm/mmu/paging.h diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index aedb8d871030..0f0c3ebfcf51 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -53,8 +53,6 @@ #include #include "trace.h" =20 -#include "paging.h" - extern bool itlb_multihit_kvm_mitigation; =20 int __read_mostly nx_huge_pages =3D -1; diff --git a/arch/x86/kvm/mmu/paging.h b/arch/x86/kvm/mmu/paging.h deleted file mode 100644 index 9de4976b2d46..000000000000 --- a/arch/x86/kvm/mmu/paging.h +++ /dev/null @@ -1,9 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* Shadow paging constants/helpers that don't need to be #undef'd. */ -#ifndef __KVM_X86_PAGING_H -#define __KVM_X86_PAGING_H - -#define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1= )) - -#endif /* __KVM_X86_PAGING_H */ - diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 4087e58e2232..1f0dbc31e5d4 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -25,7 +25,6 @@ #define pt_element_t u64 #define guest_walker guest_walker64 #define FNAME(name) paging##64_##name - #define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK #define PT_LEVEL_BITS 9 #define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT #define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT @@ -39,7 +38,6 @@ #define pt_element_t u32 #define guest_walker guest_walker32 #define FNAME(name) paging##32_##name - #define PT_BASE_ADDR_MASK ((pt_element_t)PAGE_MASK) #define PT_LEVEL_BITS PT32_LEVEL_BITS #define PT_MAX_FULL_LEVELS 2 #define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT @@ -55,7 +53,6 @@ #define pt_element_t u64 #define guest_walker guest_walkerEPT #define FNAME(name) ept_##name - #define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK #define PT_LEVEL_BITS 9 #define PT_GUEST_DIRTY_SHIFT 9 #define PT_GUEST_ACCESSED_SHIFT 8 @@ -66,6 +63,7 @@ #endif =20 /* Common logic, but per-type values. These also need to be undefined. */ +#define PT_BASE_ADDR_MASK ((pt_element_t)(((1ULL << 52) - 1) & ~(u64)(PAGE= _SIZE-1))) #define PT_LVL_ADDR_MASK(lvl) __PT_LVL_ADDR_MASK(PT_BASE_ADDR_MASK, lvl, P= T_LEVEL_BITS) #define PT_LVL_OFFSET_MASK(lvl) __PT_LVL_OFFSET_MASK(PT_BASE_ADDR_MASK, lv= l, PT_LEVEL_BITS) #define PT_INDEX(addr, lvl) __PT_INDEX(addr, lvl, PT_LEVEL_BITS) --=20 2.36.1.476.g0c4daa206d-goog