From nobody Tue Jun 23 21:23:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B739C433EF for ; Fri, 25 Feb 2022 18:23:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232630AbiBYSXe (ORCPT ); Fri, 25 Feb 2022 13:23:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232179AbiBYSXZ (ORCPT ); Fri, 25 Feb 2022 13:23:25 -0500 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 764E66E78C for ; Fri, 25 Feb 2022 10:22:52 -0800 (PST) Received: by mail-pf1-x449.google.com with SMTP id g72-20020a62524b000000b004f3e21965e8so426063pfb.5 for ; Fri, 25 Feb 2022 10:22:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=tEpUVd4jPoK9N2ecaXKtLW2f0KgIvioRCgiZkrx72eI=; b=OWOv2+M6YRuwOcezy+sXd6L23PlvZMFz0Dr8BW1S+7jDP5EiWUOkGH/ETk1JZyA7um qWXPDYWwxfky114NKGB3NXrOuVafjRsJ4REWEfm1SmqED4mNeo9TQbCr+PMnVm/LIB6M spWJ2y/b0Zj4/vMcD1BpCka8UERO/EGcTZT78DxcIuBAyZKjxc/NwCgUU65BEYDGFJUv lGlYYG5/tZZzwWoMXF9XWalPdoHPTZhqkb557RJl/vaHBT23v5q4IYfBebf86Llmos7q BAiIFGIjV/Q7Ev9uiF/tWhjt/okkT8Fg9jbhysn5iGxXLfjALWqOxeqSgBZCsXIfHvqz T0sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=tEpUVd4jPoK9N2ecaXKtLW2f0KgIvioRCgiZkrx72eI=; b=hm/cA/Sx3IC9oSocboy2xXbdB1/r0DYM2GbdAdLcn48bdutXopwgxjz7PhK1n8uq+f c3zebs4Ei8nLWRVG+TCU5U+FUVddEgu42IbDzs8PNOwHNCPeZIy7Sh0hjMlhggfawUIL oBO92BU3yDn5sp2KWD3NnBY483EsEbe53WxZBqN59KjENhll2Y6805SCJqoY+gnHkOiL IRPo/PAU0nxrmW4bcNp3zJgYV2Ab8WxZ2J2vqLxiEVql55nvemB6MpVbSi/xsLmx+bF3 RyuPf9VTNxHCv2jSeqjP9xj+wWwFQ4BRkzDtLBKbqRT9c/7r5biM2kcwmEPuqnu9c/h4 737w== X-Gm-Message-State: AOAM533mgi3oaL/jKBN6nEDqw7xHwEaAQmyRTU68ZJZIprqfAe7nGLuE aNZDSCNWd8fHSEb+32nurm19HCgKcxY= X-Google-Smtp-Source: ABdhPJzPC25kc1BUDo4Eqxg1mvuOtgWUpXZtFNWPs1qsLhYYWgU6hsM2J1mzu+ugmnwlNkFsi0Yuc/ZEMjc= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:aa7:9382:0:b0:4e0:cf48:e564 with SMTP id t2-20020aa79382000000b004e0cf48e564mr632110pfe.15.1645813371930; Fri, 25 Feb 2022 10:22:51 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 25 Feb 2022 18:22:42 +0000 In-Reply-To: <20220225182248.3812651-1-seanjc@google.com> Message-Id: <20220225182248.3812651-2-seanjc@google.com> Mime-Version: 1.0 References: <20220225182248.3812651-1-seanjc@google.com> X-Mailer: git-send-email 2.35.1.574.g5d30c73bfb-goog Subject: [PATCH v2 1/7] KVM: x86: Remove spurious whitespaces from kvm_post_set_cr4() From: Sean Christopherson To: Paolo Bonzini , Christian Borntraeger , Janosch Frank Cc: David Hildenbrand , Claudio Imbrenda , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove leading spaces that snuck into kvm_post_set_cr4(), fixing the KVM_REQ_TLB_FLUSH_CURRENT request in particular is helpful as it unaligns the body of the if-statement from the condition check. Fixes: f4bc051fc91a ("KVM: x86: flush TLB separately from MMU reset") Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6552360d8888..2157284d05b0 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1089,7 +1089,7 @@ void kvm_post_set_cr4(struct kvm_vcpu *vcpu, unsigned= long old_cr4, unsigned lon */ if (((cr4 ^ old_cr4) & X86_CR4_PGE) || (!(cr4 & X86_CR4_PCIDE) && (old_cr4 & X86_CR4_PCIDE))) - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); =20 /* * The TLB has to be flushed for the current PCID if any of the @@ -1099,7 +1099,7 @@ void kvm_post_set_cr4(struct kvm_vcpu *vcpu, unsigned= long old_cr4, unsigned lon */ else if (((cr4 ^ old_cr4) & X86_CR4_PAE) || ((cr4 & X86_CR4_SMEP) && !(old_cr4 & X86_CR4_SMEP))) - kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); + kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); =20 } EXPORT_SYMBOL_GPL(kvm_post_set_cr4); --=20 2.35.1.574.g5d30c73bfb-goog From nobody Tue Jun 23 21:23:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D322C433FE for ; Fri, 25 Feb 2022 18:23:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232888AbiBYSXh (ORCPT ); Fri, 25 Feb 2022 13:23:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232280AbiBYSX1 (ORCPT ); Fri, 25 Feb 2022 13:23:27 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48555F1EB1 for ; Fri, 25 Feb 2022 10:22:54 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id a12-20020a65640c000000b003756296df5cso3044781pgv.19 for ; Fri, 25 Feb 2022 10:22:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=VKMBlWNx4jK1iDVhpsaRZcvzkSOAxKFhhP9NDDoxkLA=; b=Fdd0ugG9W3f4EJeeUYsCgb8W9Ynd5MF6AKEcW9LnQwoRU3YEau0D2ZwFmpoNIOuvd1 J6HppSKpX/QYcTmEka6QjbKpAhYpeQgkge2amIHPVjFS9MehVv+XZZd30rcmdCiItSeY 0l6W7scfFbqHQAkX4U1MAUk+AlREKywJdodZSG3B2SIZQTZmBi9wJiPUXWioIjh7j1ya JnMh5Rv9Ap/QzJQDQXG1x8HwsQ92lTRv/4ln8Vo1StPl4s4CSvXq3bZmdHl7jANIHTPt MgOgzVCSzc4MgjTWgS8zkx+e8N0Wo/mApBy98n4knaA8odwbLo7GDc3PWVYrNry/ub6K cg+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=VKMBlWNx4jK1iDVhpsaRZcvzkSOAxKFhhP9NDDoxkLA=; b=f0tdVP1c5p5SHelKJCG/pZYTVCXFZbpZfAzz/bnFYLT+WxKWhUa27+3w1fRwnfaASB 8nLOxv7jF99dOlEf60Ly8497XfwllexW4ysxC5Fk/V6q9jRKgMEGjBv8Hi4fN652z0lH +uKYS4HJwDJdtPGud1VzXijjkz0JlttAhaiIn7FKF3bvgN2vJXTNYs8iFvH5bx01iiEe UAn6qynU/vLi0bL5lbxgY5k/THq7uxUdXSJijUPAfsMgxPd405zHflPwKZHl5ha9nqmP ZIhF/GiE7iKSDMreAgcEvjnEA1grOG7yhpp+ZPK1XQJBlulBKiaf/fGs0SqMcpXVqtRT 2OmQ== X-Gm-Message-State: AOAM533XxKYgKMYRDmNXzFc8OgY51xa5BPnfbeXnKOjWErWL5DuGkHJL 2zQHpGPTxb2AOKMveo7H43zF8zcABOk= X-Google-Smtp-Source: ABdhPJxoyxmqX3mxfoFgGAjyblYLJ3fu4AUs1RNVx87TMDdvK+Ro/2u7tKzWPg4caY5UCaJlP+QbMSmje30= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:90b:4f43:b0:1bc:7e5c:e024 with SMTP id pj3-20020a17090b4f4300b001bc7e5ce024mr26606pjb.0.1645813373353; Fri, 25 Feb 2022 10:22:53 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 25 Feb 2022 18:22:43 +0000 In-Reply-To: <20220225182248.3812651-1-seanjc@google.com> Message-Id: <20220225182248.3812651-3-seanjc@google.com> Mime-Version: 1.0 References: <20220225182248.3812651-1-seanjc@google.com> X-Mailer: git-send-email 2.35.1.574.g5d30c73bfb-goog Subject: [PATCH v2 2/7] KVM: x86: Invoke kvm_mmu_unload() directly on CR4.PCIDE change From: Sean Christopherson To: Paolo Bonzini , Christian Borntraeger , Janosch Frank Cc: David Hildenbrand , Claudio Imbrenda , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace a KVM_REQ_MMU_RELOAD request with a direct kvm_mmu_unload() call when the guest's CR4.PCIDE changes. This will allow tweaking the logic of KVM_REQ_MMU_RELOAD to free only obsolete/invalid roots, which is the historical intent of KVM_REQ_MMU_RELOAD. The recent PCIDE behavior is the only user of KVM_REQ_MMU_RELOAD that doesn't mark affected roots as obsolete, needs to unconditionally unload the entire MMU, _and_ affects only the current vCPU. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2157284d05b0..579b26ffc124 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1077,7 +1077,7 @@ void kvm_post_set_cr4(struct kvm_vcpu *vcpu, unsigned= long old_cr4, unsigned lon */ if (!tdp_enabled && (cr4 & X86_CR4_PCIDE) && !(old_cr4 & X86_CR4_PCIDE)) - kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu); + kvm_mmu_unload(vcpu); =20 /* * The TLB has to be flushed for all PCIDs if any of the following --=20 2.35.1.574.g5d30c73bfb-goog From nobody Tue Jun 23 21:23:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08441C433F5 for ; Fri, 25 Feb 2022 18:23:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232930AbiBYSXr (ORCPT ); Fri, 25 Feb 2022 13:23:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232308AbiBYSX3 (ORCPT ); Fri, 25 Feb 2022 13:23:29 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A81991029E5 for ; Fri, 25 Feb 2022 10:22:55 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id bj8-20020a056a02018800b0035ec8c16f0bso3059185pgb.11 for ; Fri, 25 Feb 2022 10:22:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=uyp+YpR2LCYPHpmAeEJtjNTYELJqSix3hhuKEFkcmAA=; b=K/9NSmTITbjmwLiBOKORmckVTpBtCOjqt8lM1ExKYGpgAuDfMMyKDuPAoxGFdQQmyO Cl/ZSWyRgjEsfD324NAAE7370ShErimNCl8bDVTbiDi9haRm21z+Lfzg6UO7EVjC4QIz 0b5azA44faCG+4MUUYxMY7rRTfxPaiN3PdpTmzbZkXD97uZboDH6t+NKrk3XYbUoN+fN ZGgkN7NdIl00CZWchQwR+ORPvF4iJZwkdc941znlbhq3bqKlFB8bhW7HtC2n9abovu6x ccCEz7dl+1u3mgWVgvF/H8fzUWaYgKpUgHEKHklpNZi3oWF/2sS4xwue0wqCIGhAIwAM zn6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=uyp+YpR2LCYPHpmAeEJtjNTYELJqSix3hhuKEFkcmAA=; b=U/uJzOFLf7RLTCpRR9AXy2opZnWP+uocr9HT1D1fuQXyikZcNwxzjzUZ1914RUGtxH f2awZUikZQitz5Fqof/xiYkPnYahZCyoG4k/VacQy0a2nmCIBVW+QF8geP5qqCJZ6S2A ScwNQuJGK7hDJJyZu+VTcdipQWgBFtwKH0ufjwtALtPJjmueS8if0eBvKYhK87Vcbkij wQFzhg5EdixdaCc3QYWBOe51uUOdMtH8vYkzMHO/zDYs2UwSdCVBRnTacxnQu0II4NBq U72ePMwKhbBeClkIMpTXy45U8I53s/oBakJUnoiaKrfN3gfu9R/NJf7Tvp/MTJvzTpR4 VHMA== X-Gm-Message-State: AOAM531ku6+IyVvidC8HZeOSzOBNvV725wcX2T5ehq/wqHdl0Yph+mGu 7SrkQRov/F6aH/nJVJndn6+24qqqAqg= X-Google-Smtp-Source: ABdhPJyOha9usWYX4MGUAJDF97U64lo1sBxpvSHGuIbz/lOMQKj21CeheradWAoSeC33U/vg9C8Yo7X2eLE= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a62:770a:0:b0:4e0:2547:9219 with SMTP id s10-20020a62770a000000b004e025479219mr8651634pfc.43.1645813375172; Fri, 25 Feb 2022 10:22:55 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 25 Feb 2022 18:22:44 +0000 In-Reply-To: <20220225182248.3812651-1-seanjc@google.com> Message-Id: <20220225182248.3812651-4-seanjc@google.com> Mime-Version: 1.0 References: <20220225182248.3812651-1-seanjc@google.com> X-Mailer: git-send-email 2.35.1.574.g5d30c73bfb-goog Subject: [PATCH v2 3/7] KVM: Drop kvm_reload_remote_mmus(), open code request in x86 users From: Sean Christopherson To: Paolo Bonzini , Christian Borntraeger , Janosch Frank Cc: David Hildenbrand , Claudio Imbrenda , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the generic kvm_reload_remote_mmus() and open code its functionality into the two x86 callers. x86 is (obviously) the only architecture that uses the hook, and is also the only architecture that uses KVM_REQ_MMU_RELOAD in a way that's consistent with the name. That will change in a future patch, as x86's usage when zapping a single shadow page x86 doesn't actually _need_ to reload all vCPUs' MMUs, only MMUs whose root is being zapped actually need to be reloaded. s390 also uses KVM_REQ_MMU_RELOAD, but for a slightly different purpose. Drop the generic code in anticipation of implementing s390 and x86 arch specific requests, which will allow dropping KVM_REQ_MMU_RELOAD entirely. Opportunistically reword the x86 TDP MMU comment to avoid making references to functions (and requests!) when possible, and to remove the rather ambiguous "this". No functional change intended. Cc: Ben Gardon Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 14 +++++++------- include/linux/kvm_host.h | 1 - virt/kvm/kvm_main.c | 5 ----- 3 files changed, 7 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b2c1c4eb6007..32c6d4b33d03 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2353,7 +2353,7 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kv= m, * treats invalid shadow pages as being obsolete. */ if (!is_obsolete_sp(kvm, sp)) - kvm_reload_remote_mmus(kvm); + kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_RELOAD); } =20 if (sp->lpage_disallowed) @@ -5639,11 +5639,11 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm) */ kvm->arch.mmu_valid_gen =3D kvm->arch.mmu_valid_gen ? 0 : 1; =20 - /* In order to ensure all threads see this change when - * handling the MMU reload signal, this must happen in the - * same critical section as kvm_reload_remote_mmus, and - * before kvm_zap_obsolete_pages as kvm_zap_obsolete_pages - * could drop the MMU lock and yield. + /* + * In order to ensure all vCPUs drop their soon-to-be invalid roots, + * invalidating TDP MMU roots must be done while holding mmu_lock for + * write and in the same critical section as making the reload request, + * e.g. before kvm_zap_obsolete_pages() could drop mmu_lock and yield. */ if (is_tdp_mmu_enabled(kvm)) kvm_tdp_mmu_invalidate_all_roots(kvm); @@ -5656,7 +5656,7 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm) * Note: we need to do this under the protection of mmu_lock, * otherwise, vcpu would purge shadow page but miss tlb flush. */ - kvm_reload_remote_mmus(kvm); + kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_RELOAD); =20 kvm_zap_obsolete_pages(kvm); =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index f11039944c08..0aeb47cffd43 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1325,7 +1325,6 @@ int kvm_vcpu_yield_to(struct kvm_vcpu *target); void kvm_vcpu_on_spin(struct kvm_vcpu *vcpu, bool usermode_vcpu_not_eligib= le); =20 void kvm_flush_remote_tlbs(struct kvm *kvm); -void kvm_reload_remote_mmus(struct kvm *kvm); =20 #ifdef KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE int kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int min); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 83c57bcc6eb6..66bb1631cb89 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -354,11 +354,6 @@ void kvm_flush_remote_tlbs(struct kvm *kvm) EXPORT_SYMBOL_GPL(kvm_flush_remote_tlbs); #endif =20 -void kvm_reload_remote_mmus(struct kvm *kvm) -{ - kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_RELOAD); -} - #ifdef KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE static inline void *mmu_memory_cache_alloc_obj(struct kvm_mmu_memory_cache= *mc, gfp_t gfp_flags) --=20 2.35.1.574.g5d30c73bfb-goog From nobody Tue Jun 23 21:23:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12530C433EF for ; Fri, 25 Feb 2022 18:23:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232910AbiBYSXk (ORCPT ); Fri, 25 Feb 2022 13:23:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232328AbiBYSX3 (ORCPT ); Fri, 25 Feb 2022 13:23:29 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A8C56E8CE for ; Fri, 25 Feb 2022 10:22:57 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id b5-20020a631b05000000b00373bd90134dso3040556pgb.22 for ; Fri, 25 Feb 2022 10:22:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=TGmyZn+f+4hIZ/LUaz810TG9h9t+KkL+VEw1/fW9xZE=; b=l764Sm9hNNUi5MdxS2VNyezNzrSlGCVCo9C+zq2apXu0W1+0BXCc6ljJQVUvHvgteL qEOQKVZVi2DJsqkB8BPkvybHNaCO2+7pNHN2/pHqeG29cTD+a3bfmsGeXKRlcCqTQagq uGrFW3Xzzn3VwZT1pZ4F8sMtQfxDlv2be9nnTRj5fsS+RcUvUjHS9kHDBnoNFiYMG7XA rpdbotPAqD/rZVRoc4Za1Yvr5AaiF1x7voLdebaFmKigmdjcZxAhBHob7Ho29I6QpHaa 4RTrKpvOP/eMjnPut7lHucTw1uGORrD/YYsrIERRxJhieThe3AoVyhUBgELliTQz3sCD Uv9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=TGmyZn+f+4hIZ/LUaz810TG9h9t+KkL+VEw1/fW9xZE=; b=blkzoE927v1h9Cg1gKCBLLJQm7Bd4IoKDPgzMF5rGaUl11V344lkButhQBjPhr3RaX RNnxQ3a5zB8d87U0gGS55PMib4TQ5EjTxF1nwWV7U59T+ukVPm4GQ/3iOD+xrmRz7Pm+ Z416MrOo4Gh4WVzZUwtzPWpCPVN2A3xBm0UDcTixCSGnPI4SVCb6g9ms+r3pF/CW5Y7Z 5fu5Lvpn+IyWkUndJaFvTSmCf7emX/qn+1sq26OMbpbDL3AJcSmkHqjRakFSWCEzDhti NttKuFt7FDilx6nVfBTIx5rIgtHgrZqIwsnmp0J074LKjpT3S2bB38hMN9GgZ/do1ox/ NVWg== X-Gm-Message-State: AOAM531gUdr6lxZKBdh5MsHyM+1QP59ikY4j+HLhpk3Yvlumw4Bt03gy 4nezE9t0rPurTYjF31Arf2uWg2aVTlg= X-Google-Smtp-Source: ABdhPJwLaz0GsL6EieN8giNkqv3Js05m/fcvm98V2qIpCHzZYOedItWZtug/UoBLqIl1xU34AjpupMWfF2M= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:90a:3d0f:b0:1bb:80e9:3b45 with SMTP id h15-20020a17090a3d0f00b001bb80e93b45mr4298597pjc.31.1645813376623; Fri, 25 Feb 2022 10:22:56 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 25 Feb 2022 18:22:45 +0000 In-Reply-To: <20220225182248.3812651-1-seanjc@google.com> Message-Id: <20220225182248.3812651-5-seanjc@google.com> Mime-Version: 1.0 References: <20220225182248.3812651-1-seanjc@google.com> X-Mailer: git-send-email 2.35.1.574.g5d30c73bfb-goog Subject: [PATCH v2 4/7] KVM: x86/mmu: Zap only obsolete roots if a root shadow page is zapped From: Sean Christopherson To: Paolo Bonzini , Christian Borntraeger , Janosch Frank Cc: David Hildenbrand , Claudio Imbrenda , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Zap only obsolete roots when responding to zapping a single root shadow page. Because KVM keeps root_count elevated when stuffing a previous root into its PGD cache, shadowing a 64-bit guest means that zapping any root causes all vCPUs to reload all roots, even if their current root is not affected by the zap. For many kernels, zapping a single root is a frequent operation, e.g. in Linux it happens whenever an mm is dropped, e.g. process exits, etc... Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon --- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 65 +++++++++++++++++++++++++++++---- arch/x86/kvm/x86.c | 4 +- 4 files changed, 63 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 713e08f62385..343041e892c6 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -102,6 +102,8 @@ #define KVM_REQ_MSR_FILTER_CHANGED KVM_ARCH_REQ(29) #define KVM_REQ_UPDATE_CPU_DIRTY_LOGGING \ KVM_ARCH_REQ_FLAGS(30, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \ + KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) =20 #define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \ diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 1d0c1904d69a..bf8dbc4bb12a 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -80,6 +80,7 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 erro= r_code, =20 int kvm_mmu_load(struct kvm_vcpu *vcpu); void kvm_mmu_unload(struct kvm_vcpu *vcpu); +void kvm_mmu_free_obsolete_roots(struct kvm_vcpu *vcpu); void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu); void kvm_mmu_sync_prev_roots(struct kvm_vcpu *vcpu); =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 32c6d4b33d03..825996408465 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2310,7 +2310,7 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kv= m, struct list_head *invalid_list, int *nr_zapped) { - bool list_unstable; + bool list_unstable, zapped_root =3D false; =20 trace_kvm_mmu_prepare_zap_page(sp); ++kvm->stat.mmu_shadow_zapped; @@ -2352,14 +2352,20 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *= kvm, * in kvm_mmu_zap_all_fast(). Note, is_obsolete_sp() also * treats invalid shadow pages as being obsolete. */ - if (!is_obsolete_sp(kvm, sp)) - kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_RELOAD); + zapped_root =3D !is_obsolete_sp(kvm, sp); } =20 if (sp->lpage_disallowed) unaccount_huge_nx_page(kvm, sp); =20 sp->role.invalid =3D 1; + + /* + * Make the request to free obsolete roots after marking the root + * invalid, otherwise other vCPUs may not see it as invalid. + */ + if (zapped_root) + kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_FREE_OBSOLETE_ROOTS); return list_unstable; } =20 @@ -3947,7 +3953,7 @@ static bool is_page_fault_stale(struct kvm_vcpu *vcpu, * previous root, then __kvm_mmu_prepare_zap_page() signals all vCPUs * to reload even if no vCPU is actively using the root. */ - if (!sp && kvm_test_request(KVM_REQ_MMU_RELOAD, vcpu)) + if (!sp && kvm_test_request(KVM_REQ_MMU_FREE_OBSOLETE_ROOTS, vcpu)) return true; =20 return fault->slot && @@ -4180,8 +4186,8 @@ void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new= _pgd) /* * It's possible that the cached previous root page is obsolete because * of a change in the MMU generation number. However, changing the - * generation number is accompanied by KVM_REQ_MMU_RELOAD, which will - * free the root set here and allocate a new one. + * generation number is accompanied by KVM_REQ_MMU_FREE_OBSOLETE_ROOTS, + * which will free the root set here and allocate a new one. */ kvm_make_request(KVM_REQ_LOAD_MMU_PGD, vcpu); =20 @@ -5085,6 +5091,51 @@ void kvm_mmu_unload(struct kvm_vcpu *vcpu) vcpu_clear_mmio_info(vcpu, MMIO_GVA_ANY); } =20 +static bool is_obsolete_root(struct kvm *kvm, hpa_t root_hpa) +{ + struct kvm_mmu_page *sp; + + if (!VALID_PAGE(root_hpa)) + return false; + + /* + * When freeing obsolete roots, treat roots as obsolete if they don't + * have an associated shadow page. This does mean KVM will get false + * positives and free roots that don't strictly need to be freed, but + * such false positives are relatively rare: + * + * (a) only PAE paging and nested NPT has roots without shadow pages + * (b) remote reloads due to a memslot update obsoletes _all_ roots + * (c) KVM doesn't track previous roots for PAE paging, and the guest + * is unlikely to zap an in-use PGD. + */ + sp =3D to_shadow_page(root_hpa); + return !sp || is_obsolete_sp(kvm, sp); +} + +static void __kvm_mmu_free_obsolete_roots(struct kvm *kvm, struct kvm_mmu = *mmu) +{ + unsigned long roots_to_free =3D 0; + int i; + + if (is_obsolete_root(kvm, mmu->root.hpa)) + roots_to_free |=3D KVM_MMU_ROOT_CURRENT; + + for (i =3D 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) { + if (is_obsolete_root(kvm, mmu->root.hpa)) + roots_to_free |=3D KVM_MMU_ROOT_PREVIOUS(i); + } + + if (roots_to_free) + kvm_mmu_free_roots(kvm, mmu, roots_to_free); +} + +void kvm_mmu_free_obsolete_roots(struct kvm_vcpu *vcpu) +{ + __kvm_mmu_free_obsolete_roots(vcpu->kvm, &vcpu->arch.root_mmu); + __kvm_mmu_free_obsolete_roots(vcpu->kvm, &vcpu->arch.guest_mmu); +} + static bool need_remote_flush(u64 old, u64 new) { if (!is_shadow_present_pte(old)) @@ -5656,7 +5707,7 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm) * Note: we need to do this under the protection of mmu_lock, * otherwise, vcpu would purge shadow page but miss tlb flush. */ - kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_RELOAD); + kvm_make_all_cpus_request(kvm, KVM_REQ_MMU_FREE_OBSOLETE_ROOTS); =20 kvm_zap_obsolete_pages(kvm); =20 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 579b26ffc124..d6bf0562c4c4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9856,8 +9856,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) goto out; } } - if (kvm_check_request(KVM_REQ_MMU_RELOAD, vcpu)) - kvm_mmu_unload(vcpu); + if (kvm_check_request(KVM_REQ_MMU_FREE_OBSOLETE_ROOTS, vcpu)) + kvm_mmu_free_obsolete_roots(vcpu); if (kvm_check_request(KVM_REQ_MIGRATE_TIMER, vcpu)) __kvm_migrate_timers(vcpu); if (kvm_check_request(KVM_REQ_MASTERCLOCK_UPDATE, vcpu)) --=20 2.35.1.574.g5d30c73bfb-goog From nobody Tue Jun 23 21:23:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22E66C433EF for ; Fri, 25 Feb 2022 18:23:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232366AbiBYSXv (ORCPT ); Fri, 25 Feb 2022 13:23:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232183AbiBYSXb (ORCPT ); Fri, 25 Feb 2022 13:23:31 -0500 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7ED61029E5 for ; Fri, 25 Feb 2022 10:22:58 -0800 (PST) Received: by mail-pg1-x549.google.com with SMTP id n188-20020a6340c5000000b003747606cb0dso3054220pga.6 for ; Fri, 25 Feb 2022 10:22:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=yJjWTGvj3xklna+YumgfwjW/70CkdMmVGz7JULQJc9w=; b=CQmcUVe96I5qFAe4jXJ6N24Z0ricyasxuL9MqcTlxQ9hAjj2K5BzVagYNRYvvxHXDU RHryQu0J0EwAqWcJ6xQ524QpN6qApH6AraFM+RRtmeZ51JPN604CQvnR6RnU5V8Jh4RM FtLlQ4SLJCco5XLwRzMTYTp73sE15/eYjZ0uqBkUA3cdFA38FG4iLC+dRoBixOVkjR/t kXMeIDO99qxMbkgRdnAVKdYm9kSstrFQbh7UpolsLsBcOZoJCZVAVLJgtUrxKPvSVJJQ j9WRUPkxwqU+/YEmmsiWMZQ+lt5O3vitEpjhyqm41PhfcRdodpnz7ihoFJaMS850sNz5 FGAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=yJjWTGvj3xklna+YumgfwjW/70CkdMmVGz7JULQJc9w=; b=g9R2nxD+RRCQY8yIDs0kIxYD58JeoqXRsszh5JTrPCeuGlDdt3qCsvEJLPOcsDYSRg CMG+T5PoyWnq0VUMbE3Gv5GvmuvPURn/xu3dKDyU4BI6D2PuH9sssYpbgHuJyoyQ4HIT IN9eIpTKXh0Oar33i8l+lIx0s0hU9Q9F57WavBwBg1uTPPNdT66+HsmxZPAjMxTQKxmx O1ZH3RyCY5dgLquUmSZ4Qsh9oEJ+gb3apXU14KMmUKat4NQlXJVkvV858Am3vq5Sy46S qrRRjX1i2FT373o8ab1yAXK4jCxVuN3k10GKRj4q9u9ejezqqbHGeiT6Zkf9jw9Zs5CP VPzQ== X-Gm-Message-State: AOAM532IoP0H9gU0LGBiQw1fZyMRtpur2n13nBc9rY8dCumjrQd4vLDN P9VhUVoVFz+RHdKyHKxu2A4p/tt9Lbo= X-Google-Smtp-Source: ABdhPJwqiIRfaVu24g/26s1NG9RDLXkL5Soz2O1/mDxe1b9k+mFjwrTrkF1v4W13AzJ6xBLpa8bZDPY/cKc= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:90a:67c1:b0:1bc:f062:62e8 with SMTP id g1-20020a17090a67c100b001bcf06262e8mr3951454pjm.86.1645813378248; Fri, 25 Feb 2022 10:22:58 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 25 Feb 2022 18:22:46 +0000 In-Reply-To: <20220225182248.3812651-1-seanjc@google.com> Message-Id: <20220225182248.3812651-6-seanjc@google.com> Mime-Version: 1.0 References: <20220225182248.3812651-1-seanjc@google.com> X-Mailer: git-send-email 2.35.1.574.g5d30c73bfb-goog Subject: [PATCH v2 5/7] KVM: s390: Replace KVM_REQ_MMU_RELOAD usage with arch specific request From: Sean Christopherson To: Paolo Bonzini , Christian Borntraeger , Janosch Frank Cc: David Hildenbrand , Claudio Imbrenda , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an arch request, KVM_REQ_REFRESH_GUEST_PREFIX, to deal with guest prefix changes instead of piggybacking KVM_REQ_MMU_RELOAD. This will allow for the removal of the generic KVM_REQ_MMU_RELOAD, which isn't actually used by generic KVM. No functional change intended. Reviewed-by: Claudio Imbrenda Reviewed-by: Janosch Frank Signed-off-by: Sean Christopherson --- arch/s390/include/asm/kvm_host.h | 2 ++ arch/s390/kvm/kvm-s390.c | 8 ++++---- arch/s390/kvm/kvm-s390.h | 2 +- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_h= ost.h index a22c9266ea05..766028d54a3e 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -45,6 +45,8 @@ #define KVM_REQ_START_MIGRATION KVM_ARCH_REQ(3) #define KVM_REQ_STOP_MIGRATION KVM_ARCH_REQ(4) #define KVM_REQ_VSIE_RESTART KVM_ARCH_REQ(5) +#define KVM_REQ_REFRESH_GUEST_PREFIX \ + KVM_ARCH_REQ_FLAGS(6, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) =20 #define SIGP_CTRL_C 0x80 #define SIGP_CTRL_SCN_MASK 0x3f diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 577f1ead6a51..db8c113562cf 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -3394,7 +3394,7 @@ static void kvm_gmap_notifier(struct gmap *gmap, unsi= gned long start, if (prefix <=3D end && start <=3D prefix + 2*PAGE_SIZE - 1) { VCPU_EVENT(vcpu, 2, "gmap notifier for %lx-%lx", start, end); - kvm_s390_sync_request(KVM_REQ_MMU_RELOAD, vcpu); + kvm_s390_sync_request(KVM_REQ_REFRESH_GUEST_PREFIX, vcpu); } } } @@ -3796,19 +3796,19 @@ static int kvm_s390_handle_requests(struct kvm_vcpu= *vcpu) if (!kvm_request_pending(vcpu)) return 0; /* - * We use MMU_RELOAD just to re-arm the ipte notifier for the + * If the guest prefix changed, re-arm the ipte notifier for the * guest prefix page. gmap_mprotect_notify will wait on the ptl lock. * This ensures that the ipte instruction for this request has * already finished. We might race against a second unmapper that * wants to set the blocking bit. Lets just retry the request loop. */ - if (kvm_check_request(KVM_REQ_MMU_RELOAD, vcpu)) { + if (kvm_check_request(KVM_REQ_REFRESH_GUEST_PREFIX, vcpu)) { int rc; rc =3D gmap_mprotect_notify(vcpu->arch.gmap, kvm_s390_get_prefix(vcpu), PAGE_SIZE * 2, PROT_WRITE); if (rc) { - kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu); + kvm_make_request(KVM_REQ_REFRESH_GUEST_PREFIX, vcpu); return rc; } goto retry; diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h index 098831e815e6..45b7c1edd85f 100644 --- a/arch/s390/kvm/kvm-s390.h +++ b/arch/s390/kvm/kvm-s390.h @@ -105,7 +105,7 @@ static inline void kvm_s390_set_prefix(struct kvm_vcpu = *vcpu, u32 prefix) prefix); vcpu->arch.sie_block->prefix =3D prefix >> GUEST_PREFIX_SHIFT; kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); - kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu); + kvm_make_request(KVM_REQ_REFRESH_GUEST_PREFIX, vcpu); } =20 static inline u64 kvm_s390_get_base_disp_s(struct kvm_vcpu *vcpu, u8 *ar) --=20 2.35.1.574.g5d30c73bfb-goog From nobody Tue Jun 23 21:23:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B344BC433F5 for ; Fri, 25 Feb 2022 18:23:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232504AbiBYSXz (ORCPT ); Fri, 25 Feb 2022 13:23:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232566AbiBYSXd (ORCPT ); Fri, 25 Feb 2022 13:23:33 -0500 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA49712A75B for ; Fri, 25 Feb 2022 10:23:00 -0800 (PST) Received: by mail-pl1-x64a.google.com with SMTP id x18-20020a170902b41200b0014fc2665bddso3408248plr.0 for ; Fri, 25 Feb 2022 10:23:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=OKpSENCWyXwHBdjGJ3gByFACo60D6cjEVL83LoqsyAI=; b=b9RI+WDrYKbDwWxRpfBrGKJ+M0uWUYG/R9HMlU4/mEqPTWu7oVtS1VUBgFgWjZ++ZP qR7+DT+WJGieOarv8OsgBF6aZHUnul3W3dcki20lr/dtvbRybp2muAitL2U05f51Ishp 6keyKWiJjZBWG/rIx5ewTqUShB0uvTvoNxYZOCz/NIiY5uOzPagL0MB+A1+anitssEbE NmT/DbUfJ8sAXFaBQmTOfpKmmqacNgBbRIxzbZPqI7vmHCB5kmBDot/QnQIzwiitk4ob cDTNizSkwp2Xg707XqCJDUTVTz0k3eXvcUkGGZ76FFJ+7R81vfgTkmat4/9b4wuNQGJK tPqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=OKpSENCWyXwHBdjGJ3gByFACo60D6cjEVL83LoqsyAI=; b=OPK2gwwwioNBBV/gFJeQA9fZz9kLCcIFocGO72tHGd8EXO0fW18BEX++SP+HnO5RyJ 7aZJQ1629lSol305xgBvc696TkRpVAJJiIL8slK8VB7oLG12PVfNA1o3CgWzTBzKgKtZ JpU3dHA0WxqRx/s6kCCxIp/6KCIqbc0CSmY/WbIK7fNzEnGSVu6ESECedpmWwBxIhZAp xNM+Xgctn7B7j62pIi+lZyTbK0dMTk1Jt3FfWRmpMk/aq4ITUv8xOXQTiUHRpQQzJ1Ep DWxU2lrvVtZ/QX63uMQyps6y1qQQNs6t64wD5YLIikDaee3x9k25slSRPCYqV3uzt0Xf cPCg== X-Gm-Message-State: AOAM532401niM7VIBKScxKk49U+Qj2ApUfuw3AGWKzxB1qr1vVkmEVDN s7RLtX3AmCLaYXfs97A9dlpZkIArcAw= X-Google-Smtp-Source: ABdhPJwUNaTUFvmEZHBOnSpxwiw/lZ+XqgCu1OS+l26XAQ4480iv62UAgTn9TFAeEJvztnJGJ5LksjK+jVc= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a17:90b:3c01:b0:1bc:b160:7811 with SMTP id pb1-20020a17090b3c0100b001bcb1607811mr4342903pjb.164.1645813379912; Fri, 25 Feb 2022 10:22:59 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 25 Feb 2022 18:22:47 +0000 In-Reply-To: <20220225182248.3812651-1-seanjc@google.com> Message-Id: <20220225182248.3812651-7-seanjc@google.com> Mime-Version: 1.0 References: <20220225182248.3812651-1-seanjc@google.com> X-Mailer: git-send-email 2.35.1.574.g5d30c73bfb-goog Subject: [PATCH v2 6/7] KVM: Drop KVM_REQ_MMU_RELOAD and update vcpu-requests.rst documentation From: Sean Christopherson To: Paolo Bonzini , Christian Borntraeger , Janosch Frank Cc: David Hildenbrand , Claudio Imbrenda , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the now unused KVM_REQ_MMU_RELOAD, shift KVM_REQ_VM_DEAD into the unoccupied space, and update vcpu-requests.rst, which was missing an entry for KVM_REQ_VM_DEAD. Switching KVM_REQ_VM_DEAD to entry '1' also fixes the stale comment about bits 4-7 being reserved. Reviewed-by: Claudio Imbrenda Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon --- Documentation/virt/kvm/vcpu-requests.rst | 7 +++---- include/linux/kvm_host.h | 3 +-- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/Documentation/virt/kvm/vcpu-requests.rst b/Documentation/virt/= kvm/vcpu-requests.rst index ad2915ef7020..b61d48aec36c 100644 --- a/Documentation/virt/kvm/vcpu-requests.rst +++ b/Documentation/virt/kvm/vcpu-requests.rst @@ -112,11 +112,10 @@ KVM_REQ_TLB_FLUSH choose to use the common kvm_flush_remote_tlbs() implementation will need to handle this VCPU request. =20 -KVM_REQ_MMU_RELOAD +KVM_REQ_VM_DEAD =20 - When shadow page tables are used and memory slots are removed it's - necessary to inform each VCPU to completely refresh the tables. This - request is used for that. + This request informs all VCPUs that the VM is dead and unusable, e.g. du= e to + fatal error or because the VM's state has been intentionally destroyed. =20 KVM_REQ_UNBLOCK =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 0aeb47cffd43..9536ffa0473b 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -153,10 +153,9 @@ static inline bool is_error_page(struct page *page) * Bits 4-7 are reserved for more arch-independent bits. */ #define KVM_REQ_TLB_FLUSH (0 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_W= AKEUP) -#define KVM_REQ_MMU_RELOAD (1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_W= AKEUP) +#define KVM_REQ_VM_DEAD (1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_W= AKEUP) #define KVM_REQ_UNBLOCK 2 #define KVM_REQ_UNHALT 3 -#define KVM_REQ_VM_DEAD (4 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_W= AKEUP) #define KVM_REQ_GPC_INVALIDATE (5 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_W= AKEUP) #define KVM_REQUEST_ARCH_BASE 8 =20 --=20 2.35.1.574.g5d30c73bfb-goog From nobody Tue Jun 23 21:23:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04C36C433EF for ; Fri, 25 Feb 2022 18:23:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232969AbiBYSX6 (ORCPT ); Fri, 25 Feb 2022 13:23:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232733AbiBYSXf (ORCPT ); Fri, 25 Feb 2022 13:23:35 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35DCF16DAD2 for ; Fri, 25 Feb 2022 10:23:02 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id bh9-20020a056a02020900b0036c0d29eb3eso3061450pgb.9 for ; Fri, 25 Feb 2022 10:23:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=L4MqIEc/JbsQtnFnEaV173FD1GR2zeygOeGJKcIQHTM=; b=j/8rVgwYSLF3S4KclnS2WyxYGKq78Q2Pofky6B4ClrHiTlJMvAxQKPBPDGdnxYTHsb hWa5sLtrz+TbRfNlOWBc4U5GVLwtNNCOpckkzebNdjafWEgSepQR/Tx3PVA74hnePoYe xP/FJLDwab7dkTjeOQtTw4XYYbSSTdfn5l26SMVawmzCvxGn0PsUaQtrYTzTAf2eHRhv UHh/Fs9+sDwrPhDKZ8dAZ86rPtP+W71Tf2mhqhXbk+bdF02skbuCTWcGfkdJyOEd2qae vRIBLgSiv09yctbEh50CsDFVMftonFAzi5fVFFtJUei2H/ei/AK4uCh73Fj+BEMFRqMP WtEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=L4MqIEc/JbsQtnFnEaV173FD1GR2zeygOeGJKcIQHTM=; b=Z3weMugoWGa4mUXRDCMm9HqqD69jCUo98JTXXEx98CRjZtZwulekK1b7diW2qe584G yFliRQKU1m46ApjawooRMRLAcxU9Wy+ckbXG3elR37QFii67p4g+NDsA7fqfCFKeaEns s3hMvtCoe2hyPLHRpIN1m5GmM4OtGj4Gfj+UUSNuvObj32G3qPy33F+U/RNKLZMUViuO PWhpUwalbqUPMmbomnmZXWOalmBkCZJbjTbVc0LiptjxVh9RbsLk0YGl/x/AmwZTrDrB xfYwqOoTmGpWGXEJ1JfA0M849Pw62gtYzhnZWOf6EJ9TxC/9Znv56zzxd+RZV6BAeOQq gmFA== X-Gm-Message-State: AOAM532zfy9mCEt2Qv3bOgi8n9Js1z+WIqhm+a8bXGtX3CO/QIx6fvCs OvJiQUopbAt6rqxZ3DZGXTLMCz8BKFA= X-Google-Smtp-Source: ABdhPJw93c17M/Wa7z94CmL+MMkOWMzcXaEGjy6d/i23LqFNc4Iax0+2elbEtSqmyNeMnzOh+XEeqm0YPKo= X-Received: from seanjc.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e5]) (user=seanjc job=sendgmr) by 2002:a62:84d3:0:b0:4e1:b5c:1dd4 with SMTP id k202-20020a6284d3000000b004e10b5c1dd4mr8798283pfd.20.1645813381710; Fri, 25 Feb 2022 10:23:01 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 25 Feb 2022 18:22:48 +0000 In-Reply-To: <20220225182248.3812651-1-seanjc@google.com> Message-Id: <20220225182248.3812651-8-seanjc@google.com> Mime-Version: 1.0 References: <20220225182248.3812651-1-seanjc@google.com> X-Mailer: git-send-email 2.35.1.574.g5d30c73bfb-goog Subject: [PATCH v2 7/7] KVM: WARN if is_unsync_root() is called on a root without a shadow page From: Sean Christopherson To: Paolo Bonzini , Christian Borntraeger , Janosch Frank Cc: David Hildenbrand , Claudio Imbrenda , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" WARN and bail if is_unsync_root() is passed a root for which there is no shadow page, i.e. is passed the physical address of one of the special roots, which do not have an associated shadow page. The current usage squeaks by without bug reports because neither kvm_mmu_sync_roots() nor kvm_mmu_sync_prev_roots() calls the helper with pae_root or pml4_root, and 5-level AMD CPUs are not generally available, i.e. no one can coerce KVM into calling is_unsync_root() on pml5_root. Note, this doesn't fix the mess with 5-level nNPT, it just (hopefully) prevents KVM from crashing. Cc: Lai Jiangshan Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 825996408465..3e7c8ad5bed9 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3634,6 +3634,14 @@ static bool is_unsync_root(hpa_t root) */ smp_rmb(); sp =3D to_shadow_page(root); + + /* + * PAE roots (somewhat arbitrarily) aren't backed by shadow pages, the + * PDPTEs for a given PAE root need to be synchronized individually. + */ + if (WARN_ON_ONCE(!sp)) + return false; + if (sp->unsync || sp->unsync_children) return true; =20 --=20 2.35.1.574.g5d30c73bfb-goog