From nobody Wed Dec 17 19:02:34 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B36011F5EA for ; Sat, 15 Mar 2025 02:40:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742006416; cv=none; b=J6DGEyBESpwIfShvYYKMl5CQkxeCv8bB4H4XpWrKPM0D4qAShmsyijBSia38cuXiWv76sZE8UX2cVpAcP3NiZJJrenq+A4Ip13dp3annNQkwpQ6pvlIyv9DN3MVompYKvToYuNYskErUZzA5jnPs1nCtZX+N1Vplh6HHaRVNZz4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742006416; c=relaxed/simple; bh=hHQMhqoJXwkU2iQFu0bAessW5nDhl6z0kwwuyI+P114=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YfkbI5sxXwdQxspZK94nk+UcmYIS2FPoxR3v79rzNQUd+tBfsNmBhC7a2hheoghuNk+Of1i6snUVG8hkshNV6TRO/kT1af3GqioDLgJQgE1Ac+0Od1WKqeNbkFapSU+rn1tVXJci4ndzy+OJWhBWdbcjTLtPK7M5bbZzz24kBbU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XNAC9G4O; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XNAC9G4O" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2217a4bfcc7so44172565ad.3 for ; Fri, 14 Mar 2025 19:40:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742006414; x=1742611214; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8AnAaiMp1pogiyQq/P5ffzpAjmvR1I8H8po4dFjhB2Q=; b=XNAC9G4Oj7OlOkf8yfqTMuh05Pdg2+fSBDlx8HIkh3Gjq65VQvIev61W+TRcmFbnJp 6v4GyPWctwnvTcsaQpSFnQHlXuNDY/qZHV5wsC2ZPnO9gjqyGk1UisJ828yPN/yJl2MT b2kL/AiGSjAZlTi22YTcrIle4pkvyE5SkYvRgT6+9kbiIT2YTr9pUjDIH7u739uK775d faxlXNuuvrU1aOA0WKgitMnURQY7VoEiqaxMg+Pgxji8lfO0d4w3c+evYXGodpQTlj84 LvT6y4ICpCAYbIZlRd0ynecOE74iGtsos17PblTxcVyxFt0HNBJT/iGVcu4QuGmCWWq2 Qpnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742006414; x=1742611214; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8AnAaiMp1pogiyQq/P5ffzpAjmvR1I8H8po4dFjhB2Q=; b=YGUwfHZuuwXBOLT1L/2ZHMizYIoMVKTL1T/FjBbRVeVJkVu8LOjapfBmcQrJbsDHF8 l7oQehd9fC6RyKghLTBYLCZ4LcG5Say9D1Z0eTdipqwxj+kzzcs4yZjP28AAupfNHzWC KG4QDh/YfdBZWwm5b9/BFfBeKd1Yrga+qWbb/T2IL500Rf84HFON86kto0H/MuNj3Q97 64JQdo3b4yzIJ9adLoqrI4v8UYY3RceOAea+adYGM67BaKGpg9vTlU/X2TOL0hSb+GxV ZkiT3XnpxStm3tBndp6A8ausHYjSr0KtUUWd27V63WAnbQ3ZeOovMgl6vbnuiDdKPSIJ 5lag== X-Forwarded-Encrypted: i=1; AJvYcCUbxeNsUtkgQan8F0jWos3xT6WpOPwtXIQnkqs+CBGtyOJuixfa39uCva7uBMJVNCDKfn0YXbgzDq2VFJs=@vger.kernel.org X-Gm-Message-State: AOJu0YxBPT9gz/H+/3+Ytgpo2zQDsIGDiBUvUAyrp9wWp3YgcCc08TSo BuhpDHRh1UPe9ftZoKE+mITzRc4+OZbToI2UqJ25w+5VdMYeRNvQYdiJ5SPQI8y4o1UgTxBePCI M6w== X-Google-Smtp-Source: AGHT+IEuUNeaevdUXvWFAIJ8JsyZ358tix7OrYJ/fGDFTE4Ye0y79h20U/9yKG7qAiftx1GmhHfquVa3dWI= X-Received: from pjbqn6.prod.google.com ([2002:a17:90b:3d46:b0:2f8:4024:b59a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:190:b0:223:525b:2a7 with SMTP id d9443c01a7336-225e0a4f8demr55785875ad.15.1742006414027; Fri, 14 Mar 2025 19:40:14 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 14 Mar 2025 19:40:08 -0700 In-Reply-To: <20250315024010.2360884-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250315024010.2360884-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250315024010.2360884-2-seanjc@google.com> Subject: [PATCH 1/3] KVM: x86/mmu: Dynamically allocate shadow MMU's hashed page list From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Dynamically allocate the (massive) array of hashed lists used to track shadow pages, as the array itself is 32KiB, i.e. is an order-3 allocation all on its own, and is *exactly* an order-3 allocation. Dynamically allocating the array will allow allocating "struct kvm" using regular kmalloc(), and will also allow deferring allocation of the array until it's actually needed, i.e. until the first shadow root is allocated. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++++++++++- arch/x86/kvm/x86.c | 5 ++++- 3 files changed, 26 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index d881e7d276b1..6ead9e57446a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1332,7 +1332,7 @@ struct kvm_arch { bool has_private_mem; bool has_protected_state; bool pre_fault_allowed; - struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES]; + struct hlist_head *mmu_page_hash; struct list_head active_mmu_pages; /* * A list of kvm_mmu_page structs that, if zapped, could possibly be @@ -1984,7 +1984,7 @@ void kvm_mmu_vendor_module_exit(void); =20 void kvm_mmu_destroy(struct kvm_vcpu *vcpu); int kvm_mmu_create(struct kvm_vcpu *vcpu); -void kvm_mmu_init_vm(struct kvm *kvm); +int kvm_mmu_init_vm(struct kvm *kvm); void kvm_mmu_uninit_vm(struct kvm *kvm); =20 void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 63bb77ee1bb1..b878f2e89dec 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3880,6 +3880,18 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *v= cpu) return r; } =20 +static int kvm_mmu_alloc_page_hash(struct kvm *kvm) +{ + typeof(kvm->arch.mmu_page_hash) h; + + h =3D kcalloc(KVM_NUM_MMU_PAGES, sizeof(*h), GFP_KERNEL_ACCOUNT); + if (!h) + return -ENOMEM; + + kvm->arch.mmu_page_hash =3D h; + return 0; +} + static int mmu_first_shadow_root_alloc(struct kvm *kvm) { struct kvm_memslots *slots; @@ -6673,13 +6685,19 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm) kvm_tdp_mmu_zap_invalidated_roots(kvm, true); } =20 -void kvm_mmu_init_vm(struct kvm *kvm) +int kvm_mmu_init_vm(struct kvm *kvm) { + int r; + kvm->arch.shadow_mmio_value =3D shadow_mmio_value; INIT_LIST_HEAD(&kvm->arch.active_mmu_pages); INIT_LIST_HEAD(&kvm->arch.possible_nx_huge_pages); spin_lock_init(&kvm->arch.mmu_unsync_pages_lock); =20 + r =3D kvm_mmu_alloc_page_hash(kvm); + if (r) + return r; + if (tdp_mmu_enabled) kvm_mmu_init_tdp_mmu(kvm); =20 @@ -6690,6 +6708,7 @@ void kvm_mmu_init_vm(struct kvm *kvm) =20 kvm->arch.split_desc_cache.kmem_cache =3D pte_list_desc_cache; kvm->arch.split_desc_cache.gfp_zero =3D __GFP_ZERO; + return 0; } =20 static void mmu_free_vm_memory_caches(struct kvm *kvm) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 69c20a68a3f0..a1d85740d6e7 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12704,7 +12704,9 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long= type) if (ret) goto out; =20 - kvm_mmu_init_vm(kvm); + ret =3D kvm_mmu_init_vm(kvm); + if (ret) + goto out_cleanup_page_track; =20 ret =3D kvm_x86_call(vm_init)(kvm); if (ret) @@ -12757,6 +12759,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long= type) =20 out_uninit_mmu: kvm_mmu_uninit_vm(kvm); +out_cleanup_page_track: kvm_page_track_cleanup(kvm); out: return ret; --=20 2.49.0.rc1.451.g8f38331e32-goog From nobody Wed Dec 17 19:02:34 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 543F618B47C for ; Sat, 15 Mar 2025 02:40:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742006417; cv=none; b=XknOOkhHUi+wvFQCuxT+lYvVkW8pmyeleHtQkWzJw7XVvh/uo4CdH+w2uYzzCMkFeT7OD98ndDHVpEt7TKJXsGUNLzBua4CHvmXSQn9qibVGb7XY7lpZR/CtWl42Fpb93ZbSZkznuCWSFFpTSo+2ijNZYPxrJnJilfewxJxrDVo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742006417; c=relaxed/simple; bh=U84p6SZxKcFGqIOb3edMWdKOJC3NG2/MOklZXIgpKbI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qyXVcE7GqJJbt/4/3A39QcXRJD9KR6lIAsgy/98KXVWqOrbiPWFWDIGRVSAVum0PCZmhIfczbafAZHkhqxtiNjz2pZZKuNsU4oV4izGYMb5pp4KFEpojGnaaH/to7duWc2VhSKjCMttfcXC42oPMfqeqF94iqGEtbQEARpASnCs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=36b36eS+; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="36b36eS+" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff854a2541so472762a91.0 for ; Fri, 14 Mar 2025 19:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742006415; x=1742611215; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=bMQrAPcfd3nTq7FKRl/uWgH64wo3Ilzhc0co1/LF5U0=; b=36b36eS+Wrm1W4zM1fRaS094HChbvPEUTq/Y+5Iez9EN6HGpBwTIY/20ADlhOpVCDT DN+fbeKBBglsM4OOaZ893t/VuhdyvlHhI+Yr68II9Hhm1teyhIAoMU/RRqr/Xg++nMr4 FbqFnFAn5jEktuYJbHLX3BO1jEw8/ZpMfg1PapU8uh97z1D7VSgW/I/HiJp9qHcaZbat ezWmTnphzb9UpKL3T7WLF/0EXYOUmmgzUmIm1KCrPAQLzZO7KAuiDy5R1Y+alrMZwvcU 8gowYILrflgLSWpFoBtWIbOIgkWN6cl4XUJalzssDw4yoYjMq1fkq8rOtgBWic6LRKxN e9Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742006415; x=1742611215; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bMQrAPcfd3nTq7FKRl/uWgH64wo3Ilzhc0co1/LF5U0=; b=sRyqb0S3t6aouIRh2hmVRqR2IQnDGp1agE9HuGkjXsl7rTpHeQXYM8wm/SCn0H6Edb KfuXiM2UeDQFN0EXtC/ZTctk0Vfo2qtdbVz5zm54Gq79NVFB6/u44Dbis169PksOyDQI QEMKZO3cX/1/jKBumPWKYNWlv5RjIBONGNqE9nimeHmdbNIBjIxSkxGKzwX62Ni43ObI 2lgLRoP+oDBc5J4CooBa+LKu0nCN0qnSnwP89/93RfGHpcJ6V9wB70KJkh4zNKJLzG3I oQDVKSAYHuYok6vvAZy+OgDSbwej+skOhnAVhCon8wjBZQGpLpCtzs7/zhvQutMCMr/5 TLhQ== X-Forwarded-Encrypted: i=1; AJvYcCUw2pAtNlwselVquLfYpNqNYks7i4kopDRy3GSsrkR/mHvIS883NRn85Ev/ijrwSAVt6jVu8iG5c/tWB1o=@vger.kernel.org X-Gm-Message-State: AOJu0YytBamesAhl3wMXYY39vSFrC03RAhwLo/gyGv6KjATG1KTYMeSx bvcDk7QcjeMU/Ivpnae5zaXOgTSjF365tcjaKewNvgGAYcxGAAYumUXBLDwBImrhVmv8dBgesMd qbw== X-Google-Smtp-Source: AGHT+IFb6MPgfB+uXSTMtqja083E/Cm6ACXuiGQKGyeJpKw+4RppFmo0v95KG0xwXnNHJcF8PwFt1kR+4Ms= X-Received: from pjbok3.prod.google.com ([2002:a17:90b:1d43:b0:2fb:fac8:f45b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3a10:b0:301:1bce:c26f with SMTP id 98e67ed59e1d1-30151c7a17dmr5547488a91.3.1742006415689; Fri, 14 Mar 2025 19:40:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 14 Mar 2025 19:40:09 -0700 In-Reply-To: <20250315024010.2360884-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250315024010.2360884-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250315024010.2360884-3-seanjc@google.com> Subject: [PATCH 2/3] KVM: x86: Allocate kvm_vmx/kvm_svm structures using kzalloc() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the size of "struct kvm" is less than 2KiB, switch back to using kzalloc() to allocate the VM structures. Add compile-time assertions in vendor code to ensure the size is an order-0 allocation, i.e. to prevent unknowingly letting the size balloon in the future. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/svm/svm.c | 1 + arch/x86/kvm/vmx/vmx.c | 1 + 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 6ead9e57446a..04e6c5604bc3 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1939,7 +1939,7 @@ void kvm_x86_vendor_exit(void); #define __KVM_HAVE_ARCH_VM_ALLOC static inline struct kvm *kvm_arch_alloc_vm(void) { - return __vmalloc(kvm_x86_ops.vm_size, GFP_KERNEL_ACCOUNT | __GFP_ZERO); + return kzalloc(kvm_x86_ops.vm_size, GFP_KERNEL_ACCOUNT); } =20 #define __KVM_HAVE_ARCH_VM_FREE diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 8abeab91d329..589adc5f92e0 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -5536,6 +5536,7 @@ static int __init svm_init(void) if (r) goto err_kvm_init; =20 + BUILD_BUG_ON(get_order(sizeof(struct kvm_svm) !=3D 0)); return 0; =20 err_kvm_init: diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index b70ed72c1783..01264842bf45 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8755,6 +8755,7 @@ static int __init vmx_init(void) if (r) goto err_kvm_init; =20 + BUILD_BUG_ON(get_order(sizeof(struct kvm_vmx) !=3D 0)); return 0; =20 err_kvm_init: --=20 2.49.0.rc1.451.g8f38331e32-goog From nobody Wed Dec 17 19:02:34 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDE4618DB26 for ; Sat, 15 Mar 2025 02:40:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742006419; cv=none; b=pGvsQR+5hslOdfX9bY2as7eUsqwMeGV2udmgGaaw6nsKKwG1jEHcdY+F5XgtT1aBaG6yRGPXTx+pv2/d3F1Ve1/ll5tpJfrW6rXa2XilYdS0woxOH7nrvnlS0Z1fzfX/0BxNLzhobNs2E5yV1ydKrVh0BJjcMpjpFlbKaK15JIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742006419; c=relaxed/simple; bh=csXcvVUGODyh39WQeJCJvvBrtei4rk6c+z8kvW6MpJM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gV6DhlfpeEMmxrHF6sqURLl5g+jFpkKpcezlGtjhA0/XBlcC0rvmeAmY75x3G+oZTsqyk4SLKpaL3rTrFuLDAErUBgT7JSKZbcjoX3ppeMu0EGQWoUnMWtQnYKSUsZMALTy7+aKx9GQ5r8Uhhh0W0MDysOLLB3S/fPnKjJSg+uA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NKh5F7OZ; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NKh5F7OZ" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ff82dd6de0so439438a91.0 for ; Fri, 14 Mar 2025 19:40:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742006417; x=1742611217; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Sy3DZMUWr7rSQ3fTWMshpO7vJyF2FiO+hW5xt0RQaio=; b=NKh5F7OZDDAXwedLK+UNsDYfqXSvmn07JTm0jfMyFG7y107zu7l/2Zm51EVBI7xDoe R4+lcZnGSGbrR5lBw9fe+qDpqLOSgIF7Lm0mmwir8LHXje7A2K1i72TSXYLLHevWROEA wR53lByQ+fCj8eHMcOwAAe9GgO9B9HqfmDVQCnRpQO+CpRJT0L1ZmgTghT0oI6x+ggm+ DJd7pRPhy6TnHVtYKm4TfDJCN+sSS7Sgd0Yseh2d94Mx8IpdsISuYwKO8oXewZNsd87F KHFPt5mzJlAaaqF9597rOCGrq7Jiv8O/fAIWm4WmcZbFaolEqfpicPkJ03GIgyPQvd5b Nybg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742006417; x=1742611217; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Sy3DZMUWr7rSQ3fTWMshpO7vJyF2FiO+hW5xt0RQaio=; b=Ig8qszBDMpRWpvjRUssCuzJKu5etxOUfqRC0Hi3fSg3X6PnEsKobhiDLSELIDkWi/n HQtrhI0qXTitJ/EE/IbBRAdOh7ObUeVsyBk8YMr4CPwgUtyBgpjr13ANEp64JQAwBF03 NwDHsPye6TXTwD4DJ/B8C03pfrm5n9lVF6A8sUVOnC3BsqS32GtwplVdtYqfsC/GLbP8 +OjPorm8d174FrNccPM6EroXSvbVYkysQ1JuJ8KiItwTrcIdbZ7Iz6grnpmGi1zOO+p5 jHVB8opyJILTllAoFAcIv2va5H/gO9G0eQn+sLsO1PMmEn0Y3yrCq/yX7Ll+5maGVZLu GnoQ== X-Forwarded-Encrypted: i=1; AJvYcCWCjQji/BbaVgPE4VYEQKdpBY4mV+NDYLYS0SQikT8JM7NCOsNanUo7/G8mwaflfBsnfRmdhEkXmszwvLk=@vger.kernel.org X-Gm-Message-State: AOJu0YwsXudiOLpCgeG0t4X2PAez+emmwot7KVsoo5hgK0odIYd87col x5lqREStZ+2g/vlqqf+ltmk9AkmiamSDcYOMptzkorrmHhvLzpptxCYpJ8VZXgSnmH8vuFyCUKi g6A== X-Google-Smtp-Source: AGHT+IG+EY2vHoY9F8/B8pXlSFd1fbS2p5geTEG1LvjxFKSZnZyvPp3RwszbiRtlmWwsDlNu/jnAcshVEQg= X-Received: from pjbph6.prod.google.com ([2002:a17:90b:3bc6:b0:2fc:11a0:c549]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:180f:b0:2ee:f22a:61dd with SMTP id 98e67ed59e1d1-30151dd2784mr5391135a91.32.1742006417396; Fri, 14 Mar 2025 19:40:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 14 Mar 2025 19:40:10 -0700 In-Reply-To: <20250315024010.2360884-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250315024010.2360884-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250315024010.2360884-4-seanjc@google.com> Subject: [PATCH 3/3] KVM: x86/mmu: Defer allocation of shadow MMU's hashed page list From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the TDP MMU is enabled, i.e. when the shadow MMU isn't used until a nested TDP VM is run, defer allocation of the array of hashed lists used to track shadow MMU pages until the first shadow root is allocated. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b878f2e89dec..3765d7abc2cc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1982,14 +1982,25 @@ static bool sp_has_gptes(struct kvm_mmu_page *sp) return true; } =20 +static __ro_after_init HLIST_HEAD(empty_page_hash); + +static struct hlist_head *kvm_get_mmu_page_hash(struct kvm *kvm, gfn_t gfn) +{ + struct hlist_head *page_hash =3D READ_ONCE(kvm->arch.mmu_page_hash); + + if (!page_hash) + return &empty_page_hash; + + return &page_hash[kvm_page_table_hashfn(gfn)]; +} + #define for_each_valid_sp(_kvm, _sp, _list) \ hlist_for_each_entry(_sp, _list, hash_link) \ if (is_obsolete_sp((_kvm), (_sp))) { \ } else =20 #define for_each_gfn_valid_sp_with_gptes(_kvm, _sp, _gfn) \ - for_each_valid_sp(_kvm, _sp, \ - &(_kvm)->arch.mmu_page_hash[kvm_page_table_hashfn(_gfn)]) \ + for_each_valid_sp(_kvm, _sp, kvm_get_mmu_page_hash(_kvm, _gfn)) \ if ((_sp)->gfn !=3D (_gfn) || !sp_has_gptes(_sp)) {} else =20 static bool kvm_sync_page_check(struct kvm_vcpu *vcpu, struct kvm_mmu_page= *sp) @@ -2357,6 +2368,7 @@ static struct kvm_mmu_page *__kvm_mmu_get_shadow_page= (struct kvm *kvm, struct kvm_mmu_page *sp; bool created =3D false; =20 + BUG_ON(!kvm->arch.mmu_page_hash); sp_list =3D &kvm->arch.mmu_page_hash[kvm_page_table_hashfn(gfn)]; =20 sp =3D kvm_mmu_find_shadow_page(kvm, vcpu, gfn, sp_list, role); @@ -3884,11 +3896,14 @@ static int kvm_mmu_alloc_page_hash(struct kvm *kvm) { typeof(kvm->arch.mmu_page_hash) h; =20 + if (kvm->arch.mmu_page_hash) + return 0; + h =3D kcalloc(KVM_NUM_MMU_PAGES, sizeof(*h), GFP_KERNEL_ACCOUNT); if (!h) return -ENOMEM; =20 - kvm->arch.mmu_page_hash =3D h; + WRITE_ONCE(kvm->arch.mmu_page_hash, h); return 0; } =20 @@ -3911,9 +3926,13 @@ static int mmu_first_shadow_root_alloc(struct kvm *k= vm) if (kvm_shadow_root_allocated(kvm)) goto out_unlock; =20 + r =3D kvm_mmu_alloc_page_hash(kvm); + if (r) + goto out_unlock; + /* - * Check if anything actually needs to be allocated, e.g. all metadata - * will be allocated upfront if TDP is disabled. + * Check if memslot metadata actually needs to be allocated, e.g. all + * metadata will be allocated upfront if TDP is disabled. */ if (kvm_memslots_have_rmaps(kvm) && kvm_page_track_write_tracking_enabled(kvm)) --=20 2.49.0.rc1.451.g8f38331e32-goog