From nobody Thu Apr 10 20:31:53 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 145A520C489 for ; Tue, 1 Apr 2025 15:57:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743523040; cv=none; b=rDxT5mJBXPN8+q9+SoT/iPrWVYQQTEZK+fSS5KbaTZwVAIcrc5tpXBZe/PR0azWS8QXpcMB354fjQWV8/B0fO+mg4SEj2Af2svx1rW7qiMHduGSuqp5ZYdF5oKlvRxARmVBYqp6Zp6Ig7Zrl8Yj79IPGnZBBebOQ/4Zk5gE+CCA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743523040; c=relaxed/simple; bh=XFMN/1TjDkt0VHZXeAmqM88pM/Yy9sojmu6ZVcp+G5A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QrKKBUHefiWrVGfb+1mjaaXm3BrOC9jpqd/aPjLAmoTnTGBxrNLlSxzfVgF4gJI9f2oWO2iStYwXVEJ2Q1oIVmfHDS0hSOdLUhTLOF8gOIbEcMKsTZrmShO9isBkLQK+GgfS7HALao0LqVBDtLgaLiUamrxmv1TaG4DP6UEXMko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=32CcTCKq; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="32CcTCKq" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff799be8f5so9534983a91.1 for ; Tue, 01 Apr 2025 08:57:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743523038; x=1744127838; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=XfD+/olMCjjTlr+W07rgl23+xXZX9pNmPmnT7WHnWhU=; b=32CcTCKqt2w3lQA/Q/QL5MQ4SJaL6xsMoDUbEf2R6UjFalcXqi/fpiUPyNUX8bY848 DGUE+Kz9cN0kMDVnB6H2Jb/Ppm2ZfHJbRPAGQp9aabNhzW2KnkkYQCvmIAwnfUjca2cm 6y5OLvBYqyT+rU2ePgqaLLBJaiHBu51b3oJ+pYIb83ReCiGlwOWGcFP7liyLqLQMzMsG KmVjkka4s3MYHeF7T3sz/eNhyn8LKoxqsmkZJ04G2crkAX85uf5J1sujsA+qHdoeVYwh LjWprGBws2qA/8eTlqJPVi8MwtlxKI8i641nqQRBFYTvAFROZI++hkIRBsdpjVlSCPLW 4/Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743523038; x=1744127838; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XfD+/olMCjjTlr+W07rgl23+xXZX9pNmPmnT7WHnWhU=; b=Z34423uxinP79BLThALehkX5wzOrLQQTHET0xxyIEiy2KACmujJnOQfcU3+t9C4by6 wmO3VxAlmEyTDuj4ZFpscOZKyHZXtx8RSk1AudSMIlhwdZUBQjwx7YlG6cq4zpwd/V9I mWjD/kBVM0yOBRAkm66aIqYJ1AvcKKMXf3f2F9PWFB6XNfUL9Z5y4CLGvwgE93sAi3Ys FNHA+UHVOrcaniVewf16VUEQK/Zk91b3AkcvY1ngAjx9eeM9k81aBe4PtV9HFoYMpmB8 B3/4TY9GUFlGs97L6mOzgcqabvi2eVL+MYyyNBITU1zcvqit/sB12Om/kNvo1JFpqIee RgTQ== X-Forwarded-Encrypted: i=1; AJvYcCU7yvXSk48h5CHlvEVGV8zzIDIJX2c4w3TSo+zB2m4dq4w6p1wXeF0C8hKwDBtDk/4ts8Z7n/emCOVv3QA=@vger.kernel.org X-Gm-Message-State: AOJu0Yy293GSg+rPCj6vr+FqG5pycqTOngCw/rS6IW/Sd3NzzBPO41wr d4H0uCkw9gB0aXfVlYuayzHqVtZHRL/nY5rFNfY0jfesSvWcrqxCwMIx714ugs6d1DeQ1sn+cGy Yeg== X-Google-Smtp-Source: AGHT+IGCmi+AM1Kq1m2h89y+RWMJjrT8URQip51SgDFCFbGhP2V4jxmx+W28wYOkSn98iKOm32ePbQd01rw= X-Received: from pjyp8.prod.google.com ([2002:a17:90a:e708:b0:2ef:78ff:bc3b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3b85:b0:2ff:6e72:b8e9 with SMTP id 98e67ed59e1d1-30532154023mr19843757a91.25.1743523038368; Tue, 01 Apr 2025 08:57:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 1 Apr 2025 08:57:12 -0700 In-Reply-To: <20250401155714.838398-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401155714.838398-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401155714.838398-2-seanjc@google.com> Subject: [PATCH v2 1/3] KVM: x86/mmu: Dynamically allocate shadow MMU's hashed page list From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Dynamically allocate the (massive) array of hashed lists used to track shadow pages, as the array itself is 32KiB, i.e. is an order-3 allocation all on its own, and is *exactly* an order-3 allocation. Dynamically allocating the array will allow allocating "struct kvm" using regular kmalloc(), and will also allow deferring allocation of the array until it's actually needed, i.e. until the first shadow root is allocated. Cc: Vipin Sharma Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/mmu/mmu.c | 23 ++++++++++++++++++++++- arch/x86/kvm/x86.c | 5 ++++- 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index a884ab544335..e523d7d8a107 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1333,7 +1333,7 @@ struct kvm_arch { bool has_private_mem; bool has_protected_state; bool pre_fault_allowed; - struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES]; + struct hlist_head *mmu_page_hash; struct list_head active_mmu_pages; /* * A list of kvm_mmu_page structs that, if zapped, could possibly be @@ -1985,7 +1985,7 @@ void kvm_mmu_vendor_module_exit(void); =20 void kvm_mmu_destroy(struct kvm_vcpu *vcpu); int kvm_mmu_create(struct kvm_vcpu *vcpu); -void kvm_mmu_init_vm(struct kvm *kvm); +int kvm_mmu_init_vm(struct kvm *kvm); void kvm_mmu_uninit_vm(struct kvm *kvm); =20 void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 63bb77ee1bb1..6b9c72405860 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3880,6 +3880,18 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *v= cpu) return r; } =20 +static int kvm_mmu_alloc_page_hash(struct kvm *kvm) +{ + typeof(kvm->arch.mmu_page_hash) h; + + h =3D kcalloc(KVM_NUM_MMU_PAGES, sizeof(*h), GFP_KERNEL_ACCOUNT); + if (!h) + return -ENOMEM; + + kvm->arch.mmu_page_hash =3D h; + return 0; +} + static int mmu_first_shadow_root_alloc(struct kvm *kvm) { struct kvm_memslots *slots; @@ -6673,13 +6685,19 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm) kvm_tdp_mmu_zap_invalidated_roots(kvm, true); } =20 -void kvm_mmu_init_vm(struct kvm *kvm) +int kvm_mmu_init_vm(struct kvm *kvm) { + int r; + kvm->arch.shadow_mmio_value =3D shadow_mmio_value; INIT_LIST_HEAD(&kvm->arch.active_mmu_pages); INIT_LIST_HEAD(&kvm->arch.possible_nx_huge_pages); spin_lock_init(&kvm->arch.mmu_unsync_pages_lock); =20 + r =3D kvm_mmu_alloc_page_hash(kvm); + if (r) + return r; + if (tdp_mmu_enabled) kvm_mmu_init_tdp_mmu(kvm); =20 @@ -6690,6 +6708,7 @@ void kvm_mmu_init_vm(struct kvm *kvm) =20 kvm->arch.split_desc_cache.kmem_cache =3D pte_list_desc_cache; kvm->arch.split_desc_cache.gfp_zero =3D __GFP_ZERO; + return 0; } =20 static void mmu_free_vm_memory_caches(struct kvm *kvm) @@ -6701,6 +6720,8 @@ static void mmu_free_vm_memory_caches(struct kvm *kvm) =20 void kvm_mmu_uninit_vm(struct kvm *kvm) { + kfree(kvm->arch.mmu_page_hash); + if (tdp_mmu_enabled) kvm_mmu_uninit_tdp_mmu(kvm); =20 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c841817a914a..4070f9d34521 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12721,7 +12721,9 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long= type) if (ret) goto out; =20 - kvm_mmu_init_vm(kvm); + ret =3D kvm_mmu_init_vm(kvm); + if (ret) + goto out_cleanup_page_track; =20 ret =3D kvm_x86_call(vm_init)(kvm); if (ret) @@ -12774,6 +12776,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long= type) =20 out_uninit_mmu: kvm_mmu_uninit_vm(kvm); +out_cleanup_page_track: kvm_page_track_cleanup(kvm); out: return ret; --=20 2.49.0.472.ge94155a9ec-goog From nobody Thu Apr 10 20:31:53 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C683A20D4E1 for ; Tue, 1 Apr 2025 15:57:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743523042; cv=none; b=Fr/GVNlv6HTrtLKNStf4L/hIJFPE8YyKnSOIFfokN+ldk5d9+8cG5TtGpj5zcW7kpbcmRtblKjNj/QEyGAm1Q8ZsXxuEa7F+b6ZeV8P3rVJL57cAoEYl0FhpJLWuPLDOil5UNYPqjdF1JCMEFJll4n0eH1wIPKIyEGChASD/B6U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743523042; c=relaxed/simple; bh=0ZZQsy5iR/tBBwSV2RsNwEBEpyYjNX9rkaOvEJOwHEo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IDtuMb38XO1jdf6Hkmfl+HEbxU1BEcVqKyMte1F8sE42fUGA5V316Lt9PwqD7cyk4Ku0pqE23o7FZ7DfGk7wRdlIHOhNMEc1tH6fTMv0KoekPnjeCCbqmY+ap896ZxbKLZPkNMjvuUagVAALVsStNP+Wsl6wUr9eLQc3ygZL7Xk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MR5rVmpw; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MR5rVmpw" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ff4b130bb2so10362275a91.0 for ; Tue, 01 Apr 2025 08:57:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743523040; x=1744127840; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=qA3FLGTWw4iTy7QH63sW/7/WyR1uRH4cACOa/5NAWCI=; b=MR5rVmpwhfFU7Kk7zoQdF/DA7BYy8CS10sch71A0/6TPL43vYo0VWwoM/ZRTon0rci 2N1wOvqVAe/mVqt9s5HPj8UJ2QqPQc8jMCAJ3wifadS3C88mQ5Rw/xzcl3OC4g6tX805 Lx6ljp3SIYVYm/W0tH9ZyWdLtB6cimXdE86W5D0wGocMK8RTu4IXY3wEZ/94nb1yqaX7 I+7GBBjtrR4h69qKA9JIDRf7TYYxao1/46lpx05/qLW14/yz/Lbx1Q4UQ9lVp4m+2SEm XMfNoD7qfov2HFIBfb4NzVa3bXzUfmbGQV2tzutopo4YQPnmDlzkOnAPdN+f/MXPqlEU vbxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743523040; x=1744127840; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qA3FLGTWw4iTy7QH63sW/7/WyR1uRH4cACOa/5NAWCI=; b=wB/kOkbrXqAEf4qBq4EjbExRiqCNZJ7gQnhWqLQ+LRr6bY0+Rl9yW6f5Lf6dfAVxZA QQdcgUXZPrbnEx/bA01di/6oMhaFxFvH/Y/Y8xD31CubLl85G7EslUs3n5S9Kix+TlwB +4o5f3sNQs0T9/xOf/meN8nRy7krnASgmNpVSIHjikEu2XO78pusdSdGsT+nz50XFxzk ndDB+lF1qT3E33VGmrAWusepA9j5KAeh1UQRnmAOt74SWQemFpQbeohxqw8D69ymLCpu CnVjnrpnvmoHOJstTZLFe7He/aT2FxJDqnUwZ17UWaE8J9ySC4wzoCXLF+7ArnlgTPfV 3vnA== X-Forwarded-Encrypted: i=1; AJvYcCXdZRgptefRfC7KcbkgHUVGRdS7JROzgIbjK/CKQ8kdXOZEIbYTJwIlvmENw57fN4KlpFAEvwVuyzFyk1A=@vger.kernel.org X-Gm-Message-State: AOJu0Yyy1gbFyqEw2pdMcWGoLH8qZRgRn8ziZuz3mbLTlUd8NlaruSpf sVtUrGKKC1yMotsO0vY0CXsFZyJQMpn/Wy/RaUTEpTlm+4I3OO+wVV7f4upkhyVFLmFMjZU+Zfq VAQ== X-Google-Smtp-Source: AGHT+IFfRx5K6oY4RAb5pZkeJwowkj1BgVmibF8BfPMQekfaCGm2fIiGn7Kg30MfKpJzguRbFKy2Jg4G68k= X-Received: from pfbli7.prod.google.com ([2002:a05:6a00:7187:b0:736:a983:dc43]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:c889:b0:1f5:6c94:2cc1 with SMTP id adf61e73a8af0-2009f640589mr24122182637.21.1743523040124; Tue, 01 Apr 2025 08:57:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 1 Apr 2025 08:57:13 -0700 In-Reply-To: <20250401155714.838398-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401155714.838398-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401155714.838398-3-seanjc@google.com> Subject: [PATCH v2 2/3] KVM: x86: Allocate kvm_vmx/kvm_svm structures using kzalloc() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the size of "struct kvm" is less than 2KiB, switch back to using kzalloc() to allocate the VM structures. Add compile-time assertions in vendor code to ensure the size is an order-0 allocation, i.e. to prevent unknowingly letting the size balloon in the future. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/svm/svm.c | 1 + arch/x86/kvm/vmx/vmx.c | 1 + 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index e523d7d8a107..6c7fd7db6f11 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1940,7 +1940,7 @@ void kvm_x86_vendor_exit(void); #define __KVM_HAVE_ARCH_VM_ALLOC static inline struct kvm *kvm_arch_alloc_vm(void) { - return __vmalloc(kvm_x86_ops.vm_size, GFP_KERNEL_ACCOUNT | __GFP_ZERO); + return kzalloc(kvm_x86_ops.vm_size, GFP_KERNEL_ACCOUNT); } =20 #define __KVM_HAVE_ARCH_VM_FREE diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 8abeab91d329..589adc5f92e0 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -5536,6 +5536,7 @@ static int __init svm_init(void) if (r) goto err_kvm_init; =20 + BUILD_BUG_ON(get_order(sizeof(struct kvm_svm) !=3D 0)); return 0; =20 err_kvm_init: diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index b70ed72c1783..01264842bf45 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8755,6 +8755,7 @@ static int __init vmx_init(void) if (r) goto err_kvm_init; =20 + BUILD_BUG_ON(get_order(sizeof(struct kvm_vmx) !=3D 0)); return 0; =20 err_kvm_init: --=20 2.49.0.472.ge94155a9ec-goog From nobody Thu Apr 10 20:31:53 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16C2120D51D for ; Tue, 1 Apr 2025 15:57:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743523043; cv=none; b=JNFj9Lbig0IJ6vTV/KZQwu5P8AffFz5eQ3eshNh3r1wD/enhzyhLG+Prhse/5rqdodhPqB+mBmjg+IWc1aRHGBKf/iVqdezg/JFm3xpNc91jQxnmNP71OJdBUf3pM5MRJeCPuJ4B4Noa4kwkz7gyCOSBHVOA84Xob3IjGs1ocFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743523043; c=relaxed/simple; bh=9xW9GrSKCT/xwLFYa59X9SWljYJetUhynN30KptsTI4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ArqPyeb9eW/Q5TnnGcj7kyvbQI8Hw8fsNrmXbEFOiVv0QREPMpZBltGYi5/z2oKi6rP6xPcE5oucigoLTSvs+BLB0fJBVqAaZjmNpsMrJVf2ta1Hot0sRSMArgICzdr/sPbKwjIBFHiJ+rSN+SricrrDsSKn1t+BBqRO84MMpcc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VeqA0S6S; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VeqA0S6S" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-30566e34290so970434a91.3 for ; Tue, 01 Apr 2025 08:57:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743523041; x=1744127841; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OcWZ8FomvSCbpj1QhcSsf3iHQ4ToTVlAqE/iZm3XJpQ=; b=VeqA0S6SCrWkkrdcdFt3WlTvC+rz5BGjdUxqJfeRjoOkC/0FG6k6UZ2n30Ebu8wa0V N5iyLrEK4DpyIVTKRXNt9mnI1sr06sZGUcPBgoyjZvXcyks+WT0LTgNoS15vgB7vIX8z 2lVtuvxvEVnBVvVQo+vUk+cF+OaC2MbvnQNsd1Tkt3goB2BoWLwBtMvKWxc47G+GmXTd sZofK4erYCdQ86CvkiWraHHE34rAYC8x7RUxSj/dmCIOg3yC0PM/B8yvexVixzao9VAh HguZfbRGrFpSBeiPT+3FfzEItHvJ5Umx9rqHwmEREuavj9pW0dzYOwuVY24Cn03uXfmL He9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743523041; x=1744127841; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OcWZ8FomvSCbpj1QhcSsf3iHQ4ToTVlAqE/iZm3XJpQ=; b=l4vlzAaZYfIm13MWizPsif1/NYyQGPRMq+iN5oGpU0elPv2GliYlNDVkYTGUGOeSMZ 7H/C67SiNiYt6wmP2InYcz9eLd0dLZrixxLE6fFzetMC3i4uejVzb4aVqKbWQPestcvp 4//XRX479+Uh9XeC6SaiefRmBOoqHRl4jqCOxGy2eOlJCzxpojxYDraHByfveYjd6qMv xg61qpvjxDlytWiGAx5e/90DKE8T/buX57GhuTc35NskotAWB0B5HbNCw2tYe7wPsI2U nrxzmu4XZSvo7zwVTf22L2Lo+XcNZy2wvUM5odrE1rigX0cawJ6RZMtnq2rxbC/OL52o hNDg== X-Forwarded-Encrypted: i=1; AJvYcCVi+qbNqjhLzDbUcYgeWK10uDS+HdnGy/sQ52QDa8fmAVQfGjikmvzimAnt8jNCUJlBHt4NgY9UXkg4Bxk=@vger.kernel.org X-Gm-Message-State: AOJu0Yz9uxHv9rep0OywNhAV6yqG/DEcHZpj8hboxOwAHMFzrxFOSXM3 JkHL48x1ridEeSf6bExo89Z3KlTAfua+aE+dXEbuS2F93IlsfHz6u55n6mppfGXNZ7l7nL0TB55 YRg== X-Google-Smtp-Source: AGHT+IHu9Zv08nz2HYgQrk1iOQEUOZWiNQiFOu+C1kdTrpqwaD1bTf9yPuSC+oGjaWR9wby7kt+YZPyycSE= X-Received: from pfbli7.prod.google.com ([2002:a05:6a00:7187:b0:736:a983:dc43]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:aa7:88c9:0:b0:736:34ff:be7 with SMTP id d2e1a72fcca58-7398044e159mr16761821b3a.15.1743523041623; Tue, 01 Apr 2025 08:57:21 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 1 Apr 2025 08:57:14 -0700 In-Reply-To: <20250401155714.838398-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401155714.838398-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401155714.838398-4-seanjc@google.com> Subject: [PATCH v2 3/3] KVM: x86/mmu: Defer allocation of shadow MMU's hashed page list From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the TDP MMU is enabled, i.e. when the shadow MMU isn't used until a nested TDP VM is run, defer allocation of the array of hashed lists used to track shadow MMU pages until the first shadow root is allocated. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6b9c72405860..213009cdba15 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1982,14 +1982,25 @@ static bool sp_has_gptes(struct kvm_mmu_page *sp) return true; } =20 +static __ro_after_init HLIST_HEAD(empty_page_hash); + +static struct hlist_head *kvm_get_mmu_page_hash(struct kvm *kvm, gfn_t gfn) +{ + struct hlist_head *page_hash =3D READ_ONCE(kvm->arch.mmu_page_hash); + + if (!page_hash) + return &empty_page_hash; + + return &page_hash[kvm_page_table_hashfn(gfn)]; +} + #define for_each_valid_sp(_kvm, _sp, _list) \ hlist_for_each_entry(_sp, _list, hash_link) \ if (is_obsolete_sp((_kvm), (_sp))) { \ } else =20 #define for_each_gfn_valid_sp_with_gptes(_kvm, _sp, _gfn) \ - for_each_valid_sp(_kvm, _sp, \ - &(_kvm)->arch.mmu_page_hash[kvm_page_table_hashfn(_gfn)]) \ + for_each_valid_sp(_kvm, _sp, kvm_get_mmu_page_hash(_kvm, _gfn)) \ if ((_sp)->gfn !=3D (_gfn) || !sp_has_gptes(_sp)) {} else =20 static bool kvm_sync_page_check(struct kvm_vcpu *vcpu, struct kvm_mmu_page= *sp) @@ -2357,6 +2368,7 @@ static struct kvm_mmu_page *__kvm_mmu_get_shadow_page= (struct kvm *kvm, struct kvm_mmu_page *sp; bool created =3D false; =20 + BUG_ON(!kvm->arch.mmu_page_hash); sp_list =3D &kvm->arch.mmu_page_hash[kvm_page_table_hashfn(gfn)]; =20 sp =3D kvm_mmu_find_shadow_page(kvm, vcpu, gfn, sp_list, role); @@ -3884,11 +3896,14 @@ static int kvm_mmu_alloc_page_hash(struct kvm *kvm) { typeof(kvm->arch.mmu_page_hash) h; =20 + if (kvm->arch.mmu_page_hash) + return 0; + h =3D kcalloc(KVM_NUM_MMU_PAGES, sizeof(*h), GFP_KERNEL_ACCOUNT); if (!h) return -ENOMEM; =20 - kvm->arch.mmu_page_hash =3D h; + WRITE_ONCE(kvm->arch.mmu_page_hash, h); return 0; } =20 @@ -3911,9 +3926,13 @@ static int mmu_first_shadow_root_alloc(struct kvm *k= vm) if (kvm_shadow_root_allocated(kvm)) goto out_unlock; =20 + r =3D kvm_mmu_alloc_page_hash(kvm); + if (r) + goto out_unlock; + /* - * Check if anything actually needs to be allocated, e.g. all metadata - * will be allocated upfront if TDP is disabled. + * Check if memslot metadata actually needs to be allocated, e.g. all + * metadata will be allocated upfront if TDP is disabled. */ if (kvm_memslots_have_rmaps(kvm) && kvm_page_track_write_tracking_enabled(kvm)) @@ -6694,12 +6713,13 @@ int kvm_mmu_init_vm(struct kvm *kvm) INIT_LIST_HEAD(&kvm->arch.possible_nx_huge_pages); spin_lock_init(&kvm->arch.mmu_unsync_pages_lock); =20 - r =3D kvm_mmu_alloc_page_hash(kvm); - if (r) - return r; - - if (tdp_mmu_enabled) + if (tdp_mmu_enabled) { kvm_mmu_init_tdp_mmu(kvm); + } else { + r =3D kvm_mmu_alloc_page_hash(kvm); + if (r) + return r; + } =20 kvm->arch.split_page_header_cache.kmem_cache =3D mmu_page_header_cache; kvm->arch.split_page_header_cache.gfp_zero =3D __GFP_ZERO; --=20 2.49.0.472.ge94155a9ec-goog