From nobody Fri Dec 19 21:55:43 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C92321D799D for ; Sat, 8 Nov 2025 01:36:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762565766; cv=none; b=FLW/evchLPHseeIu/3q5rB6v2BXHZnVuYIQNgpDhdOPj0FZ9GmFHE+YduQDM27dspEgE9ZbAEk7ddfh3rSI/3CIC7NHh1AhksN48EOXgnjnjvF6aIHn9fEYFwHqIHBO4gdGzb/o/UJm5lACTjAiggpgPSCxRj5oeB0zCU89kkRM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762565766; c=relaxed/simple; bh=fuyWxbBhFRzVlhEb1d7Ax5pvnMcMxgWPJBvojd6LEcA=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=sgyoV7pr2jlADGGSH05AiWwUV4YqfxFrDviZe5BJC+qyto9o71Dtfj5URZ4aSC1MPDj3Sea9Or8dFSwKSKzHDwblMmYu3TWlfJnH9zW8xcIl+O7LAAmDUMVvVueetcRb6du9MzNHjoAAW4vVZymcu5Q6XUZdHH8hdtaZ+PKj6Lc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=sJPtxveR; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sJPtxveR" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-297e1cf9aedso8656455ad.2 for ; Fri, 07 Nov 2025 17:36:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1762565764; x=1763170564; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:reply-to:from:to:cc :subject:date:message-id:reply-to; bh=u0gSVyaHm/RFZz43PnqJfIa7tkp5FemZN1Pt+86XgAQ=; b=sJPtxveRIB/QtPb0+MW2+77KSTfQUZUg+Rz8TGpAsS3InNOb6UyckDYdkeAND5v9UE e4qh2z5RX+KJR79kRJI2Ta/kJHtbSIGvx8bNXo3DScXvsxUPTY76EB15vQKIGT/36wsj wtfxymlk5xogJiqEy6dfFrkgu2OwHFZJndy4BNSyTHoknitUGkzX2unH2PoiIfBAcTMX tNaqrzRhpEh3JkvpM1MJHWar5I6DPODxr+O64ZoibPcMVwtInUojr4HDJ2sGOvYd6tC7 7ggdY1qi31hxqyLnKRmnVePF2VP152grAXA/JmhmA4eZIvqWXT7KQ3jt02nFiHq/BPZW a2aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762565764; x=1763170564; h=cc:to:from:subject:message-id:mime-version:date:reply-to :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=u0gSVyaHm/RFZz43PnqJfIa7tkp5FemZN1Pt+86XgAQ=; b=iK24iaKl959Be5dYtntaetbveC0gWjw5xVeG1wnJ9nQHD9ngf2A6UR+4oGjpvYWWLl FpFsKnn6RyizMJP2FvRoh/WxYBk3fN4Xu9/zTFSGuoBCfpk375tHQX2UB+R5ylhyCubm PM04oIm4DGG3WUAGzLZffNwYBGYzXeuC7jxj2oTlCZ3uSWrI7ks4ardW2TprOyNMVxGK DrkeQE53f6FNuSyi53WZOuvsW1gQfZU1GxVteOYthcS6uunLZF6BkfHJZIUo6DUVNYKN vVmcK+iVs+r4jzl8LgXabM8DPIpVwUCwhLiEQDBcdlC5ZSARh2B9AO3oSP8kNCtNukkt 8kQA== X-Forwarded-Encrypted: i=1; AJvYcCVpH3tmROFDFDIpM+C2Ubp83G6kFTStX3E4nEaIyAgllfmDfARv4wXDIYtq3XRYKPkmgfqpZt1tX5kVuzw=@vger.kernel.org X-Gm-Message-State: AOJu0YxWB9wMGP0dJqvYDolJYxwWeBG3BcD+FFY0eHpzyuW4K54AwrBS G8mMrnATeAznGchW3gNk1XO1ieQrXXhxXSZ4PFTGsUl3JSV6ySbqfnMlSU0b84ULRyzJi+0oQD0 DXg1Jig== X-Google-Smtp-Source: AGHT+IHpztq6CsL06buHflHDop5NaQ1I+bKM6en+6ybPVhUoSJG/Csnx0VRAnVSCj9mVmv1MaDNZn1NHdwg= X-Received: from pgac20.prod.google.com ([2002:a05:6a02:2954:b0:bac:6acd:8180]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:b715:b0:295:8db9:305f with SMTP id d9443c01a7336-297e56be263mr8254205ad.34.1762565764001; Fri, 07 Nov 2025 17:36:04 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 7 Nov 2025 17:36:01 -0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.51.2.1041.gc1ab5b90ca-goog Message-ID: <20251108013601.902918-1-seanjc@google.com> Subject: [PATCH] KVM: x86: Allocate/free user_return_msrs at kvm.ko (un)loading time From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Gao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Chao Gao Move user_return_msrs allocation/free from vendor modules (kvm-intel.ko and kvm-amd.ko) (un)loading time to kvm.ko's to make it less risky to access user_return_msrs in kvm.ko. Tying the lifetime of user_return_msrs to vendor modules makes every access to user_return_msrs prone to use-after-free issues as vendor modules may be unloaded at any time. Opportunistically turn the per-CPU variable into full structs, as there's no practical difference between statically allocating the memory and allocating it unconditionally during module_init(). Zero out kvm_nr_uret_msrs on vendor module exit to further minimize the chances of consuming stale data, and WARN on vendor module load if KVM thinks there are existing user-return MSRs. Note! The user-return MSRs also need to be "destroyed" if ops->hardware_setup() fails, as both SVM and VMX expect common KVM to clean up (because common code, not vendor code, is responsible for kvm_nr_uret_msrs). Signed-off-by: Chao Gao Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 46 ++++++++++++++++------------------------------ 1 file changed, 16 insertions(+), 30 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9c2e28028c2b..24dba35f3217 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -212,7 +212,7 @@ struct kvm_user_return_msrs { u32 __read_mostly kvm_nr_uret_msrs; EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_nr_uret_msrs); static u32 __read_mostly kvm_uret_msrs_list[KVM_MAX_NR_USER_RETURN_MSRS]; -static struct kvm_user_return_msrs __percpu *user_return_msrs; +static DEFINE_PER_CPU(struct kvm_user_return_msrs, user_return_msrs); =20 #define KVM_SUPPORTED_XCR0 (XFEATURE_MASK_FP | XFEATURE_MASK_SSE \ | XFEATURE_MASK_YMM | XFEATURE_MASK_BNDREGS \ @@ -575,25 +575,14 @@ static inline void kvm_async_pf_hash_reset(struct kvm= _vcpu *vcpu) vcpu->arch.apf.gfns[i] =3D ~0; } =20 -static int kvm_init_user_return_msrs(void) +static void kvm_destroy_user_return_msrs(void) { - user_return_msrs =3D alloc_percpu(struct kvm_user_return_msrs); - if (!user_return_msrs) { - pr_err("failed to allocate percpu user_return_msrs\n"); - return -ENOMEM; - } + int cpu; + + for_each_possible_cpu(cpu) + WARN_ON_ONCE(per_cpu(user_return_msrs, cpu).registered); + kvm_nr_uret_msrs =3D 0; - return 0; -} - -static void kvm_free_user_return_msrs(void) -{ - int cpu; - - for_each_possible_cpu(cpu) - WARN_ON_ONCE(per_cpu_ptr(user_return_msrs, cpu)->registered); - - free_percpu(user_return_msrs); } =20 static void kvm_on_user_return(struct user_return_notifier *urn) @@ -656,7 +645,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_find_user_return_msr= ); =20 static void kvm_user_return_msr_cpu_online(void) { - struct kvm_user_return_msrs *msrs =3D this_cpu_ptr(user_return_msrs); + struct kvm_user_return_msrs *msrs =3D this_cpu_ptr(&user_return_msrs); u64 value; int i; =20 @@ -678,7 +667,7 @@ static void kvm_user_return_register_notifier(struct kv= m_user_return_msrs *msrs) =20 int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask) { - struct kvm_user_return_msrs *msrs =3D this_cpu_ptr(user_return_msrs); + struct kvm_user_return_msrs *msrs =3D this_cpu_ptr(&user_return_msrs); int err; =20 value =3D (value & mask) | (msrs->values[slot].host & ~mask); @@ -696,13 +685,13 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_user_return_ms= r); =20 u64 kvm_get_user_return_msr(unsigned int slot) { - return this_cpu_ptr(user_return_msrs)->values[slot].curr; + return this_cpu_ptr(&user_return_msrs)->values[slot].curr; } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_get_user_return_msr); =20 static void drop_user_return_notifiers(void) { - struct kvm_user_return_msrs *msrs =3D this_cpu_ptr(user_return_msrs); + struct kvm_user_return_msrs *msrs =3D this_cpu_ptr(&user_return_msrs); =20 if (msrs->registered) kvm_on_user_return(&msrs->urn); @@ -10077,13 +10066,9 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *o= ps) return -ENOMEM; } =20 - r =3D kvm_init_user_return_msrs(); - if (r) - goto out_free_x86_emulator_cache; - r =3D kvm_mmu_vendor_module_init(); if (r) - goto out_free_percpu; + goto out_free_x86_emulator_cache; =20 kvm_caps.supported_vm_types =3D BIT(KVM_X86_DEFAULT_VM); kvm_caps.supported_mce_cap =3D MCG_CTL_P | MCG_SER_P; @@ -10108,6 +10093,8 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *op= s) if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES)) rdmsrq(MSR_IA32_ARCH_CAPABILITIES, kvm_host.arch_capabilities); =20 + WARN_ON_ONCE(kvm_nr_uret_msrs); + r =3D ops->hardware_setup(); if (r !=3D 0) goto out_mmu_exit; @@ -10180,9 +10167,8 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *op= s) kvm_x86_ops.enable_virtualization_cpu =3D NULL; kvm_x86_call(hardware_unsetup)(); out_mmu_exit: + kvm_destroy_user_return_msrs(); kvm_mmu_vendor_module_exit(); -out_free_percpu: - kvm_free_user_return_msrs(); out_free_x86_emulator_cache: kmem_cache_destroy(x86_emulator_cache); return r; @@ -10210,8 +10196,8 @@ void kvm_x86_vendor_exit(void) cancel_work_sync(&pvclock_gtod_work); #endif kvm_x86_call(hardware_unsetup)(); + kvm_destroy_user_return_msrs(); kvm_mmu_vendor_module_exit(); - kvm_free_user_return_msrs(); kmem_cache_destroy(x86_emulator_cache); #ifdef CONFIG_KVM_XEN static_key_deferred_flush(&kvm_xen_enabled); base-commit: 9052f4f6c539ea1fb7b282a34e6bb33154ce0b63 --=20 2.51.2.1041.gc1ab5b90ca-goog