From nobody Sat Feb 7 08:43:58 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9F40EAD5 for ; Thu, 28 Nov 2024 00:43:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754631; cv=none; b=aFXF4jVdxZXVJSoQZq3Sc63YKTyE5zR9ESTXLKcQB++sUHJQKhz/p+QUcEWo0q8/qjQKKb6Jpc4cBDQOWrX43Xej5Wpfm0PPQYuISWyHBUu98qNpTQRxDQHuqjwfqot2HeaM4geJZKaxJuH/NVb+byzQIOpBQYnk6mCWaG8eu8Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754631; c=relaxed/simple; bh=EACB2yPyU5Yw8+WwKYitNwB9tkKiB/UcMdmwylhn5ac=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=glqqbo59FRrZe0K7RPIjPwKPMZr33NEsKgsdxBB5dMwnHEhTSlKwbHRdZ1mtvB9r/4NVxxC/rOFARyW92jOImNrYkdtUE05cPUVCJoYB6CSunVm4hhtqtYXMC/BCeCp/OM5R3t+SVw7NaC4MuSKX5jBZjWfEM345n/0BvDh3s68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=v7XxW7Eo; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="v7XxW7Eo" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-21147fea103so3437955ad.2 for ; Wed, 27 Nov 2024 16:43:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754629; x=1733359429; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=H6v00VWPrFlV9G8PXjNGHz1lcbArd14LtnA+J13xoHI=; b=v7XxW7EoO+ncriNDu2gIHvbXy3S5xYONjO+Tf0yNsXsxEn8y6N3rh5wETBW6Tkebqa /RSErLXc8BX0QLx/r/i3LGRs4MJwvLuPUkvCxE8EQGrWfd2IM1FVHou0vWBKhB4qHfnR DC626wTtM/z4pDK4dzZeBj3uzSy3Mv7fOWtdC/65d9bGU9oseiO0LmGVrUQQwnSExSCZ SBpi/PSVK5jclZsl47hQHhddfM3q+PapNBtwbBWpJgqV4X+J7d+5jAYGYDr3OYfCJ+SV Hhe1gqn/npwqYlC37ShZibIpIqaM6lA8kjTUVNHXtr0M0gSeI98t6iGOy+81tlouIU9S Y9Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754629; x=1733359429; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H6v00VWPrFlV9G8PXjNGHz1lcbArd14LtnA+J13xoHI=; b=YXthap4BWmSbnbxauzyOJPmvX+UhKvnOeIq7sST+As7evW4elJaYeIqMLb/JOlFpBj V5LL5NPiex39sw5aEL5vG7ChCarjiDU3bA5XVj4LlqAh//Pq5AFY9caeSfBuMfnUD1ja R3OdgibRBcjA4M9LMETeBiqCXpvu4WvoQuZRlbCW0BCugyB9GdBpTLLAITdsU1toW8aV WcdJEHtVFlXVoGXtscCVlTqFwk5N9Zj6spHlSUa3zCPa4ad1RqBlPuus4hU2DjOzfek1 HLtfR6nlhGGyPdqtxhUzqew8SYbuKXKFAxWcpYVfFfheh/U3i1UELTS+t6AZKNytbM4o zpmg== X-Forwarded-Encrypted: i=1; AJvYcCVwlic9QMA26GuPqygiB6CjX2aaHcPhmPMmNB9Q9vu3+2nDoN+yiF6GQzxtNO/+KLaaosp13OPhuMBxRiE=@vger.kernel.org X-Gm-Message-State: AOJu0YwiUnooXaozHtGOpU95oxvzCVy039oTlGAWTQKh9zyiO5PECDMS VstFvzmUPPSPuXmZ/e995gEKaYG0Fs+KESkRLwHmegDIfJ9eBoZeJ1zN5YMQdfItO1CsEtd5PbB h3g== X-Google-Smtp-Source: AGHT+IEK9pbCeaFZWXX0PLadQxkBbAx1o35IfYFdtf0yqxaDLee26TJikEv6kZT/3lUfdcYbGJKEkLcA4Vc= X-Received: from pjbpd1.prod.google.com ([2002:a17:90b:1dc1:b0:2ea:448a:8cd1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4d0d:b0:2ea:bf1c:1e3a with SMTP id 98e67ed59e1d1-2ee08eb2ae0mr7730683a91.12.1732754629027; Wed, 27 Nov 2024 16:43:49 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:39 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-2-seanjc@google.com> Subject: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable. Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN: ------------[ cut here ]------------ WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercal= l_exit+0x44/0xe0 [kvm] Modules linked in: kvm_amd kvm ... [last unloaded: kvm] CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--= 392e932fa0f3-feat #470 Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024 RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm] Call Trace: kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm] kvm_vcpu_ioctl+0x54f/0x630 [kvm] __se_sys_ioctl+0x6b/0xc0 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e ---[ end trace 0000000000000000 ]--- Fixes: b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with p= rotected state") Cc: stable@vger.kernel.org Cc: Tom Lendacky Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Kai Huang Reviewed-by: Nikunj A Dadhania Reviewed-by: Tom Lendacky Reviewed-by: Xiaoyao Li --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2e713480933a..0b2fe4aa04a2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *v= cpu) { u64 ret =3D vcpu->run->hypercall.ret; =20 - if (!is_64_bit_mode(vcpu)) + if (!is_64_bit_hypercall(vcpu)) ret =3D (u32)ret; kvm_rax_write(vcpu, ret); ++vcpu->stat.hypercalls; --=20 2.47.0.338.g60cca15819-goog From nobody Sat Feb 7 08:43:58 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65AC31A270 for ; Thu, 28 Nov 2024 00:43:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754632; cv=none; b=agoGq4NBTGqDFAkCudrt6FWHf7cZiN84gKaQottRsrKRq7VUpQNuRlt6WWw5h7C91T9H2A73lT/WWS/jLN41reYh48RZw4awY17vyzrszQOMUZuLFndyd9IS5kbJ83EeymfzZFh7Vp0JwBArTxroCeWl1rY2OIGi1HOou/jy48Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754632; c=relaxed/simple; bh=FRVS2g+fG0f7ZYJIgFJENJzs0OeU6x/7e3vS+WJBqvk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Zfp+MQD8seJtckNP8T8pCOP8tEcURspqeGvcOxke7rAgcWql+miVqQqySX4pomlHt/yiLqPaR8v/T4MUmj6c2CItr4dGjtTQXFTw0FDwlSPiOfPo2+DYttSG5LwBnVPzUi3AhDqenzpGhge5d21pKzu1OgzQsMzJM+bpVAIQ8Xs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vVv115pt; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vVv115pt" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ea50564395so1155477a91.0 for ; Wed, 27 Nov 2024 16:43:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754631; x=1733359431; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=wXSz7L/2gLB1nXVbSFWAt9DGwp25yrxYRtHtDBXMeOs=; b=vVv115ptRwEuNlic1gAJJK4/+kcJkoiMNDcKeiFEAhXkDWXrppTg3cdfvlZ3ZWb57z FXp/BWwPcJckC/DVW/m7sQkZU9oFLUA4zyonj2q+6IbknIDQJ3WBWjEDKnDXWP7+NNcJ sleNVJbXc/mqQCDP8TmNhOKSc4RlvfDZLq9GuAu44sHuvSYWHygpT06NVMbeaZ1Qw8db /SA5KGsFa3eQ68PDbGN5Kb2XPI2lI/Jw44wrhJ3XUhIN6Cv+SsI5QykUqp/vMdtE57Fk 6mXa3sGUIXTydl4aXVTxyP6/93gpwzNefzBN+ns7JqN8ynbaKNoNKvJ0N90Toaz8qJaw 7yHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754631; x=1733359431; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wXSz7L/2gLB1nXVbSFWAt9DGwp25yrxYRtHtDBXMeOs=; b=nszNW7SecXQPvLRTm+f7ZiuMzme6vVOPmmV+TV4j6Knu1L238jZz3fAUonUJYb0F5i Zl55XTKiBYTfQ2UCE95t4zhBfy50CP5xuSRz9Wby9c+wPfSNqCDK4+RhV5AiXIKFMKH+ XeldVxP9Y+yHHXq9l8kbIkxXAqkOiolVWJvUJsiOKujYhoTnnzJa1g9ZfKLdf/zUY46W uiareOw760HfLjSCl01fblcXuR4elWIdlm8vjlQgaf+kLwPi6Fp0eVglYaBV7XjmAHQb Mu0dUrzgvSULkyxszQJYMISG9BOJsjC+pMv1MBLIVGgeXn3CymwwQCfPDcbcWRw6Fv/d 8FKw== X-Forwarded-Encrypted: i=1; AJvYcCUCGNUUoQj4ai1sxpRlQIN8+GR4Cw9UvWcfRGtJrFzKK57nUAIeG5LsL+U3VwQqrJ50Ufb8wl16zc395EA=@vger.kernel.org X-Gm-Message-State: AOJu0YzwE5+Wuvy5S0fVxl4NaIBGA5M6tnaK7w4cuTMiep5a+mMc0RnS r9bmB88/seQxZqQYCm7T3H9XcFVl6g0DcfWeSL6vYoytdxb7yx5qk6EHLfxVfFiaexRK79rWTE+ YOQ== X-Google-Smtp-Source: AGHT+IHtc6j3/IEkFk2MarwmYZptNoBmam/jKP/ZpVGuOWQAsWGuBBWLJG0GeFfwVxzsBB4VbyBz5vgvYEs= X-Received: from pjbhl3.prod.google.com ([2002:a17:90b:1343:b0:2e5:ef8a:48c4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:ad8d:b0:2d8:85fc:464c with SMTP id 98e67ed59e1d1-2ee25b3a158mr2128961a91.11.1732754630678; Wed, 27 Nov 2024 16:43:50 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:40 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-3-seanjc@google.com> Subject: [PATCH v4 2/6] KVM: x86: Add a helper to check for user interception of KVM hypercalls From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Binbin Wu Add and use user_exit_on_hypercall() to check if userspace wants to handle a KVM hypercall instead of open-coding the logic everywhere. No functional change intended. Signed-off-by: Binbin Wu Reviewed-by: Isaku Yamahata Reviewed-by: Kai Huang Reviewed-by: Xiaoyao Li [sean: squash into one patch, keep explicit KVM_HC_MAP_GPA_RANGE check] Signed-off-by: Sean Christopherson Reviewed-by: Tom Lendacky --- arch/x86/kvm/svm/sev.c | 4 ++-- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/x86.h | 5 +++++ 3 files changed, 8 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 72674b8825c4..6ac6312c4d57 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3640,7 +3640,7 @@ static int snp_begin_psc_msr(struct vcpu_svm *svm, u6= 4 ghcb_msr) return 1; /* resume guest */ } =20 - if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE= ))) { + if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) { set_ghcb_msr(svm, GHCB_MSR_PSC_RESP_ERROR); return 1; /* resume guest */ } @@ -3723,7 +3723,7 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct= psc_buffer *psc) bool huge; u64 gfn; =20 - if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE= ))) { + if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) { snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC); return 1; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0b2fe4aa04a2..13fe5d6eb8f3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10041,7 +10041,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vc= pu *vcpu, unsigned long nr, u64 gpa =3D a0, npages =3D a1, attrs =3D a2; =20 ret =3D -KVM_ENOSYS; - if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANG= E))) + if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) break; =20 if (!PAGE_ALIGNED(gpa) || !npages || diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index ec623d23d13d..45dd53284dbd 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -612,4 +612,9 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigne= d int size, unsigned int port, void *data, unsigned int count, int in); =20 +static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long h= c_nr) +{ + return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); +} + #endif --=20 2.47.0.338.g60cca15819-goog From nobody Sat Feb 7 08:43:58 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D5B922615 for ; Thu, 28 Nov 2024 00:43:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754634; cv=none; b=btcjDpURjtY6NRsjck4gACgo2tfKO4p11Rz2wYc7sC53jmIggHrZHBiHpK3pqciI8cHEmeCHRHOm7qOlNo5UXie2asPQSSP9K3befSBOzLuBaMa3e8PFkT9T5s1IUomQxBfbBzj7PWZ+1plzfcB2D6ssNGKpsQWoVqrdpsRY4g4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754634; c=relaxed/simple; bh=jyPX3MdAUnX1bDy5TiI1V6ZoW1meXejrHFasqL8eDK4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RG6doMLeEt/WeYYip9mzQznH6vhkm04n3Pz6PO22zQN51yXq8O2/aTOTnI8ZdzllKTislwPg4Zsw1XdvKlVM+4+sc+aoHjpB7U23klhzqDPoDIuyaNIRE/bVwOMPWPhHs6u4uqCo5qmuOs6/8BaFVdRQiIHAtApyZBtl+DU8wcY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LRbohuJA; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LRbohuJA" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ee0c9962daso311236a91.2 for ; Wed, 27 Nov 2024 16:43:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754632; x=1733359432; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=1w6JVsg4OUo1AnaxThquyWJswnSGtvGXHMhG9ntSUjE=; b=LRbohuJAneS6Fzwyo0Nu1qq3tkK9wU4TjGm/exnwHhou4HP0qz9ZFla55/i07tyYm0 TlueK+Uk8TDzc9+KfSvPEs3vi+tnaa1jSWLw33uhrBHngTEnQutuPbYrnV50SlOBNfcp aH05IDaep7q9HEE84f0Z2fPgtvY0BzBo1XLza0Rpu7yNgQn3lYFmUiBAyluZWXdgI6M3 uQwDmlvxVqRniicympzZmF0roTylHiFgvy90VFHSUzq6/pM2vbLlEXXyhUOj4+hpHMfV nHsCgtgz7St2OdhdD3x+Hd3PaBmZ2OXQqoRZ3lQEcRgkibYVavk8ZOsx+VI/b6vM0Ow1 55hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754632; x=1733359432; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1w6JVsg4OUo1AnaxThquyWJswnSGtvGXHMhG9ntSUjE=; b=Wlx8Cbs/qDvDS5zQX3Cu3C1KYdXkpPBLz3YnkZzfUMvLCqdvFlqjM8C7qOwZCjrgxT Ht+AF6qbVcbJCx6NaQfghmjqqvsSn6FAMahdtMAvY1yyyMq1u0cy9Ip58HE90Ssv28vj Dq+Pv4Rpz3u5YG0WSudic165N0vxD7aequK0g7mEMxf4UmWjJvRtu2rMK/gaDW4opoIG CB+oKuHVYNFGC9GDNj4EKfHGomSclz5JnFHGi3XaUNHzDfm5THpTE+2wuSyQkVy9/y3N OsZgsV8EO2Uy3skoeCkvhUzwNGqIIxpHIFd70ZE618wgFPjlr9BkR2TWoN7I7pwR0Vgc +mEw== X-Forwarded-Encrypted: i=1; AJvYcCVyK3BLxIT0ZrmVe20kHmd9qJa6IObRdrqXyo/1PqmNY4XvJNoaNXeEpQQwic9201Mx8fwloPojL856YGk=@vger.kernel.org X-Gm-Message-State: AOJu0Yzoqmh9bpaDDhh1PZO+mLsQ3a8zhkqa2oMLZfuW1NZHyMA1Slbe AXuwpkukymTs+UPkL2TxBfoEnNMtw+gl48DmO/I8E9tdnW/+asBzBNx3PzZ/1UQkriGWydvz2wI pGQ== X-Google-Smtp-Source: AGHT+IEQTw6NycY68SHBk7WI4L2KbulF4hVElsyaIaLj1wJbiWRwmM/PjWD0Btsx9ro7hcsaxrPuaWvf6Js= X-Received: from pjbli14.prod.google.com ([2002:a17:90b:48ce:b0:2e2:9f67:1ca3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d12:b0:2ea:4c5f:923e with SMTP id 98e67ed59e1d1-2ee08e9a0dbmr6652768a91.5.1732754632474; Wed, 27 Nov 2024 16:43:52 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:41 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-4-seanjc@google.com> Subject: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the declarations for the hypercall emulation APIs to x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor modules, i.e. don't need to exposed to the kernel at-large. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Kai Huang Reviewed-by: Tom Lendacky Reviewed-by: Xiaoyao Li --- arch/x86/include/asm/kvm_host.h | 6 ------ arch/x86/kvm/x86.h | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index e159e44a6a1b..c1251b371421 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kv= m *kvm, kvm_set_or_clear_apicv_inhibit(kvm, reason, false); } =20 -unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long= nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl); -int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); - int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_= code, void *insn, int insn_len); void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg= ); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 45dd53284dbd..6db13b696468 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *= kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } =20 +unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long= nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl); +int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); + #endif --=20 2.47.0.338.g60cca15819-goog From nobody Sat Feb 7 08:43:58 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B74503F9CC for ; Thu, 28 Nov 2024 00:43:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754636; cv=none; b=TJFkaVQBKYAqRq/GG+zA5DFFiJ+pduL7RNr2kHk6cyElR7xnmKNF/SwQyUWuszpZk7aX8vjjPL7u+KjM7n87X3vraS4ppZg0ugPqeQq++HIcGDZqUrXiA8e48aSEanar8XZHmlcFQKQrOgrPrb4to7Lhhmv0lyFS9AYRyUwtLXE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754636; c=relaxed/simple; bh=AgF1nLFRJT2jGgGNEvzsKs6m0B0/bKeLHYnQ3NQUIic=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ua263tfaBbwWUIqCSJhXjvioRNr4Ufxa6DGn7a/9O+QMjvKHr0UwlZhl+O9OnleWb2R4gSdM9/KizxIWgHfenUdUfi9qcdap9GvDVDQE9obNR7tyTCjW47oWtYM26qr/F3EF+hmwo7m0kC9CMGE6r/5ztoCuQq3tvyCaxzHRzvw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iMcPu1mW; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iMcPu1mW" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ea2dd29d16so307992a91.0 for ; Wed, 27 Nov 2024 16:43:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754634; x=1733359434; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dDZT1bmtmeMkj/4PbsRTAxMKbyk+c3Rb1LRLdcLSYzc=; b=iMcPu1mW5FyKzgEe/m80oOhGh0/ZDI2Ang0Nn0OEB2CphL89imrYEyCXvd+/PP/mxT 0zstjftCgJVBzenM9yIhb71rtHgX5cHWNIJ+OJrf1vD1KyXBEOEpKprsPFQe73bfkEW1 FiIliBS7fGZVwTCObDuwgxAOD8kslb8+Ks6nrJ8W80tQhI8qKzE+e4DAJ8uOzgn86eBs dNgwI3BWSD6/fb3cOHl54oXzMo0iz0yW87Vhife8aEtQZ4jVsVepYt83AqveVaYNs+ef SBmxzFLpyD4jnJWcz9d2SrusZNrY22scmALgyqzK8P9K4WMt4nsfxOT79fnUZDoxbIhp BebQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754634; x=1733359434; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dDZT1bmtmeMkj/4PbsRTAxMKbyk+c3Rb1LRLdcLSYzc=; b=bcrBlS6uiZ2li4jUEL50C8A95vVzfcdG6UZ869OOmPM9xssOX5M+OzgxQ3KSH8cXXM 0eGx77h2yKjHr8I6M2lamjNH9gFAcgbM6IupzcFaK7eTT/Ms/fF6WpEMNq33IAiDM6dn e66+yLAmoTx5hl2TVT10d4qLLf+z2XKXM17QTnQxD63JxJe6u30r8c4qDP0g+Ct4u3qs NsQDLey8a1OInDD77n7tblUoVD1Ety1VTGtj7SLcW9kqS9+0IecTKbeIdWU+u5wjkhoC JbwrirqGukNDNVNBzy/H0rvtdqTdhGbWizZasiXEdrlWZne4U4tZGxeXf4dlkIIpcZ5V oacA== X-Forwarded-Encrypted: i=1; AJvYcCUk0tYcMScB2433Q9ZpNbR541siUkawkMtecMBlGmH7Kwn/PZnryF6AjTvsud6cpmwvUdeTMngQOJ8AQ/Y=@vger.kernel.org X-Gm-Message-State: AOJu0Yw2sdxjrC7ldqq3vuFuVWxtJjFOJm3dgn3dS3fmJ1HsgELZ513t MBhmwhAL+uBB9jCZBdh8vwRK2tnGP1o1DeZsv74oWIDSyw7Zgqd1AXsBCeKjyPn/Fh5ayMMHcv4 TdQ== X-Google-Smtp-Source: AGHT+IFvrKrb++Xh6P0AIe+4NdzweBJ/xiuvfkWJ8bzV4vHyK2OmIv7X9EgrfXBEvIn57suFkD1uefwv9ZA= X-Received: from pjbsq16.prod.google.com ([2002:a17:90b:5310:b0:2ea:22cd:7ff1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4c85:b0:2ea:aa69:106a with SMTP id 98e67ed59e1d1-2ee08e9bd37mr6163133a91.2.1732754634166; Wed, 27 Nov 2024 16:43:54 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:42 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-5-seanjc@google.com> Subject: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits. Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences). Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Kai Huang Reviewed-by: Tom Lendacky Reviewed-by: Xiaoyao Li --- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 13fe5d6eb8f3..11434752b467 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *v= cpu) if (!is_64_bit_hypercall(vcpu)) ret =3D (u32)ret; kvm_rax_write(vcpu, ret); - ++vcpu->stat.hypercalls; return kvm_skip_emulated_instruction(vcpu); } =20 @@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu= *vcpu, unsigned long nr, { unsigned long ret; =20 + ++vcpu->stat.hypercalls; + trace_kvm_hypercall(nr, a0, a1, a2, a3); =20 if (!op_64_bit) { @@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vc= pu *vcpu, unsigned long nr, } =20 out: - ++vcpu->stat.hypercalls; return ret; } EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall); --=20 2.47.0.338.g60cca15819-goog From nobody Sat Feb 7 08:43:58 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57A5EEEDD for ; Thu, 28 Nov 2024 00:43:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754637; cv=none; b=Os3kZFUu4iLkmRC9w/5ZAAYEMcdWlr8lmfEJc7jschMNYEL71pDbYWrI0fYemHiwoQt6xpaYrMOsjl5s8GA1MFmvij8un1uOYne5a3vGtWOVEPUStBVNMV1wRrP/d7GN/Cve8fWhebEevksbjNgEMroeU4Lk3/hhthUGgouR7ZM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754637; c=relaxed/simple; bh=AEDUeCL/zaARx8onXji3t/YtfDw26liZdxyZL4gPBeI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sg7HWlP00ml01SPsumLgVxgZi29ofuI98YrW8rJ3X8QeDNdKErfmPtoCAUgHlrmf9R5qyUfAJfmia7U0BTM9D/oS8VtQ4w3CKI6pq05tkgK3ZBi4UVPEzoRcLRgBMK8quoGNPOAQusrTb1/0+q4XGK/X9P2Ol7zU/8TZgQNKJco= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MZtoZRpu; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MZtoZRpu" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e95713327eso346464a91.1 for ; Wed, 27 Nov 2024 16:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754636; x=1733359436; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=b+5GMpw/w1WHRIobpEWm5+/qy2gHIU4YsE7Y113p5dg=; b=MZtoZRpu338MJJmHtfYKDG3uwOHPGZhCnZm8GIf4tqm7RH1/zF1uxd19ucKx+2kR7I Z2TZ0yj/J4FGEjMswLH5BOp1Jo9br5CPk3AfnAEK4fBNt0KLywDDACvH8iGZ2r+GHO1s AX8dNAqsPiFUnYjKSG2fObPhN+6HwSzLo3V9YGRk7t3hpxTbcRCzGI10T8V0moGMxqUA MQjVNzYVELrzaoWSGPH7KIH9i/BttK1uuMvWSwbQsA81SMvFPfm7g9njcYMj8Nrqur9C qV1tjaKctHyQecApKVZIsMMpIVF64gXGEHRp+GVZYA0PY0jLacygQGGqxyczhEAON5M2 afVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754636; x=1733359436; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b+5GMpw/w1WHRIobpEWm5+/qy2gHIU4YsE7Y113p5dg=; b=YNlIPxnoVVZ1rDbwu5TjQRgi7gdCgEX04asuVTGNCClbNM9vDUUIY4BozVihckXvU9 YaNFr4/An1SZSLB0uvXqozC1ffbBshmffsYEASILc8wfIEWvnKrABE7ag248DbTpFDL+ /ONanasUeGS7tWI/ycuwT4e2mFBI0iNxho0s89Sb1SPoRVlPJzVRYRVTr30sJACPfLfF SELq4LktZi7oyr74tao78KH47NNoyVFn7OPKf6R/dz6actRtqXcOkvawjd7vGcHtonGw o0It/UuUR45jktjRolX3nKd2fvjmELGX0IFIpl08Oh1pitPMEPexfvEZ6K3D8lxM132z Lg+w== X-Forwarded-Encrypted: i=1; AJvYcCUZW2cuCv91JQKib3b7PsMqM0Sy0JwNxOpoVda26bxjPWchm1dRRZ6krGnsvrzyvG9pJFEPAdo95zuGz3E=@vger.kernel.org X-Gm-Message-State: AOJu0YyvpN9GmO/E0B34aLLmHTqHLNcG41Ho3K16o5oQarYZL5RF8sU1 67lzNaLSqLWWluDRCML+ahsS9iVFGqKpYERodYOWbTMnWqgN0TOU7LxqoCm+tj5XZMPBUHvpdpX 3lA== X-Google-Smtp-Source: AGHT+IGosbHKAV+rkitlr6k2kWiYww6vT/dEz80J0WwFs8XqNGHk+Uosu6KGRZs/IjB6xkH6D7120BnwA0M= X-Received: from pjbli14.prod.google.com ([2002:a17:90b:48ce:b0:2e2:9f67:1ca3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:48c1:b0:2ea:9f38:993c with SMTP id 98e67ed59e1d1-2ee08ecc4d9mr6651891a91.18.1732754635965; Wed, 27 Nov 2024 16:43:55 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:43 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-6-seanjc@google.com> Subject: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Finish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest. (Ab)Use vcpu->run->hypercall.ret to propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback. Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for KVM_HC_MAP_GPA_RANGE (and only that hypercall) means "exit to userspace". Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro. Suggested-by: Binbin Wu Suggested-by: Kai Huang Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/x86.c | 29 ++++++++++++----------------- arch/x86/kvm/x86.h | 10 ++++++---- 2 files changed, 18 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 11434752b467..39be2a891ab4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu = *vcpu) return kvm_skip_emulated_instruction(vcpu); } =20 -unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long= nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl) +int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)) { unsigned long ret; =20 @@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vc= pu *vcpu, unsigned long nr, vcpu->run->hypercall.flags |=3D KVM_EXIT_HYPERCALL_LONG_MODE; =20 WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ); - vcpu->arch.complete_userspace_io =3D complete_hypercall_exit; + vcpu->arch.complete_userspace_io =3D complete_hypercall; /* stat is incremented on completion. */ return 0; } @@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_= vcpu *vcpu, unsigned long nr, } =20 out: - return ret; + vcpu->run->hypercall.ret =3D ret; + complete_hypercall(vcpu); + return 1; } EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall); =20 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) { - unsigned long nr, a0, a1, a2, a3, ret; + unsigned long nr, a0, a1, a2, a3; int op_64_bit; int cpl; =20 @@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) op_64_bit =3D is_64_bit_hypercall(vcpu); cpl =3D kvm_x86_call(get_cpl)(vcpu); =20 - ret =3D __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl); - if (nr =3D=3D KVM_HC_MAP_GPA_RANGE && !ret) - /* MAP_GPA tosses the request to the user space. */ - return 0; - - if (!op_64_bit) - ret =3D (u32)ret; - kvm_rax_write(vcpu, ret); - - return kvm_skip_emulated_instruction(vcpu); + return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, + complete_hypercall_exit); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall); =20 diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 6db13b696468..28adc8ea04bf 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm = *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } =20 -unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long= nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl); +int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)); + int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); =20 #endif --=20 2.47.0.338.g60cca15819-goog From nobody Sat Feb 7 08:43:58 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 363BF83CC7 for ; Thu, 28 Nov 2024 00:43:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754639; cv=none; b=R7o+11QBObH/7BZtTfvsDtC0+PY8gX11RuPUZZOhCtclSo8awhT5oAaL6hkcuESPe6kEIH+Jp+6oY/TbQqSv00vy5ObjB4Jae9psPBkNxC77SuXPQ+NYd7qNJqJ2OKMWbo8vCxMb/4A3NCySv1VH8Gp9mZ39aF8Yen7GHahRD0Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754639; c=relaxed/simple; bh=iMEWMeSgtpX2vDiZAZHNPxwDcscsHshHwaR+WGYfMz8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ALFV+umhKMhrwwQjJDSovaNH16x3t568O/91h2Fib3zohnJl8TAj/q1SEV0AtjjRb7jRowHgbIbV01RICuDsAG37QlOj5FevjdnSq7Sk/SwsmGqUaXpnLk1aEaZohcsMGuRT4cb1oTriaQ6Dqb4DWkVg8UTH6xoibfsoRzpkI2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Dm4qnYCo; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Dm4qnYCo" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7ea69eeb659so196135a12.0 for ; Wed, 27 Nov 2024 16:43:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754637; x=1733359437; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Kb2dr0T5bp6rNnjcREbp+SqCdYs3HsUs4UOFpqH2Uio=; b=Dm4qnYCoQ441Mvab8cdSfmnRZWVfohDjPcUmSdbu3CGH5T+rzr0H/TNYK8pfLsIyPn 9qyKQgNv1A28bdHGrAFS5t2DXlW6fkEKWu4fHfjf2Zr7rKQwFidhdpic+AB1cXVRiyS9 P+8G754SxNVhIkU0YICN9NuNpmTecZIaoNEzbQj1WoZ5tnJRzd74K7xu5wySExxuxDH3 kKXVYWRotgqYaO4rGcU7/wN328Reo7sMuIqT99csvMEH2+Ir7GmVaD1cj4M/Ws5ShoV3 XLJQ4WFY2Ua+9oB1ksyEJ8tUyQ6vBPzkL9hXcX3UkNIKuHep3D3CjxkmVz7IhMCEQnK2 6/5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754637; x=1733359437; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Kb2dr0T5bp6rNnjcREbp+SqCdYs3HsUs4UOFpqH2Uio=; b=tZDbmDS8cIXeFuA5imbUQFLZ6Q17lAmhFtzwEkveT03GX6hU/O/NtwCXGMkb1wBU3l SsKXdMDIf/w5tkxpQLCpBZ4gKKUhJg6zFAW5NpwVQK6AtxeECky7DGKLJhjiJQzw+x3f 4YvhCP4IiNR43g+I8Q47PlCYo9+w//5oluhWYB3yWxiQN6Dg1KYUgyfzY4KtPRgsrJ8K XCCdKZumumcOV7FJiJIIsG2B1d3lcqxkcaxObDuk0dGpm5hw0dhPsDy1RP0vOZhd5S7J d2wAPd5dQ/mDPzVHHsJaBE4+cXWn7YEG+OyeHobwX3YuTbjur4OaThRIQTDB2TMf7yaO VjSg== X-Forwarded-Encrypted: i=1; AJvYcCVA7Dj0mrPAHqAz5kjvGinMUgXbDISR8gvZBhvTe5csl/lilrYIhgMBqPAaNvIeKIqS3aKVibNqxg6M8m8=@vger.kernel.org X-Gm-Message-State: AOJu0YwLGA4kRZercsVCt4isyP18oJnI/aKz07dfigBRmU+yn+8UxbtC noKWw0BwumPbvBhKST28RtXgjIQ8TR9Q+lDj8kdqXZ2Hq7NT3YcalUmHeqJytz8SPxiCEaoabcA lTA== X-Google-Smtp-Source: AGHT+IFAR5YUseLMEmNyaxfAMVHlZLF3OQCHcVX3UaR+ESdLafctVjBieHniv07m1mzMJyd37ZzdGkN19OE= X-Received: from pjd6.prod.google.com ([2002:a17:90b:54c6:b0:2ea:7174:2101]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:734a:b0:1d9:4837:ada2 with SMTP id adf61e73a8af0-1e0e0b7e3a4mr7225792637.35.1732754637627; Wed, 27 Nov 2024 16:43:57 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:44 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-7-seanjc@google.com> Subject: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rework __kvm_emulate_hypercall() into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=3Dy. Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available). Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Kai Huang --- arch/x86/kvm/x86.c | 29 +++++++++-------------------- arch/x86/kvm/x86.h | 25 ++++++++++++++++++++----- 2 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 39be2a891ab4..fef8b4e63d25 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu = *vcpu) return kvm_skip_emulated_instruction(vcpu); } =20 -int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl, - int (*complete_hypercall)(struct kvm_vcpu *)) +int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)) { unsigned long ret; =20 @@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu= , unsigned long nr, =20 out: vcpu->run->hypercall.ret =3D ret; - complete_hypercall(vcpu); return 1; } -EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall); +EXPORT_SYMBOL_GPL(____kvm_emulate_hypercall); =20 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) { - unsigned long nr, a0, a1, a2, a3; - int op_64_bit; - int cpl; - if (kvm_xen_hypercall_enabled(vcpu->kvm)) return kvm_xen_hypercall(vcpu); =20 if (kvm_hv_hypercall_enabled(vcpu)) return kvm_hv_hypercall(vcpu); =20 - nr =3D kvm_rax_read(vcpu); - a0 =3D kvm_rbx_read(vcpu); - a1 =3D kvm_rcx_read(vcpu); - a2 =3D kvm_rdx_read(vcpu); - a3 =3D kvm_rsi_read(vcpu); - op_64_bit =3D is_64_bit_hypercall(vcpu); - cpl =3D kvm_x86_call(get_cpl)(vcpu); - - return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, + return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi, + is_64_bit_hypercall(vcpu), + kvm_x86_call(get_cpl)(vcpu), complete_hypercall_exit); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 28adc8ea04bf..ad6fe6159dea 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm = *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } =20 -int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl, - int (*complete_hypercall)(struct kvm_vcpu *)); +int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)); + +#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,= complete_hypercall) \ +({ \ + int __ret; \ + \ + __ret =3D ____kvm_emulate_hypercall(_vcpu, \ + kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ + kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ + kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + complete_hypercall(_vcpu); \ + __ret; \ +}) =20 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); =20 --=20 2.47.0.338.g60cca15819-goog