From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DCD573D092F for ; Thu, 14 May 2026 21:53:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795640; cv=none; b=PqhHycfH1IOsypCC99JzA9/viX8fCqz36Q/uYBR3SbefLnxfGsqzglFbmwOVQptMyaxYI6bA+lWKyIEVsHfvb7C+K3Ehp51NAp1l5qgRA/O4SaQZtCnDitueCLWzuy1t1yFLiEcdFFUUAMEE3lce7cFYd0SEGRkxQkxHmheNUGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795640; c=relaxed/simple; bh=h1qTMfKuEFDk+2bv6Vhwh7IxdwliAQW7/9Hv05WYO48=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jMHGkRUNe5ppYgTTwetGOR4csA8SuoS+YeSr4+9qFCYAdHiYTRNf+NcJvYimbIX6+Xsk960YnWx7WdtiHZ0tvNh37bgc2aGoozoCLcWtIgtY0+6Sk4JEbU/9Jxc97FSLIUAJm0WkdVDG1Z8h3ugRBbBQwM6XnnRccECQ81fHaKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KBYjegZZ; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KBYjegZZ" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-82f74bcfb86so8824564b3a.0 for ; Thu, 14 May 2026 14:53:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795638; x=1779400438; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=6BsK2zhxiwWVCIABmK0YYfn1Tg3z/0TREkGbcHGBLTQ=; b=KBYjegZZjVcKIl1uxAOZdD+CjhXJoBmbE7Nk5yax3ptbHhTZYa0GeNgUu3Q1yY/2jL mHmBYVHxt7sEH/QU+/8aN0axwEXs6vMIbvKtcV6WZ+QRKsTjzn+l5uh6AZHtTimubUvl Cd+83QV5ptgGiuBKbmlOMiRqqk8meg2TzVX1DX8mX8x6lyLoDAnVjwPJrJXNV+/SAT8s +zaphDLcv5miqGlqZuGX9/du7AzG/j91mwQKayTqdS/J55YDjTZAr6A6Eje2LxpWa0ZY 5IXSdj25Qg0J8FyoYxClmG13M64ZBJfDwg5GaumQx7stAY8SODwBDU9MkfDNR3MenB93 d3nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795638; x=1779400438; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6BsK2zhxiwWVCIABmK0YYfn1Tg3z/0TREkGbcHGBLTQ=; b=FYoajp7AdhFGjbGwnsRPO5rPpbbB8DJMwbgf9p8hO2dr2opqQWBFVd1UUy4MEbzK6w 4Mw5cYtP/PuxTgW5zxYJZKBG+e/saJ1YTjQwYEyAzxgjw4SzkashGcS0hWu86k9pfDwU YjKrvXGRBIPcQGgL2HtIjh5r2ziba9vuuXt8+RPDlxDSIuGCPeFVrGkRDJEoH12JUICA nE7PDbv5xhdckElxWiIqsNrdrfU+l3HLSgOr4ilqWUIz7bKoqGkILmzvkUs4D2+ksHNo tx+Wl4hixUo9NIozB5Iu7+oFX4jry0qZrhWKVN+x+udc6RcHDwjez9u9wQfAPMrSxKbX 4lJg== X-Forwarded-Encrypted: i=1; AFNElJ+8OMokCpB8eo3FyzWXj5621MVYQ8GwZahlUrk4ajoXk7b2Cxyb7+S0uzATok2oM33J3I/W2jGpLkDCfjo=@vger.kernel.org X-Gm-Message-State: AOJu0Yzv9l2DPDHRm/aBVRgbnf+4tu0EEA0AZvBF949kaW99k4Zis0Vb ydcMkwUqLY+Ac0U/e9cy9Uro2jY1hEYfB6gaAMNTA7bUT6MPBFNntJ1s2r4qoXXxBxqPC3ClBCI 85B8mLQ== X-Received: from pfbly26.prod.google.com ([2002:a05:6a00:759a:b0:82f:a4cc:2fb3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2308:b0:827:447f:7dbd with SMTP id d2e1a72fcca58-83f33f28f4dmr1261393b3a.46.1778795638008; Thu, 14 May 2026 14:53:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:41 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-2-seanjc@google.com> Subject: [PATCH v2 01/15] KVM: SVM: Truncate INVLPGA address in compatibility mode From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Check for full 64-bit mode, not just long mode, when truncating the virtual address as part of INVLPGA emulation. Compatibility mode doesn't support 64-bit addressing. Note, the FIXME still applies, e.g. if the guest deliberately targeted EAX while in 64-bit via an address size override. That flaw isn't worth fixing as it would require decoding the code stream, which would open a an entirely different can of worms, and in practice no sane guest would shove garbage into RAX[63:32] and execute INVLPGA. Note #2, VMSAVE, VMLOAD, and VMRUN all suffer from the same architectural flaw of not providing the full linear address in a VMCB exit information field, because, quoting the APM verbatim: the linear address is available directly from the guest rAX register (VMSAVE, VMLOAD, and VMRUN take a physical address, but they're behavior with respect to rAX is otherwise identical). Fixes: bc9eff67fc35 ("KVM: SVM: Use default rAX size for INVLPGA emulation") Reviewed-by: Yosry Ahmed Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/svm/svm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index e74fcde6155e..4ad87f8df392 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2415,7 +2415,7 @@ static int invlpga_interception(struct kvm_vcpu *vcpu) return 1; =20 /* FIXME: Handle an address size prefix. */ - if (!is_long_mode(vcpu)) + if (!is_64_bit_mode(vcpu)) gva =3D (u32)gva; =20 trace_kvm_invlpga(to_svm(vcpu)->vmcb->save.rip, asid, gva); --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 067E83D0C17 for ; Thu, 14 May 2026 21:53:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795641; cv=none; b=fJ/8U28m6yVRCrD/9I5ho3dCsvS4MpjA8RspL/5jkUSVE2izfmIoPooqQfRhqGmjFEtNkWtZjktpV4LtyGCldndJY7GGaV4QF6+tqNS3JxkjVEsht85CFHXor7Zo2HJtiqgXTu4e46bm42PMr+k10BfhNvLe7tz3HTjl2Uf8d4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795641; c=relaxed/simple; bh=GvpQ3pTchK+W71vrs1lzAcwN2FhY6JiRsu+aYrsO2y8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=G5u9cSH5zadzDKnI9N+8aZXFJ1IrgleyOvzPlrd/BSXWpJZlAbYi2vkYp4wKxNlIacLiUDvp6w6MEMaxqTTX0U7Wa4AoCfVQD+mH1/3Ss3S5Cbdf1RNOl3NNM9bB7LY6HQZmL6uz820tN1GctgAjcLdegpHYiD5CrrDHIBE3Rjg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ibZKTRd8; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ibZKTRd8" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b458add85aso88920805ad.2 for ; Thu, 14 May 2026 14:53:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795639; x=1779400439; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=g2qOpbXP0VS/5/XuoZ/aUTxf2CmnbBXQvkx3/ML8B9c=; b=ibZKTRd8Iz6ZjnMN7o+8dn41kItXMavo/kbCMI/6TrFpOWDVLSTJX7WM6dT8nhxLQK jpU3rQohRpdUlRUEAfkWrLqhfJYS6pOTqythHmyTFmXodGMmXrgK62tPoY09lUHxx4Bo Eyli2f9A6PhIg3itzs7vLo98lYquRZ/lJ8tn/AdqRkGWPDWIL8zokU0PfscYQ8s9Dc7U ehNqYjLvIyY1mqgaCVChAoN9hTgE0CGR/fJj6cpQ0TPDNf//TCwpUNR8DHw9ORgVs/iX HzwuSHPRNsLFoJ16sQD8YDKqFg5vN2o7SOkm6ErqmaDMsG/u7DJyjIt30GTBRMsrVUuU YjHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795639; x=1779400439; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=g2qOpbXP0VS/5/XuoZ/aUTxf2CmnbBXQvkx3/ML8B9c=; b=G3EcU7zARAxIdzSPopUgDOijf8uuoQ0a1eRRAC2h7rbHpuMg7bntSwZET1bV6FZZ6L cRhLXgv6dDD8zCPf/HUgMZ3C0cYAU8uTCj1rl7T7h+OY8egUgbL1FP9mugCfyWhVa37v sJbXPG5ZJyGGwpmpE93wdOyLOxzf49Iq444oFz16QG9mjiJ3owv5ti7H++AHqfzpLxBV NXsbVe5aPHxB+OMo8V0wbwfpyaOL9JI3Jocj4jGF3LLX+CcwQMRNDtqT30YhtijGRzwu ryn4053Uo1hv6HG3Q+y7I+3CF2cDEwT6o3ElCetkOmvNL/++UrX6qXtKXdUmMUSwlH47 L2WQ== X-Forwarded-Encrypted: i=1; AFNElJ/RXO+Rd8vQW/lbPj59OGT9gWBUukh/ebMqqGJZAr4PRnN0KCKRKGt6zj//oiD2CVTfxvF1xRDNi+W3cgU=@vger.kernel.org X-Gm-Message-State: AOJu0YwZLOsjzfwdAq/Hpho5Ry5kdQf38KDomWin+TGUJ1tWYIRD6nK/ b6Sz9TFV7XJI4e6PLzRkTydtJ98Yc8hwS5CEMWqtDuV/xDh1hjyhQqrmcrsND5jOuqchA43u/eV ADIR/Tg== X-Received: from pldt9.prod.google.com ([2002:a17:903:40c9:b0:2b4:62bc:c2a9]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1ae7:b0:2b2:49a7:a5bc with SMTP id d9443c01a7336-2bd7e9e5458mr11350815ad.39.1778795639070; Thu, 14 May 2026 14:53:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:42 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-3-seanjc@google.com> Subject: [PATCH v2 02/15] KVM: x86/xen: Bug the VM if 32-bit KVM observes a 64-bit mode hypercall From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Bug the VM if 32-bit KVM attempts to handle a 64-bit hypercall, primarily so that a future change to set "input" in mode-specific code doesn't trigger a false positive warn=3D>error: arch/x86/kvm/xen.c:1687:6: error: variable 'input' is used uninitialized whenever 'if' condition is false [-Werr= or,-Wsometimes-uninitialized] 1687 | if (!longmode) { | ^~~~~~~~~ arch/x86/kvm/xen.c:1708:31: note: uninitialized use occurs here 1708 | trace_kvm_xen_hypercall(cpl, input, params[0], params[1],= params[2], | ^~~~~ x86/kvm/xen.c:1687:2: note: remove the 'if' if its condition is always tr= ue 1687 | if (!longmode) { | ^~~~~~~~~~~~~~ arch/x86/kvm/xen.c:1677:11: note: initialize the variable 'input' to sile= nce this warning 1677 | u64 input, params[6], r =3D -ENOSYS; | ^ 1 error generated. Note, params[] also has the same flaw, but -Wsometimes-uninitialized doesn't seem to be enforced for arrays, presumably because it's difficult to avoid false positives on specific entries. Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/xen.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 91fd3673c09a..6d9be74bb673 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -1694,16 +1694,19 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) params[4] =3D (u32)kvm_rdi_read(vcpu); params[5] =3D (u32)kvm_rbp_read(vcpu); } -#ifdef CONFIG_X86_64 else { +#ifdef CONFIG_X86_64 params[0] =3D (u64)kvm_rdi_read(vcpu); params[1] =3D (u64)kvm_rsi_read(vcpu); params[2] =3D (u64)kvm_rdx_read(vcpu); params[3] =3D (u64)kvm_r10_read(vcpu); params[4] =3D (u64)kvm_r8_read(vcpu); params[5] =3D (u64)kvm_r9_read(vcpu); - } +#else + KVM_BUG_ON(1, vcpu->kvm); + return -EIO; #endif + } cpl =3D kvm_x86_call(get_cpl)(vcpu); trace_kvm_xen_hypercall(cpl, input, params[0], params[1], params[2], params[3], params[4], params[5]); --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 565713D0C1C for ; Thu, 14 May 2026 21:54:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795644; cv=none; b=g28wPd+QBgP23WPS2ALmWPFZovB8+WkpPVDdY+FXYi8w/59npq7y3aZu3jvAJntDI9VeHf81AGXO/wkF7jMIe2hvfaliHVqUszCeLCj6Yqf77bEgrULJ1IGeuxVaBM/AGNMMHgMKT4vSQaHRYvNS+5vTv/uYhQau7MEWrcZDJYY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795644; c=relaxed/simple; bh=alOj0ulkorrTMebS435FX8VSWcJfgxZS2VP50saGg/c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NI8J4D0lZuc4ifvjtfGFFVZOj6+pEnFqE9f1Zhm3BdC6owylLE+/5T26Sqcvbofu2QMjNddtuczdQHARWTWUKkg3HQNDtNVGvmgAAlntp1IRQlRAtanzoRG6CFbM7X1SgarWmdS8pJPFQXwUaJz5+oACkaVaHydYZ2HQZKpklNQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BAKVKFXr; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BAKVKFXr" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-83536dc3be5so9464221b3a.1 for ; Thu, 14 May 2026 14:54:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795640; x=1779400440; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OSIIeDvi5cUuvLPtbPSIJ0wDOFHmUc0VllK+0XWeleY=; b=BAKVKFXr5kMm11OUaGbZK+Yu0SKersIp2codoWihh5yDxxjmr6aTQetszrD8ngytgS aCN2cAjLsnWryhltZvgCswV5ryjrmUbjKaNIdn3O7KPPYE3rb4JilBTrVNTyC9404wwR Zcd8FsieMcBDAOeuHr37eEQDsgpmZx+YFfeXZlVabq2oKMkfb99q8TPY3Pm2cKQRXaK/ UyMYQhAQp414LnLy76HgohfGAjxjqFC7xjAdVTfSzE+pcVNoAyqUZ9O6fiNglAyDZQxM i/TcL1c2emC+SVPWVzAeGoeprlNGdHs0ovZp6J/2o8IIig+sdKoWpkzAfSUnDA8WBwEV C6ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795640; x=1779400440; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OSIIeDvi5cUuvLPtbPSIJ0wDOFHmUc0VllK+0XWeleY=; b=QC3HYcWPGHgujAyuP9aISE7gZMJIpEWaM6hFmJUPPkFenAi67//zdzx889K7Rkzbi8 XBHZFFGqeLPSS6YO5UddzTXUsu9uBFQ0bgvmziGJXJ9leTMQjsmRi8BB05oLBq2a38xY oyILWUEvVBivl/tsIxT4yjp0Lg+pnqB8drBcBdjUBz5p5r3e5oFMYVNC34r+NRA2Aelw pcGm4IbmYCtz0Eb2ijxjtl/+RR04RiPDNSFbAQh4FAahg8SofN+GwCVwEhy/vKtssjsL kUFKVALZ1sXriE8t9bpN4x6+FyjKsZ3JULPpqadSiv+gfeamqFB2Ddza6LOW+5edRudP DOYA== X-Forwarded-Encrypted: i=1; AFNElJ/YoqfY0ES8nauwbL7q551vO8H7IvP2wFY2KHaYvuaCyMzNrfPfM7FnOBh5nxWB9z7e96ERoqBZucxQkQc=@vger.kernel.org X-Gm-Message-State: AOJu0YxvH5VsHOW4aij6DdEojbDJ6WgPhBHuXQE4uzifFCFdHdoY3tM0 wtxleIU0VNzofsflCJXWxeDjS5dbYoL+y4Jy6k2AZRlku+Ub0CFPoZAlfytqDlC49LxjjVt0wW3 f3QICQQ== X-Received: from pflb19.prod.google.com ([2002:a05:6a00:a93:b0:835:4315:3453]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3e22:b0:823:d2c:b156 with SMTP id d2e1a72fcca58-83f33c4f71cmr1244822b3a.5.1778795640146; Thu, 14 May 2026 14:54:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:43 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-4-seanjc@google.com> Subject: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't truncate RAX when handling a Xen hypercall for a guest with protected state, as KVM's ABI is to assume the guest is in 64-bit for such cases (the guest leaving garbage in 63:32 after a transition to 32-bit mode is far less likely than 63:32 being necessary to complete the hypercall). Fixes: b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with p= rotected state") Signed-off-by: Sean Christopherson Reviewed-by: David Woodhouse --- arch/x86/kvm/xen.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 6d9be74bb673..895095dc684e 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -1678,15 +1678,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) bool handled =3D false; u8 cpl; =20 - input =3D (u64)kvm_register_read(vcpu, VCPU_REGS_RAX); - /* Hyper-V hypercalls get bit 31 set in EAX */ - if ((input & 0x80000000) && + if ((kvm_rax_read(vcpu) & 0x80000000) && kvm_hv_hypercall_enabled(vcpu)) return kvm_hv_hypercall(vcpu); =20 longmode =3D is_64_bit_hypercall(vcpu); if (!longmode) { + input =3D (u32)kvm_rax_read(vcpu); params[0] =3D (u32)kvm_rbx_read(vcpu); params[1] =3D (u32)kvm_rcx_read(vcpu); params[2] =3D (u32)kvm_rdx_read(vcpu); @@ -1696,6 +1695,7 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) } else { #ifdef CONFIG_X86_64 + input =3D (u64)kvm_rax_read(vcpu); params[0] =3D (u64)kvm_rdi_read(vcpu); params[1] =3D (u64)kvm_rsi_read(vcpu); params[2] =3D (u64)kvm_rdx_read(vcpu); --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F1763D1701 for ; Thu, 14 May 2026 21:54:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795643; cv=none; b=PYhnZklOyUVJqX+4lS2dkpammpid8c/tLovMC+Gl3O/uk2DKb1SQc2ci3kgKF2l21PnVlQ88NnQrW8XENsGeflzYqwqMTiNqpaXTO9zjy+FOwEwL84h2n+NsoXS44XIZRVmyM84cZulr2mY8sp0uG+GuRX/SEFqPjVivLrPejFg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795643; c=relaxed/simple; bh=LgzQvsxxX3nzX18piIgpsaez26+Gbcq9HNZDoLa+NpM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qw6CcShoVxl8wfqkwiwnp5Vpb/xS2SZJB3/wKr2tIJx7gwgo866F21iLM6Y3+qv3GRASD2PfNBM56eWCdW8P97K05lolnEmLnZWYO4ngl4LYrDPk+ZOxLL0ytVJpEBkhfh9m1l9G8kx+DeSluyEeRjGDGBflxJMwzgOubcW60LY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kq0SKok+; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kq0SKok+" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82fa366fb79so9224059b3a.2 for ; Thu, 14 May 2026 14:54:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795642; x=1779400442; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=AuBt9TLpeBgSZbTV0sAjcKLbSpi+4/r6Mj25KjhoAAo=; b=kq0SKok++v+nT5w6jtIWcXEnKjVL4z1ZTtlSGtCs651URDxt0Rs+zAym4Hu6adjIgB g6SON9Gyy8tgG2P10+vUUiTKAm+ej6vk1Y6BuwdbQNG1YttjA0JOP3bM6g0AVjHhFDKX 0KVjPRfmXBS5KNjIfE3PtCwIbdW/+97Rnb71BO51b2WRFc61ffBRLP0rVEtINfm/lzc4 4TGLfEthUpPKJQnX3GCZgMyMQC09rP+9/5RIIpl5BHy9rVPGnUwlkTjZ3zzmTFH7b2dw H28ZILmK7pEKfQ2bVD1qcqrSDoRdijjG+XreozgxVC56Z62dqgfjKzZ83wXOEJi7uWJL e49g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795642; x=1779400442; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AuBt9TLpeBgSZbTV0sAjcKLbSpi+4/r6Mj25KjhoAAo=; b=RbGMWO7aJxXX/TjvIq1hIHOP9f7MLunsiBNVqkKNGRWKf9kWQOMq9oCJPqRhYPFd3a cwrcxccfOjAarGEFdfxmxincB7ZylgL6SVqCiGds1uvDU/MDnntHhlzF1WmLaSObgSsG GTMYLcYhlG9219kwqRv5qxaeEXgVhgmEqU4zpag/B/vkrPwZSbk0jx/3/ojrf9Iw7VHC gXFBT+SInbVcOoZ1vQN6oQXjO3IKATkLd+9TbBOpzdfpTFB0Doj4hAr+b+St85GCO/4Z sTk82vK07wtwHO5VTVbzwucmPqenqpZu4FPhgKXvMASl3iZdPXpuFx5p+NmP4P/UlHyu zVOg== X-Forwarded-Encrypted: i=1; AFNElJ/HzK7uChwDDAaOoEqSBAvVdV//NLHjDsK2LGRe2057w3kO5iJJIBUY+s/BU0dFgHJ37bg2jpfoR4PSuc8=@vger.kernel.org X-Gm-Message-State: AOJu0YyM+Oz0Hv+GTjbgBm77LTEaKOXkFIqM013Xzj9wNtlVQEO9I5Hk dutu3PG4qiPVzHAtHZ0SeLNO9w9lM8RhGe+hlbJ86BwoxnOKyZ0s7jtTaRQObIIlLHGmyTrCEfh Xnk0laA== X-Received: from pfhh12.prod.google.com ([2002:a05:6a00:230c:b0:838:1469:2431]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:198f:b0:82c:20ba:1570 with SMTP id d2e1a72fcca58-83f33d9828dmr1291294b3a.29.1778795641340; Thu, 14 May 2026 14:54:01 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:44 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-5-seanjc@google.com> Subject: [PATCH v2 04/15] KVM: VMX: Read 32-bit GPR values for ENCLS instructions outside of 64-bit mode From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When getting register values for ENCLS emulation, use kvm_register_read() instead of kvm__read() so that bits 63:32 of the register are dropped if the guest is in 32-bit mode. Note, the misleading/surprising behavior of kvm__read() being "raw" variants under the hood will be addressed once all non-benign bugs are fixed. Fixes: 70210c044b4e ("KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce C= PUID restrictions") Fixes: b6f084ca5538 ("KVM: VMX: Add ENCLS[EINIT] handler to support SGX Lau= nch Control (LC)") Acked-by: Kai Huang Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/vmx/sgx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index df1d0cf76947..4c61fc33f764 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -225,8 +225,8 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu) struct x86_exception ex; int r; =20 - if (sgx_get_encls_gva(vcpu, kvm_rbx_read(vcpu), 32, 32, &pageinfo_gva) || - sgx_get_encls_gva(vcpu, kvm_rcx_read(vcpu), 4096, 4096, &secs_gva)) + if (sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RBX), 32, 3= 2, &pageinfo_gva) || + sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RCX), 4096,= 4096, &secs_gva)) return 1; =20 /* @@ -302,9 +302,9 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu) gpa_t sig_gpa, secs_gpa, token_gpa; int ret, trapnr; =20 - if (sgx_get_encls_gva(vcpu, kvm_rbx_read(vcpu), 1808, 4096, &sig_gva) || - sgx_get_encls_gva(vcpu, kvm_rcx_read(vcpu), 4096, 4096, &secs_gva) || - sgx_get_encls_gva(vcpu, kvm_rdx_read(vcpu), 304, 512, &token_gva)) + if (sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RBX), 1808,= 4096, &sig_gva) || + sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RCX), 4096,= 4096, &secs_gva) || + sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RDX), 304, = 512, &token_gva)) return 1; =20 /* --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B9A43D0C03 for ; Thu, 14 May 2026 21:54:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795647; cv=none; b=qKYRT9B9Jt4kImoliJ2yB6O6uEpRy1dhWrBRgBbO5opxuBfncucnylVlSms7prVNU64opOsCpgOXtZjTi9zboFBcK+26at3508RABAqPZ3YzR07yhkgNeRHQx9uH9BlwGmvXZaobaxT5k+In1DafAmCfjh68JK44/BASgDS0q7w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795647; c=relaxed/simple; bh=dTA/3VR9EH1sMFnfWIiA6Io9L1zUxuNockTCKHiOFxY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Pg692R7zO1+u/BwMJywbr14rIc2gccWFpSkb2qPSw0HV3KqDk14sr0yOfUmA/PYp7G/jC9jUPagLCPizmaOwxfr6y3+ia+ofdGZg8yU7PYqlcHma/3drqY4meS2c4aCcbT4lTtqBbF/LNoiW/IrBHyGD5Mqix/XWl9WjvhrsYXc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SzNulJe+; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SzNulJe+" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2bd5b20aaa6so13346585ad.2 for ; Thu, 14 May 2026 14:54:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795643; x=1779400443; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9oYhOatBSTmYuzHR8azx31RA/Ba7pTesr7fCEDkJz5c=; b=SzNulJe+2zTaydoWpaVvUlC2Z4r9xumpue/Tl7hqVsY5/Zf4IIvB2JQS7pkKk3LJGa xM7uHjcYpuKBDq32bVqFLtaaC1XB1H2titmu3nLNNO+YxdCXoYQAEBzU5sdTt+uUqjk2 UgXMInHWc/lGhpNgZsyoDn1Fmaf+bKbPkdaFjw/VqCfPbB5tG+u6dYD4RRcY593yjDfJ VG8igONsAtTaTmKj6r270UpdbFZTXHSukLEzm3BJ5HB7Ez/JXymNpBH5JLTx59S+nD7n ++hYltk+EBpspg6t2Ppdt6IvP765Uya7d7uZp2xHsuNbmFDBLTfKM+kg27B+4yZmBWVw x6VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795643; x=1779400443; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9oYhOatBSTmYuzHR8azx31RA/Ba7pTesr7fCEDkJz5c=; b=kKhnJDC5zuNtJ/lK67WSq0gImoNxE+VZka5lnpeU9/Byp38H+7GguLWoX0z2jKe3Gu LkIAlfCCtTNkslFd1HZutGER4R8Wm9zo2+USkIgeD7/yTyp1HYe6S3Cz5uRm5UM/KbgC 303da3Pn3Yp5s1AU+MckEUpuC9nSn+BOU+6GYf/eNlA+cCfruvKSN8HwHt6jFx/pmobm 0UW1l+zTpkSdD5WNk9gArkWMA0DOA92JGtEvvY8+4+wqI0VYy+ty4Q/rgOmBYs/x5ZFL DgfBuVFZknA4EORiCX4kjx4oWemCVy4IqrLdQpiAWYfbJt7fSMGag51F9US85YtiFKEY WPhQ== X-Forwarded-Encrypted: i=1; AFNElJ+TYArK0GKb7WTbwa1ItjT7/jmVdBHzc/vlmuiW5r3eqfimYEIMvwBc31DA3mAj4Sj/eAGlz0FfMv2eSTo=@vger.kernel.org X-Gm-Message-State: AOJu0Yy/d1eoDp3Xd91u6Zm53UGum9+13w/mAWY9JFt6kAp0xdV+cXFq iIMk0k3RN7vIXI6tHfYCSSxgEJGx8faBm0q4WOd5DSN2vIvAOAXtU/ixsmdw/IUPKhSThFaKawJ Sxu+Yew== X-Received: from pgbdn13.prod.google.com ([2002:a05:6a02:e0d:b0:c82:7498:e3b5]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:244d:b0:3b1:884a:c3d5 with SMTP id adf61e73a8af0-3b22ebd8ce1mr963273637.24.1778795642494; Thu, 14 May 2026 14:54:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:45 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-6-seanjc@google.com> Subject: [PATCH v2 05/15] KVM: x86: Trace hypercall register *after* truncating values for 32-bit From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When tracing hypercalls, invoke the tracepoint *after* truncating the register values for 32-bit guests so as not to record unused garbage (in the extremely unlikely scenario that the guest left garbage in a register after transitioning from 64-bit mode to 32-bit mode). Fixes: 229456fc34b1 ("KVM: convert custom marker based tracing to event tra= ces") Reviewed-by: Yosry Ahmed Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 209eae67ab18..23b3957b9ae0 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10430,8 +10430,6 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu= , int cpl, =20 ++vcpu->stat.hypercalls; =20 - trace_kvm_hypercall(nr, a0, a1, a2, a3); - if (!op_64_bit) { nr &=3D 0xFFFFFFFF; a0 &=3D 0xFFFFFFFF; @@ -10440,6 +10438,8 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu= , int cpl, a3 &=3D 0xFFFFFFFF; } =20 + trace_kvm_hypercall(nr, a0, a1, a2, a3); + if (cpl) { ret =3D -KVM_EPERM; goto out; --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4B013D1ABD for ; Thu, 14 May 2026 21:54:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795648; cv=none; b=JV7MelGPwo8dxIkdbw4VYI8LBje0syhGPhgI7aYFHknTlr+1UpCT2DMcriWWmW6uhupv/q5n5yiFHXAjKwV70P6G/pGcYYzuSjQNVV1EDiSVIuct07YKiix53cH6LA3znzAfGtwlEcMWBiRhJYlmglsdixrIoWk3Js0MZewoTuM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795648; c=relaxed/simple; bh=G1jqVtRGMHHU2pYLXqlDWhngOFL0xaMmd6fXfWamkcQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=CwaZghyTA58LMP92n9a2bPZe0qnOJZhR5o4wkHU68byxGk2yeIetrUsipMonY8XkhAyGSGNbNH5Gv1oM2tiK+HP6y38bnTflZRqxcC5AxszZktAA+io/nYmB4+mQ6qZWbu1BHREpyr+kwSSVQZNw4BqsndLN/r+Rzsnwy6REoHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=B86sVLIA; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="B86sVLIA" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82fa1c94b37so249506b3a.0 for ; Thu, 14 May 2026 14:54:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795644; x=1779400444; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uJv9wi2J1zMACcV+TLkHPu/SNZnF0/LBSZ3jiwDXSHs=; b=B86sVLIAji13Vhv8ayDrTAAyqeHmgRAxOi9nPMRs7EoF2uvuXMJk9LJ5AbYs7Ftf8K WgvgF++rSUHDl+uIenBIuB3qSxq0Rlk30gKZ8qGi0xGqad72nQX2mK/XSmDlJy90nweK It8t8jGrCL6eCS34HASDbiD8UrKZ4Mpprh27iXHPFXQtOaBcnY9sxLhvywSuBLG2qpjU TwSBACfcrATQ0fw+DAm7KipoSdjNGvx59lPcwYMWZ81hmfzAvLclR4I78A521BgN2uaQ 3HEthi9uCpsW80BeQBXJfunHnAP2JY+V+QFqSpVPAnJacYmoqPTV6bSyKpG3nMHu1RD7 4Dug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795644; x=1779400444; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uJv9wi2J1zMACcV+TLkHPu/SNZnF0/LBSZ3jiwDXSHs=; b=Oh5doQLzUvLyhs7wkLo27Ghlj3Q6y2byIt7YlVotj8ICyl/PV5C5DSUUV2Qx07LyE1 AMoQqGptZKLos9Aw1EDvxxRTOxQnmrpW9eSmlFg/2B/RW46NY7VNIETk8EhbwdVXozN4 JkO7XQhB441t4dS9AqozyiD2i73Oasfm5Ucg4kX9K9fFWwALdplqXF8Xi7RLzGm8LUhR sj5nH09eXVNkPT8t4Mk7+MU32cEruk4ZUBbevsYTEs9cUzkmTvTiHZkA0lAbhRe/LOM4 Px8cV5OaXyaxrg+xnZWQResqsn88FbPOyd+dLcLxbLo8GFVqbSHqks/+FCoh0oQrQyBn F4GQ== X-Forwarded-Encrypted: i=1; AFNElJ8lpAAZ/PIOMIuAX3I24VD4QyU+phC658VnyvfupBoebSCyKIFxeFM4xJeQ9wfb24s/S/3vZ7VklCwPtaw=@vger.kernel.org X-Gm-Message-State: AOJu0YwCvvWYq8SjKrFwNcy6wNER3GZPlWJxFAxflGE4cEfs6OHYvgg1 MC9GnYwXNG22/89pMBIKyvtS9gK8oddPNfsBEAW2X7oX++2pxtoJNY6/8Q6vZcBjRue0xhKODcW c+a1yHw== X-Received: from pfblg3.prod.google.com ([2002:a05:6a00:7083:b0:82e:7f41:df84]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:a227:b0:82c:9c90:6ea0 with SMTP id d2e1a72fcca58-83f33ad8782mr1023609b3a.3.1778795643702; Thu, 14 May 2026 14:54:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:46 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-7-seanjc@google.com> Subject: [PATCH v2 06/15] KVM: x86: Rename kvm_cache_regs.h => regs.h From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename kvm_cache_regs.h to simply regs.h, as the "cache" nomenclature is already a lie (the file deals with state/registers that aren't cached per se), and so that more code/functionality can be landed in the header without making it a truly horrible misnomer. Deliberately drop the kvm_ prefix/namespace to align with other "local" headers, and to further differentiate regs.h from the public/global arch/x86/include/asm/kvm_vcpu_regs.h, which sadly needs to stay in asm/ so that the number of registers can be referenced by kvm_vcpu_arch. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Yosry Ahmed --- arch/x86/kvm/emulate.c | 2 +- arch/x86/kvm/lapic.c | 2 +- arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/{kvm_cache_regs.h =3D> regs.h} | 4 ++-- arch/x86/kvm/smm.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/vmx/nested.h | 2 +- arch/x86/kvm/vmx/sgx.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/x86.h | 2 +- 14 files changed, 15 insertions(+), 15 deletions(-) rename arch/x86/kvm/{kvm_cache_regs.h =3D> regs.h} (99%) diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c index 8013dccb3110..6e64761f64b1 100644 --- a/arch/x86/kvm/emulate.c +++ b/arch/x86/kvm/emulate.c @@ -20,7 +20,7 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 #include -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include #include diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 4078e624ca66..d8dbfb107bfb 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -37,7 +37,7 @@ #include #include #include -#include "kvm_cache_regs.h" +#include "regs.h" #include "irq.h" #include "ioapic.h" #include "trace.h" diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index ddf4e467c071..e1bb663ebbd5 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -3,7 +3,7 @@ #define __KVM_X86_MMU_H =20 #include -#include "kvm_cache_regs.h" +#include "regs.h" #include "x86.h" #include "cpuid.h" =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c87c26bf4149..b8f2edf2cfeb 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -22,7 +22,7 @@ #include "mmu_internal.h" #include "tdp_mmu.h" #include "x86.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "smm.h" #include "kvm_emulate.h" #include "page_track.h" diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/regs.h similarity index 99% rename from arch/x86/kvm/kvm_cache_regs.h rename to arch/x86/kvm/regs.h index 2ae492ad6412..4440f3992fce 100644 --- a/arch/x86/kvm/kvm_cache_regs.h +++ b/arch/x86/kvm/regs.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifndef ASM_KVM_CACHE_REGS_H -#define ASM_KVM_CACHE_REGS_H +#ifndef ARCH_X86_KVM_REGS_H +#define ARCH_X86_KVM_REGS_H =20 #include =20 diff --git a/arch/x86/kvm/smm.c b/arch/x86/kvm/smm.c index f623c5986119..a446487bdd5c 100644 --- a/arch/x86/kvm/smm.c +++ b/arch/x86/kvm/smm.c @@ -3,7 +3,7 @@ =20 #include #include "x86.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include "smm.h" #include "cpuid.h" diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 4ad87f8df392..be775d285ce7 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4,7 +4,7 @@ =20 #include "irq.h" #include "mmu.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "x86.h" #include "smm.h" #include "cpuid.h" diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 2b6733dffd76..b8c7f4535691 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -23,7 +23,7 @@ #include =20 #include "cpuid.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "x86.h" =20 /* diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h index 213a448104af..6d6cd5904ddf 100644 --- a/arch/x86/kvm/vmx/nested.h +++ b/arch/x86/kvm/vmx/nested.h @@ -2,7 +2,7 @@ #ifndef __KVM_X86_VMX_NESTED_H #define __KVM_X86_VMX_NESTED_H =20 -#include "kvm_cache_regs.h" +#include "regs.h" #include "hyperv.h" #include "vmcs12.h" #include "vmx.h" diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index 4c61fc33f764..66c315554b46 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -6,7 +6,7 @@ #include =20 #include "x86.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "nested.h" #include "sgx.h" #include "vmx.h" diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index b02d176800f8..67bc6edfd856 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -59,7 +59,7 @@ #include "hyperv.h" #include "kvm_onhyperv.h" #include "irq.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "lapic.h" #include "mmu.h" #include "nested.h" diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index daedf663c0a9..de9de0d2016c 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -10,7 +10,7 @@ #include =20 #include "capabilities.h" -#include "../kvm_cache_regs.h" +#include "../regs.h" #include "pmu_intel.h" #include "vmcs.h" #include "vmx_ops.h" diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 23b3957b9ae0..ab13aed2cbd0 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -23,7 +23,7 @@ #include "mmu.h" #include "i8254.h" #include "tss.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include "mmu/page_track.h" #include "x86.h" diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 38a905fa86de..2bbecc83ecc2 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -6,7 +6,7 @@ #include #include #include -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include "cpuid.h" =20 --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A3113D1CB0 for ; Thu, 14 May 2026 21:54:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795650; cv=none; b=IiKQrPxS50z1UbIl5bsimwq9Anaraw7iU0z32QnhoJYRT3mQO/hL/nlzhIylTs0hKwXgX80MgnNxharjNMRc9v4M7eLDiqG7y36NICnAgjU98TpIpq7J83BHHRndnY++fULEIXpnq0fdxuuYlgvqC/8AzOUIiNJvjJDEBAq7qEQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795650; c=relaxed/simple; bh=u7t/6O99gKqVMGcNhRnugOCAcBoXyxs2DtCzobtXGNc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bBFiilvIHJqHhE6vAPMjZoPd/otiU+ooLDWqYd+EkqIZ79U6NWZBK0UhC6lkWUkY6NaACT0egK3KVlFNVygfQcD4oGguyv1i4Hv+YInN3Yvi+bdsc1Vje5gBVkyxmfM992DSlheDC1Tr1t0euyyJ3JheTtPSVita90i270wPQ74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XviSiJfq; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XviSiJfq" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c82bd90afbbso673229a12.0 for ; Thu, 14 May 2026 14:54:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795645; x=1779400445; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=kj3u+QM9+85lpIpuH+gR93oa8wLXvI0R29z+6Y1bnto=; b=XviSiJfqMjtclFqtSrajLAhPXDCY6sugwktQ/ZVAOzva3I8H3a+pBesyPjbHeWShj2 OVyaAJXpAQbML/Tm4EGHw/5VbBuhi+nFePzBHg88mF59vMz2ZBNz694cW4E5T87r4eGp 1pnB25KLXoHd3tGSGdmb/gaM0Qbg6VpNN1dcA32aJXYH408ARnKf4mxOSMbE9nFoKS78 CqHX4tV6lqUvFTfN5fHSuIn0wc4eiEc2JYuiFSaDanp6+yPM82Bw1HcaYcrW60te5OMJ wtqEhb+WpWBgXCiwuqmdyz5iDcaXyClEOVk02Q39IAA9BFHnS2rAhXMrVBrpYIXV4BkX pNLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795645; x=1779400445; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kj3u+QM9+85lpIpuH+gR93oa8wLXvI0R29z+6Y1bnto=; b=ZLupISvJGbA/IpMjHnCoCIIO8ZUExm+ZmCM+YbATTjr2MHwA8p0aEjcVZvif+sOtHx fQdP1p8CYu5QgKP+ZBns75oJrCP5oeMG2EnUcWrJ7URQTrAJuIh4hDgafO//VQiBdn5O 2PB/OaO5IwlxjNadP+ny6zo8o543LxNky92KI4I4OMHnK+ZSKcqKOveMF+M/4b2E+zyT m6ow4ZPd4x4mZFLSC1dT/mzTgkGrTcReU3pPpwVZ/GC6nLFNa31G5j2oP7l5LMCyutu3 dkoci2ruNVTZ+i12JZitz6u4bI6MABno/WmwXkGY10PAzTwo6vheOjpn+qr+D65ehREa MWaw== X-Forwarded-Encrypted: i=1; AFNElJ8ICuLPzuCEHXTwvF7UwF1sVTUkbE6Q6a1qLLSR6Q7G0bE19NsOn9usIdWmdajcyMXTREohIWNzermtnVU=@vger.kernel.org X-Gm-Message-State: AOJu0Yw5xEkNFA0P8lJuLLjaCG61j1MmfA6ajvx3aqtnhcZosyWZrB/4 Xnj2vvVkiEqlRySc81ewOEPR4PcGaAN7miAoj7ktVoAMkAneUgE5jEdBiFOiTEft/isgENGyd7i w6ZwH7Q== X-Received: from pgbbc18.prod.google.com ([2002:a65:6d92:0:b0:c73:7b68:90d9]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:394b:b0:3ab:1692:fe91 with SMTP id adf61e73a8af0-3b22ec63d85mr1000346637.41.1778795644789; Thu, 14 May 2026 14:54:04 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:47 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-8-seanjc@google.com> Subject: [PATCH v2 07/15] KVM: x86: Move inlined CR and DR helpers from x86.h to regs.h From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move inlined Control Register and Debug Register helpers from x86.h to the aptly named regs.h, to help trim down x86.h (and x86.c in the future). Move select EFER functionality, but leave behind all other MSR handling, There is more than enough MSR code to carve out msr.{c,h} in the future. Give EFER special treatment as it's an "MSR" in name only, e.g. it's has far more in common with CR4 than it does with any MSR. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Yosry Ahmed --- arch/x86/kvm/regs.h | 108 ++++++++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/x86.h | 102 ----------------------------------------- 2 files changed, 105 insertions(+), 105 deletions(-) diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h index 4440f3992fce..ecc66b577e82 100644 --- a/arch/x86/kvm/regs.h +++ b/arch/x86/kvm/regs.h @@ -16,6 +16,37 @@ =20 static_assert(!(KVM_POSSIBLE_CR0_GUEST_BITS & X86_CR0_PDPTR_BITS)); =20 +static inline bool is_long_mode(struct kvm_vcpu *vcpu) +{ +#ifdef CONFIG_X86_64 + return !!(vcpu->arch.efer & EFER_LMA); +#else + return false; +#endif +} + +static inline bool is_64_bit_mode(struct kvm_vcpu *vcpu) +{ + int cs_db, cs_l; + + WARN_ON_ONCE(vcpu->arch.guest_state_protected); + + if (!is_long_mode(vcpu)) + return false; + kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); + return cs_l; +} + +static inline bool is_64_bit_hypercall(struct kvm_vcpu *vcpu) +{ + /* + * If running with protected guest state, the CS register is not + * accessible. The hypercall register values will have had to been + * provided in 64-bit mode, so assume the guest is in 64-bit. + */ + return vcpu->arch.guest_state_protected || is_64_bit_mode(vcpu); +} + #define BUILD_KVM_GPR_ACCESSORS(lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *v= cpu)\ { \ @@ -177,6 +208,12 @@ static inline void kvm_rsp_write(struct kvm_vcpu *vcpu= , unsigned long val) kvm_register_write_raw(vcpu, VCPU_REGS_RSP, val); } =20 +static inline u64 kvm_read_edx_eax(struct kvm_vcpu *vcpu) +{ + return (kvm_rax_read(vcpu) & -1u) + | ((u64)(kvm_rdx_read(vcpu) & -1u) << 32); +} + static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index) { might_sleep(); /* on svm */ @@ -243,10 +280,75 @@ static inline ulong kvm_read_cr4(struct kvm_vcpu *vcp= u) return kvm_read_cr4_bits(vcpu, ~0UL); } =20 -static inline u64 kvm_read_edx_eax(struct kvm_vcpu *vcpu) +static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long= cr4) { - return (kvm_rax_read(vcpu) & -1u) - | ((u64)(kvm_rdx_read(vcpu) & -1u) << 32); + return !(cr4 & vcpu->arch.cr4_guest_rsvd_bits); +} + +#define __cr4_reserved_bits(__cpu_has, __c) \ +({ \ + u64 __reserved_bits =3D CR4_RESERVED_BITS; \ + \ + if (!__cpu_has(__c, X86_FEATURE_XSAVE)) \ + __reserved_bits |=3D X86_CR4_OSXSAVE; \ + if (!__cpu_has(__c, X86_FEATURE_SMEP)) \ + __reserved_bits |=3D X86_CR4_SMEP; \ + if (!__cpu_has(__c, X86_FEATURE_SMAP)) \ + __reserved_bits |=3D X86_CR4_SMAP; \ + if (!__cpu_has(__c, X86_FEATURE_FSGSBASE)) \ + __reserved_bits |=3D X86_CR4_FSGSBASE; \ + if (!__cpu_has(__c, X86_FEATURE_PKU)) \ + __reserved_bits |=3D X86_CR4_PKE; \ + if (!__cpu_has(__c, X86_FEATURE_LA57)) \ + __reserved_bits |=3D X86_CR4_LA57; \ + if (!__cpu_has(__c, X86_FEATURE_UMIP)) \ + __reserved_bits |=3D X86_CR4_UMIP; \ + if (!__cpu_has(__c, X86_FEATURE_VMX)) \ + __reserved_bits |=3D X86_CR4_VMXE; \ + if (!__cpu_has(__c, X86_FEATURE_PCID)) \ + __reserved_bits |=3D X86_CR4_PCIDE; \ + if (!__cpu_has(__c, X86_FEATURE_LAM)) \ + __reserved_bits |=3D X86_CR4_LAM_SUP; \ + if (!__cpu_has(__c, X86_FEATURE_SHSTK) && \ + !__cpu_has(__c, X86_FEATURE_IBT)) \ + __reserved_bits |=3D X86_CR4_CET; \ + __reserved_bits; \ +}) + +static inline bool is_protmode(struct kvm_vcpu *vcpu) +{ + return kvm_is_cr0_bit_set(vcpu, X86_CR0_PE); +} + +static inline bool is_pae(struct kvm_vcpu *vcpu) +{ + return kvm_is_cr4_bit_set(vcpu, X86_CR4_PAE); +} + +static inline bool is_pse(struct kvm_vcpu *vcpu) +{ + return kvm_is_cr4_bit_set(vcpu, X86_CR4_PSE); +} + +static inline bool is_paging(struct kvm_vcpu *vcpu) +{ + return likely(kvm_is_cr0_bit_set(vcpu, X86_CR0_PG)); +} + +static inline bool is_pae_paging(struct kvm_vcpu *vcpu) +{ + return !is_long_mode(vcpu) && is_pae(vcpu) && is_paging(vcpu); +} + +static inline bool kvm_dr7_valid(u64 data) +{ + /* Bits [63:32] are reserved */ + return !(data >> 32); +} +static inline bool kvm_dr6_valid(u64 data) +{ + /* Bits [63:32] are reserved */ + return !(data >> 32); } =20 static inline void enter_guest_mode(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 2bbecc83ecc2..16d1c3c1a2d9 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -243,42 +243,6 @@ static inline bool kvm_exception_is_soft(unsigned int = nr) return (nr =3D=3D BP_VECTOR) || (nr =3D=3D OF_VECTOR); } =20 -static inline bool is_protmode(struct kvm_vcpu *vcpu) -{ - return kvm_is_cr0_bit_set(vcpu, X86_CR0_PE); -} - -static inline bool is_long_mode(struct kvm_vcpu *vcpu) -{ -#ifdef CONFIG_X86_64 - return !!(vcpu->arch.efer & EFER_LMA); -#else - return false; -#endif -} - -static inline bool is_64_bit_mode(struct kvm_vcpu *vcpu) -{ - int cs_db, cs_l; - - WARN_ON_ONCE(vcpu->arch.guest_state_protected); - - if (!is_long_mode(vcpu)) - return false; - kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); - return cs_l; -} - -static inline bool is_64_bit_hypercall(struct kvm_vcpu *vcpu) -{ - /* - * If running with protected guest state, the CS register is not - * accessible. The hypercall register values will have had to been - * provided in 64-bit mode, so assume the guest is in 64-bit. - */ - return vcpu->arch.guest_state_protected || is_64_bit_mode(vcpu); -} - static inline bool x86_exception_has_error_code(unsigned int vector) { static u32 exception_has_error_code =3D BIT(DF_VECTOR) | BIT(TS_VECTOR) | @@ -293,26 +257,6 @@ static inline bool mmu_is_nested(struct kvm_vcpu *vcpu) return vcpu->arch.walk_mmu =3D=3D &vcpu->arch.nested_mmu; } =20 -static inline bool is_pae(struct kvm_vcpu *vcpu) -{ - return kvm_is_cr4_bit_set(vcpu, X86_CR4_PAE); -} - -static inline bool is_pse(struct kvm_vcpu *vcpu) -{ - return kvm_is_cr4_bit_set(vcpu, X86_CR4_PSE); -} - -static inline bool is_paging(struct kvm_vcpu *vcpu) -{ - return likely(kvm_is_cr0_bit_set(vcpu, X86_CR0_PG)); -} - -static inline bool is_pae_paging(struct kvm_vcpu *vcpu) -{ - return !is_long_mode(vcpu) && is_pae(vcpu) && is_paging(vcpu); -} - static inline u8 vcpu_virt_addr_bits(struct kvm_vcpu *vcpu) { return kvm_is_cr4_bit_set(vcpu, X86_CR4_LA57) ? 57 : 48; @@ -630,17 +574,6 @@ static inline bool kvm_pat_valid(u64 data) return (data | ((data & 0x0202020202020202ull) << 1)) =3D=3D data; } =20 -static inline bool kvm_dr7_valid(u64 data) -{ - /* Bits [63:32] are reserved */ - return !(data >> 32); -} -static inline bool kvm_dr6_valid(u64 data) -{ - /* Bits [63:32] are reserved */ - return !(data >> 32); -} - /* * Trigger machine check on the host. We assume all the MSRs are already s= et up * by the CPU and that we still run on the same CPU as the MCE occurred on. @@ -687,41 +620,6 @@ enum kvm_msr_access { #define KVM_MSR_RET_UNSUPPORTED 2 #define KVM_MSR_RET_FILTERED 3 =20 -static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long= cr4) -{ - return !(cr4 & vcpu->arch.cr4_guest_rsvd_bits); -} - -#define __cr4_reserved_bits(__cpu_has, __c) \ -({ \ - u64 __reserved_bits =3D CR4_RESERVED_BITS; \ - \ - if (!__cpu_has(__c, X86_FEATURE_XSAVE)) \ - __reserved_bits |=3D X86_CR4_OSXSAVE; \ - if (!__cpu_has(__c, X86_FEATURE_SMEP)) \ - __reserved_bits |=3D X86_CR4_SMEP; \ - if (!__cpu_has(__c, X86_FEATURE_SMAP)) \ - __reserved_bits |=3D X86_CR4_SMAP; \ - if (!__cpu_has(__c, X86_FEATURE_FSGSBASE)) \ - __reserved_bits |=3D X86_CR4_FSGSBASE; \ - if (!__cpu_has(__c, X86_FEATURE_PKU)) \ - __reserved_bits |=3D X86_CR4_PKE; \ - if (!__cpu_has(__c, X86_FEATURE_LA57)) \ - __reserved_bits |=3D X86_CR4_LA57; \ - if (!__cpu_has(__c, X86_FEATURE_UMIP)) \ - __reserved_bits |=3D X86_CR4_UMIP; \ - if (!__cpu_has(__c, X86_FEATURE_VMX)) \ - __reserved_bits |=3D X86_CR4_VMXE; \ - if (!__cpu_has(__c, X86_FEATURE_PCID)) \ - __reserved_bits |=3D X86_CR4_PCIDE; \ - if (!__cpu_has(__c, X86_FEATURE_LAM)) \ - __reserved_bits |=3D X86_CR4_LAM_SUP; \ - if (!__cpu_has(__c, X86_FEATURE_SHSTK) && \ - !__cpu_has(__c, X86_FEATURE_IBT)) \ - __reserved_bits |=3D X86_CR4_CET; \ - __reserved_bits; \ -}) - int kvm_sev_es_mmio(struct kvm_vcpu *vcpu, bool is_write, gpa_t gpa, unsigned int bytes, void *data); int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4C2E3D25BB for ; Thu, 14 May 2026 21:54:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795649; cv=none; b=P1Hbyx4bKgDWZIh5/ZzbjE7NmPrtOkkQpP7H8yr94IqZt8LCwy3PwM4TSE4XF4eBIfXvXwx0T2G6GjXk6PT3zrDklbC8EOi1Kb+0kSSRsPcguCTUdkbu0prxN5VnMrTeogb/tBF9wKLCihbRybEH8Xh7EtUxdDSqK67EyEDsZ4E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795649; c=relaxed/simple; bh=/7WGcr+w6dGw7gtqTCNGgHkdnFAOxbkW6ggtoAdN6/g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XzM+tz6jm3eFEtW9j7OVrn7qm2XcrKINOn3fFuuT+RwDF5mG4qxI0kWNAU74NLLtMEvqMR8SFP3QZItxEDpT4i5f8XI8WlQ7cs/aWxSFWldYfGKOk7IGAMUfZI+FuWT+NchWIqwc4oBTKQzOvSOq0qzuOeYpAvkRHXFqZ6mqk1M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cXz6/xw5; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cXz6/xw5" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-8397b14a689so5256711b3a.2 for ; Thu, 14 May 2026 14:54:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795646; x=1779400446; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=tsZR7o5U5njspjjsGG2ZXE4g2wdS0yroCDi4c1UBTZ0=; b=cXz6/xw5JPBKATRrbN3lSPvx9zsdS//YdGckby1S7+P55UtaJv56UBes4pcV2xDdBG V5YIp8F2HTLdjuOJ64ktiJbs3Rrpi3OmMfzoDUNARwXHYEvocJi3GJ6lod7S1RqVNzns JpLKESYbYju1Rw7g9w1C+Jl4jzDhjjXBk/RLf6Q4Sm7d0CuhUOwVxCPd/CToEPjPcnqd oIa0iUCXk66l9BgiGB3VW7fCWk4ZjjGeJn4F8hRoyQ0/+W8DjSUla7evWisLvy2MyXi3 ymeICmbGioeTmRnGkxlvdGX4LQi+CMQpGkCEDqb3wfEjguwvxIDdx0Xx/Zw20if1JwDt 6tuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795646; x=1779400446; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tsZR7o5U5njspjjsGG2ZXE4g2wdS0yroCDi4c1UBTZ0=; b=Xg7GSCmSKX3z7owoTkwdSit1DYECNXGDcC2yfHHI4DonQVYea/g3XhArJCaiypQz1T 3soyekLX5WG0QktgzsdWB6kHgLH134eXhlFX8ekKlCVLIrgpvSantSUEJnMpQvQrYU2/ RcijEkS/fsd6aKZInHkBYlrk4Nn0WXm6cpDvFZtRdurn7XQxmvpLZz9CfY33AKTDnHXZ +UuFFt/iFbtGtYTTQgSLbIV0zqhovXRZ3DXT6cCpsovtSATKxNsMNZQaHsJMtdD5TUer DYf0ay894dMG99I6vZUzUKKzSVfjQFNrYoruwtWZhcRvtk56auTLM8vJgpDtopeD9uTk ydsw== X-Forwarded-Encrypted: i=1; AFNElJ9nEYyJC5v/V1QCLTGLItGMCaWQ+wikuSNZOszhktwkeUfqmWIvtOpIgAdiDLT3yvsvTFvv4LKaD+XeK5c=@vger.kernel.org X-Gm-Message-State: AOJu0Yz+ujcC+97D+ivPItH4+7pBZou/B8Q+EBQPPFBmoXfLzguo+Hfn MMW8y+3LFKwwHKev3brHKF0roKHQ4zrBMyr9g/EgZQZB/crsYghYWaB/rzc8cJVUAqHSZW7JyH3 xuAV8vw== X-Received: from pfblj2.prod.google.com ([2002:a05:6a00:71c2:b0:83e:c835:edc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4484:b0:829:8cfb:df45 with SMTP id d2e1a72fcca58-83f33c99b07mr1271607b3a.15.1778795645919; Thu, 14 May 2026 14:54:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:48 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-9-seanjc@google.com> Subject: [PATCH v2 08/15] KVM: x86: Add mode-aware versions of kvm__{read,write}() helpers From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make kvm__{read,write}() mode-aware (where the value is truncated to 32 bits if the vCPU isn't in 64-bit mode), and convert all the intentional "raw" accesses to kvm__{read,write}_raw() versions. To avoid confusion and bikeshedding over whether or not explicit 32-bit accesses should use the "raw" or mode-aware variants, add and use "e" versions, e.g. for things like RDMSR, WRMSR, and CPUID, where the instruction uses only only bits 31:0, regardless of mode. No functional change intended (all use of "e" versions is for cases where the value is already truncated due to bouncing through a u32). Cc: Binbin Wu Cc: Kai Huang Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/cpuid.c | 12 ++-- arch/x86/kvm/hyperv.c | 21 +++---- arch/x86/kvm/hyperv.h | 4 +- arch/x86/kvm/regs.h | 80 +++++++++++++++++-------- arch/x86/kvm/svm/nested.c | 6 +- arch/x86/kvm/svm/svm.c | 13 ++-- arch/x86/kvm/vmx/nested.c | 8 +-- arch/x86/kvm/vmx/sgx.c | 4 +- arch/x86/kvm/vmx/tdx.c | 18 +++--- arch/x86/kvm/x86.c | 121 +++++++++++++++++++------------------- arch/x86/kvm/x86.h | 8 +-- arch/x86/kvm/xen.c | 32 +++++----- 12 files changed, 173 insertions(+), 154 deletions(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index e69156b54cff..fe765f1c3b15 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -2165,13 +2165,13 @@ int kvm_emulate_cpuid(struct kvm_vcpu *vcpu) !kvm_require_cpl(vcpu, 0)) return 1; =20 - eax =3D kvm_rax_read(vcpu); - ecx =3D kvm_rcx_read(vcpu); + eax =3D kvm_eax_read(vcpu); + ecx =3D kvm_ecx_read(vcpu); kvm_cpuid(vcpu, &eax, &ebx, &ecx, &edx, false); - kvm_rax_write(vcpu, eax); - kvm_rbx_write(vcpu, ebx); - kvm_rcx_write(vcpu, ecx); - kvm_rdx_write(vcpu, edx); + kvm_eax_write(vcpu, eax); + kvm_ebx_write(vcpu, ebx); + kvm_ecx_write(vcpu, ecx); + kvm_edx_write(vcpu, edx); return kvm_skip_emulated_instruction(vcpu); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_emulate_cpuid); diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 015c6947b462..3551af9a9453 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2377,10 +2377,10 @@ static void kvm_hv_hypercall_set_result(struct kvm_= vcpu *vcpu, u64 result) =20 longmode =3D is_64_bit_hypercall(vcpu); if (longmode) - kvm_rax_write(vcpu, result); + kvm_rax_write_raw(vcpu, result); else { - kvm_rdx_write(vcpu, result >> 32); - kvm_rax_write(vcpu, result & 0xffffffff); + kvm_edx_write(vcpu, result >> 32); + kvm_eax_write(vcpu, result); } } =20 @@ -2544,18 +2544,15 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) =20 #ifdef CONFIG_X86_64 if (is_64_bit_hypercall(vcpu)) { - hc.param =3D kvm_rcx_read(vcpu); - hc.ingpa =3D kvm_rdx_read(vcpu); - hc.outgpa =3D kvm_r8_read(vcpu); + hc.param =3D kvm_rcx_read_raw(vcpu); + hc.ingpa =3D kvm_rdx_read_raw(vcpu); + hc.outgpa =3D kvm_r8_read_raw(vcpu); } else #endif { - hc.param =3D ((u64)kvm_rdx_read(vcpu) << 32) | - (kvm_rax_read(vcpu) & 0xffffffff); - hc.ingpa =3D ((u64)kvm_rbx_read(vcpu) << 32) | - (kvm_rcx_read(vcpu) & 0xffffffff); - hc.outgpa =3D ((u64)kvm_rdi_read(vcpu) << 32) | - (kvm_rsi_read(vcpu) & 0xffffffff); + hc.param =3D ((u64)kvm_edx_read(vcpu) << 32) | kvm_eax_read(vcpu); + hc.ingpa =3D ((u64)kvm_ebx_read(vcpu) << 32) | kvm_ecx_read(vcpu); + hc.outgpa =3D ((u64)kvm_edi_read(vcpu) << 32) | kvm_esi_read(vcpu); } =20 hc.code =3D hc.param & 0xffff; diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 6301f79fcbae..65e89ed65349 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -232,8 +232,8 @@ static inline bool kvm_hv_is_tlb_flush_hcall(struct kvm= _vcpu *vcpu) if (!hv_vcpu) return false; =20 - code =3D is_64_bit_hypercall(vcpu) ? kvm_rcx_read(vcpu) : - kvm_rax_read(vcpu); + code =3D is_64_bit_hypercall(vcpu) ? kvm_rcx_read_raw(vcpu) : + kvm_eax_read(vcpu); =20 return (code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE || code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h index ecc66b577e82..b28e71caed25 100644 --- a/arch/x86/kvm/regs.h +++ b/arch/x86/kvm/regs.h @@ -47,32 +47,61 @@ static inline bool is_64_bit_hypercall(struct kvm_vcpu = *vcpu) return vcpu->arch.guest_state_protected || is_64_bit_mode(vcpu); } =20 -#define BUILD_KVM_GPR_ACCESSORS(lname, uname) \ -static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *v= cpu)\ -{ \ - return vcpu->arch.regs[VCPU_REGS_##uname]; \ -} \ -static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, = \ - unsigned long val) \ -{ \ - vcpu->arch.regs[VCPU_REGS_##uname] =3D val; \ +static __always_inline unsigned long kvm_reg_mode_mask(struct kvm_vcpu *vc= pu) +{ +#ifdef CONFIG_X86_64 + return is_64_bit_mode(vcpu) ? GENMASK(63, 0) : GENMASK(31, 0); +#else + return GENMASK(31, 0); +#endif +} + +#define __BUILD_KVM_GPR_ACCESSORS(lname, uname) \ +static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *v= cpu) \ +{ \ + return vcpu->arch.regs[VCPU_REGS_##uname] & kvm_reg_mode_mask(vcpu); \ +} \ +static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ + unsigned long val) \ +{ \ + vcpu->arch.regs[VCPU_REGS_##uname] =3D val & kvm_reg_mode_mask(vcpu); \ +} \ +static __always_inline unsigned long kvm_##lname##_read_raw(struct kvm_vcp= u *vcpu) \ +{ \ + return vcpu->arch.regs[VCPU_REGS_##uname]; \ +} \ +static __always_inline void kvm_##lname##_write_raw(struct kvm_vcpu *vcpu,= \ + unsigned long val) \ +{ \ + vcpu->arch.regs[VCPU_REGS_##uname] =3D val; \ } -BUILD_KVM_GPR_ACCESSORS(rax, RAX) -BUILD_KVM_GPR_ACCESSORS(rbx, RBX) -BUILD_KVM_GPR_ACCESSORS(rcx, RCX) -BUILD_KVM_GPR_ACCESSORS(rdx, RDX) -BUILD_KVM_GPR_ACCESSORS(rbp, RBP) -BUILD_KVM_GPR_ACCESSORS(rsi, RSI) -BUILD_KVM_GPR_ACCESSORS(rdi, RDI) +#define BUILD_KVM_GPR_ACCESSORS(lname, uname) \ +static __always_inline u32 kvm_e##lname##_read(struct kvm_vcpu *vcpu) \ +{ \ + return vcpu->arch.regs[VCPU_REGS_##uname]; \ +} \ +static __always_inline void kvm_e##lname##_write(struct kvm_vcpu *vcpu, u3= 2 val) \ +{ \ + vcpu->arch.regs[VCPU_REGS_##uname] =3D val; \ +} \ +__BUILD_KVM_GPR_ACCESSORS(r##lname, uname) + +BUILD_KVM_GPR_ACCESSORS(ax, RAX) +BUILD_KVM_GPR_ACCESSORS(bx, RBX) +BUILD_KVM_GPR_ACCESSORS(cx, RCX) +BUILD_KVM_GPR_ACCESSORS(dx, RDX) +BUILD_KVM_GPR_ACCESSORS(bp, RBP) +BUILD_KVM_GPR_ACCESSORS(si, RSI) +BUILD_KVM_GPR_ACCESSORS(di, RDI) #ifdef CONFIG_X86_64 -BUILD_KVM_GPR_ACCESSORS(r8, R8) -BUILD_KVM_GPR_ACCESSORS(r9, R9) -BUILD_KVM_GPR_ACCESSORS(r10, R10) -BUILD_KVM_GPR_ACCESSORS(r11, R11) -BUILD_KVM_GPR_ACCESSORS(r12, R12) -BUILD_KVM_GPR_ACCESSORS(r13, R13) -BUILD_KVM_GPR_ACCESSORS(r14, R14) -BUILD_KVM_GPR_ACCESSORS(r15, R15) +__BUILD_KVM_GPR_ACCESSORS(r8, R8) +__BUILD_KVM_GPR_ACCESSORS(r9, R9) +__BUILD_KVM_GPR_ACCESSORS(r10, R10) +__BUILD_KVM_GPR_ACCESSORS(r11, R11) +__BUILD_KVM_GPR_ACCESSORS(r12, R12) +__BUILD_KVM_GPR_ACCESSORS(r13, R13) +__BUILD_KVM_GPR_ACCESSORS(r14, R14) +__BUILD_KVM_GPR_ACCESSORS(r15, R15) #endif =20 /* @@ -210,8 +239,7 @@ static inline void kvm_rsp_write(struct kvm_vcpu *vcpu,= unsigned long val) =20 static inline u64 kvm_read_edx_eax(struct kvm_vcpu *vcpu) { - return (kvm_rax_read(vcpu) & -1u) - | ((u64)(kvm_rdx_read(vcpu) & -1u) << 32); + return kvm_eax_read(vcpu) | (u64)(kvm_edx_read(vcpu)) << 32; } =20 static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 4ef9bc6a553f..7b2d804ef2b0 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -778,7 +778,7 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm = *svm) =20 svm->vcpu.arch.cr2 =3D save->cr2; =20 - kvm_rax_write(vcpu, save->rax); + kvm_rax_write_raw(vcpu, save->rax); kvm_rsp_write(vcpu, save->rsp); kvm_rip_write(vcpu, save->rip); =20 @@ -1244,7 +1244,7 @@ static int nested_svm_vmexit_update_vmcb12(struct kvm= _vcpu *vcpu) vmcb12->save.rflags =3D kvm_get_rflags(vcpu); vmcb12->save.rip =3D kvm_rip_read(vcpu); vmcb12->save.rsp =3D kvm_rsp_read(vcpu); - vmcb12->save.rax =3D kvm_rax_read(vcpu); + vmcb12->save.rax =3D kvm_rax_read_raw(vcpu); vmcb12->save.dr7 =3D vmcb02->save.dr7; vmcb12->save.dr6 =3D svm->vcpu.arch.dr6; vmcb12->save.cpl =3D vmcb02->save.cpl; @@ -1394,7 +1394,7 @@ void nested_svm_vmexit(struct vcpu_svm *svm) svm_set_efer(vcpu, vmcb01->save.efer); svm_set_cr0(vcpu, vmcb01->save.cr0 | X86_CR0_PE); svm_set_cr4(vcpu, vmcb01->save.cr4); - kvm_rax_write(vcpu, vmcb01->save.rax); + kvm_rax_write_raw(vcpu, vmcb01->save.rax); kvm_rsp_write(vcpu, vmcb01->save.rsp); kvm_rip_write(vcpu, vmcb01->save.rip); =20 diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index be775d285ce7..02fb9560c26e 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2408,15 +2408,12 @@ static int clgi_interception(struct kvm_vcpu *vcpu) =20 static int invlpga_interception(struct kvm_vcpu *vcpu) { - gva_t gva =3D kvm_rax_read(vcpu); - u32 asid =3D kvm_rcx_read(vcpu); - - if (nested_svm_check_permissions(vcpu)) - return 1; - /* FIXME: Handle an address size prefix. */ - if (!is_64_bit_mode(vcpu)) - gva =3D (u32)gva; + gva_t gva =3D kvm_rax_read(vcpu); + u32 asid =3D kvm_ecx_read(vcpu); + + if (nested_svm_check_permissions(vcpu)) + return 1; =20 trace_kvm_invlpga(to_svm(vcpu)->vmcb->save.rip, asid, gva); =20 diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 4690a4d23709..20d75bf0a455 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -6148,7 +6148,7 @@ static int handle_invvpid(struct kvm_vcpu *vcpu) static int nested_vmx_eptp_switching(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12) { - u32 index =3D kvm_rcx_read(vcpu); + u32 index =3D kvm_ecx_read(vcpu); u64 new_eptp; =20 if (WARN_ON_ONCE(!nested_cpu_has_ept(vmcs12))) @@ -6182,7 +6182,7 @@ static int handle_vmfunc(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx =3D to_vmx(vcpu); struct vmcs12 *vmcs12; - u32 function =3D kvm_rax_read(vcpu); + u32 function =3D kvm_eax_read(vcpu); =20 /* * VMFUNC should never execute cleanly while L1 is active; KVM supports @@ -6304,7 +6304,7 @@ static bool nested_vmx_exit_handled_msr(struct kvm_vc= pu *vcpu, exit_reason.basic =3D=3D EXIT_REASON_MSR_WRITE_IMM) msr_index =3D vmx_get_exit_qual(vcpu); else - msr_index =3D kvm_rcx_read(vcpu); + msr_index =3D kvm_ecx_read(vcpu); =20 /* * The MSR_BITMAP page is divided into four 1024-byte bitmaps, @@ -6414,7 +6414,7 @@ static bool nested_vmx_exit_handled_encls(struct kvm_= vcpu *vcpu, !nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING)) return false; =20 - encls_leaf =3D kvm_rax_read(vcpu); + encls_leaf =3D kvm_eax_read(vcpu); if (encls_leaf > 62) encls_leaf =3D 63; return vmcs12->encls_exiting_bitmap & BIT_ULL(encls_leaf); diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index 66c315554b46..2f5a1c58f3c5 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -352,7 +352,7 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu) rflags &=3D ~X86_EFLAGS_ZF; vmx_set_rflags(vcpu, rflags); =20 - kvm_rax_write(vcpu, ret); + kvm_eax_write(vcpu, ret); return kvm_skip_emulated_instruction(vcpu); } =20 @@ -380,7 +380,7 @@ static inline bool sgx_enabled_in_guest_bios(struct kvm= _vcpu *vcpu) =20 int handle_encls(struct kvm_vcpu *vcpu) { - u32 leaf =3D (u32)kvm_rax_read(vcpu); + u32 leaf =3D kvm_eax_read(vcpu); =20 if (!enable_sgx || !guest_cpu_cap_has(vcpu, X86_FEATURE_SGX) || !guest_cpu_cap_has(vcpu, X86_FEATURE_SGX1)) { diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index f97bcf580e6d..ec88b58e2b27 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1163,11 +1163,11 @@ static int complete_hypercall_exit(struct kvm_vcpu = *vcpu) =20 static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu) { - kvm_rax_write(vcpu, to_tdx(vcpu)->vp_enter_args.r10); - kvm_rbx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r11); - kvm_rcx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r12); - kvm_rdx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r13); - kvm_rsi_write(vcpu, to_tdx(vcpu)->vp_enter_args.r14); + kvm_rax_write_raw(vcpu, to_tdx(vcpu)->vp_enter_args.r10); + kvm_rbx_write_raw(vcpu, to_tdx(vcpu)->vp_enter_args.r11); + kvm_rcx_write_raw(vcpu, to_tdx(vcpu)->vp_enter_args.r12); + kvm_rdx_write_raw(vcpu, to_tdx(vcpu)->vp_enter_args.r13); + kvm_rsi_write_raw(vcpu, to_tdx(vcpu)->vp_enter_args.r14); =20 return __kvm_emulate_hypercall(vcpu, 0, complete_hypercall_exit); } @@ -2028,12 +2028,12 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath= _t fastpath) case EXIT_REASON_IO_INSTRUCTION: return tdx_emulate_io(vcpu); case EXIT_REASON_MSR_READ: - kvm_rcx_write(vcpu, tdx->vp_enter_args.r12); + kvm_ecx_write(vcpu, tdx->vp_enter_args.r12); return kvm_emulate_rdmsr(vcpu); case EXIT_REASON_MSR_WRITE: - kvm_rcx_write(vcpu, tdx->vp_enter_args.r12); - kvm_rax_write(vcpu, tdx->vp_enter_args.r13 & -1u); - kvm_rdx_write(vcpu, tdx->vp_enter_args.r13 >> 32); + kvm_ecx_write(vcpu, tdx->vp_enter_args.r12); + kvm_eax_write(vcpu, tdx->vp_enter_args.r13); + kvm_edx_write(vcpu, tdx->vp_enter_args.r13 >> 32); return kvm_emulate_wrmsr(vcpu); case EXIT_REASON_EPT_MISCONFIG: return tdx_emulate_mmio(vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ab13aed2cbd0..b958521bc81f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1319,7 +1319,7 @@ int kvm_emulate_xsetbv(struct kvm_vcpu *vcpu) { /* Note, #UD due to CR4.OSXSAVE=3D0 has priority over the intercept. */ if (kvm_x86_call(get_cpl)(vcpu) !=3D 0 || - __kvm_set_xcr(vcpu, kvm_rcx_read(vcpu), kvm_read_edx_eax(vcpu))) { + __kvm_set_xcr(vcpu, kvm_ecx_read(vcpu), kvm_read_edx_eax(vcpu))) { kvm_inject_gp(vcpu, 0); return 1; } @@ -1608,7 +1608,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_get_dr); =20 int kvm_emulate_rdpmc(struct kvm_vcpu *vcpu) { - u32 pmc =3D kvm_rcx_read(vcpu); + u32 pmc =3D kvm_ecx_read(vcpu); u64 data; =20 if (kvm_pmu_rdpmc(vcpu, pmc, &data)) { @@ -1616,8 +1616,8 @@ int kvm_emulate_rdpmc(struct kvm_vcpu *vcpu) return 1; } =20 - kvm_rax_write(vcpu, (u32)data); - kvm_rdx_write(vcpu, data >> 32); + kvm_eax_write(vcpu, data); + kvm_edx_write(vcpu, data >> 32); return kvm_skip_emulated_instruction(vcpu); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_emulate_rdpmc); @@ -2064,8 +2064,8 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_emulate_msr_write); static void complete_userspace_rdmsr(struct kvm_vcpu *vcpu) { if (!vcpu->run->msr.error) { - kvm_rax_write(vcpu, (u32)vcpu->run->msr.data); - kvm_rdx_write(vcpu, vcpu->run->msr.data >> 32); + kvm_eax_write(vcpu, vcpu->run->msr.data); + kvm_edx_write(vcpu, vcpu->run->msr.data >> 32); } } =20 @@ -2146,8 +2146,8 @@ static int __kvm_emulate_rdmsr(struct kvm_vcpu *vcpu,= u32 msr, int reg, trace_kvm_msr_read(msr, data); =20 if (reg < 0) { - kvm_rax_write(vcpu, data & -1u); - kvm_rdx_write(vcpu, (data >> 32) & -1u); + kvm_eax_write(vcpu, data); + kvm_edx_write(vcpu, data >> 32); } else { kvm_register_write(vcpu, reg, data); } @@ -2164,7 +2164,7 @@ static int __kvm_emulate_rdmsr(struct kvm_vcpu *vcpu,= u32 msr, int reg, =20 int kvm_emulate_rdmsr(struct kvm_vcpu *vcpu) { - return __kvm_emulate_rdmsr(vcpu, kvm_rcx_read(vcpu), -1, + return __kvm_emulate_rdmsr(vcpu, kvm_ecx_read(vcpu), -1, complete_fast_rdmsr); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_emulate_rdmsr); @@ -2200,7 +2200,7 @@ static int __kvm_emulate_wrmsr(struct kvm_vcpu *vcpu,= u32 msr, u64 data) =20 int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu) { - return __kvm_emulate_wrmsr(vcpu, kvm_rcx_read(vcpu), + return __kvm_emulate_wrmsr(vcpu, kvm_ecx_read(vcpu), kvm_read_edx_eax(vcpu)); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_emulate_wrmsr); @@ -2310,7 +2310,7 @@ static fastpath_t __handle_fastpath_wrmsr(struct kvm_= vcpu *vcpu, u32 msr, u64 da =20 fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu) { - return __handle_fastpath_wrmsr(vcpu, kvm_rcx_read(vcpu), + return __handle_fastpath_wrmsr(vcpu, kvm_ecx_read(vcpu), kvm_read_edx_eax(vcpu)); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(handle_fastpath_wrmsr); @@ -9691,7 +9691,7 @@ static int complete_fast_pio_out(struct kvm_vcpu *vcp= u) static int kvm_fast_pio_out(struct kvm_vcpu *vcpu, int size, unsigned short port) { - unsigned long val =3D kvm_rax_read(vcpu); + unsigned long val =3D kvm_rax_read_raw(vcpu); int ret =3D emulator_pio_out(vcpu, size, port, &val, 1); =20 if (ret) @@ -9727,10 +9727,10 @@ static int complete_fast_pio_in(struct kvm_vcpu *vc= pu) } =20 /* For size less than 4 we merge, else we zero extend */ - val =3D (vcpu->arch.pio.size < 4) ? kvm_rax_read(vcpu) : 0; + val =3D (vcpu->arch.pio.size < 4) ? kvm_rax_read_raw(vcpu) : 0; =20 complete_emulator_pio_in(vcpu, &val); - kvm_rax_write(vcpu, val); + kvm_rax_write_raw(vcpu, val); =20 return kvm_skip_emulated_instruction(vcpu); } @@ -9742,11 +9742,11 @@ static int kvm_fast_pio_in(struct kvm_vcpu *vcpu, i= nt size, int ret; =20 /* For size less than 4 we merge, else we zero extend */ - val =3D (size < 4) ? kvm_rax_read(vcpu) : 0; + val =3D (size < 4) ? kvm_rax_read_raw(vcpu) : 0; =20 ret =3D emulator_pio_in(vcpu, size, port, &val, 1); if (ret) { - kvm_rax_write(vcpu, val); + kvm_rax_write_raw(vcpu, val); return ret; } =20 @@ -10413,29 +10413,30 @@ static int complete_hypercall_exit(struct kvm_vcp= u *vcpu) =20 if (!is_64_bit_hypercall(vcpu)) ret =3D (u32)ret; - kvm_rax_write(vcpu, ret); + kvm_rax_write_raw(vcpu, ret); return kvm_skip_emulated_instruction(vcpu); } =20 int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl, int (*complete_hypercall)(struct kvm_vcpu *)) { - unsigned long ret; - unsigned long nr =3D kvm_rax_read(vcpu); - unsigned long a0 =3D kvm_rbx_read(vcpu); - unsigned long a1 =3D kvm_rcx_read(vcpu); - unsigned long a2 =3D kvm_rdx_read(vcpu); - unsigned long a3 =3D kvm_rsi_read(vcpu); int op_64_bit =3D is_64_bit_hypercall(vcpu); + unsigned long ret, nr, a0, a1, a2, a3; =20 ++vcpu->stat.hypercalls; =20 - if (!op_64_bit) { - nr &=3D 0xFFFFFFFF; - a0 &=3D 0xFFFFFFFF; - a1 &=3D 0xFFFFFFFF; - a2 &=3D 0xFFFFFFFF; - a3 &=3D 0xFFFFFFFF; + if (op_64_bit) { + nr =3D kvm_rax_read_raw(vcpu); + a0 =3D kvm_rbx_read_raw(vcpu); + a1 =3D kvm_rcx_read_raw(vcpu); + a2 =3D kvm_rdx_read_raw(vcpu); + a3 =3D kvm_rsi_read_raw(vcpu); + } else { + nr =3D kvm_eax_read(vcpu); + a0 =3D kvm_ebx_read(vcpu); + a1 =3D kvm_ecx_read(vcpu); + a2 =3D kvm_edx_read(vcpu); + a3 =3D kvm_esi_read(vcpu); } =20 trace_kvm_hypercall(nr, a0, a1, a2, a3); @@ -12133,23 +12134,23 @@ static void __get_regs(struct kvm_vcpu *vcpu, str= uct kvm_regs *regs) emulator_writeback_register_cache(vcpu->arch.emulate_ctxt); vcpu->arch.emulate_regs_need_sync_to_vcpu =3D false; } - regs->rax =3D kvm_rax_read(vcpu); - regs->rbx =3D kvm_rbx_read(vcpu); - regs->rcx =3D kvm_rcx_read(vcpu); - regs->rdx =3D kvm_rdx_read(vcpu); - regs->rsi =3D kvm_rsi_read(vcpu); - regs->rdi =3D kvm_rdi_read(vcpu); + regs->rax =3D kvm_rax_read_raw(vcpu); + regs->rbx =3D kvm_rbx_read_raw(vcpu); + regs->rcx =3D kvm_rcx_read_raw(vcpu); + regs->rdx =3D kvm_rdx_read_raw(vcpu); + regs->rsi =3D kvm_rsi_read_raw(vcpu); + regs->rdi =3D kvm_rdi_read_raw(vcpu); regs->rsp =3D kvm_rsp_read(vcpu); - regs->rbp =3D kvm_rbp_read(vcpu); + regs->rbp =3D kvm_rbp_read_raw(vcpu); #ifdef CONFIG_X86_64 - regs->r8 =3D kvm_r8_read(vcpu); - regs->r9 =3D kvm_r9_read(vcpu); - regs->r10 =3D kvm_r10_read(vcpu); - regs->r11 =3D kvm_r11_read(vcpu); - regs->r12 =3D kvm_r12_read(vcpu); - regs->r13 =3D kvm_r13_read(vcpu); - regs->r14 =3D kvm_r14_read(vcpu); - regs->r15 =3D kvm_r15_read(vcpu); + regs->r8 =3D kvm_r8_read_raw(vcpu); + regs->r9 =3D kvm_r9_read_raw(vcpu); + regs->r10 =3D kvm_r10_read_raw(vcpu); + regs->r11 =3D kvm_r11_read_raw(vcpu); + regs->r12 =3D kvm_r12_read_raw(vcpu); + regs->r13 =3D kvm_r13_read_raw(vcpu); + regs->r14 =3D kvm_r14_read_raw(vcpu); + regs->r15 =3D kvm_r15_read_raw(vcpu); #endif =20 regs->rip =3D kvm_rip_read(vcpu); @@ -12173,23 +12174,23 @@ static void __set_regs(struct kvm_vcpu *vcpu, str= uct kvm_regs *regs) vcpu->arch.emulate_regs_need_sync_from_vcpu =3D true; vcpu->arch.emulate_regs_need_sync_to_vcpu =3D false; =20 - kvm_rax_write(vcpu, regs->rax); - kvm_rbx_write(vcpu, regs->rbx); - kvm_rcx_write(vcpu, regs->rcx); - kvm_rdx_write(vcpu, regs->rdx); - kvm_rsi_write(vcpu, regs->rsi); - kvm_rdi_write(vcpu, regs->rdi); + kvm_rax_write_raw(vcpu, regs->rax); + kvm_rbx_write_raw(vcpu, regs->rbx); + kvm_rcx_write_raw(vcpu, regs->rcx); + kvm_rdx_write_raw(vcpu, regs->rdx); + kvm_rsi_write_raw(vcpu, regs->rsi); + kvm_rdi_write_raw(vcpu, regs->rdi); kvm_rsp_write(vcpu, regs->rsp); - kvm_rbp_write(vcpu, regs->rbp); + kvm_rbp_write_raw(vcpu, regs->rbp); #ifdef CONFIG_X86_64 - kvm_r8_write(vcpu, regs->r8); - kvm_r9_write(vcpu, regs->r9); - kvm_r10_write(vcpu, regs->r10); - kvm_r11_write(vcpu, regs->r11); - kvm_r12_write(vcpu, regs->r12); - kvm_r13_write(vcpu, regs->r13); - kvm_r14_write(vcpu, regs->r14); - kvm_r15_write(vcpu, regs->r15); + kvm_r8_write_raw(vcpu, regs->r8); + kvm_r9_write_raw(vcpu, regs->r9); + kvm_r10_write_raw(vcpu, regs->r10); + kvm_r11_write_raw(vcpu, regs->r11); + kvm_r12_write_raw(vcpu, regs->r12); + kvm_r13_write_raw(vcpu, regs->r13); + kvm_r14_write_raw(vcpu, regs->r14); + kvm_r15_write_raw(vcpu, regs->r15); #endif =20 kvm_rip_write(vcpu, regs->rip); @@ -13092,7 +13093,7 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool ini= t_event) * on RESET. But, go through the motions in case that's ever remedied. */ cpuid_0x1 =3D kvm_find_cpuid_entry(vcpu, 1); - kvm_rdx_write(vcpu, cpuid_0x1 ? cpuid_0x1->eax : 0x600); + kvm_edx_write(vcpu, cpuid_0x1 ? cpuid_0x1->eax : 0x600); =20 kvm_x86_call(vcpu_reset)(vcpu, init_event); =20 diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 16d1c3c1a2d9..bd4423e82b02 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -367,17 +367,13 @@ static inline bool vcpu_match_mmio_gpa(struct kvm_vcp= u *vcpu, gpa_t gpa) =20 static inline unsigned long kvm_register_read(struct kvm_vcpu *vcpu, int r= eg) { - unsigned long val =3D kvm_register_read_raw(vcpu, reg); - - return is_64_bit_mode(vcpu) ? val : (u32)val; + return kvm_register_read_raw(vcpu, reg) & kvm_reg_mode_mask(vcpu); } =20 static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg, unsigned long val) { - if (!is_64_bit_mode(vcpu)) - val =3D (u32)val; - return kvm_register_write_raw(vcpu, reg, val); + return kvm_register_write_raw(vcpu, reg, val & kvm_reg_mode_mask(vcpu)); } =20 static inline bool kvm_check_has_quirk(struct kvm *kvm, u64 quirk) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 895095dc684e..694b31c1fcc9 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -1408,7 +1408,7 @@ int kvm_xen_hvm_config(struct kvm *kvm, struct kvm_xe= n_hvm_config *xhc) =20 static int kvm_xen_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) { - kvm_rax_write(vcpu, result); + kvm_rax_write_raw(vcpu, result); return kvm_skip_emulated_instruction(vcpu); } =20 @@ -1679,29 +1679,29 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) u8 cpl; =20 /* Hyper-V hypercalls get bit 31 set in EAX */ - if ((kvm_rax_read(vcpu) & 0x80000000) && + if ((kvm_rax_read_raw(vcpu) & 0x80000000) && kvm_hv_hypercall_enabled(vcpu)) return kvm_hv_hypercall(vcpu); =20 longmode =3D is_64_bit_hypercall(vcpu); if (!longmode) { - input =3D (u32)kvm_rax_read(vcpu); - params[0] =3D (u32)kvm_rbx_read(vcpu); - params[1] =3D (u32)kvm_rcx_read(vcpu); - params[2] =3D (u32)kvm_rdx_read(vcpu); - params[3] =3D (u32)kvm_rsi_read(vcpu); - params[4] =3D (u32)kvm_rdi_read(vcpu); - params[5] =3D (u32)kvm_rbp_read(vcpu); + input =3D kvm_eax_read(vcpu); + params[0] =3D kvm_ebx_read(vcpu); + params[1] =3D kvm_ecx_read(vcpu); + params[2] =3D kvm_edx_read(vcpu); + params[3] =3D kvm_esi_read(vcpu); + params[4] =3D kvm_edi_read(vcpu); + params[5] =3D kvm_ebp_read(vcpu); } else { #ifdef CONFIG_X86_64 - input =3D (u64)kvm_rax_read(vcpu); - params[0] =3D (u64)kvm_rdi_read(vcpu); - params[1] =3D (u64)kvm_rsi_read(vcpu); - params[2] =3D (u64)kvm_rdx_read(vcpu); - params[3] =3D (u64)kvm_r10_read(vcpu); - params[4] =3D (u64)kvm_r8_read(vcpu); - params[5] =3D (u64)kvm_r9_read(vcpu); + input =3D (u64)kvm_rax_read_raw(vcpu); + params[0] =3D (u64)kvm_rdi_read_raw(vcpu); + params[1] =3D (u64)kvm_rsi_read_raw(vcpu); + params[2] =3D (u64)kvm_rdx_read_raw(vcpu); + params[3] =3D (u64)kvm_r10_read_raw(vcpu); + params[4] =3D (u64)kvm_r8_read_raw(vcpu); + params[5] =3D (u64)kvm_r9_read_raw(vcpu); #else KVM_BUG_ON(1, vcpu->kvm); return -EIO; --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C58F3D301A for ; Thu, 14 May 2026 21:54:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795650; cv=none; b=OhZpilhtPoy1iEjS0cv8OIfbo1I8GygZW2cZFG4gXe0V5cX03xK93SzahAetpQY2+QwO3251z/SrhDPDTsZqw5YNtIfmUvERB+3lnL5vOux286APMo0reCVIRaJmiS/LlbBRw0mknf/ilgP8PjYnw3m7n7sp/g2++Vf/QagWBXA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795650; c=relaxed/simple; bh=N92fSgTbRoFdGmTmYT2jdWK5Axf4rwJn0qavUPuZUdQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ioWXQTp5Wu8oEDQoABbdaZ0652E/LKZE5wXCYR5+lmZnublKhom4lq8elOLJE//Y/NtgFe8GR+2l3NAkIA2vOnMLthM/GWxTQ9J+JxFGIxqtXuG5gBgVBjr7c4IXRI5DOWv64lYR85k8X6UHrjfQjaX7nehGe9Ys5WX8oKodgt0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=f8ZREpaL; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="f8ZREpaL" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-c82c4772950so137707a12.1 for ; Thu, 14 May 2026 14:54:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795648; x=1779400448; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0lOseohsrh45pecTnMcOmR3eyEgHoLMULb1l21GrEmk=; b=f8ZREpaLXWyQDHDJYHpLcAmpEf4LcE0pNwlP8kI7Us/EIPJhByHl1uvY0WRmBBUKNO ++gdM20ruv58rywqunMiMfVi/omxkALz/E+BcSR2Te4r19h/uGuRnAQUTWcD7AMWT0Ak GG2TqyEYg3RfS7J2nUfh4k7p5YhHqTGbfK3QV4NAR7ExMA3233W54gBdcpU0js0O3t19 TD3WP9H70mlVuZeuDifxNy2ziUQMYhlMI96TcyBVkT1FtmwpnfZymLUeIYl1LmphSwWv JoNN+3PnI0/mwDi3o+BDMb4xsCQvZSeBei+OOOo/bbrVan8s/PUEY1vsnOPPSN0I9/rm /WXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795648; x=1779400448; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0lOseohsrh45pecTnMcOmR3eyEgHoLMULb1l21GrEmk=; b=K5/0VSIbW20sFuTXlR9R0GFoawo6j+12pfRMgIRIZU6PbEfVz+iugWM127uK0pmGO1 HiebfBLeus9DWy66ek3nElucitIKd+k5MiO0jKzHjrym9+boyQselAjIIy6oC7D0sIwf 9K/VyGNzGJjdHjTq1K5uXK5uXkQAElRuN7lVzMInd2xWXzyvLh4rFoTAXi+svGUtHp3L vFLZXPO8V7jht6VxuWqhYjPFQ8YkvkNPVttDaKrdYUjZ2ULUuyJEonJISZPZbtBOpQ61 Imnukt1PWNLzq4DcuKyj6SDJbEUnAOlmtiBCHxqAVdnQs7h67Ykjfm2xwoeUYTah2gDS a77A== X-Forwarded-Encrypted: i=1; AFNElJ/ZPpQMmxQDu9RH2l3mALnZFfJtqSiMr97UuNDYXqn7i4FQzSdlJgmgHEYdvx1d6IrbbZT/o6u2Y87Ol4E=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4cq/vOUhUGAOXTQy/rL5HfcW2RIJr8QDCEOzZhDsU7mvNCpsd z05r+fDgRobE0+avWzlEA7d56BBAinFjJ3smBXPfI5+Qd+ce2iuL62ksrHAC/XeSxO5xwvy5t87 oNtiolQ== X-Received: from pgna27.prod.google.com ([2002:a63:705b:0:b0:c82:7a7f:9bf6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:7349:b0:39f:3559:750d with SMTP id adf61e73a8af0-3b0c0abccb2mr4934624637.31.1778795647024; Thu, 14 May 2026 14:54:07 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:49 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-10-seanjc@google.com> Subject: [PATCH v2 09/15] KVM: x86: Drop non-raw kvm__write() helpers From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the non-raw, mode-aware kvm__write() helpers as there is no usage in KVM, and in all likelihood there will never be usage in KVM as use of hardcoded registers in instructions is uncommon, and *modifying* hardcoded registers is practically unheard of. While there are a few instructions that modify registers in mode-aware ways, e.g. REP string and some ENCLS varieties, the odds of KVM needing to emulate such instructions (outside of the fully emulator) are vanishingly small. Drop kvm__write() to prevent incorrect usage; _if_ a new instruction comes along that needs to modify a hardcoded register, this can be reverted. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/regs.h | 5 ----- 1 file changed, 5 deletions(-) diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h index b28e71caed25..52bed14f43e3 100644 --- a/arch/x86/kvm/regs.h +++ b/arch/x86/kvm/regs.h @@ -61,11 +61,6 @@ static __always_inline unsigned long kvm_##lname##_read(= struct kvm_vcpu *vcpu) { \ return vcpu->arch.regs[VCPU_REGS_##uname] & kvm_reg_mode_mask(vcpu); \ } \ -static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ - unsigned long val) \ -{ \ - vcpu->arch.regs[VCPU_REGS_##uname] =3D val & kvm_reg_mode_mask(vcpu); \ -} \ static __always_inline unsigned long kvm_##lname##_read_raw(struct kvm_vcp= u *vcpu) \ { \ return vcpu->arch.regs[VCPU_REGS_##uname]; \ --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74CF33D34A6 for ; Thu, 14 May 2026 21:54:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795651; cv=none; b=nn+T9OLASYz3pEQ3aAkbsAi4PcIxsuyuitpHUkGKU1h8Vdu7YSeFE5SU1vo6nJWidjMr3t2rzzKLw5cZtIh2aB1oXYtKeh+4P1k/qDGPVFrF4ZcA2tjvdiV1TxJt+KvG5I09RRqUQ4s5aNERZ6nrleGl0NWLBO886zWFsLEVlck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795651; c=relaxed/simple; bh=FsAq7KEuatX1Pn52xDkKa5S9Ii59O/tYrQdckY5bUaw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=l0aT06RbKsPk8BBGycc3uI4irN2DgA8vBAAYWwbaQptY/S6l0RUZr8WG1RMMgia9AgkztRnPAISjXIYRCI+9W0ywmdk+IKbPlY4is102mvIzvWIFgwbimm/479ku2z9Ee2dJxLWNCGpGQuNLxvp/9McNx1VehZK694OLx13LfvM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=s6hx7fWG; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="s6hx7fWG" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82fa7c6699fso10915678b3a.1 for ; Thu, 14 May 2026 14:54:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795649; x=1779400449; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=g9vonHXs6r1LN2kGlbzZfFmfFgWCjNXLKuYTABKAIiY=; b=s6hx7fWGUrmKMVbwXM37Ehqom4y48epHMsLjCmeIGb3CxN77F3xZd6W7ISncA5RHub aDSxH/pKMEKh3Qcq1wQB4yWepmQONhpHLAyguy60XZ7iX8AnEFYf1kP903HZevWY5se3 6psq9Br1E5ZnHIsLGUttid74bzsWyvHVBxTXMVPIAdCpYnZfVHS+HiPo7DxocDmTKyi8 jnccU91a6EnhVWZ7P1Dl/ETEXwJvZc1rWygU8NriXo6sWfcud6MWQ3hOFNmhUZW3EcAy c4OVo5TNS0AQVWMvqOMc193hwWzCD0jE14uY2LesCiTyO/4mLOP9Fw+k3xHQVpCcxcuV beyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795649; x=1779400449; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=g9vonHXs6r1LN2kGlbzZfFmfFgWCjNXLKuYTABKAIiY=; b=hBvLG/yRojGtnftEGZcUW/IIxREvfqfIjjG/cOMVslbN8miYMFSASeF93VUII9gwHe 06TqrOe8TATgr94XjrX903ntg+5OHySWujfZANA60Hx1Xdt6n6YkKiHKc6ga/m5zl/IJ B5/6HPPQoND7RseE81FMElVoOsyTkvFH55/ILOFr06QuilJQpzdraBSnf2+V7rrjmbVj tEGgx9ikOAiag+QEklebQEUwtmjcPGo8jtDIf+bIWCZjL2FiDPjJVAZRtMV8M1JIzhV1 4GBZD/iXfmRVVPhVYDg6OH6muAtN6L8kML4hrpCtOHARNFz+/959EA153nXXnHvT0PFM F9ag== X-Forwarded-Encrypted: i=1; AFNElJ+mm3ULXSspv6y0SuPlUv0cI99EMrgjBhbJKiAxQ7gf1n+R192hkD0PE+ol20mwVLvkcLrteIauZtvte/U=@vger.kernel.org X-Gm-Message-State: AOJu0YyGuGb6Un2IYVJqu6cN8TPB6SIOapT+yh3ZvUA5FZxownMF8w4q XAuRU2UGPXtTHF2ZaZ31VM9K3J1/t88Iu2dIRM2iuAFmpKdQVeuLXszzNzxRekJPa33m6sR8dgz eGRVAyw== X-Received: from pfbem25.prod.google.com ([2002:a05:6a00:3759:b0:835:2a09:221c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3a25:b0:82f:1b42:11d0 with SMTP id d2e1a72fcca58-83f33c9bdc4mr1362504b3a.15.1778795648217; Thu, 14 May 2026 14:54:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:50 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-11-seanjc@google.com> Subject: [PATCH v2 10/15] KVM: nSVM: Use kvm_rax_read() now that it's mode-aware From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that kvm_rax_read() truncates the output value to 32 bits if the vCPU isn't in 64-bit mode, use it instead of the more verbose (and very technically slower) kvm_register_read(). Note! VMLOAD, VMSAVE, and VMRUN emulation are still technically buggy, as they can use EAX (versus RAX) in 64-bit mode via an operand size prefix. Don't bother trying to handle that case, as it would require decoding the code stream, which would open an entirely different can of worms, and in practice no sane guest would shove garbage into RAX[63:32] and then execute VMLOAD/VMSAVE/VMRUN with just EAX. No functional change intended. Cc: Yosry Ahmed Reviewed-by: Yosry Ahmed Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/nested.c | 2 +- arch/x86/kvm/svm/svm.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 7b2d804ef2b0..4b1259eecec5 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -1119,7 +1119,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) if (WARN_ON_ONCE(!svm->nested.initialized)) return -EINVAL; =20 - vmcb12_gpa =3D kvm_register_read(vcpu, VCPU_REGS_RAX); + vmcb12_gpa =3D kvm_rax_read(vcpu); if (!page_address_valid(vcpu, vmcb12_gpa)) { kvm_inject_gp(vcpu, 0); return 1; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 02fb9560c26e..6379c389d811 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2217,7 +2217,7 @@ static int intr_interception(struct kvm_vcpu *vcpu) =20 static int vmload_vmsave_interception(struct kvm_vcpu *vcpu, bool vmload) { - u64 vmcb12_gpa =3D kvm_register_read(vcpu, VCPU_REGS_RAX); + u64 vmcb12_gpa =3D kvm_rax_read(vcpu); struct vcpu_svm *svm =3D to_svm(vcpu); struct vmcb *vmcb12; struct kvm_host_map map; @@ -2325,7 +2325,7 @@ static int gp_interception(struct kvm_vcpu *vcpu) if (nested_svm_check_permissions(vcpu)) return 1; =20 - if (!page_address_valid(vcpu, kvm_register_read(vcpu, VCPU_REGS_RAX))) + if (!page_address_valid(vcpu, kvm_rax_read(vcpu))) goto reinject; =20 /* --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 744D83D412A for ; Thu, 14 May 2026 21:54:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795652; cv=none; b=SX1A6x4L/D2B0E7ED5po9+8ag3q1V8cBBYwHtU8sd9SBBuCzv0gj/15QSAji6cF/KBeWq8Uc/vSwV9S0SBl/lYwR8ZaCXKLgt8ZCXxKR2LNkhShY4IRxXZ75hdMN8S/oD0v4WeTCd++Ku8tHFavcb/wwtJcc5xVUFKhzHj2zrcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795652; c=relaxed/simple; bh=kPzayqgTX4AYKj05HSSQLjERNpIEzXUE+TYLElLBau0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=s8L10ToJzO4lm49mtPzRMUvFIWN0C8swKSg8JdoHH3hKlEvCV40tEVkYQpXaoHoEyEPId5MCTy4jT4pQrQCI0hEmTmMaG79A6CukknPh7RbkmH/Xldt0dvAHkaeTZ7SeUVzogA6PIvzbNAdwz6ElONIo3oUHaEI384oaxBv4Oh0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oBRYB03I; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oBRYB03I" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c8276c91addso3515995a12.2 for ; Thu, 14 May 2026 14:54:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795650; x=1779400450; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=LhpTli2hbTyfQzA9vyBgyGyJFOk8w+Rac8Zy54UhAOk=; b=oBRYB03IbWkiuAdyITc7VfVBAxwdxEjmOKUdwv0f51j5TXFJImuceM8LqgVdIhA1Km CTeiOOdlsdKeo9gapDtvijjgEKpQSog5vpPgj8xwvjbvOWkxUZKch7WzxXrWIQ3/jamE 4vdPZLvBf4tkZaYtovqXOMo97jpK8uGHOgmc5JJZS39i8Qe+p2dp+l4KIuakFS9+HMgf XHuEL8O942FP+AiLA0fGU3pXT4pMoVbpvjjGY9ynsp8xpnMdrEbXr8kuloWbbA8M4b45 0wHtI4bK5AbXPV/ybPaeziAFM+SwujD8r0dl+xiz4tekgjpHJl2hNlSiAatI2to9YJxe smzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795650; x=1779400450; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LhpTli2hbTyfQzA9vyBgyGyJFOk8w+Rac8Zy54UhAOk=; b=iThmXiUh1e8hFPC0SzrwbwZAndvp+SEyeIxiMmf2GsEIiOmEGGoEdJpinblOOJfHZo qX2pzMWgwpzbG8tG2J8HDobz+IAmSpekVRHF5cERZfJNPuC11BdBCh24SWNMquPAia0G GFjuxiRkjBLzJagDukCSbkQQwNnW+jNgZxfv71A0Q4CSNdUIjVgppaqZ73LVFAAH1+4H m2Lu8J/M1QoGpMp0XE63TLLk4ud1z334JGlpaOscf0Vyb0Mtor2D5eCQ5YDfMdch2OMZ eYOq2ifTJj8q5UOKZgLZDfVRLRkfslkXH/wkpEFEWXCYNlBmWTwCOVvdpIlBwfF6SeLY Up2Q== X-Forwarded-Encrypted: i=1; AFNElJ9KmbUAWxI76ZJmG8kLB6j9ualgh46a7x3yYQm1bIdWM/eGba68zJrq6kp7dcfiLQi1+fTuNYnCIlWP3xU=@vger.kernel.org X-Gm-Message-State: AOJu0YzGKCxdIAPqP2MiIx07SsUFyN7HqG5pkWq+2sY3/f/Qlo28RtMT MRsxW9qjm2lz9bKpRrr1svDRN/UyLUg5rfOPQihyjuEigGkMgplWkdDZVxQNL+s3OdBihIaVUx8 v2cIZUw== X-Received: from pfmy21.prod.google.com ([2002:aa7:8055:0:b0:83f:2ce7:48ce]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1954:b0:835:366f:5da2 with SMTP id d2e1a72fcca58-83f33d80937mr1240198b3a.37.1778795649298; Thu, 14 May 2026 14:54:09 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:51 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-12-seanjc@google.com> Subject: [PATCH v2 11/15] Revert "KVM: VMX: Read 32-bit GPR values for ENCLS instructions outside of 64-bit mode" From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that kvm__read() are mode aware, i.e. are functionally equivalent to kvm_register_read(), revert aback to the less verbose versions. No functional change intended. This reverts commit 60919eccf6764c71cef31a1afeaa1a36b8e5ab85. Acked-by: Kai Huang Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/vmx/sgx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index 2f5a1c58f3c5..876dc2814108 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -225,8 +225,8 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu) struct x86_exception ex; int r; =20 - if (sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RBX), 32, 3= 2, &pageinfo_gva) || - sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RCX), 4096,= 4096, &secs_gva)) + if (sgx_get_encls_gva(vcpu, kvm_rbx_read(vcpu), 32, 32, &pageinfo_gva) || + sgx_get_encls_gva(vcpu, kvm_rcx_read(vcpu), 4096, 4096, &secs_gva)) return 1; =20 /* @@ -302,9 +302,9 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu) gpa_t sig_gpa, secs_gpa, token_gpa; int ret, trapnr; =20 - if (sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RBX), 1808,= 4096, &sig_gva) || - sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RCX), 4096,= 4096, &secs_gva) || - sgx_get_encls_gva(vcpu, kvm_register_read(vcpu, VCPU_REGS_RDX), 304, = 512, &token_gva)) + if (sgx_get_encls_gva(vcpu, kvm_rbx_read(vcpu), 1808, 4096, &sig_gva) || + sgx_get_encls_gva(vcpu, kvm_rcx_read(vcpu), 4096, 4096, &secs_gva) || + sgx_get_encls_gva(vcpu, kvm_rdx_read(vcpu), 304, 512, &token_gva)) return 1; =20 /* --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A10D3D47CB for ; Thu, 14 May 2026 21:54:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795653; cv=none; b=YpEdsmx0kTjmd4z+JuS/RVQhH/6an2HmqPPRpOrJpxsTB6Hwx8/h21VHVceC1KO9MvjB9wsuwHIlBcjVnIClRHoQDulWMf/l+FpXZX6KYh1gD8psIar1OGpX+BMXwV4glzApfszGO0Q/xASWvDU+0BwyaXbsXyWiYqJm5yGHZcc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795653; c=relaxed/simple; bh=ASa0qONy08n1jZW2D9s2UlvBL3Zl9eZuoZRjbijA0sY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=m7vQ7Ub8fggrnNclZcF6ibRDnW+2wwQhg5eSaofjcXUj5LxkBJVBI3LuT/1oyrLdVJVP96a4Wri2z3nK4BKybWyMXDyhNT1Leg8EWwtCIvM8YBjN+4l0FmLPq1x89TjbB/F5eJvNL173jZj5luNMJgYWu7ZIaQUf6/BVbmaGs2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WaDdmZOc; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WaDdmZOc" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-82fa860e71eso4811707b3a.0 for ; Thu, 14 May 2026 14:54:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795651; x=1779400451; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=AMywqrk2MGSBbKp3e1jShOoo/2aA5jpTeXkhhSsyu/E=; b=WaDdmZOcicsLIuhNVUrPH+WXJzlYDCXXjySdy2ZLYth+3kZOZ1N1KE03KflCbxuzF6 xtVgvL4iiK5INZ4KWZyNm4PIpMB8pe4mQqqjPrJ59s+CO5AAOqZ/nL9lSTnsCDkiUbDW lNsnArax/RWeEmL8FoBnn9Z6Sb6CbFA3z9tYifz77RdbwV3psgYI/QGIrwyBS+TiY3Sp uSaO04P8BXEyowN0rQoABP86it+wqq2ELlR+4aXLGsIaj1BrnTgbpyu7GzDuJWfknP0R yE6cVAkLIhJyW1O9s/icvmrVgWK6vYli5d+OTkqQKKvk6fxPbbvzY4/wPHka2KcPRoLN t1Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795651; x=1779400451; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AMywqrk2MGSBbKp3e1jShOoo/2aA5jpTeXkhhSsyu/E=; b=Psdpg7NhzxQ9L3Kj8NAmbncxC8t0V0GcU8+IwTsbhrpbuVpgv6cGzwMr3n3Q6ZYuLx WAzFvDxm0iGcCbC6dMvTRL33JKNfahORB8KVxtNVjjVwcD/aQVCwVh5qjYAIvmNKx3fe PcZSA6yau34UZi9FW7JhUs1jYGfMqCXL0G9IzVp9hQQBtxiuqJCBxTbhO1MrvampgtfF 9EYi+Il/eGsbUPkXFVGkUGiKcUQfNaeXMB+pwFn21+dJvqqUXrIFgVMjIwwADIndmJ/2 4RV2ZYx0JQ0cKGETiLIeONCBZmnNP0NjXxVt+b/m4VA4VKa/+fZl7QQX348h6UrGC9GN a3TA== X-Forwarded-Encrypted: i=1; AFNElJ87WntEsGQNjPxznBy3rAZzLyQZbVEn0ynQzwYkbD50MIYyHeVJZ6FK67B8wsMiuKFzYw290E8V8LBTZAI=@vger.kernel.org X-Gm-Message-State: AOJu0Yz20vZeMD17jpW93R5SuGkAN23zTvQjfI4/t3S05bBPQ9kczULo QrIa78wnjWuiikG729/7PW4bKn+BopnmiZfeDZHIBSb40KsxIZnu+FjtKNpUVU0rz/9lPRGYPjN cIiwkWw== X-Received: from pfnj26.prod.google.com ([2002:aa7:83da:0:b0:838:ec6d:449c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:12c2:b0:82c:225e:63c5 with SMTP id d2e1a72fcca58-83f33b34c12mr1218784b3a.18.1778795650326; Thu, 14 May 2026 14:54:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:52 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-13-seanjc@google.com> Subject: [PATCH v2 12/15] KVM: x86: Harden is_64_bit_hypercall() against bugs on 32-bit kernels From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Unconditionally return %false for is_64_bit_hypercall() on 32-bit kernels to guard against incorrectly setting guest_state_protected, and because in a (very) hypothetical world where 32-bit KVM supports protected guests, assuming a hypercall was made in 64-bit mode is flat out wrong. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu --- arch/x86/kvm/regs.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h index 52bed14f43e3..d4d2a47a4968 100644 --- a/arch/x86/kvm/regs.h +++ b/arch/x86/kvm/regs.h @@ -39,12 +39,16 @@ static inline bool is_64_bit_mode(struct kvm_vcpu *vcpu) =20 static inline bool is_64_bit_hypercall(struct kvm_vcpu *vcpu) { +#ifdef CONFIG_X86_64 /* * If running with protected guest state, the CS register is not * accessible. The hypercall register values will have had to been * provided in 64-bit mode, so assume the guest is in 64-bit. */ return vcpu->arch.guest_state_protected || is_64_bit_mode(vcpu); +#else + return false; +#endif } =20 static __always_inline unsigned long kvm_reg_mode_mask(struct kvm_vcpu *vc= pu) --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88F5D3D1AB1 for ; Thu, 14 May 2026 21:54:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795655; cv=none; b=BQmzAEgDsHQK/S6ZntMXVx9mVdyI7+sEGB3uLi+16ipYUwBEKQO0iGFndtoauAVOhZZ9L9G+FoWZD7uBgzt7uzWi6psQ0YJYBAexcxJ1ORAG6/fPsLFGQn5ivBXnhT2kSQbUUwr8VubbPzZ6cErWjsUxYcTvYjiYmOw+iE4Wy9o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795655; c=relaxed/simple; bh=KrQzVOQV563oIUJ/VEU8+6nkqUmBj6iF52H6K+QKjMQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=peh1J4QpgXl4WIV0Dz4mFkINeBJesXhwkQNYGRKOLN3+IvaDtVYYkYKIbqdBWwYMURi+AdTbekGKcxTcUZrt3Elv9171vVyd4v1vznBALiaoeP/0LKLOHy8KTt6Z1kTU+IUx2hG0f5wnfXVRVOnk4V+lFgzUAFcrSXISycbMNfE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uPvE02p4; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uPvE02p4" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-836cfd84728so135053b3a.0 for ; Thu, 14 May 2026 14:54:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795652; x=1779400452; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=PkJ0oOSgDIB4In8kN5PJvcBuLqM4XeIONdFdM8EJTow=; b=uPvE02p4vfBu15gmndiBLJfRSMClma2TVGrcvSq7BST8Agi4lEUT4NYTJCcbD4xxQY CEP6jw3zb62TFkHCmXVVp979sHIReQuIYJCLlt7T4JdsVYf9AUv3hn+ypmVcfR5d9Dmz 3nPD4JBqz7cN0h2CQLcsA5JYmNCDrhcFbRwqDf3PWzKAMVufghg0BrEKRJ9ys8q2NlE/ Soxqzp8quWriHLFhu3VVHk+AVBj3IdAlh4FKSVoLNebopQKdGggeBxdBome8nnKkuO/d ZVXtTWMIIz/gABF2VhETL7mwEVcVopoZVi7EYgbuKrj4a1hJlf2mj8DWVEBpNKUKSP4I hxZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795652; x=1779400452; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=PkJ0oOSgDIB4In8kN5PJvcBuLqM4XeIONdFdM8EJTow=; b=OrkH7t3c7Kp1ywiOoKXGstFUxf+1Mhqn6G0EYc/7Szz4dJLVvSe4/y84b7NYciWy+I P1nqqtWokXM7pUc1msVh6HT/lSRWo4qvYdDh0Ek4pkCchr4sClnjv67L49o3SK3irpm1 EdMQNGNDMCnGuo21+TgZsXvIZMr49ifCFbUHZzI4+N2DNeqx6MM95tDToYAdvfZU0wOZ nb7lQ4lPHjs/pnBp21tkf2RjPyN0OSaAhD9rLXetHp+HdqfL1oJml5jd5cOXOugj0un7 G1ZMoYz9thyEqzpd9HC3PtBO0Y5OpxzDG5VpB0USEWEhU2A3lG/dutsFCBvl51sOcvoF UPwg== X-Forwarded-Encrypted: i=1; AFNElJ8BGFKhAuIMwC3yR19lTP7KEjdBFvvYCJQuPVF78HJo7/72hidbGp5fjT6CJ7JA13NEmedtXs8KTOlM0vM=@vger.kernel.org X-Gm-Message-State: AOJu0YziYrdJhoc2mJNNQCcrcteEyXc9Du87W6qrHtNKIjf+U4qs/Lfx YQa0mE5ak8VFg5risffXfHCZp3ZRIFCCwq1tdLT0p1p5awZmi5IeizB87XrgHF2hC2vbEMIl/gi /HP8P2Q== X-Received: from pfhh2.prod.google.com ([2002:a05:6a00:2302:b0:82f:7163:35c4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:240e:b0:82f:6d4a:df3e with SMTP id d2e1a72fcca58-83f33ad8a57mr1146074b3a.5.1778795651528; Thu, 14 May 2026 14:54:11 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:53 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-14-seanjc@google.com> Subject: [PATCH v2 13/15] KVM: x86: Move update_cr8_intercept() to lapic.c From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move update_cr8_intercept() to lapic.c so that it's globally visible in anticipation of extracting most of the register-specific code out of x86.c and into a new compilation unit. Opportunistically prefix the helper kvm_lapic_ to make its role/scope more obvious. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 26 ++++++++++++++++++++++++++ arch/x86/kvm/lapic.h | 1 + arch/x86/kvm/x86.c | 34 +++------------------------------- 3 files changed, 30 insertions(+), 31 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index d8dbfb107bfb..27cca31308bd 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2744,6 +2744,32 @@ u64 kvm_lapic_get_cr8(struct kvm_vcpu *vcpu) return (tpr & 0xf0) >> 4; } =20 +void kvm_lapic_update_cr8_intercept(struct kvm_vcpu *vcpu) +{ + int max_irr, tpr; + + if (!kvm_x86_ops.update_cr8_intercept) + return; + + if (!lapic_in_kernel(vcpu)) + return; + + if (vcpu->arch.apic->apicv_active) + return; + + if (!vcpu->arch.apic->vapic_addr) + max_irr =3D kvm_lapic_find_highest_irr(vcpu); + else + max_irr =3D -1; + + if (max_irr !=3D -1) + max_irr >>=3D 4; + + tpr =3D kvm_lapic_get_cr8(vcpu); + + kvm_x86_call(update_cr8_intercept)(vcpu, tpr, max_irr); +} + static void __kvm_apic_set_base(struct kvm_vcpu *vcpu, u64 value) { u64 old_value =3D vcpu->arch.apic_base; diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 274885af4ebc..533581d06151 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -100,6 +100,7 @@ int kvm_apic_accept_events(struct kvm_vcpu *vcpu); void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event); u64 kvm_lapic_get_cr8(struct kvm_vcpu *vcpu); void kvm_lapic_set_tpr(struct kvm_vcpu *vcpu, unsigned long cr8); +void kvm_lapic_update_cr8_intercept(struct kvm_vcpu *vcpu); void kvm_lapic_set_eoi(struct kvm_vcpu *vcpu); void kvm_apic_set_version(struct kvm_vcpu *vcpu); void kvm_apic_after_set_mcg_cap(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b958521bc81f..1113a31978dd 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -128,7 +128,6 @@ static u64 __read_mostly efer_reserved_bits =3D ~((u64)= EFER_SCE); KVM_X2APIC_ENABLE_SUPPRESS_EOI_BROADCAST | \ KVM_X2APIC_DISABLE_SUPPRESS_EOI_BROADCAST) =20 -static void update_cr8_intercept(struct kvm_vcpu *vcpu); static void process_nmi(struct kvm_vcpu *vcpu); static void __kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags); static void store_regs(struct kvm_vcpu *vcpu); @@ -5342,7 +5341,7 @@ static int kvm_vcpu_ioctl_set_lapic(struct kvm_vcpu *= vcpu, r =3D kvm_apic_set_state(vcpu, s); if (r) return r; - update_cr8_intercept(vcpu); + kvm_lapic_update_cr8_intercept(vcpu); =20 return 0; } @@ -10583,33 +10582,6 @@ static void post_kvm_run_save(struct kvm_vcpu *vcp= u) kvm_run->flags |=3D KVM_RUN_X86_GUEST_MODE; } =20 -static void update_cr8_intercept(struct kvm_vcpu *vcpu) -{ - int max_irr, tpr; - - if (!kvm_x86_ops.update_cr8_intercept) - return; - - if (!lapic_in_kernel(vcpu)) - return; - - if (vcpu->arch.apic->apicv_active) - return; - - if (!vcpu->arch.apic->vapic_addr) - max_irr =3D kvm_lapic_find_highest_irr(vcpu); - else - max_irr =3D -1; - - if (max_irr !=3D -1) - max_irr >>=3D 4; - - tpr =3D kvm_lapic_get_cr8(vcpu); - - kvm_x86_call(update_cr8_intercept)(vcpu, tpr, max_irr); -} - - int kvm_check_nested_events(struct kvm_vcpu *vcpu) { if (kvm_test_request(KVM_REQ_TRIPLE_FAULT, vcpu)) { @@ -11350,7 +11322,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) kvm_x86_call(enable_irq_window)(vcpu); =20 if (kvm_lapic_enabled(vcpu)) { - update_cr8_intercept(vcpu); + kvm_lapic_update_cr8_intercept(vcpu); kvm_lapic_sync_to_vapic(vcpu); } } @@ -12496,7 +12468,7 @@ static int __set_sregs_common(struct kvm_vcpu *vcpu= , struct kvm_sregs *sregs, kvm_set_segment(vcpu, &sregs->tr, VCPU_SREG_TR); kvm_set_segment(vcpu, &sregs->ldt, VCPU_SREG_LDTR); =20 - update_cr8_intercept(vcpu); + kvm_lapic_update_cr8_intercept(vcpu); =20 /* Older userspace won't unhalt the vcpu on reset. */ if (kvm_vcpu_is_bsp(vcpu) && kvm_rip_read(vcpu) =3D=3D 0xfff0 && --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC4E03D1CD4 for ; Thu, 14 May 2026 21:54:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795655; cv=none; b=ZfYSFIRy8Phs8op+nqrVd+Md93emAof2IOGART+PT7WXqwyalr+CpB1Lnq/u5/2/y6+JH5UdCaTwl10E3L7CdIgOM7+WYa6ckWNWsm9IdzuV0ryL49IKd/zJR0nxrHyCnc8cwcNYilh8pQHu9igXHMivqcY0NBx+BCP4U9Jdoys= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795655; c=relaxed/simple; bh=DkYUZVCFxypYvjE+ECVjC8MTu+dU12DtdKyLPawOsdk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NRidlvpBfIszKcNPKY/Ok0yXHF20INnmOYz6yiU4GwvV/fiDzvmIXsbUC2OtgH8eufXRFb9nmF4z6W2gPRBJs2+9INO9PKu0FyCHmsTYVlcp5rsMSSLZSyF2opTk0NnxS2dAeoJlv7lqt67LhlOabHerNJrCN+xh7kl88bp0D9Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PH8qIL3C; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PH8qIL3C" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c828cee4fcdso3182830a12.3 for ; Thu, 14 May 2026 14:54:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795653; x=1779400453; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=brzyKoCEspwI0RWW9AziHqTrLPrrZVq/O92aCbmnfFc=; b=PH8qIL3CLryxPTb8Mv/w0e4IyPfIyUjFd5EoOPinf95d/zvvnNSDu5m4PRcAgva5Gn D6LKHngyMjGvLARykoqlTl+YdJHW2qzqCQt4ntaAtPTqgbstFXn481iPXUTI4xL4Jk8+ 1BFMiQkqR2lrL6B2TETYK53MnVaeODOh+0HqhLQkJ8yjj67sG+n1k5/RX+k3WVZ5EHGH EW09wm+UScPMndYzCTHE6+d69RmaVuf5uBqLa3V5B+4F2AvgYdpJ2UHBoSLeYXM756wX k4dUV/nWTQeaINvjN6LOYj+NAO2zWcovLix5c7fnItCOPWHtlnQvJWhrW7gHt07HLRC1 y0UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795653; x=1779400453; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=brzyKoCEspwI0RWW9AziHqTrLPrrZVq/O92aCbmnfFc=; b=nR2HHZEiODHi69Zi3F8uRmZ9OrGTGsabJHWs8bJYWVPcJ92tOTCEMJSci4CBWJ1DaR ooYRoeiz/X49VWcfW8b3qeLm8pUVrt/KtSTP2ZNedUngbOhlqBA5owEsJmUF1AvhnqJi SIMFrDaDJDLY/ODSh3jNhWjdt0p/CM6wWgg40OvKPLM/htgVRlu2VMgJhqpvdY7f5397 IYmP2lJXC/tFHZcDzvOc/5XZpo34q+wqyGGDDLg4QR2a6v1Jmn1QgPwq/UqQMpY+uTca zAxVmQFqyAvwZV1Pzxnaz0TrrNRFBzKMLrVvPLndGye1DYAXuDvUrRdspjwMSCq1zCzJ ihFA== X-Forwarded-Encrypted: i=1; AFNElJ/hFuIuTHdW8HJrvB9G7inbk3DWHYS0S+3jSRFXsmmQ635Pok4bJ6ePIvSNuGvfTGuCFJ+kAdiWbJW6TLE=@vger.kernel.org X-Gm-Message-State: AOJu0YyO35x0XCet52HsyNl8EBQdIR1MfvNhKmaG1K4cv/Q8payenuT1 9Ul2YmZJN4Kte75VzRaRO853hEUu3aVCZBFKozyxQghLmkJBVPnBerN4zXqvz2hB/5Jty0vrn2a 3zzxDXQ== X-Received: from pfoo22.prod.google.com ([2002:a05:6a00:1a16:b0:836:d115:1e44]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:94c4:b0:82c:212a:a9b5 with SMTP id d2e1a72fcca58-83f33d52d3cmr1204761b3a.36.1778795652618; Thu, 14 May 2026 14:54:12 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:54 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-15-seanjc@google.com> Subject: [PATCH v2 14/15] KVM: x86: Move kvm_pv_async_pf_enabled() to x86.h (as an inline) From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move kvm_pv_async_pf_enabled() in anticipation of extracting the majority of register specific code out of x86.c. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 12 ------------ arch/x86/kvm/x86.h | 12 ++++++++++++ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1113a31978dd..e664e874973b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1042,18 +1042,6 @@ bool kvm_require_dr(struct kvm_vcpu *vcpu, int dr) } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_require_dr); =20 -static bool __kvm_pv_async_pf_enabled(u64 data) -{ - u64 mask =3D KVM_ASYNC_PF_ENABLED | KVM_ASYNC_PF_DELIVERY_AS_INT; - - return (data & mask) =3D=3D mask; -} - -static bool kvm_pv_async_pf_enabled(struct kvm_vcpu *vcpu) -{ - return __kvm_pv_async_pf_enabled(vcpu->arch.apf.msr_en_val); -} - static inline u64 pdptr_rsvd_bits(struct kvm_vcpu *vcpu) { return vcpu->arch.reserved_gpa_bits | rsvd_bits(5, 8) | rsvd_bits(1, 2); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index bd4423e82b02..185062a26924 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -570,6 +570,18 @@ static inline bool kvm_pat_valid(u64 data) return (data | ((data & 0x0202020202020202ull) << 1)) =3D=3D data; } =20 +static inline bool __kvm_pv_async_pf_enabled(u64 data) +{ + u64 mask =3D KVM_ASYNC_PF_ENABLED | KVM_ASYNC_PF_DELIVERY_AS_INT; + + return (data & mask) =3D=3D mask; +} + +static inline bool kvm_pv_async_pf_enabled(struct kvm_vcpu *vcpu) +{ + return __kvm_pv_async_pf_enabled(vcpu->arch.apf.msr_en_val); +} + /* * Trigger machine check on the host. We assume all the MSRs are already s= et up * by the CPU and that we still run on the same CPU as the MCE occurred on. --=20 2.54.0.563.g4f69b47b94-goog From nobody Fri Jun 12 12:44:30 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C11C63D7D7D for ; Thu, 14 May 2026 21:54:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795659; cv=none; b=EwzXlRjz9bl9ZsZPWl45nUFAjdqzBfZ/SYyyry5Z/pxiIdiJHd66nzN1PzaWYv8B4CV9cbsYMyAjFXAMvdOHI7748ngYtFTWRaRZHaKurbIwyQBq+lm+3J6i/MjGDO40EaPPnWP71CaJ0vw4YnCkSd49HXkr6pbuFff03kEVUoU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778795659; c=relaxed/simple; bh=FW9r5K/eMLMJVMpwkUNBgMdoN5Y09QF0Niqq560hV8M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=r4nuRlcGQ1BASqUB88Tnat/UyU9QNfcK9rLiyVo1Qk5I6KSv7qpPJUFzvFBYIsnqc7T5dFFoExUWdDTa1Rdb1JSDUufOfb8v/qmuusTf5FWU4OZahMBGzdeJygL4TgXrPCXa0/fArwGf2iI2tj8GhnGZFl00ugXS/rsOsw99Gp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JKxxXsVI; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JKxxXsVI" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2b2ecc96a9aso88995695ad.1 for ; Thu, 14 May 2026 14:54:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1778795654; x=1779400454; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=S9rq+9EQyy0Grumxb4oMswzj8RHSPFqJq9fyNVfdwiY=; b=JKxxXsVINlFVHMOo/al+BTmkVIUuVQ3VHOQA0RxLw82VeOred1hdW4sk5+zUIcesKM CaEnDx8mZpcDqAy7RrBC9Uaneu369pOCy/kJsKMvnWFnLNOuDRzwY+GHTxjZelp7UJhU 2XrdsGh5gPUqvUJpvPMxJtimXiQZwnhVPabRyQLBKjvLj0EhppJk8zmDLhm12FheI0yw y2olNGRc3XiLDbQ0trwp6O9uzn3lUUszsZNOT3RDAAmKxswwMA0cE2ztEHlh6w5BMxm4 LCAfNgugnOV5r5ByEiv+1YdypLiHC59X5pmEiOCs9FV7rtb1xK/796XG2AZfk+Fitnkt c4sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778795654; x=1779400454; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=S9rq+9EQyy0Grumxb4oMswzj8RHSPFqJq9fyNVfdwiY=; b=nGlZaHOfD9bzWoNMGhn03/Nr6P1pctmX+li0KTQiIHNeWyYgTYu3lqrzrWiNM5+Czf i+99/O+3DlB9DreS8bKGcrF0g2X7nR14hccJboJB5hyJc2R+L/IjDfD4wDtyzWFWBscy FSuwudyrG1KtOBbTErreKpOGcFj/kjpxy4k+GOwQif5s1qLcdBoe1Y2zgG+4RBnDZM3v xaQnEME3u2n6FuajIh2jJskKKn0KauUgt2YZlBsbWWcRzN9Q653Zj6SVkeCq0V7BEtN5 lgStWafGk7G8rOKeT5myve7pRcVYHoTTOWMWhNjh9tCmYXhWFhEXjyBcuJItjdbr+OX8 Qztw== X-Forwarded-Encrypted: i=1; AFNElJ8mnc4Tplrn65QkG6nQ3FLGgZhsTlBXf4yevpCD2t2umfrI0dMaVPHNFwxR86zHnbRAVfiY8ygwVL90ObA=@vger.kernel.org X-Gm-Message-State: AOJu0YwVk8SB1ZE9sj2cL5hD6VfWC+ZkwXxUcHGJvoedh9Mju1hFPIfm BQz++BO2sjqHP5DuUhslhh8qQWouYLZ536eL25cxWQQWEwWi+njZUVcpYtO9D3VyvW4pWOo6niZ jHJkDkQ== X-Received: from plbi11.prod.google.com ([2002:a17:903:20cb:b0:2b2:4611:5de2]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:f7d1:b0:2b2:67ca:5fff with SMTP id d9443c01a7336-2bd7e9adfafmr8714765ad.28.1778795653789; Thu, 14 May 2026 14:54:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 14 May 2026 14:53:55 -0700 In-Reply-To: <20260514215355.1648463-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260514215355.1648463-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.563.g4f69b47b94-goog Message-ID: <20260514215355.1648463-16-seanjc@google.com> Subject: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Vitaly Kuznetsov , Kiryl Shutsemau , David Woodhouse , Paul Durrant Cc: Dave Hansen , Rick Edgecombe , kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yosry Ahmed , Kai Huang , Binbin Wu Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce regs.c, and move the vast majority of register specific code out of x86.c and into regs.c. Deliberately leave behind MSR code (except for EFER, which can hardly be called an MSR), as KVM's MSR support is complex enough to warrant its own compilation unit, and doesn't have much in common with the other register code. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 2 - arch/x86/kvm/Makefile | 4 +- arch/x86/kvm/regs.c | 829 ++++++++++++++++++++++++++++++++ arch/x86/kvm/regs.h | 16 + arch/x86/kvm/x86.c | 824 +------------------------------ arch/x86/kvm/x86.h | 2 + 6 files changed, 856 insertions(+), 821 deletions(-) create mode 100644 arch/x86/kvm/regs.c diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 271bdd109a98..5e24987b2a94 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2326,8 +2326,6 @@ static inline int __kvm_irq_line_state(unsigned long = *irq_state, void kvm_inject_nmi(struct kvm_vcpu *vcpu); int kvm_get_nr_pending_nmis(struct kvm_vcpu *vcpu); =20 -void kvm_update_dr7(struct kvm_vcpu *vcpu); - bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or= _gpa, bool always_retry); =20 diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 77337c37324b..f39c311fd756 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -5,8 +5,8 @@ ccflags-$(CONFIG_KVM_WERROR) +=3D -Werror =20 include $(srctree)/virt/kvm/Makefile.kvm =20 -kvm-y +=3D x86.o emulate.o irq.o lapic.o cpuid.o pmu.o mtrr.o \ - debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o +kvm-y +=3D x86.o emulate.o irq.o lapic.o cpuid.o pmu.o regs.o \ + mtrr.o debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o =20 kvm-$(CONFIG_X86_64) +=3D mmu/tdp_iter.o mmu/tdp_mmu.o kvm-$(CONFIG_KVM_IOAPIC) +=3D i8259.o i8254.o ioapic.o diff --git a/arch/x86/kvm/regs.c b/arch/x86/kvm/regs.c new file mode 100644 index 000000000000..ee8a97c31d78 --- /dev/null +++ b/arch/x86/kvm/regs.c @@ -0,0 +1,829 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include + +#include "lapic.h" +#include "mmu.h" +#include "regs.h" + +static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) +{ + if (vcpu->arch.emulate_regs_need_sync_to_vcpu) { + /* + * We are here if userspace calls get_regs() in the middle of + * instruction emulation. Registers state needs to be copied + * back from emulation context to vcpu. Userspace shouldn't do + * that usually, but some bad designed PV devices (vmware + * backdoor interface) need this to work + */ + emulator_writeback_register_cache(vcpu->arch.emulate_ctxt); + vcpu->arch.emulate_regs_need_sync_to_vcpu =3D false; + } + regs->rax =3D kvm_rax_read_raw(vcpu); + regs->rbx =3D kvm_rbx_read_raw(vcpu); + regs->rcx =3D kvm_rcx_read_raw(vcpu); + regs->rdx =3D kvm_rdx_read_raw(vcpu); + regs->rsi =3D kvm_rsi_read_raw(vcpu); + regs->rdi =3D kvm_rdi_read_raw(vcpu); + regs->rsp =3D kvm_rsp_read(vcpu); + regs->rbp =3D kvm_rbp_read_raw(vcpu); +#ifdef CONFIG_X86_64 + regs->r8 =3D kvm_r8_read_raw(vcpu); + regs->r9 =3D kvm_r9_read_raw(vcpu); + regs->r10 =3D kvm_r10_read_raw(vcpu); + regs->r11 =3D kvm_r11_read_raw(vcpu); + regs->r12 =3D kvm_r12_read_raw(vcpu); + regs->r13 =3D kvm_r13_read_raw(vcpu); + regs->r14 =3D kvm_r14_read_raw(vcpu); + regs->r15 =3D kvm_r15_read_raw(vcpu); +#endif + + regs->rip =3D kvm_rip_read(vcpu); + regs->rflags =3D kvm_get_rflags(vcpu); +} + +int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) +{ + if (vcpu->kvm->arch.has_protected_state && + vcpu->arch.guest_state_protected) + return -EINVAL; + + vcpu_load(vcpu); + __get_regs(vcpu, regs); + vcpu_put(vcpu); + return 0; +} + +static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) +{ + vcpu->arch.emulate_regs_need_sync_from_vcpu =3D true; + vcpu->arch.emulate_regs_need_sync_to_vcpu =3D false; + + kvm_rax_write_raw(vcpu, regs->rax); + kvm_rbx_write_raw(vcpu, regs->rbx); + kvm_rcx_write_raw(vcpu, regs->rcx); + kvm_rdx_write_raw(vcpu, regs->rdx); + kvm_rsi_write_raw(vcpu, regs->rsi); + kvm_rdi_write_raw(vcpu, regs->rdi); + kvm_rsp_write(vcpu, regs->rsp); + kvm_rbp_write_raw(vcpu, regs->rbp); +#ifdef CONFIG_X86_64 + kvm_r8_write_raw(vcpu, regs->r8); + kvm_r9_write_raw(vcpu, regs->r9); + kvm_r10_write_raw(vcpu, regs->r10); + kvm_r11_write_raw(vcpu, regs->r11); + kvm_r12_write_raw(vcpu, regs->r12); + kvm_r13_write_raw(vcpu, regs->r13); + kvm_r14_write_raw(vcpu, regs->r14); + kvm_r15_write_raw(vcpu, regs->r15); +#endif + + kvm_rip_write(vcpu, regs->rip); + kvm_set_rflags(vcpu, regs->rflags | X86_EFLAGS_FIXED); + + vcpu->arch.exception.pending =3D false; + vcpu->arch.exception_vmexit.pending =3D false; + + kvm_make_request(KVM_REQ_EVENT, vcpu); +} + +int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) +{ + if (vcpu->kvm->arch.has_protected_state && + vcpu->arch.guest_state_protected) + return -EINVAL; + + vcpu_load(vcpu); + __set_regs(vcpu, regs); + vcpu_put(vcpu); + return 0; +} + +static inline u64 pdptr_rsvd_bits(struct kvm_vcpu *vcpu) +{ + return vcpu->arch.reserved_gpa_bits | rsvd_bits(5, 8) | rsvd_bits(1, 2); +} + +/* + * Load the pae pdptrs. Return 1 if they are all valid, 0 otherwise. + */ +int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3) +{ + struct kvm_mmu *mmu =3D vcpu->arch.walk_mmu; + gfn_t pdpt_gfn =3D cr3 >> PAGE_SHIFT; + gpa_t real_gpa; + int i; + int ret; + u64 pdpte[ARRAY_SIZE(mmu->pdptrs)]; + + /* + * If the MMU is nested, CR3 holds an L2 GPA and needs to be translated + * to an L1 GPA. + */ + real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(pdpt_gfn), + PFERR_USER_MASK | PFERR_WRITE_MASK | + PFERR_GUEST_PAGE_MASK, NULL, 0); + if (real_gpa =3D=3D INVALID_GPA) + return 0; + + /* Note the offset, PDPTRs are 32 byte aligned when using PAE paging. */ + ret =3D kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(real_gpa), pdpte, + cr3 & GENMASK(11, 5), sizeof(pdpte)); + if (ret < 0) + return 0; + + for (i =3D 0; i < ARRAY_SIZE(pdpte); ++i) { + if ((pdpte[i] & PT_PRESENT_MASK) && + (pdpte[i] & pdptr_rsvd_bits(vcpu))) { + return 0; + } + } + + /* + * Marking VCPU_REG_PDPTR dirty doesn't work for !tdp_enabled. + * Shadow page roots need to be reconstructed instead. + */ + if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) + kvm_mmu_free_roots(vcpu->kvm, mmu, KVM_MMU_ROOT_CURRENT); + + memcpy(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs)); + kvm_register_mark_dirty(vcpu, VCPU_REG_PDPTR); + kvm_make_request(KVM_REQ_LOAD_MMU_PGD, vcpu); + vcpu->arch.pdptrs_from_userspace =3D false; + + return 1; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(load_pdptrs); + +static bool kvm_is_valid_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) +{ +#ifdef CONFIG_X86_64 + if (cr0 & 0xffffffff00000000UL) + return false; +#endif + + if ((cr0 & X86_CR0_NW) && !(cr0 & X86_CR0_CD)) + return false; + + if ((cr0 & X86_CR0_PG) && !(cr0 & X86_CR0_PE)) + return false; + + return kvm_x86_call(is_valid_cr0)(vcpu, cr0); +} + +void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsign= ed long cr0) +{ + /* + * CR0.WP is incorporated into the MMU role, but only for non-nested, + * indirect shadow MMUs. If paging is disabled, no updates are needed + * as there are no permission bits to emulate. If TDP is enabled, the + * MMU's metadata needs to be updated, e.g. so that emulating guest + * translations does the right thing, but there's no need to unload the + * root as CR0.WP doesn't affect SPTEs. + */ + if ((cr0 ^ old_cr0) =3D=3D X86_CR0_WP) { + if (!(cr0 & X86_CR0_PG)) + return; + + if (tdp_enabled) { + kvm_init_mmu(vcpu); + return; + } + } + + if ((cr0 ^ old_cr0) & X86_CR0_PG) { + /* + * Clearing CR0.PG is defined to flush the TLB from the guest's + * perspective. + */ + if (!(cr0 & X86_CR0_PG)) + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + /* + * Check for async #PF completion events when enabling paging, + * as the vCPU may have previously encountered async #PFs (it's + * entirely legal for the guest to toggle paging on/off without + * waiting for the async #PF queue to drain). + */ + else if (kvm_pv_async_pf_enabled(vcpu)) + kvm_make_request(KVM_REQ_APF_READY, vcpu); + } + + if ((cr0 ^ old_cr0) & KVM_MMU_CR0_ROLE_BITS) + kvm_mmu_reset_context(vcpu); +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_post_set_cr0); + +int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) +{ + unsigned long old_cr0 =3D kvm_read_cr0(vcpu); + + if (!kvm_is_valid_cr0(vcpu, cr0)) + return 1; + + cr0 |=3D X86_CR0_ET; + + /* Write to CR0 reserved bits are ignored, even on Intel. */ + cr0 &=3D ~CR0_RESERVED_BITS; + +#ifdef CONFIG_X86_64 + if ((vcpu->arch.efer & EFER_LME) && !is_paging(vcpu) && + (cr0 & X86_CR0_PG)) { + int cs_db, cs_l; + + if (!is_pae(vcpu)) + return 1; + kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); + if (cs_l) + return 1; + } +#endif + if (!(vcpu->arch.efer & EFER_LME) && (cr0 & X86_CR0_PG) && + is_pae(vcpu) && ((cr0 ^ old_cr0) & X86_CR0_PDPTR_BITS) && + !load_pdptrs(vcpu, kvm_read_cr3(vcpu))) + return 1; + + if (!(cr0 & X86_CR0_PG) && + (is_64_bit_mode(vcpu) || kvm_is_cr4_bit_set(vcpu, X86_CR4_PCIDE))) + return 1; + + if (!(cr0 & X86_CR0_WP) && kvm_is_cr4_bit_set(vcpu, X86_CR4_CET)) + return 1; + + kvm_x86_call(set_cr0)(vcpu, cr0); + + kvm_post_set_cr0(vcpu, old_cr0, cr0); + + return 0; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr0); + +void kvm_lmsw(struct kvm_vcpu *vcpu, unsigned long msw) +{ + (void)kvm_set_cr0(vcpu, kvm_read_cr0_bits(vcpu, ~0x0eul) | (msw & 0x0f)); +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_lmsw); + +int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) +{ + bool skip_tlb_flush =3D false; + unsigned long pcid =3D 0; +#ifdef CONFIG_X86_64 + if (kvm_is_cr4_bit_set(vcpu, X86_CR4_PCIDE)) { + skip_tlb_flush =3D cr3 & X86_CR3_PCID_NOFLUSH; + cr3 &=3D ~X86_CR3_PCID_NOFLUSH; + pcid =3D cr3 & X86_CR3_PCID_MASK; + } +#endif + + /* PDPTRs are always reloaded for PAE paging. */ + if (cr3 =3D=3D kvm_read_cr3(vcpu) && !is_pae_paging(vcpu)) + goto handle_tlb_flush; + + /* + * Do not condition the GPA check on long mode, this helper is used to + * stuff CR3, e.g. for RSM emulation, and there is no guarantee that + * the current vCPU mode is accurate. + */ + if (!kvm_vcpu_is_legal_cr3(vcpu, cr3)) + return 1; + + if (is_pae_paging(vcpu) && !load_pdptrs(vcpu, cr3)) + return 1; + + if (cr3 !=3D kvm_read_cr3(vcpu)) + kvm_mmu_new_pgd(vcpu, cr3); + + vcpu->arch.cr3 =3D cr3; + kvm_register_mark_dirty(vcpu, VCPU_REG_CR3); + /* Do not call post_set_cr3, we do not get here for confidential guests. = */ + +handle_tlb_flush: + /* + * A load of CR3 that flushes the TLB flushes only the current PCID, + * even if PCID is disabled, in which case PCID=3D0 is flushed. It's a + * moot point in the end because _disabling_ PCID will flush all PCIDs, + * and it's impossible to use a non-zero PCID when PCID is disabled, + * i.e. only PCID=3D0 can be relevant. + */ + if (!skip_tlb_flush) + kvm_invalidate_pcid(vcpu, pcid); + + return 0; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr3); + +static bool kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) +{ + return __kvm_is_valid_cr4(vcpu, cr4) && + kvm_x86_call(is_valid_cr4)(vcpu, cr4); +} + +void kvm_post_set_cr4(struct kvm_vcpu *vcpu, unsigned long old_cr4, unsign= ed long cr4) +{ + if ((cr4 ^ old_cr4) & KVM_MMU_CR4_ROLE_BITS) + kvm_mmu_reset_context(vcpu); + + /* + * If CR4.PCIDE is changed 0 -> 1, there is no need to flush the TLB + * according to the SDM; however, stale prev_roots could be reused + * incorrectly in the future after a MOV to CR3 with NOFLUSH=3D1, so we + * free them all. This is *not* a superset of KVM_REQ_TLB_FLUSH_GUEST + * or KVM_REQ_TLB_FLUSH_CURRENT, because the hardware TLB is not flushed, + * so fall through. + */ + if (!tdp_enabled && + (cr4 & X86_CR4_PCIDE) && !(old_cr4 & X86_CR4_PCIDE)) + kvm_mmu_unload(vcpu); + + /* + * The TLB has to be flushed for all PCIDs if any of the following + * (architecturally required) changes happen: + * - CR4.PCIDE is changed from 1 to 0 + * - CR4.PGE is toggled + * + * This is a superset of KVM_REQ_TLB_FLUSH_CURRENT. + */ + if (((cr4 ^ old_cr4) & X86_CR4_PGE) || + (!(cr4 & X86_CR4_PCIDE) && (old_cr4 & X86_CR4_PCIDE))) + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + + /* + * The TLB has to be flushed for the current PCID if any of the + * following (architecturally required) changes happen: + * - CR4.SMEP is changed from 0 to 1 + * - CR4.PAE is toggled + */ + else if (((cr4 ^ old_cr4) & X86_CR4_PAE) || + ((cr4 & X86_CR4_SMEP) && !(old_cr4 & X86_CR4_SMEP))) + kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); + +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_post_set_cr4); + +int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) +{ + unsigned long old_cr4 =3D kvm_read_cr4(vcpu); + + if (!kvm_is_valid_cr4(vcpu, cr4)) + return 1; + + if (is_long_mode(vcpu)) { + if (!(cr4 & X86_CR4_PAE)) + return 1; + if ((cr4 ^ old_cr4) & X86_CR4_LA57) + return 1; + } else if (is_paging(vcpu) && (cr4 & X86_CR4_PAE) + && ((cr4 ^ old_cr4) & X86_CR4_PDPTR_BITS) + && !load_pdptrs(vcpu, kvm_read_cr3(vcpu))) + return 1; + + if ((cr4 & X86_CR4_PCIDE) && !(old_cr4 & X86_CR4_PCIDE)) { + /* PCID can not be enabled when cr3[11:0]!=3D000H or EFER.LMA=3D0 */ + if ((kvm_read_cr3(vcpu) & X86_CR3_PCID_MASK) || !is_long_mode(vcpu)) + return 1; + } + + if ((cr4 & X86_CR4_CET) && !kvm_is_cr0_bit_set(vcpu, X86_CR0_WP)) + return 1; + + kvm_x86_call(set_cr4)(vcpu, cr4); + + kvm_post_set_cr4(vcpu, old_cr4, cr4); + + return 0; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr4); + +int kvm_set_cr8(struct kvm_vcpu *vcpu, unsigned long cr8) +{ + if (cr8 & CR8_RESERVED_BITS) + return 1; + if (lapic_in_kernel(vcpu)) + kvm_lapic_set_tpr(vcpu, cr8); + else + vcpu->arch.cr8 =3D cr8; + return 0; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr8); + +unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) +{ + if (lapic_in_kernel(vcpu)) + return kvm_lapic_get_cr8(vcpu); + else + return vcpu->arch.cr8; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_get_cr8); + +static void __get_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sr= egs) +{ + struct desc_ptr dt; + + if (vcpu->arch.guest_state_protected) + goto skip_protected_regs; + + kvm_handle_exception_payload_quirk(vcpu); + + kvm_get_segment(vcpu, &sregs->cs, VCPU_SREG_CS); + kvm_get_segment(vcpu, &sregs->ds, VCPU_SREG_DS); + kvm_get_segment(vcpu, &sregs->es, VCPU_SREG_ES); + kvm_get_segment(vcpu, &sregs->fs, VCPU_SREG_FS); + kvm_get_segment(vcpu, &sregs->gs, VCPU_SREG_GS); + kvm_get_segment(vcpu, &sregs->ss, VCPU_SREG_SS); + + kvm_get_segment(vcpu, &sregs->tr, VCPU_SREG_TR); + kvm_get_segment(vcpu, &sregs->ldt, VCPU_SREG_LDTR); + + kvm_x86_call(get_idt)(vcpu, &dt); + sregs->idt.limit =3D dt.size; + sregs->idt.base =3D dt.address; + kvm_x86_call(get_gdt)(vcpu, &dt); + sregs->gdt.limit =3D dt.size; + sregs->gdt.base =3D dt.address; + + sregs->cr2 =3D vcpu->arch.cr2; + sregs->cr3 =3D kvm_read_cr3(vcpu); + +skip_protected_regs: + sregs->cr0 =3D kvm_read_cr0(vcpu); + sregs->cr4 =3D kvm_read_cr4(vcpu); + sregs->cr8 =3D kvm_get_cr8(vcpu); + sregs->efer =3D vcpu->arch.efer; + sregs->apic_base =3D vcpu->arch.apic_base; +} + +static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) +{ + __get_sregs_common(vcpu, sregs); + + if (vcpu->arch.guest_state_protected) + return; + + if (vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft) + set_bit(vcpu->arch.interrupt.nr, + (unsigned long *)sregs->interrupt_bitmap); +} + +int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, + struct kvm_sregs *sregs) +{ + if (vcpu->kvm->arch.has_protected_state && + vcpu->arch.guest_state_protected) + return -EINVAL; + + vcpu_load(vcpu); + __get_sregs(vcpu, sregs); + vcpu_put(vcpu); + return 0; +} + +void kvm_x86_vcpu_ioctl_get_sregs2(struct kvm_vcpu *vcpu, + struct kvm_sregs2 *sregs2) +{ + int i; + + __get_sregs_common(vcpu, (struct kvm_sregs *)sregs2); + + if (vcpu->arch.guest_state_protected) + return; + + if (is_pae_paging(vcpu)) { + kvm_vcpu_srcu_read_lock(vcpu); + for (i =3D 0 ; i < 4 ; i++) + sregs2->pdptrs[i] =3D kvm_pdptr_read(vcpu, i); + sregs2->flags |=3D KVM_SREGS2_FLAGS_PDPTRS_VALID; + kvm_vcpu_srcu_read_unlock(vcpu); + } +} + +static bool kvm_is_valid_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sr= egs) +{ + if ((sregs->efer & EFER_LME) && (sregs->cr0 & X86_CR0_PG)) { + /* + * When EFER.LME and CR0.PG are set, the processor is in + * 64-bit mode (though maybe in a 32-bit code segment). + * CR4.PAE and EFER.LMA must be set. + */ + if (!(sregs->cr4 & X86_CR4_PAE) || !(sregs->efer & EFER_LMA)) + return false; + if (!kvm_vcpu_is_legal_cr3(vcpu, sregs->cr3)) + return false; + } else { + /* + * Not in 64-bit mode: EFER.LMA is clear and the code + * segment cannot be 64-bit. + */ + if (sregs->efer & EFER_LMA || sregs->cs.l) + return false; + } + + return kvm_is_valid_cr4(vcpu, sregs->cr4) && + kvm_is_valid_cr0(vcpu, sregs->cr0); +} + +static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sre= gs, + int *mmu_reset_needed, bool update_pdptrs) +{ + int idx; + struct desc_ptr dt; + + if (!kvm_is_valid_sregs(vcpu, sregs)) + return -EINVAL; + + if (kvm_apic_set_base(vcpu, sregs->apic_base, true)) + return -EINVAL; + + if (vcpu->arch.guest_state_protected) + return 0; + + dt.size =3D sregs->idt.limit; + dt.address =3D sregs->idt.base; + kvm_x86_call(set_idt)(vcpu, &dt); + dt.size =3D sregs->gdt.limit; + dt.address =3D sregs->gdt.base; + kvm_x86_call(set_gdt)(vcpu, &dt); + + vcpu->arch.cr2 =3D sregs->cr2; + *mmu_reset_needed |=3D kvm_read_cr3(vcpu) !=3D sregs->cr3; + vcpu->arch.cr3 =3D sregs->cr3; + kvm_register_mark_dirty(vcpu, VCPU_REG_CR3); + kvm_x86_call(post_set_cr3)(vcpu, sregs->cr3); + + kvm_set_cr8(vcpu, sregs->cr8); + + *mmu_reset_needed |=3D vcpu->arch.efer !=3D sregs->efer; + kvm_x86_call(set_efer)(vcpu, sregs->efer); + + *mmu_reset_needed |=3D kvm_read_cr0(vcpu) !=3D sregs->cr0; + kvm_x86_call(set_cr0)(vcpu, sregs->cr0); + + *mmu_reset_needed |=3D kvm_read_cr4(vcpu) !=3D sregs->cr4; + kvm_x86_call(set_cr4)(vcpu, sregs->cr4); + + if (update_pdptrs) { + idx =3D srcu_read_lock(&vcpu->kvm->srcu); + if (is_pae_paging(vcpu)) { + load_pdptrs(vcpu, kvm_read_cr3(vcpu)); + *mmu_reset_needed =3D 1; + } + srcu_read_unlock(&vcpu->kvm->srcu, idx); + } + + kvm_set_segment(vcpu, &sregs->cs, VCPU_SREG_CS); + kvm_set_segment(vcpu, &sregs->ds, VCPU_SREG_DS); + kvm_set_segment(vcpu, &sregs->es, VCPU_SREG_ES); + kvm_set_segment(vcpu, &sregs->fs, VCPU_SREG_FS); + kvm_set_segment(vcpu, &sregs->gs, VCPU_SREG_GS); + kvm_set_segment(vcpu, &sregs->ss, VCPU_SREG_SS); + + kvm_set_segment(vcpu, &sregs->tr, VCPU_SREG_TR); + kvm_set_segment(vcpu, &sregs->ldt, VCPU_SREG_LDTR); + + kvm_lapic_update_cr8_intercept(vcpu); + + /* Older userspace won't unhalt the vcpu on reset. */ + if (kvm_vcpu_is_bsp(vcpu) && kvm_rip_read(vcpu) =3D=3D 0xfff0 && + sregs->cs.selector =3D=3D 0xf000 && sregs->cs.base =3D=3D 0xffff0000 = && + !is_protmode(vcpu)) + kvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE); + + return 0; +} + +static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) +{ + int pending_vec, max_bits; + int mmu_reset_needed =3D 0; + int ret =3D __set_sregs_common(vcpu, sregs, &mmu_reset_needed, true); + + if (ret) + return ret; + + if (mmu_reset_needed) { + kvm_mmu_reset_context(vcpu); + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + } + + max_bits =3D KVM_NR_INTERRUPTS; + pending_vec =3D find_first_bit( + (const unsigned long *)sregs->interrupt_bitmap, max_bits); + + if (pending_vec < max_bits) { + kvm_queue_interrupt(vcpu, pending_vec, false); + pr_debug("Set back pending irq %d\n", pending_vec); + kvm_make_request(KVM_REQ_EVENT, vcpu); + } + return 0; +} + +int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, + struct kvm_sregs *sregs) +{ + int ret; + + if (vcpu->kvm->arch.has_protected_state && + vcpu->arch.guest_state_protected) + return -EINVAL; + + vcpu_load(vcpu); + ret =3D __set_sregs(vcpu, sregs); + vcpu_put(vcpu); + return ret; +} + +int kvm_x86_vcpu_ioctl_set_sregs2(struct kvm_vcpu *vcpu, + struct kvm_sregs2 *sregs2) +{ + int mmu_reset_needed =3D 0; + bool valid_pdptrs =3D sregs2->flags & KVM_SREGS2_FLAGS_PDPTRS_VALID; + bool pae =3D (sregs2->cr0 & X86_CR0_PG) && (sregs2->cr4 & X86_CR4_PAE) && + !(sregs2->efer & EFER_LMA); + int i, ret; + + if (sregs2->flags & ~KVM_SREGS2_FLAGS_PDPTRS_VALID) + return -EINVAL; + + if (valid_pdptrs && (!pae || vcpu->arch.guest_state_protected)) + return -EINVAL; + + ret =3D __set_sregs_common(vcpu, (struct kvm_sregs *)sregs2, + &mmu_reset_needed, !valid_pdptrs); + if (ret) + return ret; + + if (valid_pdptrs) { + for (i =3D 0; i < 4 ; i++) + kvm_pdptr_write(vcpu, i, sregs2->pdptrs[i]); + + kvm_register_mark_dirty(vcpu, VCPU_REG_PDPTR); + mmu_reset_needed =3D 1; + vcpu->arch.pdptrs_from_userspace =3D true; + } + if (mmu_reset_needed) { + kvm_mmu_reset_context(vcpu); + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + } + return 0; +} + +void kvm_run_get_regs(struct kvm_vcpu *vcpu) +{ + BUILD_BUG_ON(sizeof(struct kvm_sync_regs) > SYNC_REGS_SIZE_BYTES); + + if (vcpu->run->kvm_valid_regs & KVM_SYNC_X86_REGS) + __get_regs(vcpu, &vcpu->run->s.regs.regs); + + if (vcpu->run->kvm_valid_regs & KVM_SYNC_X86_SREGS) + __get_sregs(vcpu, &vcpu->run->s.regs.sregs); +} + +int kvm_run_set_regs(struct kvm_vcpu *vcpu) +{ + if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_REGS) { + __set_regs(vcpu, &vcpu->run->s.regs.regs); + vcpu->run->kvm_dirty_regs &=3D ~KVM_SYNC_X86_REGS; + } + + if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_SREGS) { + struct kvm_sregs sregs =3D vcpu->run->s.regs.sregs; + + if (__set_sregs(vcpu, &sregs)) + return -EINVAL; + + vcpu->run->kvm_dirty_regs &=3D ~KVM_SYNC_X86_SREGS; + } + + return 0; +} + +void kvm_update_dr0123(struct kvm_vcpu *vcpu) +{ + int i; + + if (!(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP)) { + for (i =3D 0; i < KVM_NR_DB_REGS; i++) + vcpu->arch.eff_db[i] =3D vcpu->arch.db[i]; + } +} + +void kvm_update_dr7(struct kvm_vcpu *vcpu) +{ + unsigned long dr7; + + if (vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP) + dr7 =3D vcpu->arch.guest_debug_dr7; + else + dr7 =3D vcpu->arch.dr7; + kvm_x86_call(set_dr7)(vcpu, dr7); + vcpu->arch.switch_db_regs &=3D ~KVM_DEBUGREG_BP_ENABLED; + if (dr7 & DR7_BP_EN_MASK) + vcpu->arch.switch_db_regs |=3D KVM_DEBUGREG_BP_ENABLED; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_update_dr7); + +static u64 kvm_dr6_fixed(struct kvm_vcpu *vcpu) +{ + u64 fixed =3D DR6_FIXED_1; + + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_RTM)) + fixed |=3D DR6_RTM; + + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_BUS_LOCK_DETECT)) + fixed |=3D DR6_BUS_LOCK; + return fixed; +} + +int kvm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long val) +{ + size_t size =3D ARRAY_SIZE(vcpu->arch.db); + + switch (dr) { + case 0 ... 3: + vcpu->arch.db[array_index_nospec(dr, size)] =3D val; + if (!(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP)) + vcpu->arch.eff_db[dr] =3D val; + break; + case 4: + case 6: + if (!kvm_dr6_valid(val)) + return 1; /* #GP */ + vcpu->arch.dr6 =3D (val & DR6_VOLATILE) | kvm_dr6_fixed(vcpu); + break; + case 5: + default: /* 7 */ + if (!kvm_dr7_valid(val)) + return 1; /* #GP */ + vcpu->arch.dr7 =3D (val & DR7_VOLATILE) | DR7_FIXED_1; + kvm_update_dr7(vcpu); + break; + } + + return 0; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_dr); + +unsigned long kvm_get_dr(struct kvm_vcpu *vcpu, int dr) +{ + size_t size =3D ARRAY_SIZE(vcpu->arch.db); + + switch (dr) { + case 0 ... 3: + return vcpu->arch.db[array_index_nospec(dr, size)]; + case 4: + case 6: + return vcpu->arch.dr6; + case 5: + default: /* 7 */ + return vcpu->arch.dr7; + } +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_get_dr); + +int kvm_vcpu_ioctl_x86_get_debugregs(struct kvm_vcpu *vcpu, + struct kvm_debugregs *dbgregs) +{ + unsigned int i; + + if (vcpu->kvm->arch.has_protected_state && + vcpu->arch.guest_state_protected) + return -EINVAL; + + kvm_handle_exception_payload_quirk(vcpu); + + memset(dbgregs, 0, sizeof(*dbgregs)); + + BUILD_BUG_ON(ARRAY_SIZE(vcpu->arch.db) !=3D ARRAY_SIZE(dbgregs->db)); + for (i =3D 0; i < ARRAY_SIZE(vcpu->arch.db); i++) + dbgregs->db[i] =3D vcpu->arch.db[i]; + + dbgregs->dr6 =3D vcpu->arch.dr6; + dbgregs->dr7 =3D vcpu->arch.dr7; + return 0; +} + +int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, + struct kvm_debugregs *dbgregs) +{ + unsigned int i; + + if (vcpu->kvm->arch.has_protected_state && + vcpu->arch.guest_state_protected) + return -EINVAL; + + if (dbgregs->flags) + return -EINVAL; + + if (!kvm_dr6_valid(dbgregs->dr6)) + return -EINVAL; + if (!kvm_dr7_valid(dbgregs->dr7)) + return -EINVAL; + + for (i =3D 0; i < ARRAY_SIZE(vcpu->arch.db); i++) + vcpu->arch.db[i] =3D dbgregs->db[i]; + + kvm_update_dr0123(vcpu); + vcpu->arch.dr6 =3D dbgregs->dr6; + vcpu->arch.dr7 =3D dbgregs->dr7; + kvm_update_dr7(vcpu); + + return 0; +} diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h index d4d2a47a4968..875a1b66d67a 100644 --- a/arch/x86/kvm/regs.h +++ b/arch/x86/kvm/regs.h @@ -401,4 +401,20 @@ static inline bool is_guest_mode(struct kvm_vcpu *vcpu) return vcpu->arch.hflags & HF_GUEST_MASK; } =20 +void kvm_x86_vcpu_ioctl_get_sregs2(struct kvm_vcpu *vcpu, + struct kvm_sregs2 *sregs2); +int kvm_x86_vcpu_ioctl_set_sregs2(struct kvm_vcpu *vcpu, + struct kvm_sregs2 *sregs2); + +void kvm_run_get_regs(struct kvm_vcpu *vcpu); +int kvm_run_set_regs(struct kvm_vcpu *vcpu); + +void kvm_update_dr0123(struct kvm_vcpu *vcpu); +void kvm_update_dr7(struct kvm_vcpu *vcpu); +int kvm_vcpu_ioctl_x86_get_debugregs(struct kvm_vcpu *vcpu, + struct kvm_debugregs *dbgregs); +int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, + struct kvm_debugregs *dbgregs); + + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e664e874973b..4ba1e329ac68 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -134,9 +134,6 @@ static void store_regs(struct kvm_vcpu *vcpu); static int sync_regs(struct kvm_vcpu *vcpu); static int kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu); =20 -static int __set_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2); -static void __get_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2); - static DEFINE_MUTEX(vendor_module_lock); static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu); static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu); @@ -1042,170 +1039,6 @@ bool kvm_require_dr(struct kvm_vcpu *vcpu, int dr) } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_require_dr); =20 -static inline u64 pdptr_rsvd_bits(struct kvm_vcpu *vcpu) -{ - return vcpu->arch.reserved_gpa_bits | rsvd_bits(5, 8) | rsvd_bits(1, 2); -} - -/* - * Load the pae pdptrs. Return 1 if they are all valid, 0 otherwise. - */ -int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3) -{ - struct kvm_mmu *mmu =3D vcpu->arch.walk_mmu; - gfn_t pdpt_gfn =3D cr3 >> PAGE_SHIFT; - gpa_t real_gpa; - int i; - int ret; - u64 pdpte[ARRAY_SIZE(mmu->pdptrs)]; - - /* - * If the MMU is nested, CR3 holds an L2 GPA and needs to be translated - * to an L1 GPA. - */ - real_gpa =3D kvm_translate_gpa(vcpu, mmu, gfn_to_gpa(pdpt_gfn), - PFERR_USER_MASK | PFERR_WRITE_MASK | - PFERR_GUEST_PAGE_MASK, NULL, 0); - if (real_gpa =3D=3D INVALID_GPA) - return 0; - - /* Note the offset, PDPTRs are 32 byte aligned when using PAE paging. */ - ret =3D kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(real_gpa), pdpte, - cr3 & GENMASK(11, 5), sizeof(pdpte)); - if (ret < 0) - return 0; - - for (i =3D 0; i < ARRAY_SIZE(pdpte); ++i) { - if ((pdpte[i] & PT_PRESENT_MASK) && - (pdpte[i] & pdptr_rsvd_bits(vcpu))) { - return 0; - } - } - - /* - * Marking VCPU_REG_PDPTR dirty doesn't work for !tdp_enabled. - * Shadow page roots need to be reconstructed instead. - */ - if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) - kvm_mmu_free_roots(vcpu->kvm, mmu, KVM_MMU_ROOT_CURRENT); - - memcpy(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs)); - kvm_register_mark_dirty(vcpu, VCPU_REG_PDPTR); - kvm_make_request(KVM_REQ_LOAD_MMU_PGD, vcpu); - vcpu->arch.pdptrs_from_userspace =3D false; - - return 1; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(load_pdptrs); - -static bool kvm_is_valid_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) -{ -#ifdef CONFIG_X86_64 - if (cr0 & 0xffffffff00000000UL) - return false; -#endif - - if ((cr0 & X86_CR0_NW) && !(cr0 & X86_CR0_CD)) - return false; - - if ((cr0 & X86_CR0_PG) && !(cr0 & X86_CR0_PE)) - return false; - - return kvm_x86_call(is_valid_cr0)(vcpu, cr0); -} - -void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsign= ed long cr0) -{ - /* - * CR0.WP is incorporated into the MMU role, but only for non-nested, - * indirect shadow MMUs. If paging is disabled, no updates are needed - * as there are no permission bits to emulate. If TDP is enabled, the - * MMU's metadata needs to be updated, e.g. so that emulating guest - * translations does the right thing, but there's no need to unload the - * root as CR0.WP doesn't affect SPTEs. - */ - if ((cr0 ^ old_cr0) =3D=3D X86_CR0_WP) { - if (!(cr0 & X86_CR0_PG)) - return; - - if (tdp_enabled) { - kvm_init_mmu(vcpu); - return; - } - } - - if ((cr0 ^ old_cr0) & X86_CR0_PG) { - /* - * Clearing CR0.PG is defined to flush the TLB from the guest's - * perspective. - */ - if (!(cr0 & X86_CR0_PG)) - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); - /* - * Check for async #PF completion events when enabling paging, - * as the vCPU may have previously encountered async #PFs (it's - * entirely legal for the guest to toggle paging on/off without - * waiting for the async #PF queue to drain). - */ - else if (kvm_pv_async_pf_enabled(vcpu)) - kvm_make_request(KVM_REQ_APF_READY, vcpu); - } - - if ((cr0 ^ old_cr0) & KVM_MMU_CR0_ROLE_BITS) - kvm_mmu_reset_context(vcpu); -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_post_set_cr0); - -int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) -{ - unsigned long old_cr0 =3D kvm_read_cr0(vcpu); - - if (!kvm_is_valid_cr0(vcpu, cr0)) - return 1; - - cr0 |=3D X86_CR0_ET; - - /* Write to CR0 reserved bits are ignored, even on Intel. */ - cr0 &=3D ~CR0_RESERVED_BITS; - -#ifdef CONFIG_X86_64 - if ((vcpu->arch.efer & EFER_LME) && !is_paging(vcpu) && - (cr0 & X86_CR0_PG)) { - int cs_db, cs_l; - - if (!is_pae(vcpu)) - return 1; - kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); - if (cs_l) - return 1; - } -#endif - if (!(vcpu->arch.efer & EFER_LME) && (cr0 & X86_CR0_PG) && - is_pae(vcpu) && ((cr0 ^ old_cr0) & X86_CR0_PDPTR_BITS) && - !load_pdptrs(vcpu, kvm_read_cr3(vcpu))) - return 1; - - if (!(cr0 & X86_CR0_PG) && - (is_64_bit_mode(vcpu) || kvm_is_cr4_bit_set(vcpu, X86_CR4_PCIDE))) - return 1; - - if (!(cr0 & X86_CR0_WP) && kvm_is_cr4_bit_set(vcpu, X86_CR4_CET)) - return 1; - - kvm_x86_call(set_cr0)(vcpu, cr0); - - kvm_post_set_cr0(vcpu, old_cr0, cr0); - - return 0; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr0); - -void kvm_lmsw(struct kvm_vcpu *vcpu, unsigned long msw) -{ - (void)kvm_set_cr0(vcpu, kvm_read_cr0_bits(vcpu, ~0x0eul) | (msw & 0x0f)); -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_lmsw); - static void kvm_load_xfeatures(struct kvm_vcpu *vcpu, bool load_guest) { if (vcpu->arch.guest_state_protected) @@ -1315,89 +1148,7 @@ int kvm_emulate_xsetbv(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_emulate_xsetbv); =20 -static bool kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) -{ - return __kvm_is_valid_cr4(vcpu, cr4) && - kvm_x86_call(is_valid_cr4)(vcpu, cr4); -} - -void kvm_post_set_cr4(struct kvm_vcpu *vcpu, unsigned long old_cr4, unsign= ed long cr4) -{ - if ((cr4 ^ old_cr4) & KVM_MMU_CR4_ROLE_BITS) - kvm_mmu_reset_context(vcpu); - - /* - * If CR4.PCIDE is changed 0 -> 1, there is no need to flush the TLB - * according to the SDM; however, stale prev_roots could be reused - * incorrectly in the future after a MOV to CR3 with NOFLUSH=3D1, so we - * free them all. This is *not* a superset of KVM_REQ_TLB_FLUSH_GUEST - * or KVM_REQ_TLB_FLUSH_CURRENT, because the hardware TLB is not flushed, - * so fall through. - */ - if (!tdp_enabled && - (cr4 & X86_CR4_PCIDE) && !(old_cr4 & X86_CR4_PCIDE)) - kvm_mmu_unload(vcpu); - - /* - * The TLB has to be flushed for all PCIDs if any of the following - * (architecturally required) changes happen: - * - CR4.PCIDE is changed from 1 to 0 - * - CR4.PGE is toggled - * - * This is a superset of KVM_REQ_TLB_FLUSH_CURRENT. - */ - if (((cr4 ^ old_cr4) & X86_CR4_PGE) || - (!(cr4 & X86_CR4_PCIDE) && (old_cr4 & X86_CR4_PCIDE))) - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); - - /* - * The TLB has to be flushed for the current PCID if any of the - * following (architecturally required) changes happen: - * - CR4.SMEP is changed from 0 to 1 - * - CR4.PAE is toggled - */ - else if (((cr4 ^ old_cr4) & X86_CR4_PAE) || - ((cr4 & X86_CR4_SMEP) && !(old_cr4 & X86_CR4_SMEP))) - kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); - -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_post_set_cr4); - -int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) -{ - unsigned long old_cr4 =3D kvm_read_cr4(vcpu); - - if (!kvm_is_valid_cr4(vcpu, cr4)) - return 1; - - if (is_long_mode(vcpu)) { - if (!(cr4 & X86_CR4_PAE)) - return 1; - if ((cr4 ^ old_cr4) & X86_CR4_LA57) - return 1; - } else if (is_paging(vcpu) && (cr4 & X86_CR4_PAE) - && ((cr4 ^ old_cr4) & X86_CR4_PDPTR_BITS) - && !load_pdptrs(vcpu, kvm_read_cr3(vcpu))) - return 1; - - if ((cr4 & X86_CR4_PCIDE) && !(old_cr4 & X86_CR4_PCIDE)) { - /* PCID can not be enabled when cr3[11:0]!=3D000H or EFER.LMA=3D0 */ - if ((kvm_read_cr3(vcpu) & X86_CR3_PCID_MASK) || !is_long_mode(vcpu)) - return 1; - } - - if ((cr4 & X86_CR4_CET) && !kvm_is_cr0_bit_set(vcpu, X86_CR0_WP)) - return 1; - - kvm_x86_call(set_cr4)(vcpu, cr4); - - kvm_post_set_cr4(vcpu, old_cr4, cr4); - - return 0; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr4); - -static void kvm_invalidate_pcid(struct kvm_vcpu *vcpu, unsigned long pcid) +void kvm_invalidate_pcid(struct kvm_vcpu *vcpu, unsigned long pcid) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; unsigned long roots_to_free =3D 0; @@ -1440,159 +1191,6 @@ static void kvm_invalidate_pcid(struct kvm_vcpu *vc= pu, unsigned long pcid) kvm_mmu_free_roots(vcpu->kvm, mmu, roots_to_free); } =20 -int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) -{ - bool skip_tlb_flush =3D false; - unsigned long pcid =3D 0; -#ifdef CONFIG_X86_64 - if (kvm_is_cr4_bit_set(vcpu, X86_CR4_PCIDE)) { - skip_tlb_flush =3D cr3 & X86_CR3_PCID_NOFLUSH; - cr3 &=3D ~X86_CR3_PCID_NOFLUSH; - pcid =3D cr3 & X86_CR3_PCID_MASK; - } -#endif - - /* PDPTRs are always reloaded for PAE paging. */ - if (cr3 =3D=3D kvm_read_cr3(vcpu) && !is_pae_paging(vcpu)) - goto handle_tlb_flush; - - /* - * Do not condition the GPA check on long mode, this helper is used to - * stuff CR3, e.g. for RSM emulation, and there is no guarantee that - * the current vCPU mode is accurate. - */ - if (!kvm_vcpu_is_legal_cr3(vcpu, cr3)) - return 1; - - if (is_pae_paging(vcpu) && !load_pdptrs(vcpu, cr3)) - return 1; - - if (cr3 !=3D kvm_read_cr3(vcpu)) - kvm_mmu_new_pgd(vcpu, cr3); - - vcpu->arch.cr3 =3D cr3; - kvm_register_mark_dirty(vcpu, VCPU_REG_CR3); - /* Do not call post_set_cr3, we do not get here for confidential guests. = */ - -handle_tlb_flush: - /* - * A load of CR3 that flushes the TLB flushes only the current PCID, - * even if PCID is disabled, in which case PCID=3D0 is flushed. It's a - * moot point in the end because _disabling_ PCID will flush all PCIDs, - * and it's impossible to use a non-zero PCID when PCID is disabled, - * i.e. only PCID=3D0 can be relevant. - */ - if (!skip_tlb_flush) - kvm_invalidate_pcid(vcpu, pcid); - - return 0; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr3); - -int kvm_set_cr8(struct kvm_vcpu *vcpu, unsigned long cr8) -{ - if (cr8 & CR8_RESERVED_BITS) - return 1; - if (lapic_in_kernel(vcpu)) - kvm_lapic_set_tpr(vcpu, cr8); - else - vcpu->arch.cr8 =3D cr8; - return 0; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cr8); - -unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) -{ - if (lapic_in_kernel(vcpu)) - return kvm_lapic_get_cr8(vcpu); - else - return vcpu->arch.cr8; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_get_cr8); - -static void kvm_update_dr0123(struct kvm_vcpu *vcpu) -{ - int i; - - if (!(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP)) { - for (i =3D 0; i < KVM_NR_DB_REGS; i++) - vcpu->arch.eff_db[i] =3D vcpu->arch.db[i]; - } -} - -void kvm_update_dr7(struct kvm_vcpu *vcpu) -{ - unsigned long dr7; - - if (vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP) - dr7 =3D vcpu->arch.guest_debug_dr7; - else - dr7 =3D vcpu->arch.dr7; - kvm_x86_call(set_dr7)(vcpu, dr7); - vcpu->arch.switch_db_regs &=3D ~KVM_DEBUGREG_BP_ENABLED; - if (dr7 & DR7_BP_EN_MASK) - vcpu->arch.switch_db_regs |=3D KVM_DEBUGREG_BP_ENABLED; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_update_dr7); - -static u64 kvm_dr6_fixed(struct kvm_vcpu *vcpu) -{ - u64 fixed =3D DR6_FIXED_1; - - if (!guest_cpu_cap_has(vcpu, X86_FEATURE_RTM)) - fixed |=3D DR6_RTM; - - if (!guest_cpu_cap_has(vcpu, X86_FEATURE_BUS_LOCK_DETECT)) - fixed |=3D DR6_BUS_LOCK; - return fixed; -} - -int kvm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long val) -{ - size_t size =3D ARRAY_SIZE(vcpu->arch.db); - - switch (dr) { - case 0 ... 3: - vcpu->arch.db[array_index_nospec(dr, size)] =3D val; - if (!(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP)) - vcpu->arch.eff_db[dr] =3D val; - break; - case 4: - case 6: - if (!kvm_dr6_valid(val)) - return 1; /* #GP */ - vcpu->arch.dr6 =3D (val & DR6_VOLATILE) | kvm_dr6_fixed(vcpu); - break; - case 5: - default: /* 7 */ - if (!kvm_dr7_valid(val)) - return 1; /* #GP */ - vcpu->arch.dr7 =3D (val & DR7_VOLATILE) | DR7_FIXED_1; - kvm_update_dr7(vcpu); - break; - } - - return 0; -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_dr); - -unsigned long kvm_get_dr(struct kvm_vcpu *vcpu, int dr) -{ - size_t size =3D ARRAY_SIZE(vcpu->arch.db); - - switch (dr) { - case 0 ... 3: - return vcpu->arch.db[array_index_nospec(dr, size)]; - case 4: - case 6: - return vcpu->arch.dr6; - case 5: - default: /* 7 */ - return vcpu->arch.dr7; - } -} -EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_get_dr); - int kvm_emulate_rdpmc(struct kvm_vcpu *vcpu) { u32 pmc =3D kvm_ecx_read(vcpu); @@ -5544,7 +5142,7 @@ static struct kvm_queued_exception *kvm_get_exception= _to_save(struct kvm_vcpu *v return &vcpu->arch.exception; } =20 -static void kvm_handle_exception_payload_quirk(struct kvm_vcpu *vcpu) +void kvm_handle_exception_payload_quirk(struct kvm_vcpu *vcpu) { struct kvm_queued_exception *ex =3D kvm_get_exception_to_save(vcpu); =20 @@ -5748,57 +5346,6 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct= kvm_vcpu *vcpu, return 0; } =20 -static int kvm_vcpu_ioctl_x86_get_debugregs(struct kvm_vcpu *vcpu, - struct kvm_debugregs *dbgregs) -{ - unsigned int i; - - if (vcpu->kvm->arch.has_protected_state && - vcpu->arch.guest_state_protected) - return -EINVAL; - - kvm_handle_exception_payload_quirk(vcpu); - - memset(dbgregs, 0, sizeof(*dbgregs)); - - BUILD_BUG_ON(ARRAY_SIZE(vcpu->arch.db) !=3D ARRAY_SIZE(dbgregs->db)); - for (i =3D 0; i < ARRAY_SIZE(vcpu->arch.db); i++) - dbgregs->db[i] =3D vcpu->arch.db[i]; - - dbgregs->dr6 =3D vcpu->arch.dr6; - dbgregs->dr7 =3D vcpu->arch.dr7; - return 0; -} - -static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, - struct kvm_debugregs *dbgregs) -{ - unsigned int i; - - if (vcpu->kvm->arch.has_protected_state && - vcpu->arch.guest_state_protected) - return -EINVAL; - - if (dbgregs->flags) - return -EINVAL; - - if (!kvm_dr6_valid(dbgregs->dr6)) - return -EINVAL; - if (!kvm_dr7_valid(dbgregs->dr7)) - return -EINVAL; - - for (i =3D 0; i < ARRAY_SIZE(vcpu->arch.db); i++) - vcpu->arch.db[i] =3D dbgregs->db[i]; - - kvm_update_dr0123(vcpu); - vcpu->arch.dr6 =3D dbgregs->dr6; - vcpu->arch.dr7 =3D dbgregs->dr7; - kvm_update_dr7(vcpu); - - return 0; -} - - static int kvm_vcpu_ioctl_x86_get_xsave2(struct kvm_vcpu *vcpu, u8 *state, unsigned int size) { @@ -6635,7 +6182,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp, r =3D -ENOMEM; if (!u.sregs2) goto out; - __get_sregs2(vcpu, u.sregs2); + kvm_x86_vcpu_ioctl_get_sregs2(vcpu, u.sregs2); r =3D -EFAULT; if (copy_to_user(argp, u.sregs2, sizeof(struct kvm_sregs2))) goto out; @@ -6654,7 +6201,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp, u.sregs2 =3D NULL; goto out; } - r =3D __set_sregs2(vcpu, u.sregs2); + r =3D kvm_x86_vcpu_ioctl_set_sregs2(vcpu, u.sregs2); break; } case KVM_HAS_DEVICE_ATTR: @@ -12081,179 +11628,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) return r; } =20 -static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) -{ - if (vcpu->arch.emulate_regs_need_sync_to_vcpu) { - /* - * We are here if userspace calls get_regs() in the middle of - * instruction emulation. Registers state needs to be copied - * back from emulation context to vcpu. Userspace shouldn't do - * that usually, but some bad designed PV devices (vmware - * backdoor interface) need this to work - */ - emulator_writeback_register_cache(vcpu->arch.emulate_ctxt); - vcpu->arch.emulate_regs_need_sync_to_vcpu =3D false; - } - regs->rax =3D kvm_rax_read_raw(vcpu); - regs->rbx =3D kvm_rbx_read_raw(vcpu); - regs->rcx =3D kvm_rcx_read_raw(vcpu); - regs->rdx =3D kvm_rdx_read_raw(vcpu); - regs->rsi =3D kvm_rsi_read_raw(vcpu); - regs->rdi =3D kvm_rdi_read_raw(vcpu); - regs->rsp =3D kvm_rsp_read(vcpu); - regs->rbp =3D kvm_rbp_read_raw(vcpu); -#ifdef CONFIG_X86_64 - regs->r8 =3D kvm_r8_read_raw(vcpu); - regs->r9 =3D kvm_r9_read_raw(vcpu); - regs->r10 =3D kvm_r10_read_raw(vcpu); - regs->r11 =3D kvm_r11_read_raw(vcpu); - regs->r12 =3D kvm_r12_read_raw(vcpu); - regs->r13 =3D kvm_r13_read_raw(vcpu); - regs->r14 =3D kvm_r14_read_raw(vcpu); - regs->r15 =3D kvm_r15_read_raw(vcpu); -#endif - - regs->rip =3D kvm_rip_read(vcpu); - regs->rflags =3D kvm_get_rflags(vcpu); -} - -int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) -{ - if (vcpu->kvm->arch.has_protected_state && - vcpu->arch.guest_state_protected) - return -EINVAL; - - vcpu_load(vcpu); - __get_regs(vcpu, regs); - vcpu_put(vcpu); - return 0; -} - -static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) -{ - vcpu->arch.emulate_regs_need_sync_from_vcpu =3D true; - vcpu->arch.emulate_regs_need_sync_to_vcpu =3D false; - - kvm_rax_write_raw(vcpu, regs->rax); - kvm_rbx_write_raw(vcpu, regs->rbx); - kvm_rcx_write_raw(vcpu, regs->rcx); - kvm_rdx_write_raw(vcpu, regs->rdx); - kvm_rsi_write_raw(vcpu, regs->rsi); - kvm_rdi_write_raw(vcpu, regs->rdi); - kvm_rsp_write(vcpu, regs->rsp); - kvm_rbp_write_raw(vcpu, regs->rbp); -#ifdef CONFIG_X86_64 - kvm_r8_write_raw(vcpu, regs->r8); - kvm_r9_write_raw(vcpu, regs->r9); - kvm_r10_write_raw(vcpu, regs->r10); - kvm_r11_write_raw(vcpu, regs->r11); - kvm_r12_write_raw(vcpu, regs->r12); - kvm_r13_write_raw(vcpu, regs->r13); - kvm_r14_write_raw(vcpu, regs->r14); - kvm_r15_write_raw(vcpu, regs->r15); -#endif - - kvm_rip_write(vcpu, regs->rip); - kvm_set_rflags(vcpu, regs->rflags | X86_EFLAGS_FIXED); - - vcpu->arch.exception.pending =3D false; - vcpu->arch.exception_vmexit.pending =3D false; - - kvm_make_request(KVM_REQ_EVENT, vcpu); -} - -int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) -{ - if (vcpu->kvm->arch.has_protected_state && - vcpu->arch.guest_state_protected) - return -EINVAL; - - vcpu_load(vcpu); - __set_regs(vcpu, regs); - vcpu_put(vcpu); - return 0; -} - -static void __get_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sr= egs) -{ - struct desc_ptr dt; - - if (vcpu->arch.guest_state_protected) - goto skip_protected_regs; - - kvm_handle_exception_payload_quirk(vcpu); - - kvm_get_segment(vcpu, &sregs->cs, VCPU_SREG_CS); - kvm_get_segment(vcpu, &sregs->ds, VCPU_SREG_DS); - kvm_get_segment(vcpu, &sregs->es, VCPU_SREG_ES); - kvm_get_segment(vcpu, &sregs->fs, VCPU_SREG_FS); - kvm_get_segment(vcpu, &sregs->gs, VCPU_SREG_GS); - kvm_get_segment(vcpu, &sregs->ss, VCPU_SREG_SS); - - kvm_get_segment(vcpu, &sregs->tr, VCPU_SREG_TR); - kvm_get_segment(vcpu, &sregs->ldt, VCPU_SREG_LDTR); - - kvm_x86_call(get_idt)(vcpu, &dt); - sregs->idt.limit =3D dt.size; - sregs->idt.base =3D dt.address; - kvm_x86_call(get_gdt)(vcpu, &dt); - sregs->gdt.limit =3D dt.size; - sregs->gdt.base =3D dt.address; - - sregs->cr2 =3D vcpu->arch.cr2; - sregs->cr3 =3D kvm_read_cr3(vcpu); - -skip_protected_regs: - sregs->cr0 =3D kvm_read_cr0(vcpu); - sregs->cr4 =3D kvm_read_cr4(vcpu); - sregs->cr8 =3D kvm_get_cr8(vcpu); - sregs->efer =3D vcpu->arch.efer; - sregs->apic_base =3D vcpu->arch.apic_base; -} - -static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) -{ - __get_sregs_common(vcpu, sregs); - - if (vcpu->arch.guest_state_protected) - return; - - if (vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft) - set_bit(vcpu->arch.interrupt.nr, - (unsigned long *)sregs->interrupt_bitmap); -} - -static void __get_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2) -{ - int i; - - __get_sregs_common(vcpu, (struct kvm_sregs *)sregs2); - - if (vcpu->arch.guest_state_protected) - return; - - if (is_pae_paging(vcpu)) { - kvm_vcpu_srcu_read_lock(vcpu); - for (i =3D 0 ; i < 4 ; i++) - sregs2->pdptrs[i] =3D kvm_pdptr_read(vcpu, i); - sregs2->flags |=3D KVM_SREGS2_FLAGS_PDPTRS_VALID; - kvm_vcpu_srcu_read_unlock(vcpu); - } -} - -int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, - struct kvm_sregs *sregs) -{ - if (vcpu->kvm->arch.has_protected_state && - vcpu->arch.guest_state_protected) - return -EINVAL; - - vcpu_load(vcpu); - __get_sregs(vcpu, sregs); - vcpu_put(vcpu); - return 0; -} - int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, struct kvm_mp_state *mp_state) { @@ -12373,175 +11747,6 @@ int kvm_task_switch(struct kvm_vcpu *vcpu, u16 ts= s_selector, int idt_index, } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_task_switch); =20 -static bool kvm_is_valid_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sr= egs) -{ - if ((sregs->efer & EFER_LME) && (sregs->cr0 & X86_CR0_PG)) { - /* - * When EFER.LME and CR0.PG are set, the processor is in - * 64-bit mode (though maybe in a 32-bit code segment). - * CR4.PAE and EFER.LMA must be set. - */ - if (!(sregs->cr4 & X86_CR4_PAE) || !(sregs->efer & EFER_LMA)) - return false; - if (!kvm_vcpu_is_legal_cr3(vcpu, sregs->cr3)) - return false; - } else { - /* - * Not in 64-bit mode: EFER.LMA is clear and the code - * segment cannot be 64-bit. - */ - if (sregs->efer & EFER_LMA || sregs->cs.l) - return false; - } - - return kvm_is_valid_cr4(vcpu, sregs->cr4) && - kvm_is_valid_cr0(vcpu, sregs->cr0); -} - -static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sre= gs, - int *mmu_reset_needed, bool update_pdptrs) -{ - int idx; - struct desc_ptr dt; - - if (!kvm_is_valid_sregs(vcpu, sregs)) - return -EINVAL; - - if (kvm_apic_set_base(vcpu, sregs->apic_base, true)) - return -EINVAL; - - if (vcpu->arch.guest_state_protected) - return 0; - - dt.size =3D sregs->idt.limit; - dt.address =3D sregs->idt.base; - kvm_x86_call(set_idt)(vcpu, &dt); - dt.size =3D sregs->gdt.limit; - dt.address =3D sregs->gdt.base; - kvm_x86_call(set_gdt)(vcpu, &dt); - - vcpu->arch.cr2 =3D sregs->cr2; - *mmu_reset_needed |=3D kvm_read_cr3(vcpu) !=3D sregs->cr3; - vcpu->arch.cr3 =3D sregs->cr3; - kvm_register_mark_dirty(vcpu, VCPU_REG_CR3); - kvm_x86_call(post_set_cr3)(vcpu, sregs->cr3); - - kvm_set_cr8(vcpu, sregs->cr8); - - *mmu_reset_needed |=3D vcpu->arch.efer !=3D sregs->efer; - kvm_x86_call(set_efer)(vcpu, sregs->efer); - - *mmu_reset_needed |=3D kvm_read_cr0(vcpu) !=3D sregs->cr0; - kvm_x86_call(set_cr0)(vcpu, sregs->cr0); - - *mmu_reset_needed |=3D kvm_read_cr4(vcpu) !=3D sregs->cr4; - kvm_x86_call(set_cr4)(vcpu, sregs->cr4); - - if (update_pdptrs) { - idx =3D srcu_read_lock(&vcpu->kvm->srcu); - if (is_pae_paging(vcpu)) { - load_pdptrs(vcpu, kvm_read_cr3(vcpu)); - *mmu_reset_needed =3D 1; - } - srcu_read_unlock(&vcpu->kvm->srcu, idx); - } - - kvm_set_segment(vcpu, &sregs->cs, VCPU_SREG_CS); - kvm_set_segment(vcpu, &sregs->ds, VCPU_SREG_DS); - kvm_set_segment(vcpu, &sregs->es, VCPU_SREG_ES); - kvm_set_segment(vcpu, &sregs->fs, VCPU_SREG_FS); - kvm_set_segment(vcpu, &sregs->gs, VCPU_SREG_GS); - kvm_set_segment(vcpu, &sregs->ss, VCPU_SREG_SS); - - kvm_set_segment(vcpu, &sregs->tr, VCPU_SREG_TR); - kvm_set_segment(vcpu, &sregs->ldt, VCPU_SREG_LDTR); - - kvm_lapic_update_cr8_intercept(vcpu); - - /* Older userspace won't unhalt the vcpu on reset. */ - if (kvm_vcpu_is_bsp(vcpu) && kvm_rip_read(vcpu) =3D=3D 0xfff0 && - sregs->cs.selector =3D=3D 0xf000 && sregs->cs.base =3D=3D 0xffff0000 = && - !is_protmode(vcpu)) - kvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE); - - return 0; -} - -static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) -{ - int pending_vec, max_bits; - int mmu_reset_needed =3D 0; - int ret =3D __set_sregs_common(vcpu, sregs, &mmu_reset_needed, true); - - if (ret) - return ret; - - if (mmu_reset_needed) { - kvm_mmu_reset_context(vcpu); - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); - } - - max_bits =3D KVM_NR_INTERRUPTS; - pending_vec =3D find_first_bit( - (const unsigned long *)sregs->interrupt_bitmap, max_bits); - - if (pending_vec < max_bits) { - kvm_queue_interrupt(vcpu, pending_vec, false); - pr_debug("Set back pending irq %d\n", pending_vec); - kvm_make_request(KVM_REQ_EVENT, vcpu); - } - return 0; -} - -static int __set_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2) -{ - int mmu_reset_needed =3D 0; - bool valid_pdptrs =3D sregs2->flags & KVM_SREGS2_FLAGS_PDPTRS_VALID; - bool pae =3D (sregs2->cr0 & X86_CR0_PG) && (sregs2->cr4 & X86_CR4_PAE) && - !(sregs2->efer & EFER_LMA); - int i, ret; - - if (sregs2->flags & ~KVM_SREGS2_FLAGS_PDPTRS_VALID) - return -EINVAL; - - if (valid_pdptrs && (!pae || vcpu->arch.guest_state_protected)) - return -EINVAL; - - ret =3D __set_sregs_common(vcpu, (struct kvm_sregs *)sregs2, - &mmu_reset_needed, !valid_pdptrs); - if (ret) - return ret; - - if (valid_pdptrs) { - for (i =3D 0; i < 4 ; i++) - kvm_pdptr_write(vcpu, i, sregs2->pdptrs[i]); - - kvm_register_mark_dirty(vcpu, VCPU_REG_PDPTR); - mmu_reset_needed =3D 1; - vcpu->arch.pdptrs_from_userspace =3D true; - } - if (mmu_reset_needed) { - kvm_mmu_reset_context(vcpu); - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); - } - return 0; -} - -int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, - struct kvm_sregs *sregs) -{ - int ret; - - if (vcpu->kvm->arch.has_protected_state && - vcpu->arch.guest_state_protected) - return -EINVAL; - - vcpu_load(vcpu); - ret =3D __set_sregs(vcpu, sregs); - vcpu_put(vcpu); - return ret; -} - static void kvm_arch_vcpu_guestdbg_update_apicv_inhibit(struct kvm *kvm) { bool set =3D false; @@ -12699,11 +11904,7 @@ static void store_regs(struct kvm_vcpu *vcpu) { BUILD_BUG_ON(sizeof(struct kvm_sync_regs) > SYNC_REGS_SIZE_BYTES); =20 - if (vcpu->run->kvm_valid_regs & KVM_SYNC_X86_REGS) - __get_regs(vcpu, &vcpu->run->s.regs.regs); - - if (vcpu->run->kvm_valid_regs & KVM_SYNC_X86_SREGS) - __get_sregs(vcpu, &vcpu->run->s.regs.sregs); + kvm_run_get_regs(vcpu); =20 if (vcpu->run->kvm_valid_regs & KVM_SYNC_X86_EVENTS) kvm_vcpu_ioctl_x86_get_vcpu_events( @@ -12712,19 +11913,8 @@ static void store_regs(struct kvm_vcpu *vcpu) =20 static int sync_regs(struct kvm_vcpu *vcpu) { - if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_REGS) { - __set_regs(vcpu, &vcpu->run->s.regs.regs); - vcpu->run->kvm_dirty_regs &=3D ~KVM_SYNC_X86_REGS; - } - - if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_SREGS) { - struct kvm_sregs sregs =3D vcpu->run->s.regs.sregs; - - if (__set_sregs(vcpu, &sregs)) - return -EINVAL; - - vcpu->run->kvm_dirty_regs &=3D ~KVM_SYNC_X86_SREGS; - } + if (kvm_run_set_regs(vcpu)) + return -EINVAL; =20 if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_EVENTS) { struct kvm_vcpu_events events =3D vcpu->run->s.regs.events; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 185062a26924..fd55cd031b1c 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -414,6 +414,7 @@ int handle_ud(struct kvm_vcpu *vcpu); =20 void kvm_deliver_exception_payload(struct kvm_vcpu *vcpu, struct kvm_queued_exception *ex); +void kvm_handle_exception_payload_quirk(struct kvm_vcpu *vcpu); =20 int kvm_mtrr_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data); int kvm_mtrr_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata); @@ -604,6 +605,7 @@ static inline void kvm_machine_check(void) int kvm_spec_ctrl_test_value(u64 value); int kvm_handle_memory_failure(struct kvm_vcpu *vcpu, int r, struct x86_exception *e); +void kvm_invalidate_pcid(struct kvm_vcpu *vcpu, unsigned long pcid); int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gv= a); bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type); =20 --=20 2.54.0.563.g4f69b47b94-goog