From nobody Tue Jun 16 06:26:32 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ABCD39A048 for ; Thu, 16 Apr 2026 23:10:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381051; cv=none; b=sqDZ8+aQMxKHH6ZR2EL/zVyISi+RoOVPDjrf0GtNf8q7CPn5V6dkdIc3FLSrI5AyvAgTUAyI1B5IsWmIYN5qOuIilaOPTftxkZAIdyoAMMdHAq+vwqy6KVNvaAMjpsajFiDgW6Skd8VOCqqK3H317pI/CL1jZ0yFaXJRsx/4t8o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381051; c=relaxed/simple; bh=9rNPGkTreelzLILSOGZqVPCjG69Up2clw9XS0zVDc+A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bD7KBRiS/ZzbTuJwei4m0YjYPe6LWuHRhOBHWeOXt99mPEqcwfGDYlRZlYLvddYHkNIOyjxR8Ofut5zrCjwe0nSqLi0rnHTki4n8xZwkUMm4qMQuL7gBthPAkKTaBJbDPcOtXPoYV4/tn2dgpqxgxWVepPdKgIKsjQkHLk/1EDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dRrG515o; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dRrG515o" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82f896eb6faso171752b3a.3 for ; Thu, 16 Apr 2026 16:10:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776381049; x=1776985849; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9XumdmAM5k86iU84PyqR7i86FKgUtltq5DYpR2bGk2E=; b=dRrG515oZMzwI2pzmNx63Qpn13/SDeP6duvcPY15tCYOfEGk/9LhE+8bKRRpWIj5YW 3GKJ2Qibv6dJR96B2gr7g+xrovQ9gLo62mWrFwpkdOshSgXu3MbUWWUEBMDAoaiEHkFJ eparX9zkK5DZgk0MhaWh4JStTgUmlIxUVVQVGeXbd/6LttQ/lvV6AsbFu+0TTFZkFNf0 ZOh58nnJh7aU6ujEfXf07XVeb1Wvn7yMPYeVquTNn5FADnsfjZjOEs3S7+j8YvaHULUt SE4Otk8tpYyzleOC0X4JbEgY3GQAqR6uxl9jnWeX3eOPTHDAqXh9dtCJPhQ0jxylTqcZ g+pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776381049; x=1776985849; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9XumdmAM5k86iU84PyqR7i86FKgUtltq5DYpR2bGk2E=; b=bZY80IuZfZdT9UmuGLrDuOqoA60oVGjMmK0OIV79my5Z+Tnr8EE7v1jXHrvJS7IT/J TX5XNfnzOM01Y6LUOaBeVZSGSG8nbrp0OU/EWP9+kYf+OkhXrV2BSjJjLlMnpmSg2lrX iTtv/YHvX73LDm9I2aqndOu7wSrOQs5V6Wz/+Rx4TdX/AhH2yZNZbuauBbvICBA9LLh0 xb8T8XOhll32vqgvekDKGcFn89j+CB76e17nAAOQtMUoUBOuCbxLxEXJY4zNOXpSx8je K5tDLclvRPxi31HdrfSamWySjIDynTFHIAQ5HhCg1rrfIoQjceZrkZe7GrtmaiVBfEmU u6Aw== X-Forwarded-Encrypted: i=1; AFNElJ9gVwBqO79MjAtjoVGqjGCt+/S6FfIbHP8oNvIE5bnqYxQgu3hgWIXcAbTfDsFyHaJ0P0EHyvGzSwCGhc8=@vger.kernel.org X-Gm-Message-State: AOJu0Yy+alYnEqpY38bAlL/a7VYGPGrv22dwKbQ7Q8QaRiO2GtMEX+bm sM1Ar+cWo+nmDr4GgfHHi9UDW5JgPs51uBLYAGpPgDnM3N/4DMoegaJiivQJWVgdIiFMUfDl7fz J8WtM3Q== X-Received: from pfzz15.prod.google.com ([2002:a05:6a00:1ef:b0:82c:e899:f08d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3385:b0:82a:805a:7cc with SMTP id d2e1a72fcca58-82f8c7da23fmr229961b3a.4.1776381049416; Thu, 16 Apr 2026 16:10:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 16 Apr 2026 16:10:38 -0700 In-Reply-To: <20260416231043.3402410-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260416231043.3402410-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.rc1.513.gad8abe7a5a-goog Message-ID: <20260416231043.3402410-2-seanjc@google.com> Subject: [PATCH v2 1/6] KVM: SVM: Fix page overflow in sev_dbg_crypt() for ENCRYPT path From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ashutosh Desai Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ashutosh Desai In sev_dbg_crypt(), the per-iteration transfer length is bounded by the source page offset (PAGE_SIZE - s_off) but not by the destination page offset (PAGE_SIZE - d_off). When d_off > s_off, the encrypt path (__sev_dbg_encrypt_user) performs a read-modify-write using a single-page intermediate buffer (dst_tpage): 1. __sev_dbg_decrypt() expands the size to round_up(len + (d_off & 15), 1= 6) before issuing the PSP command. If len + (d_off & 15) > PAGE_SIZE, the PSP writes beyond the end of the 4096-byte dst_tpage allocation. 2. The subsequent memcpy()/copy_from_user() into page_address(dst_tpage) + (d_off & 15) of 'len' bytes overflows by up to 15 bytes under the same condition. Trigger example: s_off =3D 0, d_off =3D 1, debug.len =3D PAGE_SIZE - the PSP is instructed to write round_up(4097, 16) =3D 4112 bytes to a 4096-byte buffer. Fix by also bounding len by (PAGE_SIZE - d_off), the same check that sev_send_update_data() already performs for its single-page guest region. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D BUG: KASAN: slab-use-after-free in sev_dbg_crypt+0x993/0xd10 [kvm_amd] Write of size 4095 at addr ff110062293bb009 by task sev_dbg_test/228214 CPU: 96 UID: 0 PID: 228214 Comm: sev_dbg_test Tainted: G U W = 7.0.0-smp--5ce9b0c48211-dbg #156 PREEMPTLAZY Tainted: [U]=3DUSER, [W]=3DWARN Hardware name: Google Astoria/astoria, BIOS 0.20250817.1-0 08/25/2025 Call Trace: dump_stack_lvl+0x54/0x70 print_report+0xbc/0x260 kasan_report+0xa2/0xd0 kasan_check_range+0x25f/0x2c0 __asan_memcpy+0x40/0x70 sev_dbg_crypt+0x993/0xd10 [kvm_amd] sev_mem_enc_ioctl+0x33c/0x450 [kvm_amd] kvm_vm_ioctl+0x65d/0x6d0 [kvm] __se_sys_ioctl+0xb2/0x100 do_syscall_64+0xe8/0x870 entry_SYSCALL_64_after_hwframe+0x4b/0x53 The buggy address belongs to the physical page: page: refcount:1 mapcount:0 mapping:0000000000000000 index:0x7fe72b6a0 pfn= :0x62293bb memcg:ff11000112827d82 flags: 0x1400000000000000(node=3D1|zone=3D1) raw: 1400000000000000 0000000000000000 dead000000000122 0000000000000000 raw: 00000007fe72b6a0 0000000000000000 00000001ffffffff ff11000112827d82 page dumped because: kasan: bad access detected Memory state around the buggy address: ff110062293bbf00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff110062293bbf80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >ff110062293bc000: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ^ ff110062293bc080: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ff110062293bc100: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Disabling lock debugging due to kernel taint Fixes: 24f41fb23a39 ("KVM: SVM: Add support for SEV DEBUG_DECRYPT command") Fixes: 7d1594f5d94b ("KVM: SVM: Add support for SEV DEBUG_ENCRYPT command") Cc: stable@vger.kernel.org Signed-off-by: Ashutosh Desai [sean: add sample KASAN splat, Fixes, and stable@] Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index c2126b3c3072..b9d7bd868e0b 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -1396,6 +1396,7 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kvm_= sev_cmd *argp, bool dec) s_off =3D vaddr & ~PAGE_MASK; d_off =3D dst_vaddr & ~PAGE_MASK; len =3D min_t(size_t, (PAGE_SIZE - s_off), size); + len =3D min_t(size_t, len, PAGE_SIZE - d_off); =20 if (dec) ret =3D __sev_dbg_decrypt_user(kvm, --=20 2.54.0.rc1.513.gad8abe7a5a-goog From nobody Tue Jun 16 06:26:32 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2462F396D1E for ; Thu, 16 Apr 2026 23:10:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381053; cv=none; b=pvqY0JC4PL4qRGsnV1Ws4rxxdNRVnks+guISNq+JK/ICJsO33J2ioMmTsjCr2vtWqygURNSqLNOhlO/KkQv52+0aIqEdmGUr7QMt/Bhsp3jYprgWmFup1gnL3mnBWXfCmOgnRseqyMLVC/KZn+rMyZJRh60akIDWeDAwqzaq54Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381053; c=relaxed/simple; bh=m4XaX8Jz+FsMeYdAGEkpWFL5GaceHm5GV84xoWA3Kf0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=X7AewS12AqIQjd9Bd8GgWbsrTGuSGFZDSrg05bW7PjvFsrpySBcsQBtkUpqVdphx06k88oMbCWJVkBFHgTcOCWnqegAAofnisa/jHlIWTCzlOCZEALU+07q7+zNKTfcsMbuE7l+7VX5XPT7NdHOkuYQpPYjdLGCGbyy4LZecT2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Y7nQJHoC; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Y7nQJHoC" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2adef9d486bso516905ad.2 for ; Thu, 16 Apr 2026 16:10:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776381051; x=1776985851; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=TGFUBJqdj697HeMyWfcv7hzZssdk/CwlMwin6XkTx9U=; b=Y7nQJHoChayOxnaNGAPuqp931pFRKQWiA/vCTP6r5z6gv7z5JZi5Utz+Eky/y4eoT+ brWUc5EwlcVhYcf+ghf3oHepqaLczglG5tsF6WvrBRbUYEZOK9sGhcmqjDMikkGhcJ+4 cPipOIi7tKstO3ROqubbIzHz8k9sn0lsg2dtYN4JGD6Frj3Xk9K5HHJ0oR0/+ykvhmJU a0YIA5gQdUnBf5DdXLdS0Mtkcg93aiOcEle57kL6Pr5fuU7Zf1oBFg5p6CMs3jVnneRf Qr4JYCnKonMgS/m0fAlcGr2wWG5Cz1rUHeIPsBm7cEtsz5MH0z3J13Bje7DmDPN/YRp5 B23g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776381051; x=1776985851; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TGFUBJqdj697HeMyWfcv7hzZssdk/CwlMwin6XkTx9U=; b=J55OoqjNHQY0zFNFM6oLniXeHXZSKgIG+FuoyP4obqy0mhyiYxk1OmdgAHW5hAEkYI +73fDgr9/KRHd9UDowzI0i0mOUryyNqB2OC6U8VxiFT4fLle8Rxlfwq48YsKa3vGHzqd KgdmFWIGwWIzmRGgWU03YqzbSe8GWPVYSiMu8MocvFq1ApLyYul7BiCUnY2vAUCFLUJ4 Z3v78uqfvy1s0W4fnEYr3xSVq3nQdht+Tnz1SJGVsaZWL+F2HirdBguLFCq3NWQP6IjO M9fAzIIaatGEs0ubVfnuXadaM98FGu8JkWZdxHjVf3Ys3ygG292RluKAK50VtN73Oazi +qLA== X-Forwarded-Encrypted: i=1; AFNElJ9kbgbU+oV1CsgqH8EZT0dv7TIrxDLOucBFE31r+jqZ7C6/KaMJapQwcr9dUWJZIS9ERFNPoxqUsrFpHBY=@vger.kernel.org X-Gm-Message-State: AOJu0Yywy/cPg8WvTUDO2CYmF3RvX3A2zt443aeTWdjnlFvnQL3ICViK QnkWFiehBv5xhUh0AAXxFj89Msp0ERx7dpfHPnPV1HM86wGlTy9gXzCaWSR5N3dWQcVGucOQEQy 5uisrTw== X-Received: from pfbky41.prod.google.com ([2002:a05:6a00:6f69:b0:82f:6a88:310b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:7287:b0:39c:2a9c:bbe2 with SMTP id adf61e73a8af0-3a08d7545b4mr259210637.24.1776381051386; Thu, 16 Apr 2026 16:10:51 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 16 Apr 2026 16:10:39 -0700 In-Reply-To: <20260416231043.3402410-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260416231043.3402410-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.rc1.513.gad8abe7a5a-goog Message-ID: <20260416231043.3402410-3-seanjc@google.com> Subject: [PATCH v2 2/6] KVM: selftests: Add a test to verify SEV {en,de}crypt debug ioctls From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ashutosh Desai Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a selftest to verify KVM's handling of {de,en}crypt debug ioctls, specifically focusing on edge cases around the chunk (16 bytes) and page (4096) sizes, where KVM had multiple bugs. E.g. KVM would fail to handle small sizes that aren't naturally aligned and sized, would buffer overflow if the destination was unaligned but the source was not, etc. Attempt to strike a balance between an exhaustive test and a reasonable runtime. On a system with both SEV and SEV-ES support, the current runtime is under 45 seconds. Which isn't great, but it's tolerable, and it's not obvious which of the combinations are "better" than the others. Signed-off-by: Sean Christopherson --- tools/testing/selftests/kvm/Makefile.kvm | 1 + tools/testing/selftests/kvm/include/x86/sev.h | 24 ++++ .../testing/selftests/kvm/x86/sev_dbg_test.c | 118 ++++++++++++++++++ 3 files changed, 143 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/sev_dbg_test.c diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selft= ests/kvm/Makefile.kvm index 9118a5a51b89..82fa943b9503 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -140,6 +140,7 @@ TEST_GEN_PROGS_x86 +=3D x86/tsc_msrs_test TEST_GEN_PROGS_x86 +=3D x86/vmx_pmu_caps_test TEST_GEN_PROGS_x86 +=3D x86/xen_shinfo_test TEST_GEN_PROGS_x86 +=3D x86/xen_vmcall_test +TEST_GEN_PROGS_x86 +=3D x86/sev_dbg_test TEST_GEN_PROGS_x86 +=3D x86/sev_init2_tests TEST_GEN_PROGS_x86 +=3D x86/sev_migrate_tests TEST_GEN_PROGS_x86 +=3D x86/sev_smoke_test diff --git a/tools/testing/selftests/kvm/include/x86/sev.h b/tools/testing/= selftests/kvm/include/x86/sev.h index 008b4169f5e2..669dbec927ac 100644 --- a/tools/testing/selftests/kvm/include/x86/sev.h +++ b/tools/testing/selftests/kvm/include/x86/sev.h @@ -144,4 +144,28 @@ static inline void snp_launch_update_data(struct kvm_v= m *vm, vm_paddr_t gpa, vm_sev_ioctl(vm, KVM_SEV_SNP_LAUNCH_UPDATE, &update_data); } =20 +static inline void sev_dbg_crypt_memory(struct kvm_vm *vm, unsigned int cm= d, + void *dst, void *src, unsigned int len) +{ + struct kvm_sev_dbg dbg =3D { + .src_uaddr =3D (unsigned long)src, + .dst_uaddr =3D (unsigned long)dst, + .len =3D len, + }; + + vm_sev_ioctl(vm, cmd, &dbg); +} + +static inline void sev_decrypt_memory(struct kvm_vm *vm, void *dst, void *= src, + unsigned int len) +{ + sev_dbg_crypt_memory(vm, KVM_SEV_DBG_DECRYPT, dst, src, len); +} + +static inline void sev_encrypt_memory(struct kvm_vm *vm, void *dst, void *= src, + unsigned int len) +{ + sev_dbg_crypt_memory(vm, KVM_SEV_DBG_ENCRYPT, dst, src, len); +} + #endif /* SELFTEST_KVM_SEV_H */ diff --git a/tools/testing/selftests/kvm/x86/sev_dbg_test.c b/tools/testing= /selftests/kvm/x86/sev_dbg_test.c new file mode 100644 index 000000000000..55530cc2be05 --- /dev/null +++ b/tools/testing/selftests/kvm/x86/sev_dbg_test.c @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include + +#include "test_util.h" +#include "kvm_util.h" +#include "processor.h" +#include "sev.h" + +#define BUFFER_SIZE (PAGE_SIZE * 2) + +static u8 *data; +static u8 src[BUFFER_SIZE] __aligned(PAGE_SIZE); +static u8 dst[BUFFER_SIZE] __aligned(PAGE_SIZE); + +static void validate_dst(int i, int nr_bytes, u8 pattern) +{ + for ( ; i < nr_bytes; i++) + TEST_ASSERT(dst[i] =3D=3D pattern, + "Expected 0x%x at byte %u, got 0x%x", + pattern, i, dst[i]); +} + +static void validate_buffers(void) +{ + int i; + + for (i =3D 0; i < BUFFER_SIZE; i++) + TEST_ASSERT(src[i] =3D=3D dst[i], + "Expected src[%u] (0x%x) =3D=3D dst[%u] (0x%x)", + i, src[i], i, dst[i]); +} + +static void ____test_sev_dbg(struct kvm_vm *vm, int i, int j, int nr_bytes) +{ + u8 pattern =3D guest_random_u32(&guest_rng); + + if (i + nr_bytes > BUFFER_SIZE || j + nr_bytes > BUFFER_SIZE) + return; + + memset(&src[i], pattern, nr_bytes); + sev_encrypt_memory(vm, &data[j], &src[i], nr_bytes); + sev_decrypt_memory(vm, &dst[i], &data[j], nr_bytes); + validate_buffers(); + validate_dst(i, nr_bytes, pattern); +} + +static void __test_sev_dbg(struct kvm_vm *vm, int nr_bytes) +{ + /* + * In a perfect world, all sizes at all combinations within the buffers + * would be tested. In reality, even this much testing is quite slow. + * Target sizes and offsets around the chunk (16 bytes) and page (4096 + * bytes) sizes. + */ + int x[] =3D { 1, 8, 15, 16, 23 }; + int p =3D PAGE_SIZE - 24; + int i, j; + + ____test_sev_dbg(vm, 0, 0, nr_bytes); + + for (i =3D 0; i < ARRAY_SIZE(x); i++) { + for (j =3D 0; j < ARRAY_SIZE(x); j++) { + ____test_sev_dbg(vm, x[i], x[j], nr_bytes); + ____test_sev_dbg(vm, x[i], p + x[j], nr_bytes); + ____test_sev_dbg(vm, p + x[i], x[j], nr_bytes); + ____test_sev_dbg(vm, p + x[i], p + x[j], nr_bytes); + } + } +} + +static void test_sev_dbg(uint32_t type, uint64_t policy) +{ + int sizes[] =3D { 1, 8, 15, 16, 17, 32, 33 }; + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + int i; + + if (!(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(type))) + return; + + vm =3D vm_sev_create_with_one_vcpu(type, NULL, &vcpu); + + data =3D addr_gva2hva(vm, vm_vaddr_alloc(vm, BUFFER_SIZE, KVM_UTIL_MIN_VA= DDR)); + memset(data, 0xaa, BUFFER_SIZE); + + vm_sev_launch(vm, policy, NULL); + + sev_decrypt_memory(vm, dst, data, BUFFER_SIZE); + validate_dst(0, BUFFER_SIZE, 0xaa); + + memset(src, 0x55, BUFFER_SIZE); + sev_encrypt_memory(vm, data, src, BUFFER_SIZE); + sev_decrypt_memory(vm, dst, data, BUFFER_SIZE); + validate_dst(0, BUFFER_SIZE, 0x55); + + __test_sev_dbg(vm, PAGE_SIZE); + + for (i =3D 0; i < ARRAY_SIZE(sizes); i++) { + __test_sev_dbg(vm, sizes[i]); + __test_sev_dbg(vm, PAGE_SIZE - sizes[i]); + __test_sev_dbg(vm, PAGE_SIZE + sizes[i]); + __test_sev_dbg(vm, BUFFER_SIZE - sizes[i]); + } + + kvm_vm_free(vm); +} + +int main(int argc, char *argv[]) +{ + TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SEV)); + + /* Note, KVM doesn't support {de,en}crypt commands for SNP. */ + test_sev_dbg(KVM_X86_SEV_VM, 0); + test_sev_dbg(KVM_X86_SEV_ES_VM, SEV_POLICY_ES); + return 0; +} --=20 2.54.0.rc1.513.gad8abe7a5a-goog From nobody Tue Jun 16 06:26:32 2026 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1140E38D00A for ; Thu, 16 Apr 2026 23:10:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381055; cv=none; b=Y4Zxn0O8RTRR+lgS5wlVFatLttZ2p4sVzG+IJ9sbUWSe0sNm7vNxGk3EODwb3ejrIjCiHlrK9WSE870PlinnYeM9rkwT2ggMFw5jnyAr+CWssWlXPrfyaNU+E+C8NZaNokhLaAQH2w1n6h0RBMnMqbtwHZVQBfgdFbUsJoDcMzg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381055; c=relaxed/simple; bh=Q9UIEcAvPJORPTWD41cpIOjdp7/S4AQkA2+wGNagTA4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=k5YBrutU7KWU0GiHV+3fvJhx2imkymPTgWwlP5QzQ/1uDqPGtm2Fi3JbNZ+EB2Na1ARXttDuX4MgquzFFH2eXbRev/RU6EmM6uaDq9quzwEY7y98arq18UQRa/q5iXbgKSjcbeGzUzJAXRcdeBHTlab97PWra5G5Av9mt183AJg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QJTlx4tw; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QJTlx4tw" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-35fbb5779e8so111767a91.3 for ; Thu, 16 Apr 2026 16:10:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776381053; x=1776985853; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RgRlvg9Y1LaNRx+SazWupXuWPbXqEY3tEnJHpSwHv0E=; b=QJTlx4twRmhfrEGatktSQvd4X4v7FY/LZNnGi2n2AeU8uQ348hlrKM2le9swV5yvKh 1unkrrxesMrfjLfRGSUie0CUI7JM4Z7nLE87N3FTeGl0+xMWZXPhFGxCP3XSoOMVVj/H 7zSduyRegCDnZPaFc9cKeyiSznk+uJybE/H6i9TIOnIU7ye27rcFIJGyqAR8vMbg/psD 2qanLZfFVGBThDu/57jbYczCHHiF9zkn2NnRQtRKAOdYckpatWElzJe4VSXZViVxn/9X XScfXlNwG0C+1PZqEzGyZHI0pnuyJwC8hSIWXcXmKgNTMv+GReTyfjUsGb4tmaLAsjpD 8QUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776381053; x=1776985853; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RgRlvg9Y1LaNRx+SazWupXuWPbXqEY3tEnJHpSwHv0E=; b=k4iGA4KNPZY6Azo3LMrCkWCuAccEyj65OoFQuFy/rK19rrKCUAGC1jD6u0lKbWGLxc DDNIQ7c49MvN0ZhwOiISh3QfWpWCYI+TL8CE1p6RvuHB6NyKHOAtAGpB7KFcFBDR/XVZ jrXxOsCd2+mm8GyjvBtIxtjEephvCfxo9wvSdnVXEqRvyyq+Gq33yks2Df5mCQcRaePb inlqmskou4NWMqHg+dwqXmNSjyZnzC9IJkg12C44dKxpMXR3r40eDm8plWf48QPY27OT 4WepIHmV84F9MWZGsQDVoO9IkvCXZkrlES+nzgvrZ5mgz7Lo1kdAFYBttYBA5cKMRzPH maFQ== X-Forwarded-Encrypted: i=1; AFNElJ+4LlNE5V0G2pljvp8SJebA+lW2wohyyQ8dIL5nVN+vyEIOJ6lJImlgAdbIw7L40TXSXG6S3MPkd6t6u9Y=@vger.kernel.org X-Gm-Message-State: AOJu0YzD0yhdEt11MDhccO8l2iC3yjGPX8QJrfi5Znk2DHQ6zjt2YFdn ht1t9dBsxpYhg17fv+BtvKa20oPCcniWPMqiVZCU2H07YO6umeYUUC9N0hGYLpmiaItHSK/KtuD Qg6qlhw== X-Received: from pldt10.prod.google.com ([2002:a17:903:40ca:b0:2b2:4611:5de2]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5845:b0:35f:c46f:2b0 with SMTP id 98e67ed59e1d1-36140473f70mr272112a91.14.1776381053296; Thu, 16 Apr 2026 16:10:53 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 16 Apr 2026 16:10:40 -0700 In-Reply-To: <20260416231043.3402410-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260416231043.3402410-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.rc1.513.gad8abe7a5a-goog Message-ID: <20260416231043.3402410-4-seanjc@google.com> Subject: [PATCH v2 3/6] KVM: SEV: Explicitly validate the dst buffer for debug operations From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ashutosh Desai Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When encrypting/decrypting guest memory, explicitly check that the destination is non-NULL and doesn't wrap instead of subtly relying on sev_pin_memory() to perform the check. This will allow adding and using a more focused single-page pinning helper. Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index b9d7bd868e0b..9d2044fd910d 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -1357,9 +1357,11 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kvm= _sev_cmd *argp, bool dec) if (copy_from_user(&debug, u64_to_user_ptr(argp->data), sizeof(debug))) return -EFAULT; =20 - if (!debug.len || debug.src_uaddr + debug.len < debug.src_uaddr) + if (!debug.len || !debug.src_uaddr || !debug.dst_uaddr) return -EINVAL; - if (!debug.dst_uaddr) + + if (debug.src_uaddr + debug.len < debug.src_uaddr || + debug.dst_uaddr + debug.len < debug.dst_uaddr) return -EINVAL; =20 vaddr =3D debug.src_uaddr; --=20 2.54.0.rc1.513.gad8abe7a5a-goog From nobody Tue Jun 16 06:26:32 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 387AE38B15D for ; Thu, 16 Apr 2026 23:10:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381058; cv=none; b=F1C2iQyL3KdzsTngz4ZMyo5acr3tHleNg1SJuM/VAx/oR3TtxyUV53qAv0jLlIW2A34+OIT56bae2lASmko2nDn0LqWHvt4HHgcjH71Z2OuXByuNPDryIyn+/ezWOEHVR9WGpRlBaKW5XRrU6MEV5QepGQvMtoRdtq+8rEuaX0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381058; c=relaxed/simple; bh=FdLaYtwZwiRS1VHilQTQbfHMmhTQEAcT5xOv9STThQ4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZzlJk544PXk/iTm35uNtrZHuokLKV6GwU3FLwYZ6PXunjqEcl5XlyUZdmW33njZ5PvYxXBYd+Jjc0fUiyBHM8jPu6A8R2vbd28tXhtbBfSBmFs+PAHNcTFehcipIUi7ymANYGIS+SEkLLEnJJa//qL8ak3QVpvXhA0/1S0UTXKY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZZR55u6R; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZZR55u6R" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2b4654f9bb6so440695ad.2 for ; Thu, 16 Apr 2026 16:10:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776381057; x=1776985857; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=fJxe2OB8vPecXs/qWoD+k06aOyPltin/mVNi1viGV8M=; b=ZZR55u6RYR5Sk6C3LfteiCslxOEeL1iZpdU+gPY46n73mFX+bWeDpvrVvvFzAI43+B wPAs/dmXm47VM84VTC5IOL6q1gHQHA0QyzAcblz/3I79wlGkFIzwhgvAdeRSE7nc2uRO qupzP185/bauG9iv3fvTnvLdptgVHsE29aBmPkJAnj8Q2Tdrj4bWbZMZtHRqTt7bQ6+4 6oT3NDp19Z2TYqOmi+lcD58K+MfGHm99fMW5QhR6jLDdk3gl8iFIDmiR3RzzgqXvAoUj 1dBVFGMv+lNvXB+xYjwBAcNvzh1NzYQWukp0P6Be6GawgSF7zNlBZgRdy+yLXbvbvEMo whgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776381057; x=1776985857; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fJxe2OB8vPecXs/qWoD+k06aOyPltin/mVNi1viGV8M=; b=FKAEOoJmsfD6T8aIlBfmjFWs8FdCVQ3yY7YumX3sSBOCKIwa0Ivsj+g+UhwyKQOiQi kbUARjt1JJlNV/qwhB2vjo+sNrSfctqGqJ3Gbs351HYDu3k0sKvgf6Wkn/AMywwJJVMk BGeMLX3SFegiPYISNxAaTU5z+7xp2HtID1eU3ogxIQqTFb248KyZgIJZoQn6vvPB8On9 7LoZL0FV3ZUIpLN+sKPikazjHSIaULrTKPtZ/XMpDy9GF5MO0ECsxLD9VXT57KH4Ev3I WPjMpXwqG549o+Y7ZBJyNvcy2jlR6JNz2ftZHb0VjxNoNMrxauKPnJmIB27mvn7M7Goj c+Pw== X-Forwarded-Encrypted: i=1; AFNElJ9Fm5/yuhdPgHtbg0038MbfH3TgQgCrRTCHWFkX2zIQCHZqYxtkjbqtcWlXSvN3hY7zQPpjbj2f1Pc/BXk=@vger.kernel.org X-Gm-Message-State: AOJu0YwrBNrjPvKYkDYQqvh35UaMTqPH7PGHMe2aqb1Vjtymwnv788Kt T8NrPWrN9yDQZKlpZNWaEl5FUp8AKR67O5aTav0+/SjNV3pQtHWqQEyPrCm8seFVeyHU95zDgri Qpu6p4g== X-Received: from pleg6.prod.google.com ([2002:a17:902:e386:b0:2b0:5102:ab9e]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:3b52:b0:2b0:5cb4:d894 with SMTP id d9443c01a7336-2b5f9ed0d0amr2154685ad.13.1776381056388; Thu, 16 Apr 2026 16:10:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 16 Apr 2026 16:10:41 -0700 In-Reply-To: <20260416231043.3402410-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260416231043.3402410-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.rc1.513.gad8abe7a5a-goog Message-ID: <20260416231043.3402410-5-seanjc@google.com> Subject: [PATCH v2 4/6] KVM: SEV: Add helper function to pin/unpin a single page From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ashutosh Desai Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add helpers to pin/unpin a single page, and use it in all flows that pin exactly one page. None of the single-page users actually check that the correct number of pages was pinned, which is functionally ok, but visually jarring, especially in the decrypt/encrypt flow, which separately pins two pages, but uses a single variable to track how pages were pinned each time. Again, it's functionally ok since core mm guarantees exactly one page will be pinned on success, but it's ugly. Opportunistically use page_to_phys() instead of open coding an equivalent via page_to_pfn(). Note, all users of the single-page helper pre-validate the address and length, i.e. don't rely on the sanity check in sev_pin_memory(). No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 145 +++++++++++++++++++++++++---------------- 1 file changed, 90 insertions(+), 55 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 9d2044fd910d..4146c91788fb 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -720,14 +720,50 @@ static int sev_launch_start(struct kvm *kvm, struct k= vm_sev_cmd *argp) return ret; } =20 +static int sev_check_pin_count(struct kvm *kvm, unsigned long npages) +{ + unsigned long total_npages, lock_limit; + + total_npages =3D to_kvm_sev_info(kvm)->pages_locked + npages; + if (total_npages > totalram_pages()) + return -EINVAL; + + lock_limit =3D rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + if (total_npages > lock_limit && !capable(CAP_IPC_LOCK)) { + pr_err_ratelimited("SEV: %lu total pages would exceed the lock limit of = %lu.\n", + total_npages, lock_limit); + return -ENOMEM; + } + + return 0; +} + +static int sev_pin_user_pages(struct kvm *kvm, unsigned long addr, int npa= ges, + unsigned int gup_flags, struct page **pages) +{ + int npinned; + + lockdep_assert_held(&kvm->lock); + + npinned =3D pin_user_pages_fast(addr, npages, gup_flags, pages); + if (npinned !=3D npages) { + if (npinned > 0) + unpin_user_pages(pages, npinned); + pr_err_ratelimited("SEV: Failure locking %u pages.\n", npages); + return -ENOMEM; + } + + to_kvm_sev_info(kvm)->pages_locked +=3D npages; + return 0; +} + static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, unsigned long ulen, unsigned long *n, unsigned int flags) { - struct kvm_sev_info *sev =3D to_kvm_sev_info(kvm); - unsigned long npages, total_npages, lock_limit; + unsigned long npages; struct page **pages; - int npinned, ret; + int ret; =20 lockdep_assert_held(&kvm->lock); =20 @@ -743,16 +779,9 @@ static struct page **sev_pin_memory(struct kvm *kvm, u= nsigned long uaddr, if (npages > INT_MAX) return ERR_PTR(-EINVAL); =20 - total_npages =3D sev->pages_locked + npages; - if (total_npages > totalram_pages()) - return ERR_PTR(-EINVAL); - - lock_limit =3D rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (total_npages > lock_limit && !capable(CAP_IPC_LOCK)) { - pr_err("SEV: %lu total pages would exceed the lock limit of %lu.\n", - total_npages, lock_limit); - return ERR_PTR(-ENOMEM); - } + ret =3D sev_check_pin_count(kvm, npages); + if (ret) + return ERR_PTR(ret); =20 /* * Don't WARN if the kernel (rightly) thinks the total size is absurd, @@ -764,25 +793,14 @@ static struct page **sev_pin_memory(struct kvm *kvm, = unsigned long uaddr, if (!pages) return ERR_PTR(-ENOMEM); =20 - /* Pin the user virtual address. */ - npinned =3D pin_user_pages_fast(uaddr, npages, flags, pages); - if (npinned !=3D npages) { - pr_err("SEV: Failure locking %lu pages.\n", npages); - ret =3D -ENOMEM; - goto err; + ret =3D sev_pin_user_pages(kvm, uaddr, npages, flags, pages); + if (ret) { + kvfree(pages); + return ERR_PTR(ret); } =20 *n =3D npages; - sev->pages_locked =3D total_npages; - return pages; - -err: - if (npinned > 0) - unpin_user_pages(pages, npinned); - - kvfree(pages); - return ERR_PTR(ret); } =20 static void sev_unpin_memory(struct kvm *kvm, struct page **pages, @@ -793,6 +811,29 @@ static void sev_unpin_memory(struct kvm *kvm, struct p= age **pages, to_kvm_sev_info(kvm)->pages_locked -=3D npages; } =20 +static struct page *sev_pin_page(struct kvm *kvm, unsigned long addr, + unsigned int flags) +{ + struct page *page; + int r; + + r =3D sev_check_pin_count(kvm, 1); + if (r) + return ERR_PTR(r); + + r =3D sev_pin_user_pages(kvm, addr, 1, flags, &page); + if (r) + return ERR_PTR(r); + + return page; +} + +static void sev_unpin_page(struct kvm *kvm, struct page *page) +{ + unpin_user_pages(&page, 1); + to_kvm_sev_info(kvm)->pages_locked -=3D 1; +} + static void sev_clflush_pages(struct page *pages[], unsigned long npages) { uint8_t *page_virtual; @@ -1345,9 +1386,8 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kvm_= sev_cmd *argp, bool dec) { unsigned long vaddr, vaddr_end, next_vaddr; unsigned long dst_vaddr; - struct page **src_p, **dst_p; + struct page *src_p, *dst_p; struct kvm_sev_dbg debug; - unsigned long n; unsigned int size; int ret; =20 @@ -1373,13 +1413,13 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kv= m_sev_cmd *argp, bool dec) int len, s_off, d_off; =20 /* lock userspace source and destination page */ - src_p =3D sev_pin_memory(kvm, vaddr & PAGE_MASK, PAGE_SIZE, &n, 0); + src_p =3D sev_pin_page(kvm, vaddr & PAGE_MASK, 0); if (IS_ERR(src_p)) return PTR_ERR(src_p); =20 - dst_p =3D sev_pin_memory(kvm, dst_vaddr & PAGE_MASK, PAGE_SIZE, &n, FOLL= _WRITE); + dst_p =3D sev_pin_page(kvm, dst_vaddr & PAGE_MASK, FOLL_WRITE); if (IS_ERR(dst_p)) { - sev_unpin_memory(kvm, src_p, n); + sev_unpin_page(kvm, src_p); return PTR_ERR(dst_p); } =20 @@ -1388,8 +1428,8 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kvm_= sev_cmd *argp, bool dec) * the pages; flush the destination too so that future accesses do not * see stale data. */ - sev_clflush_pages(src_p, 1); - sev_clflush_pages(dst_p, 1); + sev_clflush_pages(&src_p, 1); + sev_clflush_pages(&dst_p, 1); =20 /* * Since user buffer may not be page aligned, calculate the @@ -1402,20 +1442,20 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kv= m_sev_cmd *argp, bool dec) =20 if (dec) ret =3D __sev_dbg_decrypt_user(kvm, - __sme_page_pa(src_p[0]) + s_off, + __sme_page_pa(src_p) + s_off, (void __user *)dst_vaddr, - __sme_page_pa(dst_p[0]) + d_off, + __sme_page_pa(dst_p) + d_off, len, &argp->error); else ret =3D __sev_dbg_encrypt_user(kvm, - __sme_page_pa(src_p[0]) + s_off, + __sme_page_pa(src_p) + s_off, (void __user *)vaddr, - __sme_page_pa(dst_p[0]) + d_off, + __sme_page_pa(dst_p) + d_off, (void __user *)dst_vaddr, len, &argp->error); =20 - sev_unpin_memory(kvm, src_p, n); - sev_unpin_memory(kvm, dst_p, n); + sev_unpin_page(kvm, src_p); + sev_unpin_page(kvm, dst_p); =20 if (ret) goto err; @@ -1698,8 +1738,7 @@ static int sev_send_update_data(struct kvm *kvm, stru= ct kvm_sev_cmd *argp) struct sev_data_send_update_data data; struct kvm_sev_send_update_data params; void *hdr, *trans_data; - struct page **guest_page; - unsigned long n; + struct page *guest_page; int ret, offset; =20 if (!sev_guest(kvm)) @@ -1723,8 +1762,7 @@ static int sev_send_update_data(struct kvm *kvm, stru= ct kvm_sev_cmd *argp) return -EINVAL; =20 /* Pin guest memory */ - guest_page =3D sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK, - PAGE_SIZE, &n, 0); + guest_page =3D sev_pin_page(kvm, params.guest_uaddr & PAGE_MASK, 0); if (IS_ERR(guest_page)) return PTR_ERR(guest_page); =20 @@ -1745,7 +1783,7 @@ static int sev_send_update_data(struct kvm *kvm, stru= ct kvm_sev_cmd *argp) data.trans_len =3D params.trans_len; =20 /* The SEND_UPDATE_DATA command requires C-bit to be always set. */ - data.guest_address =3D (page_to_pfn(guest_page[0]) << PAGE_SHIFT) + offse= t; + data.guest_address =3D page_to_phys(guest_page) + offset; data.guest_address |=3D sev_me_mask; data.guest_len =3D params.guest_len; data.handle =3D to_kvm_sev_info(kvm)->handle; @@ -1772,8 +1810,7 @@ static int sev_send_update_data(struct kvm *kvm, stru= ct kvm_sev_cmd *argp) e_free_hdr: kfree(hdr); e_unpin: - sev_unpin_memory(kvm, guest_page, n); - + sev_unpin_page(kvm, guest_page); return ret; } =20 @@ -1878,8 +1915,7 @@ static int sev_receive_update_data(struct kvm *kvm, s= truct kvm_sev_cmd *argp) struct kvm_sev_receive_update_data params; struct sev_data_receive_update_data data; void *hdr =3D NULL, *trans =3D NULL; - struct page **guest_page; - unsigned long n; + struct page *guest_page; int ret, offset; =20 if (!sev_guest(kvm)) @@ -1916,8 +1952,7 @@ static int sev_receive_update_data(struct kvm *kvm, s= truct kvm_sev_cmd *argp) data.trans_len =3D params.trans_len; =20 /* Pin guest memory */ - guest_page =3D sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK, - PAGE_SIZE, &n, FOLL_WRITE); + guest_page =3D sev_pin_page(kvm, params.guest_uaddr & PAGE_MASK, FOLL_WRI= TE); if (IS_ERR(guest_page)) { ret =3D PTR_ERR(guest_page); goto e_free_trans; @@ -1928,10 +1963,10 @@ static int sev_receive_update_data(struct kvm *kvm,= struct kvm_sev_cmd *argp) * encrypts the written data with the guest's key, and the cache may * contain dirty, unencrypted data. */ - sev_clflush_pages(guest_page, n); + sev_clflush_pages(&guest_page, 1); =20 /* The RECEIVE_UPDATE_DATA command requires C-bit to be always set. */ - data.guest_address =3D (page_to_pfn(guest_page[0]) << PAGE_SHIFT) + offse= t; + data.guest_address =3D page_to_phys(guest_page) + offset; data.guest_address |=3D sev_me_mask; data.guest_len =3D params.guest_len; data.handle =3D to_kvm_sev_info(kvm)->handle; @@ -1939,7 +1974,7 @@ static int sev_receive_update_data(struct kvm *kvm, s= truct kvm_sev_cmd *argp) ret =3D sev_issue_cmd(kvm, SEV_CMD_RECEIVE_UPDATE_DATA, &data, &argp->error); =20 - sev_unpin_memory(kvm, guest_page, n); + sev_unpin_page(kvm, guest_page); =20 e_free_trans: kfree(trans); --=20 2.54.0.rc1.513.gad8abe7a5a-goog From nobody Tue Jun 16 06:26:32 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E92003976A7 for ; Thu, 16 Apr 2026 23:10:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381060; cv=none; b=OmoIvekEwyxEVgGjYHjBuVmyunc77aeRHbjNjsKr5RFvgCWxEEwxAkHQW7G0JON7vbKY07NJ9YYYyGKfQZcAePAuKoTz4rfo0qKBlGqL9dND86qj16iztIVFlUZ7w0hNNxWQBhfUrcmEi5tA3fUJndgtu9VeR76Me6jmPpD2Fb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381060; c=relaxed/simple; bh=AqvreRcUrDcVAklDELALQJuhHMkS0iYZ2X6C9pDWxOw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sDJafiZtU7DCrv/9poUOyhFYB2pXDo6ykX1vLkMqeMRk5AguwbArMVWEQM3tD6jYdMF2BEhi8ydN5G5XHa2AxsBSEviTw2VjPxv6sXIlS+WKxknaqSZR9hGiB1mdk3Toe4bvwLoq57z/0Lm+q8Zg5zJ1tirHLyryvLbMxNx8MtA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LltwWCYG; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LltwWCYG" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-82f07078eaaso75792b3a.0 for ; Thu, 16 Apr 2026 16:10:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776381058; x=1776985858; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=eOmi0SGJneN2bKDEnlJdJDSqg2s6V8AvI55vj4MvzeU=; b=LltwWCYG4orBUo41PR5IBBC5v3U/CUyhSy/GtM8guySEFvQRQ/bHaDtrMqcAYwnJPh aledTtX415hcnJwA0RX07q0sGuWaVoncJTy2eXMw4VOmXs6Xz3RaCcBxBF706l1K/OIo zaiack5fhIjTTKFQxYerp/zla+ZfczjALys0PkGjFvN87MIAQQSYzIOdNM7IxiV24/5l VMhuZ4JMrltLtfc4LUFKrFczeWQzb4Z3xjXupq9KqSt/FzbbGddOHVdFAylP7dB9RADy LMVGXeJyRism7h+JNURfkYUL9hnJjto0zBCAvP0ufpg72tFYFwBO2y6rWNfo1w7OsMZy 06aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776381058; x=1776985858; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eOmi0SGJneN2bKDEnlJdJDSqg2s6V8AvI55vj4MvzeU=; b=cmn72ZHhpJy3HSmaGXkYg+aH4lmQ8jSPtdjZIKS5PceTxhBODOBH53YWRQGmGDdRxt KGRblOiNIDA4Ml6wVODQ66vHbmogh7VJbyFkD5h8aVUmb3C2pceCLu294NNdTCVj3pXy qvkHDgepNRMn5rQTtBJnlSyE2hlR1kERjFNxVlgOI4VFXW0seNSnFgt7RdsqeIlgQoCI FAo3ydgaCMSAE4aWCMaHzFd4YqPfohxKu2S+lOQ/wN9p6e4VwcibbQxDQZ+jyiV7ot/5 Tjaiq54RD6RFDq7Je2aulr/xzL86L4/Fg65TdKlPShbCk76vsI+YK4b00xJ84mrFBJcv KhLQ== X-Forwarded-Encrypted: i=1; AFNElJ81fo0oKP3HCSbUf4uSOu2xmJayYkg0gRVhfAc71d8KAEItnguIWLKduGeYanVY6H3zLWEtTUEVya1nbl8=@vger.kernel.org X-Gm-Message-State: AOJu0Yxs41uRbOUU+ytZ/OYeIJadEvN+2GJ4JdfAFaWgPkD/yzc3a7zG k5qd7E1AnMKBemyNjVdlwSf/U67cWx0vUvjQuNX42pcKdr2fnxxPxmqfwmvLsl5Ol4vQjr9a+ZA YU8sAyA== X-Received: from pfbdo18.prod.google.com ([2002:a05:6a00:4a12:b0:82f:75de:5da4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2d28:b0:829:8942:2c93 with SMTP id d2e1a72fcca58-82f8c7d10a9mr246512b3a.9.1776381058149; Thu, 16 Apr 2026 16:10:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 16 Apr 2026 16:10:42 -0700 In-Reply-To: <20260416231043.3402410-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260416231043.3402410-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.rc1.513.gad8abe7a5a-goog Message-ID: <20260416231043.3402410-6-seanjc@google.com> Subject: [PATCH v2 5/6] KVM: SEV: Rewrite logic to {de,en}crypt memory for debug From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ashutosh Desai Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Wholesale rewrite the guts of the debug {de,en}crypt flows, as the existing code is broken, e.g. doesn't handle cases where the access isn't naturally sized and aligned, and is so wildly flawed that attempting to salvage the current code in an iterative fashion would be more risky than a rewrite. E.g. when encrypting 9 bytes at offset 8, KVM needs to _decrypt_ destination[31:0] into a temporary buffer, buffer[31:0], then copy 9 bytes from source[8:0] to buffer[16:8], then encrypt buffer[31:0] back into destination[31:0]. The current code only ever copies 16 bytes, and bizarrely uses a temporary buffer for the source as well. To keep the code easier to read and maintain, send the unaligned cases down dedicated "slow" paths instead of trying to mix and match the possible combinations in one helper. For now, preserve the basic approach of the current code, e.g. allocate an entire page for the temporary buffer, to minimize unwanted changes in functionality. Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 306 +++++++++++++++++++---------------------- 1 file changed, 139 insertions(+), 167 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 4146c91788fb..89586f821c9c 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -1237,159 +1237,140 @@ static int sev_guest_status(struct kvm *kvm, stru= ct kvm_sev_cmd *argp) return ret; } =20 -static int __sev_issue_dbg_cmd(struct kvm *kvm, unsigned long src, - unsigned long dst, int size, - int *error, bool enc) +static int sev_issue_dbg_cmd(struct kvm *kvm, unsigned long src_pa, + unsigned long dst_pa, unsigned int size, + unsigned int ioctl, int *error) { - struct sev_data_dbg data; + int cmd =3D ioctl =3D=3D KVM_SEV_DBG_DECRYPT ? SEV_CMD_DBG_DECRYPT : + SEV_CMD_DBG_ENCRYPT; + struct sev_data_dbg data =3D { + .handle =3D to_kvm_sev_info(kvm)->handle, + .dst_addr =3D dst_pa, + .src_addr =3D src_pa, + .len =3D size, + }; + + return sev_issue_cmd(kvm, cmd, &data, error); +} + +static struct page *sev_alloc_dbg_buffer(void **buf) +{ + struct page *buf_p; =20 - data.reserved =3D 0; - data.handle =3D to_kvm_sev_info(kvm)->handle; - data.dst_addr =3D dst; - data.src_addr =3D src; - data.len =3D size; + buf_p =3D alloc_page(GFP_KERNEL); + if (!buf_p) + return NULL; + + *buf =3D kmap_local_page(buf_p); + return buf_p; +} =20 - return sev_issue_cmd(kvm, - enc ? SEV_CMD_DBG_ENCRYPT : SEV_CMD_DBG_DECRYPT, - &data, error); +static void sev_free_dbg_buffer(struct page *buf_p, void *buf) +{ + kunmap_local(buf); + __free_page(buf_p); } =20 -static int __sev_dbg_decrypt(struct kvm *kvm, unsigned long src_paddr, - unsigned long dst_paddr, int sz, int *err) +static unsigned int sev_dbg_crypt_slow_addr_and_size(struct page *page, + unsigned long __va, + unsigned int len, + unsigned long *pa) { - int offset; + /* The number of bytes to {de,en}crypt must be 16-byte aligned. */ + unsigned int nr_bytes =3D round_up(len, 16); + unsigned long va =3D ALIGN_DOWN(__va, 16); + + /* + * Increase the number of bytes to {de,en}crypt by one chunk (16 bytes) + * if the aligned address and length doesn't cover the unaligned range, + * e.g. if the address is unaligned _and_ the access will split a chunk + * at the tail. + */ + if (va + nr_bytes < __va + len) + nr_bytes +=3D 16; + + *pa =3D __sme_page_pa(page) + (va & ~PAGE_MASK); =20 /* - * Its safe to read more than we are asked, caller should ensure that - * destination has enough space. + * Sanity check that the new access won't split a page. This should + * never happen; just squash the access and let the firmware command + * fail. */ - offset =3D src_paddr & 15; - src_paddr =3D round_down(src_paddr, 16); - sz =3D round_up(sz + offset, 16); + if (WARN_ON_ONCE((*pa & PAGE_MASK) !=3D ((*pa + nr_bytes - 1) & PAGE_MASK= ))) + return 0; =20 - return __sev_issue_dbg_cmd(kvm, src_paddr, dst_paddr, sz, err, false); + return nr_bytes; } =20 -static int __sev_dbg_decrypt_user(struct kvm *kvm, unsigned long paddr, - void __user *dst_uaddr, - unsigned long dst_paddr, - int size, int *err) +static int sev_dbg_decrypt_slow(struct kvm *kvm, unsigned long src, + struct page *src_p, unsigned long dst, + unsigned int len, int *err) { - struct page *tpage =3D NULL; - int ret, offset; - - /* if inputs are not 16-byte then use intermediate buffer */ - if (!IS_ALIGNED(dst_paddr, 16) || - !IS_ALIGNED(paddr, 16) || - !IS_ALIGNED(size, 16)) { - tpage =3D (void *)alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO); - if (!tpage) - return -ENOMEM; - - dst_paddr =3D __sme_page_pa(tpage); - } - - ret =3D __sev_dbg_decrypt(kvm, paddr, dst_paddr, size, err); - if (ret) - goto e_free; - - if (tpage) { - offset =3D paddr & 15; - if (copy_to_user(dst_uaddr, page_address(tpage) + offset, size)) - ret =3D -EFAULT; - } - -e_free: - if (tpage) - __free_page(tpage); - - return ret; + unsigned int nr_bytes; + unsigned long src_pa; + struct page *buf_p; + void *buf; + int r; + + buf_p =3D sev_alloc_dbg_buffer(&buf); + if (!buf_p) + return -ENOMEM; + + nr_bytes =3D sev_dbg_crypt_slow_addr_and_size(src_p, src, len, &src_pa); + + r =3D sev_issue_dbg_cmd(kvm, src_pa, __sme_page_pa(buf_p), + nr_bytes, KVM_SEV_DBG_DECRYPT, err); + if (r) + goto out; + + if (copy_to_user((void __user *)dst, buf + (src & 15), len)) + r =3D -EFAULT; +out: + sev_free_dbg_buffer(buf_p, buf); + return r; } =20 -static int __sev_dbg_encrypt_user(struct kvm *kvm, unsigned long paddr, - void __user *vaddr, - unsigned long dst_paddr, - void __user *dst_vaddr, - int size, int *error) +static int sev_dbg_encrypt_slow(struct kvm *kvm, unsigned long src, + unsigned long dst, struct page *dst_p, + unsigned int len, int *err) { - struct page *src_tpage =3D NULL; - struct page *dst_tpage =3D NULL; - int ret, len =3D size; + unsigned int nr_bytes; + unsigned long dst_pa; + struct page *buf_p; + void *buf; + int r; =20 - /* If source buffer is not aligned then use an intermediate buffer */ - if (!IS_ALIGNED((unsigned long)vaddr, 16)) { - src_tpage =3D alloc_page(GFP_KERNEL_ACCOUNT); - if (!src_tpage) - return -ENOMEM; + buf_p =3D sev_alloc_dbg_buffer(&buf); + if (!buf_p) + return -ENOMEM; =20 - if (copy_from_user(page_address(src_tpage), vaddr, size)) { - __free_page(src_tpage); - return -EFAULT; - } + /* Decrypt the _destination_ to do a RMW on plaintext. */ + nr_bytes =3D sev_dbg_crypt_slow_addr_and_size(dst_p, dst, len, &dst_pa); =20 - paddr =3D __sme_page_pa(src_tpage); - } + r =3D sev_issue_dbg_cmd(kvm, dst_pa, __sme_page_pa(buf_p), + nr_bytes, KVM_SEV_DBG_DECRYPT, err); + if (r) + goto out; =20 /* - * If destination buffer or length is not aligned then do read-modify-wr= ite: - * - decrypt destination in an intermediate buffer - * - copy the source buffer in an intermediate buffer - * - use the intermediate buffer as source buffer + * Copy from the source into the intermediate buffer, and then + * re-encrypt the buffer into the destination. */ - if (!IS_ALIGNED((unsigned long)dst_vaddr, 16) || !IS_ALIGNED(size, 16)) { - int dst_offset; - - dst_tpage =3D alloc_page(GFP_KERNEL_ACCOUNT); - if (!dst_tpage) { - ret =3D -ENOMEM; - goto e_free; - } - - ret =3D __sev_dbg_decrypt(kvm, dst_paddr, - __sme_page_pa(dst_tpage), size, error); - if (ret) - goto e_free; - - /* - * If source is kernel buffer then use memcpy() otherwise - * copy_from_user(). - */ - dst_offset =3D dst_paddr & 15; - - if (src_tpage) - memcpy(page_address(dst_tpage) + dst_offset, - page_address(src_tpage), size); - else { - if (copy_from_user(page_address(dst_tpage) + dst_offset, - vaddr, size)) { - ret =3D -EFAULT; - goto e_free; - } - } - - paddr =3D __sme_page_pa(dst_tpage); - dst_paddr =3D round_down(dst_paddr, 16); - len =3D round_up(size, 16); - } - - ret =3D __sev_issue_dbg_cmd(kvm, paddr, dst_paddr, len, error, true); - -e_free: - if (src_tpage) - __free_page(src_tpage); - if (dst_tpage) - __free_page(dst_tpage); - return ret; + if (copy_from_user(buf + (dst & 15), (void __user *)src, len)) + r =3D -EFAULT; + else + r =3D sev_issue_dbg_cmd(kvm, __sme_page_pa(buf_p), dst_pa, + nr_bytes, KVM_SEV_DBG_ENCRYPT, err); +out: + sev_free_dbg_buffer(buf_p, buf); + return r; } =20 -static int sev_dbg_crypt(struct kvm *kvm, struct kvm_sev_cmd *argp, bool d= ec) +static int sev_dbg_crypt(struct kvm *kvm, struct kvm_sev_cmd *argp, + unsigned int cmd) { - unsigned long vaddr, vaddr_end, next_vaddr; - unsigned long dst_vaddr; - struct page *src_p, *dst_p; struct kvm_sev_dbg debug; - unsigned int size; - int ret; + unsigned int i, len; =20 if (!sev_guest(kvm)) return -ENOTTY; @@ -1404,20 +1385,29 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kv= m_sev_cmd *argp, bool dec) debug.dst_uaddr + debug.len < debug.dst_uaddr) return -EINVAL; =20 - vaddr =3D debug.src_uaddr; - size =3D debug.len; - vaddr_end =3D vaddr + size; - dst_vaddr =3D debug.dst_uaddr; + for (i =3D 0; i < debug.len; i +=3D len) { + unsigned long src =3D debug.src_uaddr + i; + unsigned long dst =3D debug.dst_uaddr + i; + unsigned long s_off =3D src & ~PAGE_MASK; + unsigned long d_off =3D dst & ~PAGE_MASK; + struct page *src_p, *dst_p; + int ret; =20 - for (; vaddr < vaddr_end; vaddr =3D next_vaddr) { - int len, s_off, d_off; + /* + * Copy as many remaining bytes as possible while staying in a + * single page for both the source and destination. + */ + len =3D min3(debug.len - i, PAGE_SIZE - s_off, PAGE_SIZE - d_off); =20 - /* lock userspace source and destination page */ - src_p =3D sev_pin_page(kvm, vaddr & PAGE_MASK, 0); + /* + * Pin the source and destination pages; firmware operates on + * physical addresses. + */ + src_p =3D sev_pin_page(kvm, src & PAGE_MASK, 0); if (IS_ERR(src_p)) return PTR_ERR(src_p); =20 - dst_p =3D sev_pin_page(kvm, dst_vaddr & PAGE_MASK, FOLL_WRITE); + dst_p =3D sev_pin_page(kvm, dst & PAGE_MASK, FOLL_WRITE); if (IS_ERR(dst_p)) { sev_unpin_page(kvm, src_p); return PTR_ERR(dst_p); @@ -1431,41 +1421,25 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kv= m_sev_cmd *argp, bool dec) sev_clflush_pages(&src_p, 1); sev_clflush_pages(&dst_p, 1); =20 - /* - * Since user buffer may not be page aligned, calculate the - * offset within the page. - */ - s_off =3D vaddr & ~PAGE_MASK; - d_off =3D dst_vaddr & ~PAGE_MASK; - len =3D min_t(size_t, (PAGE_SIZE - s_off), size); - len =3D min_t(size_t, len, PAGE_SIZE - d_off); - - if (dec) - ret =3D __sev_dbg_decrypt_user(kvm, - __sme_page_pa(src_p) + s_off, - (void __user *)dst_vaddr, - __sme_page_pa(dst_p) + d_off, - len, &argp->error); + if (IS_ALIGNED(src, 16) && IS_ALIGNED(dst, 16) && IS_ALIGNED(len, 16)) + ret =3D sev_issue_dbg_cmd(kvm, + __sme_page_pa(src_p) + s_off, + __sme_page_pa(dst_p) + d_off, + len, cmd, &argp->error); + else if (cmd =3D=3D KVM_SEV_DBG_DECRYPT) + ret =3D sev_dbg_decrypt_slow(kvm, src, src_p, dst, + len, &argp->error); else - ret =3D __sev_dbg_encrypt_user(kvm, - __sme_page_pa(src_p) + s_off, - (void __user *)vaddr, - __sme_page_pa(dst_p) + d_off, - (void __user *)dst_vaddr, - len, &argp->error); + ret =3D sev_dbg_encrypt_slow(kvm, src, dst, dst_p, + len, &argp->error); =20 sev_unpin_page(kvm, src_p); sev_unpin_page(kvm, dst_p); =20 if (ret) - goto err; - - next_vaddr =3D vaddr + len; - dst_vaddr =3D dst_vaddr + len; - size -=3D len; + return ret; } -err: - return ret; + return 0; } =20 static int sev_launch_secret(struct kvm *kvm, struct kvm_sev_cmd *argp) @@ -2727,10 +2701,8 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *= argp) r =3D sev_guest_status(kvm, &sev_cmd); break; case KVM_SEV_DBG_DECRYPT: - r =3D sev_dbg_crypt(kvm, &sev_cmd, true); - break; case KVM_SEV_DBG_ENCRYPT: - r =3D sev_dbg_crypt(kvm, &sev_cmd, false); + r =3D sev_dbg_crypt(kvm, &sev_cmd, sev_cmd.id); break; case KVM_SEV_LAUNCH_SECRET: r =3D sev_launch_secret(kvm, &sev_cmd); --=20 2.54.0.rc1.513.gad8abe7a5a-goog From nobody Tue Jun 16 06:26:32 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAA0B39C010 for ; Thu, 16 Apr 2026 23:11:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381062; cv=none; b=Ff/gwWgp243khEBTd0kDPqjwyDtmY4a0HkQwilWUhhsxH2DxGHmU3Pfbznd70s5gxjnExaIm4G1PvFFpulmEGJ/agtGstW6YL1KSAnTIYzG2asEmX0bVsWaS7uHtQyDeDknp55QBCMxaQ5oKYp4x760bWLI7ESgrFDIel9MCr1s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776381062; c=relaxed/simple; bh=85ORCOpy0tBEfyyTiUwYsB5sebkTBrSUPiI1VxHClOs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WzinQHSA4TSFnZ9cxFx4P/E/buoCYOL1BXm0NKblZos+aJYTIkzDw7JkoxN+XHX4xvyABeLpnfL3w8CqA8A8GoOmJ7hkNufLaF1GZJUiBnptIQb1uK5HAmVoL6ed+7dxS3vBa7CDNx+T9vhUtJSIYsi9HoLU+dHGiKAwdYzZde8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pBeOzKtx; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pBeOzKtx" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-358df8fbd1cso144813a91.0 for ; Thu, 16 Apr 2026 16:11:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1776381060; x=1776985860; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=jNbNgJLHKhmCy6JtsDJ8ebUiJV8vqj7Fm/YHhQltXZ0=; b=pBeOzKtx1GzaCXIhEs4pEhp0XDFfFdAvA5uC1DQ6bT72KFhA9xdDJgpTawZjTOkJrT uO1RHC1QoQh3zastL8WLb5r+zj5/1G26P4+3QnZRlXgpA1Ak1eNj4BhxfRpdYTM9tfmt Q7HenjX4lcdG0VUbLuslqCodxRTeIciBmilPpNgcW7zR6kD4mtEaIvcjRF7Ja30lK4tc zUJCqIrXj67c6YJHwbgY8AsVnMTlWg238XkrMJO0j+Vh2DXzr8TdsHiFjmdJGwa8ivLC MXfafc6IXrFNNZ+Tm8/7uA5NLjA66UWmf5QDmJf3/HqdzlZm7mSS+LdkxiNi2o1E2H50 VC8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776381060; x=1776985860; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jNbNgJLHKhmCy6JtsDJ8ebUiJV8vqj7Fm/YHhQltXZ0=; b=HlUPau7C+TBaA8O7J+xYLz0uRUQdKWDl4+u4MfHoRoMFGvdSQ/TSAYf4DhpKyxF3do nrgZ9HbcT3cbIrJaRvUKxeV9tQm7RF6hZrYITprHk0JxoQm5KBX3hHtkIY0N78AXD4uK 6KStf4hWedJH1kv9QtSZw3bM+HG87SH2E5TFr7vcUQoVEkN0LbcLfLTWeZbilOILj+1E aqObSLu4GEKqP08DfMvZxVL1DAzxFH8WssNTjhuZuOHLEhKTt0RgTohaGRr9F5f5QQgK 40VTDP83SOElQ+hwFRejwes5dTaHP1kKMF421YwiJKT0QoHXjZE/OQ6iyzwEH4SPJL4Y ZQ9Q== X-Forwarded-Encrypted: i=1; AFNElJ/EaaMBuuM0lDTlNdv2D63AuL5jGHuBqKUAn/y0XLvQnGmIHLIRk2tbCWfye4yNS5gO/ImzLZ7aINaHCdc=@vger.kernel.org X-Gm-Message-State: AOJu0YzeWPzZLxuzowVdsN/TigVHnrEVGw8iSqZdDoqf2rbYki5KzubI 6aRbcjA5EwtbxnyS+d9XP+nIF2HxCZXf7FMQZ2ziUvMrcPCZwJnq3iCqC2s6ySctnHmmsnjkqMI /HVZUOg== X-Received: from pfmm8.prod.google.com ([2002:a05:6a00:2488:b0:82f:3774:4736]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:7f88:b0:39b:e40b:fcae with SMTP id adf61e73a8af0-3a08d6f21e4mr289701637.10.1776381059837; Thu, 16 Apr 2026 16:10:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 16 Apr 2026 16:10:43 -0700 In-Reply-To: <20260416231043.3402410-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260416231043.3402410-1-seanjc@google.com> X-Mailer: git-send-email 2.54.0.rc1.513.gad8abe7a5a-goog Message-ID: <20260416231043.3402410-7-seanjc@google.com> Subject: [PATCH v2 6/6] KVM: SEV: Allocate only as many bytes as needed for temp crypt buffers From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ashutosh Desai Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When using a temporary buffer to {de,en}crypt unaligned memory for debug, allocate only the number of bytes that are needed instead of allocating an entire page. The most common case for unaligned accesses will be reading or writing less than 16 bytes. Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 69 ++++++++++++++---------------------------- 1 file changed, 22 insertions(+), 47 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 89586f821c9c..0865ce4bcecb 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -1253,53 +1253,34 @@ static int sev_issue_dbg_cmd(struct kvm *kvm, unsig= ned long src_pa, return sev_issue_cmd(kvm, cmd, &data, error); } =20 -static struct page *sev_alloc_dbg_buffer(void **buf) +static void *sev_dbg_crypt_slow_alloc(struct page *page, unsigned long __v= a, + unsigned int len, unsigned long *pa, + unsigned int *nr_bytes) { - struct page *buf_p; - - buf_p =3D alloc_page(GFP_KERNEL); - if (!buf_p) - return NULL; - - *buf =3D kmap_local_page(buf_p); - return buf_p; -} - -static void sev_free_dbg_buffer(struct page *buf_p, void *buf) -{ - kunmap_local(buf); - __free_page(buf_p); -} - -static unsigned int sev_dbg_crypt_slow_addr_and_size(struct page *page, - unsigned long __va, - unsigned int len, - unsigned long *pa) -{ - /* The number of bytes to {de,en}crypt must be 16-byte aligned. */ - unsigned int nr_bytes =3D round_up(len, 16); unsigned long va =3D ALIGN_DOWN(__va, 16); =20 + /* The number of bytes to {de,en}crypt must be 16-byte aligned. */ + *nr_bytes =3D round_up(len, 16); + /* * Increase the number of bytes to {de,en}crypt by one chunk (16 bytes) * if the aligned address and length doesn't cover the unaligned range, * e.g. if the address is unaligned _and_ the access will split a chunk * at the tail. */ - if (va + nr_bytes < __va + len) - nr_bytes +=3D 16; + if (va + *nr_bytes < __va + len) + *nr_bytes +=3D 16; =20 *pa =3D __sme_page_pa(page) + (va & ~PAGE_MASK); =20 /* * Sanity check that the new access won't split a page. This should - * never happen; just squash the access and let the firmware command - * fail. + * never happen; just pretend the allocation failed. */ - if (WARN_ON_ONCE((*pa & PAGE_MASK) !=3D ((*pa + nr_bytes - 1) & PAGE_MASK= ))) - return 0; + if (WARN_ON_ONCE((*pa & PAGE_MASK) !=3D ((*pa + *nr_bytes - 1) & PAGE_MAS= K))) + return NULL; =20 - return nr_bytes; + return kmalloc(*nr_bytes, GFP_KERNEL); } =20 static int sev_dbg_decrypt_slow(struct kvm *kvm, unsigned long src, @@ -1308,17 +1289,14 @@ static int sev_dbg_decrypt_slow(struct kvm *kvm, un= signed long src, { unsigned int nr_bytes; unsigned long src_pa; - struct page *buf_p; void *buf; int r; =20 - buf_p =3D sev_alloc_dbg_buffer(&buf); - if (!buf_p) + buf =3D sev_dbg_crypt_slow_alloc(src_p, src, len, &src_pa, &nr_bytes); + if (!buf) return -ENOMEM; =20 - nr_bytes =3D sev_dbg_crypt_slow_addr_and_size(src_p, src, len, &src_pa); - - r =3D sev_issue_dbg_cmd(kvm, src_pa, __sme_page_pa(buf_p), + r =3D sev_issue_dbg_cmd(kvm, src_pa, __sme_set(__pa(buf)), nr_bytes, KVM_SEV_DBG_DECRYPT, err); if (r) goto out; @@ -1326,7 +1304,7 @@ static int sev_dbg_decrypt_slow(struct kvm *kvm, unsi= gned long src, if (copy_to_user((void __user *)dst, buf + (src & 15), len)) r =3D -EFAULT; out: - sev_free_dbg_buffer(buf_p, buf); + kfree(buf); return r; } =20 @@ -1336,18 +1314,15 @@ static int sev_dbg_encrypt_slow(struct kvm *kvm, un= signed long src, { unsigned int nr_bytes; unsigned long dst_pa; - struct page *buf_p; void *buf; int r; =20 - buf_p =3D sev_alloc_dbg_buffer(&buf); - if (!buf_p) - return -ENOMEM; - /* Decrypt the _destination_ to do a RMW on plaintext. */ - nr_bytes =3D sev_dbg_crypt_slow_addr_and_size(dst_p, dst, len, &dst_pa); + buf =3D sev_dbg_crypt_slow_alloc(dst_p, dst, len, &dst_pa, &nr_bytes); + if (!buf) + return -ENOMEM; =20 - r =3D sev_issue_dbg_cmd(kvm, dst_pa, __sme_page_pa(buf_p), + r =3D sev_issue_dbg_cmd(kvm, dst_pa, __sme_set(__pa(buf)), nr_bytes, KVM_SEV_DBG_DECRYPT, err); if (r) goto out; @@ -1359,10 +1334,10 @@ static int sev_dbg_encrypt_slow(struct kvm *kvm, un= signed long src, if (copy_from_user(buf + (dst & 15), (void __user *)src, len)) r =3D -EFAULT; else - r =3D sev_issue_dbg_cmd(kvm, __sme_page_pa(buf_p), dst_pa, + r =3D sev_issue_dbg_cmd(kvm, __sme_set(__pa(buf)), dst_pa, nr_bytes, KVM_SEV_DBG_ENCRYPT, err); out: - sev_free_dbg_buffer(buf_p, buf); + kfree(buf); return r; } =20 --=20 2.54.0.rc1.513.gad8abe7a5a-goog