From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E912A262FD0; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=OI8RbmkskkPwiXQCJQ6dNbHoix6TSYZaoKlTyDmIB3/1edAR7wTgib/M8sa8nb8ih2ML5qrU8cTyVhomRB+GZQgeLeK1tEdphNoRx9vF5ah8W1iCznHE3oH5VpawMng6gN4TMZ+MylOmGC2ZVvSN0WLNflg2JgiJNnIRNHwcJLw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=3RDuReZQPdnpDhEdbvLR7X6xqKa7H4GlqlpiAhWp0p8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KIAV8rR01tGdgChxqaWmB9AwgjQ2Vv7ZSughvaHKGQnDGySTI/iVY9m/uY8fnlWqtZMA6rEfqx62RmnducRXSlPPzvZcPlnd2Urvz0HPB66fzHxz5qxT2OiTD8NbwgChJDP1DGGOlddgX/3d6vcAVpOqTxNLu32Uj47O+MWQEXc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g1VwRCOy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g1VwRCOy" Received: by smtp.kernel.org (Postfix) with ESMTPS id 81A38C2BCFD; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135919; bh=3RDuReZQPdnpDhEdbvLR7X6xqKa7H4GlqlpiAhWp0p8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=g1VwRCOyedOtGxbdxTdegr8DDWZU6VvjQ3ogxdfRCgtIocveYdghE/0OQCDWp6XZ6 rJaKhYTt3GLN2DGn+6EydXUuVUTpRjIdTePNulx7JT01OmNMIdkH3oo5nLuvMW87Is t65Y8a9defMIL0Y50tVPo7s+NIamuBk5pTPUd9C4AVX4XaZG9RJqTy0XdfoKk2LEgY 0/89NCMEfqpDZ0EgtTqrfIfuJEfrsPDMVpainkPKbpxxp1eSW3jh4noJIc2WV7R7FD 58/j4PMMkF5Skpcj3CRZRR3q68JjBJJVWJ4AGEApPOUfPVRFGqHTDVPWu67OjBEWfE FzaA04lPYtGag== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5981FC61DE2; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:02 -0700 Subject: [PATCH v12 01/45] KVM: guest_memfd: Optimize away conversion overheads via dead-code elimination Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-1-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=3210; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=KU5lBT3XbWircUdxqmS0fnCYMooY3DNtiddyAdBDhH4=; b=wRmS6WFV25eBLlkWhcNyTW8umJE+UPeg3uo0+Q/XM+JU1xnLi6JSPz7bg3Lq5iuNlI20eGVdN DspTez7dVYLAdbPlQJTf6Vo6/pxxxjuYSdhZdAGOK71ZI1A0hl3fCXc X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Add and use kvm_arch_has_gmem_convert() to guard guest_memfd's invocation of arch hooks related to converting memory between private and shared, as only one half of the x86 CoCo duo needs the runtime hooks (any pre-work is pure overhead for TDX). At this exact moment, the overhead is negligible, but that will change when in-place conversion comes along, at which point to-shared conversions will "need" to find all affected folios prior to calling into arch code. In quotes because very technically that work could be pushed to arch code, but that would bleed guest_memfd details into arch code and would be far worse than adding yet another kvm_arch_has... hook. Opportunistically provide the kvm_arch_gmem_make_private() declaration, and rely on dead-code elimination to eliminate the call to non-existent code when CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT=3Dn. Reported-by: Binbin Wu Closes: https://lore.kernel.org/all/1ec08cd8-3072-4753-ad5e-cd34956647f8@li= nux.intel.com Suggested-by: Ackerley Tng Signed-off-by: Sean Christopherson Signed-off-by: Ackerley Tng Reviewed-by: Binbin Wu Reviewed-by: David Hildenbrand (Arm) Reviewed-by: Fuad Tabba --- arch/x86/include/asm/kvm_host.h | 3 +++ include/linux/kvm_host.h | 3 ++- virt/kvm/guest_memfd.c | 5 ++--- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 683bb8bf43a94..6beddd8a0c749 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1854,6 +1854,9 @@ enum kvm_intr_type { #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES #define kvm_arch_has_private_mem(kvm) ((kvm)->arch.has_private_mem) #endif +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +#define kvm_arch_has_gmem_convert() (!!kvm_x86_ops.gmem_make_private) +#endif =20 #define kvm_arch_has_readonly_mem(kvm) (!(kvm)->arch.has_protected_state) =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 502465119ca0c..b7516e19ef419 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2597,9 +2597,10 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, } #endif /* CONFIG_KVM_GUEST_MEMFD */ =20 -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, kvm_pfn_t nr_pages); +#ifndef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +#define kvm_arch_has_gmem_convert() false #endif =20 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 798d91df69537..2190e4d6b6bea 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -772,11 +772,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memo= ry_slot *slot, folio_mark_uptodate(folio); } =20 -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT - if (kvm_gmem_is_private_mem(file_inode(file), index)) + if (kvm_arch_has_gmem_convert() && + kvm_gmem_is_private_mem(file_inode(file), index)) r =3D kvm_arch_gmem_make_private(kvm, gfn, *pfn, (kvm_pfn_t)1 << *max_order); -#endif =20 folio_unlock(folio); folio_put(folio); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A2F427A907; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=PCut52w2dw3UiY6xX3jTqwbkmUJ83WkdTmXpK6RRGYrHAkM41hIQWSnbvRJTMERLR2EhdqIEzncBKbD7KLkCKH6W+LJdQTvrV2DNggR+fLdb/0AcW+0jB1KUQdvgFY0rKpL6ADpRBScvQhnafLkpcnjiUDgKnILGN2mbLs8oW0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=yiocxA49550KUCzo+T/5GJg/8zdJWfhnYG1cFdaaNSU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OO8WBvGnhUrx73Jh8bInnr5bimPxzQQpOphOKDAlccrGoEs+aBGf/Y+A89mJUWdvkmo20wmwGECdsfxTCSmlqKEn/8ocNRWGaSnWI/RAs+PcEQCid6UDLodDuqa6DGEh4Kwt7wJX+rGNu/IIBdbjB6JOdM6OEcDOpE+M1gQ4UTQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Mcwrx3xV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Mcwrx3xV" Received: by smtp.kernel.org (Postfix) with ESMTPS id A6194C4AF17; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135919; bh=yiocxA49550KUCzo+T/5GJg/8zdJWfhnYG1cFdaaNSU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Mcwrx3xVKrw7T1MSZ2sW1ftA1mFi8bY/BSWRzLgPsqEPrbTFpHguRcFKPiZJdqLh3 +uMrgEzTGPEViQTFLuXMKyEJB9fdZoO5+0xGd9BGKNa4KI1gWRcAv6bFweeCqyFSrS bSvXhYUPEAWJu51b1eSpkQCMt8i3sgt0d//1XhxYni6GyF9dBWQwumlE2lNbvJVAeo CVDqOKpdsolJOeE+bunKAJYHgWIkKk9CmbiyQDvSUWYLXMjBfOcFhoAcbZKzmIiw+D KyzKr4IZ3IG+301E/VYad8pZz+LWpn4IvRJhVnG3/0bVHCvoOQM4mudvUpHPtEBzFb eATcDh+aftF7w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88F9AC61DF1; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:03 -0700 Subject: [PATCH v12 02/45] KVM: guest_memfd: Use kvm_mem_is_private() when populating guest_memfd memory Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-2-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li , Fuad Tabba X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=1236; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=J603Sitd2IbIuBFVZAUo4Wjqwpu3VfTvCGGSI7vmMZw=; b=z1YCMYJwV4u7T1sNVHAC3h7gI3Q7k56jpxSLyvJRX3D5GfhJNIFP05gy8Y0FpE9NEMLL/4MSl 1WCUEPt1h8tAH7mYqnxVUByUNSnEvOicaThuXDmg0bin61FAkF+2Srv X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Use kvm_mem_is_private() when populating guest_memfd instead of using an open coded equivalent. In addition to simplifying the populate code *now*, this avoids the need to provide a range-based gmem lookup API in the future as well. No functional change intended. Signed-off-by: Sean Christopherson Suggested-by: Xiaoyao Li Reviewed-by: Xiaoyao Li Reviewed-by: Binbin Wu Reviewed-by: Fuad Tabba Signed-off-by: Ackerley Tng Reviewed-by: David Hildenbrand (Arm) --- virt/kvm/guest_memfd.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 2190e4d6b6bea..69612bf146be1 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -805,9 +805,7 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct= kvm_memory_slot *slot, =20 folio_unlock(folio); =20 - if (!kvm_range_has_memory_attributes(kvm, gfn, gfn + 1, - KVM_MEMORY_ATTRIBUTE_PRIVATE, - KVM_MEMORY_ATTRIBUTE_PRIVATE)) { + if (!kvm_mem_is_private(kvm, gfn)) { ret =3D -EINVAL; goto out_put_folio; } --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A23D279DB6; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=hQzyd4s78jT8oGoLt+DThfsdHDNFYXf4rEYNARtcNMM7qZjDrtkjIojEXvsT1wAld16S9v2nrJXlLWk9t3qLE+1OFRuGhzICDyQtAenKoTH03KczqwNf0G7+jREYTnoeZhD6Cmmx64EiJcefUdvmhpIfq6gtlLZp3bTV2g8qCx4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=cQsJwwAdgl6as/zP1eyJ1SShuRQR+8U5D8In8m4y84k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pP9sEJIw+scAvMqSE30hZH+HMw/3XePSkGH/F7gRCCtu5DS0tL/MQSk8bXMfaC3VpqljAPoRW2/YbLlW8Hr192lJklMp2tpIdMe20ZKk8rEimuO5nko54aNbWRFuzpuMqE7OfTrS67oMD9oj2UpH1VUDz6ZAbS9VDgtN41e8jfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dnvaMDDb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dnvaMDDb" Received: by smtp.kernel.org (Postfix) with ESMTPS id B9038C4AF49; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135919; bh=cQsJwwAdgl6as/zP1eyJ1SShuRQR+8U5D8In8m4y84k=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=dnvaMDDbpagrivQOgB3jNyFMdVBElgF6q1YNlCao3e5zSBVySwrG9lIVWkyJ1prpP uiGn/0IaXbkz7rH8mLVUO+aC89ObX/XghF/G5QKfQxDQRNSYO9AbZ50mspMf8A1liM X5BYI8Ic9GvJ5ZP14E3nZgBVJFcPLBYn4B4r2RPrUUJziAP8Ro3iOpI9qutPHMj7j1 3+wUS263OTV+te4AR5K3+cP0bqK2ujOA23/go/oCKBruYmdtBF/OIE7M9rYEyxxVV7 geLWU1l4vzwilqI1C9OLoXpcxvxgYRb8vjal51LKMcCzFome99f+s+u5yYRjG5+PUt Izjx4zw0e9ZlA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1BB4C61DE4; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:04 -0700 Subject: [PATCH v12 03/45] KVM: guest_memfd: Introduce per-gmem attributes, use to guard user mappings Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-3-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=8901; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=H+Foxo2jkk0EvS1xizl1rXzAIb0v1aN2W7mi7/WZF/o=; b=iafaIhiBr5nL66w1QN+aqiUa27rAU5GRN8Uxc7LJ6x2LcNgjQzD7Z0SFlKt1Y1NG1gHPxQd9S /In+BlOkxOfAGXNJ1fFsWiBjwYZqY+KMiA+gMkTPNmnocnV3X0byT4O X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Start plumbing in guest_memfd support for in-place private<=3D>shared conversions by tracking attributes via a maple tree. KVM currently tracks private vs. shared attributes on a per-VM basis, which made sense when a guest_memfd _only_ supported private memory, but tracking per-VM simply can't work for in-place conversions as the shared/private status of a given page needs to be per-gmem_inode, not per-VM. Use the filemap invalidation lock to protect the maple tree, as taking the lock for read when faulting in memory (for userspace or the guest) isn't expected to result in meaningful contention, and using a separate lock would add significant complexity (avoiding deadlock is quite difficult). In kvm_gmem_get_pfn(), drop the folio refcount before releasing filemap_invalidate_lock(). This ensures that a competing conversion request from userspace (to be added in a later patch), which also takes the filemap_invalidate_lock(), will never see an elevated refcount due to kvm_gmem_get_pfn(). Co-developed-by: Vishal Annapurve Signed-off-by: Vishal Annapurve Co-developed-by: Fuad Tabba Signed-off-by: Fuad Tabba Signed-off-by: Sean Christopherson Tested-by: Shivank Garg Reviewed-by: Xiaoyao Li Reviewed-by: Binbin Wu Co-developed-by: Ackerley Tng Signed-off-by: Ackerley Tng --- virt/kvm/guest_memfd.c | 136 ++++++++++++++++++++++++++++++++++++++++++---= ---- 1 file changed, 119 insertions(+), 17 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 69612bf146be1..e32ac2c28cbef 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -34,6 +35,13 @@ struct gmem_inode { struct list_head gmem_file_list; =20 u64 flags; + /* + * Every index in this inode, whether memory is populated or + * not, is tracked in attributes. The entire range of indices, + * corresponding to the size of this inode, is represented in + * this maple tree. + */ + struct maple_tree attributes; }; =20 static __always_inline struct gmem_inode *GMEM_I(struct inode *inode) @@ -61,9 +69,28 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot= *slot, gfn_t gfn) return gfn - slot->base_gfn + slot->gmem.pgoff; } =20 +static u64 kvm_gmem_get_default_attributes(struct inode *inode) +{ + bool init_shared =3D GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED; + + return init_shared ? 0 : KVM_MEMORY_ATTRIBUTE_PRIVATE; +} + +static u64 kvm_gmem_get_attributes(struct inode *inode, void *entry) +{ + if (WARN_ON_ONCE(!entry)) + return kvm_gmem_get_default_attributes(inode); + + return xa_to_value(entry); +} + static bool kvm_gmem_is_private_mem(struct inode *inode, pgoff_t index) { - return !(GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED); + struct maple_tree *mt =3D &GMEM_I(inode)->attributes; + void *entry =3D mtree_load(mt, index); + + return kvm_gmem_get_attributes(inode, entry) & + KVM_MEMORY_ATTRIBUTE_PRIVATE; } =20 static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index) @@ -364,10 +391,13 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct = vm_fault *vmf) if (((loff_t)vmf->pgoff << PAGE_SHIFT) >=3D i_size_read(inode)) return VM_FAULT_SIGBUS; =20 - if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) - return VM_FAULT_SIGBUS; + filemap_invalidate_lock_shared(inode->i_mapping); + if (kvm_gmem_is_shared_mem(inode, vmf->pgoff)) + folio =3D kvm_gmem_get_folio(inode, vmf->pgoff); + else + folio =3D ERR_PTR(-EACCES); + filemap_invalidate_unlock_shared(inode->i_mapping); =20 - folio =3D kvm_gmem_get_folio(inode, vmf->pgoff); if (IS_ERR(folio)) { if (PTR_ERR(folio) =3D=3D -EAGAIN) return VM_FAULT_RETRY; @@ -520,6 +550,51 @@ bool __weak kvm_arch_supports_gmem_init_shared(struct = kvm *kvm) return true; } =20 +static int kvm_gmem_init_inode(struct inode *inode, loff_t size, u64 flags) +{ + struct gmem_inode *gi =3D GMEM_I(inode); + MA_STATE(mas, &gi->attributes, 0, (size >> PAGE_SHIFT) - 1); + u64 attrs; + int r; + + inode->i_op =3D &kvm_gmem_iops; + inode->i_mapping->a_ops =3D &kvm_gmem_aops; + inode->i_mode |=3D S_IFREG; + inode->i_size =3D size; + mapping_set_gfp_mask(inode->i_mapping, GFP_HIGHUSER); + + /* + * guest_memfd memory is neither migratable nor swappable: set + * inaccessible to gate off both. + */ + mapping_set_inaccessible(inode->i_mapping); + WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping)); + + gi->flags =3D flags; + + mt_set_external_lock(&gi->attributes, + &inode->i_mapping->invalidate_lock); + + /* + * Store default attributes for the entire gmem instance. Ensuring every + * index is represented in the maple tree at all times simplifies the + * conversion and merging logic. + */ + attrs =3D kvm_gmem_get_default_attributes(inode); + + /* + * Acquire the invalidation lock purely to make lockdep happy. The + * maple tree library expects all stores to be protected via the lock, + * and the library can't know when the tree is reachable only by the + * caller, as is the case here. + */ + filemap_invalidate_lock(inode->i_mapping); + r =3D mas_store_gfp(&mas, xa_mk_value(attrs), GFP_KERNEL); + filemap_invalidate_unlock(inode->i_mapping); + + return r; +} + static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags) { static const char *name =3D "[kvm-gmem]"; @@ -550,16 +625,9 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t s= ize, u64 flags) goto err_fops; } =20 - inode->i_op =3D &kvm_gmem_iops; - inode->i_mapping->a_ops =3D &kvm_gmem_aops; - inode->i_mode |=3D S_IFREG; - inode->i_size =3D size; - mapping_set_gfp_mask(inode->i_mapping, GFP_HIGHUSER); - mapping_set_inaccessible(inode->i_mapping); - /* Unmovable mappings are supposed to be marked unevictable as well. */ - WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping)); - - GMEM_I(inode)->flags =3D flags; + err =3D kvm_gmem_init_inode(inode, size, flags); + if (err) + goto err_inode; =20 file =3D alloc_file_pseudo(inode, kvm_gmem_mnt, name, O_RDWR, &kvm_gmem_f= ops); if (IS_ERR(file)) { @@ -763,9 +831,13 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memor= y_slot *slot, if (!file) return -EFAULT; =20 + filemap_invalidate_lock_shared(file_inode(file)->i_mapping); + folio =3D __kvm_gmem_get_pfn(file, slot, index, pfn, max_order); - if (IS_ERR(folio)) - return PTR_ERR(folio); + if (IS_ERR(folio)) { + r =3D PTR_ERR(folio); + goto out; + } =20 if (!folio_test_uptodate(folio)) { clear_highpage(folio_page(folio, 0)); @@ -780,6 +852,8 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory= _slot *slot, folio_unlock(folio); folio_put(folio); =20 +out: + filemap_invalidate_unlock_shared(file_inode(file)->i_mapping); return r; } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_get_pfn); @@ -909,6 +983,15 @@ static struct inode *kvm_gmem_alloc_inode(struct super= _block *sb) =20 mpol_shared_policy_init(&gi->policy, NULL); =20 + /* + * Memory attributes are protected by the filemap invalidation lock, but + * the lock structure isn't available at this time. Immediately mark + * maple tree as using external locking so that accessing the tree + * before it's fully initialized results in NULL pointer dereferences + * and not more subtle bugs. + */ + mt_init_flags(&gi->attributes, MT_FLAGS_LOCK_EXTERN | MT_FLAGS_USE_RCU); + gi->flags =3D 0; INIT_LIST_HEAD(&gi->gmem_file_list); return &gi->vfs_inode; @@ -916,7 +999,26 @@ static struct inode *kvm_gmem_alloc_inode(struct super= _block *sb) =20 static void kvm_gmem_destroy_inode(struct inode *inode) { - mpol_free_shared_policy(&GMEM_I(inode)->policy); + struct gmem_inode *gi =3D GMEM_I(inode); + + mpol_free_shared_policy(&gi->policy); + + /* + * Note! Checking for an empty tree is functionally necessary + * to avoid explosions if the tree hasn't been fully + * initialized, i.e. if the inode is being destroyed before + * guest_memfd can set the external lock, lockdep would find + * that the tree's internal ma_lock was not held. + */ + if (!mtree_empty(&gi->attributes)) { + /* + * Acquire the invalidation lock purely to make lockdep happy, + * the inode is unreachable at this point. + */ + filemap_invalidate_lock(inode->i_mapping); + __mt_destroy(&gi->attributes); + filemap_invalidate_unlock(inode->i_mapping); + } } =20 static void kvm_gmem_free_inode(struct inode *inode) --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48FFD2BE639; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=iPdXSTiBijiSn80JnLNR2EVMMNMxup78I2KEAYQwlDEZZxv4Cnxbcz9IAEZVcV+6kSSjDhDZFLL7z/PYzup2DGUicQjEzMhJhBDofjqunk6VLR98F4cCs9VQE+1dgAFOHf9XIptbj/Ak35qbyZqzkDb1YTdXHSubNqwm3nGlhtA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=QNEr0l6uyABTQH9EbEs520Jpiwb+hrO9V0kR/2Vx/Og=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qRf4x/x7qq7gnLUpePI8fxo9VlT1c7gAcpB3OLTbKyZL+uPD1kTZT01z7iXjh3hsGbszEZrLB8E1YWN2r/iFdT0F3/ozPByjjK+BI4lwgzRCoYsMHXsY4EjaOeOrfCbBafW/390Q/OTJL/1DtUcDG+TNzm4+U7Sj1NiJiAYH0jA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=N57NtZdN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="N57NtZdN" Received: by smtp.kernel.org (Postfix) with ESMTPS id CE41DC2BCFC; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135919; bh=QNEr0l6uyABTQH9EbEs520Jpiwb+hrO9V0kR/2Vx/Og=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=N57NtZdNg6z4P9Wu+xdoNOedRMxc3pTUfgH7SBD7QSOxW8qeVxRTkmdiP5XaPY4oJ PgtXMya2JTo924GBF0N7NXTh2LR/eq61e7EiGM14btxukiE3h7b0rxvpl6eAeCRPCV YTqM9eaXQUL9NaOSkF6qb0NWYGZF+loLdBYE8tmbKTN+RE2ufDjpU83tDkmhnyI44Q ZwNzIV/oB44ZHd+08CIA4x05CnwQXWyGfLPHvRoIV05L02gppDOk/OwlB0T7DtAlpN D6bSxo3JXCM2mPFa0WaBLT2puAJnlpyf2h7q6uilaBSArTe6Ju47leXCfkzcb+31oF TA+d0WejkUojQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7D52C61DFD; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:05 -0700 Subject: [PATCH v12 04/45] KVM: Rename KVM_GENERIC_MEMORY_ATTRIBUTES to KVM_VM_MEMORY_ATTRIBUTES Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-4-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=8898; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=gEE74in80sRXX09YhOQRdl9xeeZyKCNSW0cL6P6ue3c=; b=cRirIa0UinZmq+XAQNsLF5Ync4aqcC5gAa1NQGwk1s9s/4QdDWzGgOHJdAhXbV/Z22KT62E8A 9BO+kRzZ25CDBlzVGEro2VWYD0aXEV41GRJqhYPRCuovYF4rA0D763V X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Rename the per-VM memory attributes Kconfig to make it explicitly about per-VM attributes in anticipation of adding memory attributes support to guest_memfd, at which point it will be possible (and desirable) to have memory attributes without the per-VM support, even in x86. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Reviewed-by: Xiaoyao Li Reviewed-by: Binbin Wu Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/Kconfig | 6 +++--- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 8 ++++---- include/trace/events/kvm.h | 4 ++-- virt/kvm/Kconfig | 2 +- virt/kvm/kvm_main.c | 14 +++++++------- 8 files changed, 20 insertions(+), 20 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 6beddd8a0c749..744c1f6ff03ed 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1851,7 +1851,7 @@ enum kvm_intr_type { ((vcpu) && (vcpu)->arch.handling_intr_from_guest && \ (!!in_nmi() =3D=3D ((vcpu)->arch.handling_intr_from_guest =3D=3D KVM_HAN= DLING_NMI))) =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES #define kvm_arch_has_private_mem(kvm) ((kvm)->arch.has_private_mem) #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 538ed1e803326..2f8e1b5a8a72c 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -84,7 +84,7 @@ config KVM_SW_PROTECTED_VM bool "Enable support for KVM software-protected VMs" depends on EXPERT depends on KVM_X86 && X86_64 - select KVM_GENERIC_MEMORY_ATTRIBUTES + select KVM_VM_MEMORY_ATTRIBUTES help Enable support for KVM software-protected VMs. Currently, software- protected VMs are purely a development and testing vehicle for @@ -135,7 +135,7 @@ config KVM_INTEL_TDX bool "Intel Trust Domain Extensions (TDX) support" default y depends on INTEL_TDX_HOST - select KVM_GENERIC_MEMORY_ATTRIBUTES + select KVM_VM_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_POPULATE help Provides support for launching Intel Trust Domain Extensions (TDX) @@ -159,7 +159,7 @@ config KVM_AMD_SEV depends on KVM_AMD && X86_64 depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=3Dy && CRYPTO_DEV_CCP_DD=3Dm) select ARCH_HAS_CC_PLATFORM - select KVM_GENERIC_MEMORY_ATTRIBUTES + select KVM_VM_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 947c9cd843450..b88c00e11854a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -8097,7 +8097,7 @@ void kvm_mmu_pre_destroy_vm(struct kvm *kvm) vhost_task_stop(kvm->arch.nx_huge_page_recovery_thread); } =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static bool hugepage_test_mixed(struct kvm_memory_slot *slot, gfn_t gfn, int level) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 79468ddfe4736..2292249570314 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10140,7 +10140,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *k= vm, } } =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES kvm_mmu_init_memslot_memory_attributes(kvm, slot); #endif =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index b7516e19ef419..cffa16647f2ab 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -722,7 +722,7 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm_= vcpu *vcpu) } #endif =20 -#ifndef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifndef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static inline bool kvm_arch_has_private_mem(struct kvm *kvm) { return false; @@ -872,7 +872,7 @@ struct kvm { #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER struct notifier_block pm_notifier; #endif -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES /* Protected by slots_lock (for writes) and RCU (for reads) */ struct xarray mem_attr_array; #endif @@ -2560,7 +2560,7 @@ static inline bool kvm_memslot_is_gmem_only(const str= uct kvm_memory_slot *slot) return slot->flags & KVM_MEMSLOT_GMEM_ONLY; } =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn= _t gfn) { return xa_to_value(xa_load(&kvm->mem_attr_array, gfn)); @@ -2582,7 +2582,7 @@ static inline bool kvm_mem_is_private(struct kvm *kvm= , gfn_t gfn) { return false; } -#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ +#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ =20 #ifdef CONFIG_KVM_GUEST_MEMFD int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h index b282e3a867696..1ba72bd73ea2f 100644 --- a/include/trace/events/kvm.h +++ b/include/trace/events/kvm.h @@ -358,7 +358,7 @@ TRACE_EVENT(kvm_dirty_ring_exit, TP_printk("vcpu %d", __entry->vcpu_id) ); =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES /* * @start: Starting address of guest memory range * @end: End address of guest memory range @@ -383,7 +383,7 @@ TRACE_EVENT(kvm_vm_set_mem_attributes, TP_printk("%#016llx -- %#016llx [0x%lx]", __entry->start, __entry->end, __entry->attr) ); -#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ +#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ =20 TRACE_EVENT(kvm_unmap_hva_range, TP_PROTO(unsigned long start, unsigned long end), diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig index c3c0ee253fc77..df575449d97b5 100644 --- a/virt/kvm/Kconfig +++ b/virt/kvm/Kconfig @@ -100,7 +100,7 @@ config KVM_ELIDE_TLB_FLUSH_IF_YOUNG config KVM_MMU_LOCKLESS_AGING bool =20 -config KVM_GENERIC_MEMORY_ATTRIBUTES +config KVM_VM_MEMORY_ATTRIBUTES bool =20 config KVM_GUEST_MEMFD diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 65eb26a0520d8..7d5325e3cb44b 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1116,7 +1116,7 @@ static struct kvm *kvm_create_vm(unsigned long type, = const char *fdname) spin_lock_init(&kvm->mn_invalidate_lock); rcuwait_init(&kvm->mn_memslots_update_rcuwait); xa_init(&kvm->vcpu_array); -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES xa_init(&kvm->mem_attr_array); #endif =20 @@ -1301,7 +1301,7 @@ static void kvm_destroy_vm(struct kvm *kvm) cleanup_srcu_struct(&kvm->irq_srcu); srcu_barrier(&kvm->srcu); cleanup_srcu_struct(&kvm->srcu); -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES xa_destroy(&kvm->mem_attr_array); #endif kvm_arch_free_vm(kvm); @@ -2419,7 +2419,7 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *k= vm, } #endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */ =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static u64 kvm_supported_mem_attributes(struct kvm *kvm) { if (!kvm || kvm_arch_has_private_mem(kvm)) @@ -2624,7 +2624,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm= *kvm, =20 return kvm_vm_set_mem_attributes(kvm, start, end, attrs->attributes); } -#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ +#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ =20 struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn) { @@ -4941,7 +4941,7 @@ static int kvm_vm_ioctl_check_extension_generic(struc= t kvm *kvm, long arg) case KVM_CAP_SYSTEM_EVENT_DATA: case KVM_CAP_DEVICE_CTRL: return 1; -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES case KVM_CAP_MEMORY_ATTRIBUTES: return kvm_supported_mem_attributes(kvm); #endif @@ -5345,7 +5345,7 @@ static long kvm_vm_ioctl(struct file *filp, break; } #endif /* CONFIG_HAVE_KVM_IRQ_ROUTING */ -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES case KVM_SET_MEMORY_ATTRIBUTES: { struct kvm_memory_attributes attrs; =20 @@ -5356,7 +5356,7 @@ static long kvm_vm_ioctl(struct file *filp, r =3D kvm_vm_ioctl_set_mem_attributes(kvm, &attrs); break; } -#endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ +#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ case KVM_CREATE_DEVICE: { struct kvm_create_device cd; =20 --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40F2229D260; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=nLWTfnPwxIbOaZOoU8zznHDgQjwrBQLMNV+n6XaRjECfDYekr9PT5C57sAhb0S3F+F2iGpeQRcfVfbzksYmJiEXS/60brU+hiqkYlO/wmP1A8NIuKq/8FfKY5WiyKpxGcMr+2lgp01GS17cYIL942Q002f8Y+ygW9o5Ku1gkPKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=JofTZBDWcojn6WUeVdQV9Y+aeDYMG83KVX4oKmgX9pQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PydePSlu68aOZ4hhvCiO55P0vJhwfl5F2LaGpxJGbJKf08SH+7vKkW6muThzoF5/BraevHb7NsGBxtk/Xj3hgSbEccfQOTlMwoWpHN0g8yefaTGo7nHTbuv2YwqbpFAaAJ2n3cwQGf4byfVhtQUc3up0M5J2RwlrbgLMqAh5tHk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sIapqrWK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sIapqrWK" Received: by smtp.kernel.org (Postfix) with ESMTPS id EA848C4AF18; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=JofTZBDWcojn6WUeVdQV9Y+aeDYMG83KVX4oKmgX9pQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=sIapqrWK8NZxJH1HWgqX3hV9HnvBMlzwJNaSEcGDFZL3S1oa48uCXrhAVvMnAf3C7 rYYWKQ3swEFgW+rexkGgUsNF7Ctdj05Z6Wyza4FsMw8yi6jA5PECRcmrNTghFiC1Hq gO10D4S2dMvXVTYiWC2u27ZxbZct5K3ktddLN691Cp4qGFTb5XMLNfamUCPm7320Hz LIer4734/wS9RrlJJLF649fKrWeJBXeR7AME0HPW/pnPL2gX5wHn9TTc+7QeaJvArj 3+/RTWkvOpBkAPLdZty8pgN0zzAF5T01cR4gXXpqUiRcHesk8ghuVCq6PNsO26uyer ufMmZO+qp2P7w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD993C61DFF; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:06 -0700 Subject: [PATCH v12 05/45] KVM: Enumerate support for PRIVATE memory iff kvm_arch_has_private_mem is defined Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-5-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=2388; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=Npw45RhiuLmUU1MwINVjbSZRhJ+VqvMGA2O5ckbiMOE=; b=hcW6l//v6f2eUwZw8P7Qz727FIbcXf31OIfmzbJ/LOZhc5nkpROp8zwh5hMXWqpp7ERrejoqB F7AAbMsg+GBDHdwp37rBq/IND3M1JNVanBPieJM9VYyO2L3kDVkp0n3 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Explicitly guard reporting support for KVM_MEMORY_ATTRIBUTE_PRIVATE based on kvm_arch_has_private_mem being #defined in anticipation of tracking PRIVATE vs. SHARED state per-guest_memfd, not per-VM (to allow in-place conversion). guest_memfd support for memory attributes is expected to be unconditional to avoid yet more macros (all architectures that support guest_memfd are expected to use per-gmem attributes at some point), at which point enumerating support KVM_MEMORY_ATTRIBUTE_PRIVATE based solely on memory attributes being supported by KVM at-large would result in a system-scope check (NULL @kvm) over-reporting support on arm64. Give architectures full control over overriding the default definition of kvm_arch_has_private_mem() by removing the coupling with CONFIG_KVM_VM_MEMORY_ATTRIBUTES. In a later patch, kvm_arch_has_private_mem() will be defined based on whether architectural features are compiled in, and made orthogonal to CONFIG_KVM_VM_MEMORY_ATTRIBUTES. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Reviewed-by: Binbin Wu Tested-by: Shivank Garg Reviewed-by: Xiaoyao Li Signed-off-by: Ackerley Tng --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cffa16647f2ab..b45fb80b0aca0 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -722,7 +722,7 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm_= vcpu *vcpu) } #endif =20 -#ifndef CONFIG_KVM_VM_MEMORY_ATTRIBUTES +#ifndef kvm_arch_has_private_mem static inline bool kvm_arch_has_private_mem(struct kvm *kvm) { return false; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7d5325e3cb44b..050757544a1cb 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2422,8 +2422,10 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *= kvm, #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static u64 kvm_supported_mem_attributes(struct kvm *kvm) { +#ifdef kvm_arch_has_private_mem if (!kvm || kvm_arch_has_private_mem(kvm)) return KVM_MEMORY_ATTRIBUTE_PRIVATE; +#endif =20 return 0; } --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5520F2C08CF; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=alaQLiGqN9rHujFayIwfQfJigLVw7+ykwg8j5lkXWekwrgEsvvmr0YfPGr6XOdwjZjpbofyGX/PRvuvdQFsQfDPBItVQp2J+CtKPq9pp6uHk7w//fP8DKDD6w5uxNWEOgmVBD6YDrpCaGuFcibNYG4OTvblz/WayBVmtjjEsiag= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=SvZvS9iw5qz4DENwf1yMZryvWfhqxycGJzjcxbe+3Qg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LOmKEshboVpgDLeyOkAbNoqqzi5EjnDlSGlhyxkqOMJDEBphk5JOWfU7qaMRlOX3U6lx4DJE9NHix0qPUwUmezW73qFjCIMzb7V7muCNeaxAckOTVOn+blCDThLs1TogIMZNgXlmnPWZi4VHMbqnOc3WFt/Z6VaTPdXXMvc81Z0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=P3lVJDcx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="P3lVJDcx" Received: by smtp.kernel.org (Postfix) with ESMTPS id 0F1A8C4AF0C; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=SvZvS9iw5qz4DENwf1yMZryvWfhqxycGJzjcxbe+3Qg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=P3lVJDcxpiRVC5hXkL/EEih9ymmMyU+gDGgUd4Vx4XndiyGKuKuVSD7yRzNKhx1ul YxEBuMGnf4zvkA3RDHXiDZhnRwTQBFzxCGnK8G0fmrh7Uw8IYKPT2Pw4BBKl8PBqMl R48M7ZHGVn1fVZiii/H7vXc54NR8/FIJE3XV2nS1rj1p0IVSrc9NPJfUmZ7IKFLxPS ZZUx4TZKU64c9uxTuLxnnxQMJI75LKEwp4itRRDdy/n2p3wqe60gZ72wrH2dpg2FSk 5XzvnUcZ1vBIYxHdJlwjrkS5CdxlHPZEIhrtg7d1zXSyYarwe6uZurw/J+BiE8pRij 8iF6BS2s/IsEQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6261C61DE2; Mon, 31 Aug 2026 00:25:19 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:07 -0700 Subject: [PATCH v12 06/45] KVM: Rename memory attribute APIs to prepare for in-place gmem conversion Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-6-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=10021; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=fDMvsndjh9kiPdU+EOjmSK37yD3KVUvdGEDhGSeU9Yc=; b=lB0XddyerqtBMcJR1vCoGfSIBRTqg4NdDZJdpmaLJfB/IBI68i71qvfQ+s493eCuKY+GqFdQL ZUqx1UZYxAABwOt4BMeuLBbc0xD6bsPRxhFLtXiZ3XjBYhEJO7ifu42 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Rename memory attribute APIs to add a "vm_" in the name in anticipation of moving PRIVATE tracking into guest_memfd, to allow in-place conversion between SHARED and PRIVATE. At that point, there will effectively be two (potential) sources of memory attributes: the VM and guest_memfd. kvm_vm_set_mem_attributes() already has "vm" in the name to indicate that it is a VM ioctl. Rename it to kvm_set_vm_mem_attributes() to show that it is setting the VM's memory attributes. (Drop the VM-ioctl scoping since it is a helper local to the file.) Update the accompanying trace function to match. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Reviewed-by: Xiaoyao Li Reviewed-by: Binbin Wu Signed-off-by: Ackerley Tng --- arch/x86/kvm/mmu/mmu.c | 14 +++++++------- include/linux/kvm_host.h | 16 ++++++++-------- include/trace/events/kvm.h | 2 +- virt/kvm/kvm_main.c | 32 ++++++++++++++++---------------- 4 files changed, 32 insertions(+), 32 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b88c00e11854a..10e59507d252f 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -8116,8 +8116,8 @@ static void hugepage_set_mixed(struct kvm_memory_slot= *slot, gfn_t gfn, lpage_info_slot(gfn, slot, level)->disallow_lpage |=3D KVM_LPAGE_MIXED_FL= AG; } =20 -bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, - struct kvm_gfn_range *range) +bool kvm_arch_pre_set_vm_memory_attributes(struct kvm *kvm, + struct kvm_gfn_range *range) { struct kvm_memory_slot *slot =3D range->slot; int level; @@ -8186,18 +8186,18 @@ static bool hugepage_has_attrs(struct kvm *kvm, str= uct kvm_memory_slot *slot, const unsigned long end =3D start + KVM_PAGES_PER_HPAGE(level); =20 if (level =3D=3D PG_LEVEL_2M) - return kvm_range_has_memory_attributes(kvm, start, end, ~0, attrs); + return kvm_range_has_vm_memory_attributes(kvm, start, end, ~0, attrs); =20 for (gfn =3D start; gfn < end; gfn +=3D KVM_PAGES_PER_HPAGE(level - 1)) { if (hugepage_test_mixed(slot, gfn, level - 1) || - attrs !=3D kvm_get_memory_attributes(kvm, gfn)) + attrs !=3D kvm_get_vm_memory_attributes(kvm, gfn)) return false; } return true; } =20 -bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, - struct kvm_gfn_range *range) +bool kvm_arch_post_set_vm_memory_attributes(struct kvm *kvm, + struct kvm_gfn_range *range) { unsigned long attrs =3D range->arg.attributes; struct kvm_memory_slot *slot =3D range->slot; @@ -8290,7 +8290,7 @@ void kvm_mmu_init_memslot_memory_attributes(struct kv= m *kvm, * be manually checked as the attributes may already be mixed. */ for (gfn =3D start; gfn < end; gfn +=3D nr_pages) { - unsigned long attrs =3D kvm_get_memory_attributes(kvm, gfn); + unsigned long attrs =3D kvm_get_vm_memory_attributes(kvm, gfn); =20 if (hugepage_has_attrs(kvm, slot, gfn, level, attrs)) hugepage_clear_mixed(slot, gfn, level); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index b45fb80b0aca0..6ca81f5349db8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2561,21 +2561,21 @@ static inline bool kvm_memslot_is_gmem_only(const s= truct kvm_memory_slot *slot) } =20 #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES -static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn= _t gfn) +static inline unsigned long kvm_get_vm_memory_attributes(struct kvm *kvm, = gfn_t gfn) { return xa_to_value(xa_load(&kvm->mem_attr_array, gfn)); } =20 -bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t e= nd, - unsigned long mask, unsigned long attrs); -bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, - struct kvm_gfn_range *range); -bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, - struct kvm_gfn_range *range); +bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_= t end, + unsigned long mask, unsigned long attrs); +bool kvm_arch_pre_set_vm_memory_attributes(struct kvm *kvm, + struct kvm_gfn_range *range); +bool kvm_arch_post_set_vm_memory_attributes(struct kvm *kvm, + struct kvm_gfn_range *range); =20 static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) { - return kvm_get_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIVATE; + return kvm_get_vm_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIV= ATE; } #else static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h index 1ba72bd73ea2f..c4d514d2350cf 100644 --- a/include/trace/events/kvm.h +++ b/include/trace/events/kvm.h @@ -364,7 +364,7 @@ TRACE_EVENT(kvm_dirty_ring_exit, * @end: End address of guest memory range * @attr: The value of the attribute being set. */ -TRACE_EVENT(kvm_vm_set_mem_attributes, +TRACE_EVENT(kvm_set_vm_mem_attributes, TP_PROTO(gfn_t start, gfn_t end, unsigned long attr), TP_ARGS(start, end, attr), =20 diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 050757544a1cb..91d5af7d60d0f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2420,7 +2420,7 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *k= vm, #endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */ =20 #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES -static u64 kvm_supported_mem_attributes(struct kvm *kvm) +static u64 kvm_supported_vm_mem_attributes(struct kvm *kvm) { #ifdef kvm_arch_has_private_mem if (!kvm || kvm_arch_has_private_mem(kvm)) @@ -2434,19 +2434,19 @@ static u64 kvm_supported_mem_attributes(struct kvm = *kvm) * Returns true if _all_ gfns in the range [@start, @end) have attributes * such that the bits in @mask match @attrs. */ -bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t e= nd, - unsigned long mask, unsigned long attrs) +bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_= t end, + unsigned long mask, unsigned long attrs) { XA_STATE(xas, &kvm->mem_attr_array, start); unsigned long index; void *entry; =20 - mask &=3D kvm_supported_mem_attributes(kvm); + mask &=3D kvm_supported_vm_mem_attributes(kvm); if (attrs & ~mask) return false; =20 if (end =3D=3D start + 1) - return (kvm_get_memory_attributes(kvm, start) & mask) =3D=3D attrs; + return (kvm_get_vm_memory_attributes(kvm, start) & mask) =3D=3D attrs; =20 guard(rcu)(); if (!attrs) @@ -2517,8 +2517,8 @@ static __always_inline void kvm_handle_gfn_range(stru= ct kvm *kvm, KVM_MMU_UNLOCK(kvm); } =20 -static bool kvm_pre_set_memory_attributes(struct kvm *kvm, - struct kvm_gfn_range *range) +static bool kvm_pre_set_vm_memory_attributes(struct kvm *kvm, + struct kvm_gfn_range *range) { /* * Unconditionally add the range to the invalidation set, regardless of @@ -2533,18 +2533,18 @@ static bool kvm_pre_set_memory_attributes(struct kv= m *kvm, */ kvm_mmu_invalidate_range_add(kvm, range->start, range->end); =20 - return kvm_arch_pre_set_memory_attributes(kvm, range); + return kvm_arch_pre_set_vm_memory_attributes(kvm, range); } =20 /* Set @attributes for the gfn range [@start, @end). */ -static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t e= nd, +static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t e= nd, unsigned long attributes) { struct kvm_mmu_notifier_range pre_set_range =3D { .start =3D start, .end =3D end, .arg.attributes =3D attributes, - .handler =3D kvm_pre_set_memory_attributes, + .handler =3D kvm_pre_set_vm_memory_attributes, .on_lock =3D kvm_mmu_invalidate_start, .flush_on_ret =3D true, .may_block =3D true, @@ -2553,7 +2553,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm,= gfn_t start, gfn_t end, .start =3D start, .end =3D end, .arg.attributes =3D attributes, - .handler =3D kvm_arch_post_set_memory_attributes, + .handler =3D kvm_arch_post_set_vm_memory_attributes, .on_lock =3D kvm_mmu_invalidate_end, .may_block =3D true, }; @@ -2563,12 +2563,12 @@ static int kvm_vm_set_mem_attributes(struct kvm *kv= m, gfn_t start, gfn_t end, =20 entry =3D attributes ? xa_mk_value(attributes) : NULL; =20 - trace_kvm_vm_set_mem_attributes(start, end, attributes); + trace_kvm_set_vm_mem_attributes(start, end, attributes); =20 mutex_lock(&kvm->slots_lock); =20 /* Nothing to do if the entire range has the desired attributes. */ - if (kvm_range_has_memory_attributes(kvm, start, end, ~0, attributes)) + if (kvm_range_has_vm_memory_attributes(kvm, start, end, ~0, attributes)) goto out_unlock; =20 /* @@ -2607,7 +2607,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm= *kvm, /* flags is currently not used. */ if (attrs->flags) return -EINVAL; - if (attrs->attributes & ~kvm_supported_mem_attributes(kvm)) + if (attrs->attributes & ~kvm_supported_vm_mem_attributes(kvm)) return -EINVAL; if (attrs->size =3D=3D 0 || attrs->address + attrs->size < attrs->address) return -EINVAL; @@ -2624,7 +2624,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm= *kvm, */ BUILD_BUG_ON(sizeof(attrs->attributes) !=3D sizeof(unsigned long)); =20 - return kvm_vm_set_mem_attributes(kvm, start, end, attrs->attributes); + return kvm_set_vm_mem_attributes(kvm, start, end, attrs->attributes); } #endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ =20 @@ -4945,7 +4945,7 @@ static int kvm_vm_ioctl_check_extension_generic(struc= t kvm *kvm, long arg) return 1; #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES case KVM_CAP_MEMORY_ATTRIBUTES: - return kvm_supported_mem_attributes(kvm); + return kvm_supported_vm_mem_attributes(kvm); #endif #ifdef CONFIG_KVM_GUEST_MEMFD case KVM_CAP_GUEST_MEMFD: --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D7E32D238F; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=N7wpsruehEiPCpZG9yoJBI5oZqk+SMX3PNRQkScUVWvo9T7OVpILagvQdwlunmK5BLR8eHYjdMbcER8iV1ZV4e4HVA3dAVm2UwiOJU1JHO9YjS1JrbFhsdf0MMbr/72nnpKCRwydDFNTc87ZY9mRCEiUdhlfT85/g9CmktX2ENU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=KrGDcS6cy/AkD2yrXWcr9bAXsvRzIR53iwDweGRLZFc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HlFWim/qhbcwFrk74nEmcgrw16zOqPvBArlSG7TWplFnBFCnwqEc1177chI/nZ/kTBqMjps13cd7lIs5VD35SdgNyq+OnkQBiIvLn7qZV0I2M7StgfxvhN9Y5tGP0gEoGOqHRmc+qXqYhK6dIEpPBZz1ei7xLGZld/ZNUsL3uSE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XKbQ1hSV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XKbQ1hSV" Received: by smtp.kernel.org (Postfix) with ESMTPS id 2C195C4AF14; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=KrGDcS6cy/AkD2yrXWcr9bAXsvRzIR53iwDweGRLZFc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=XKbQ1hSVBXleF7yR17zPg82ohUAEiIS46Q0ZlPicqWp1yRIB6Z9cJq/VgdnjI88th XpAshl6srDv4EWR/vuHEIHeCL6UdcCwkZiFPvzS4nghBFASKfrgA2iDNQRp0vcOYSR MDeyAu4662sg8SjWhMyULET+EDh3TQsk34YxyBHMoxD8ESoboVqcVpNjftMQjWarrN sl2d86SpRTYeg3I/QGHwwSSgLtqfb8FfqtpJ1g7TWIwSdxeccjEvI47BONF2obyMeC K4C2610qBhHIU2XQfm/9KgeMN9qAMgYDR56XQJyOnXntujcB01zp3L99T2T08gjk4f eDQ+bpVLqSCng== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08564C61DF1; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:08 -0700 Subject: [PATCH v12 07/45] KVM: Rename kvm_mem_is_private() to kvm_is_private_gfn() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-7-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=4963; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=S5RCe4TQDZm3nHFvcXoNSPDbyF0DJXsRh4I6ecZKWnA=; b=8QOHgDh+p+0BDtEfm9STQE+bY0kNr2gTTBgbFMNHSzegw2N+ibVWgWMTF9CwpV50TAvoJ8ISN enLTjDrdcz9ByffglEZQ7oQcBo8ZjVcRqMu4mx1BIJ44peRDH7MnQlf X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Rename kvm_mem_is_private() to kvm_is_private_gfn() to prepare for in-place conversion, where there will be two lookup functions, kvm_vm_is_private_gfn() and kvm_gmem_is_private_gfn(). This renaming allows consistent prefixing of "vm" vs "gmem" for kvm_*_is_private_gfn(), as opposed to kvm_gmem_mem_is_private(), which looks like a typo. Suggested-by: Sean Christopherson Signed-off-by: Ackerley Tng Reviewed-by: Binbin Wu Reviewed-by: Fuad Tabba --- arch/x86/kvm/mmu/mmu.c | 12 ++++++------ arch/x86/kvm/svm/sev.c | 2 +- include/linux/kvm_host.h | 4 ++-- virt/kvm/guest_memfd.c | 2 +- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 10e59507d252f..62f751952ad8a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3421,7 +3421,7 @@ int kvm_mmu_max_mapping_level(struct kvm *kvm, struct= kvm_page_fault *fault, is_private =3D fault->is_private; } else { max_level =3D PG_LEVEL_NUM; - is_private =3D kvm_mem_is_private(kvm, gfn); + is_private =3D kvm_is_private_gfn(kvm, gfn); } =20 max_level =3D min(max_level, max_huge_page_level); @@ -3634,13 +3634,13 @@ static bool page_fault_can_be_fast(struct kvm *kvm,= struct kvm_page_fault *fault * guest spinning on a #PF indefinitely, so don't attempt the fast path * in this case. * - * Note that the kvm_mem_is_private() check might race with an + * Note that the kvm_is_private_gfn() check might race with an * attribute update, but this will either result in the guest spinning * on RET_PF_SPURIOUS until the update completes, or an actual spurious * case might go down the slow path. Either case will resolve itself. */ if (kvm->arch.has_private_mem && - fault->is_private !=3D kvm_mem_is_private(kvm, fault->gfn)) + fault->is_private !=3D kvm_is_private_gfn(kvm, fault->gfn)) return false; =20 /* @@ -4708,7 +4708,7 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * Now that we have a snapshot of mmu_invalidate_seq we can check for a * private vs. shared mismatch. */ - if (fault->is_private !=3D kvm_mem_is_private(kvm, fault->gfn)) { + if (fault->is_private !=3D kvm_is_private_gfn(kvm, fault->gfn)) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); return -EFAULT; } @@ -5111,7 +5111,7 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *= vcpu, =20 direct_bits =3D 0; if (kvm_arch_has_private_mem(vcpu->kvm) && - kvm_mem_is_private(vcpu->kvm, gpa_to_gfn(range->gpa))) + kvm_is_private_gfn(vcpu->kvm, gpa_to_gfn(range->gpa))) error_code |=3D PFERR_PRIVATE_ACCESS; else direct_bits =3D gfn_to_gpa(kvm_gfn_direct_bits(vcpu->kvm)); @@ -6584,7 +6584,7 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu= , gpa_t cr2_or_gpa, u64 err if (IS_ENABLED(CONFIG_KVM_SW_PROTECTED_VM) && !(error_code & PFERR_RSVD_MASK) && vcpu->kvm->arch.vm_type =3D=3D KVM_X86_SW_PROTECTED_VM && - kvm_mem_is_private(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) + kvm_is_private_gfn(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) error_code |=3D PFERR_PRIVATE_ACCESS; =20 r =3D RET_PF_INVALID; diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 0a887f8e05d3a..dff684b88f153 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -5030,7 +5030,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_= t gpa, u64 error_code) * userspace via KVM_EXIT_MEMORY_FAULT events, however, so RMP faults * for shared pages should not end up here. */ - if (!kvm_mem_is_private(kvm, gfn)) { + if (!kvm_is_private_gfn(kvm, gfn)) { pr_warn_ratelimited("SEV: Unexpected RMP fault for non-private GPA 0x%ll= x\n", gpa); return; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 6ca81f5349db8..9b97266f61ec8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2573,12 +2573,12 @@ bool kvm_arch_pre_set_vm_memory_attributes(struct k= vm *kvm, bool kvm_arch_post_set_vm_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); =20 -static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) +static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) { return kvm_get_vm_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIV= ATE; } #else -static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) +static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) { return false; } diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index e32ac2c28cbef..ce5efcacda36e 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -879,7 +879,7 @@ static long __kvm_gmem_populate(struct kvm *kvm, struct= kvm_memory_slot *slot, =20 folio_unlock(folio); =20 - if (!kvm_mem_is_private(kvm, gfn)) { + if (!kvm_is_private_gfn(kvm, gfn)) { ret =3D -EINVAL; goto out_put_folio; } --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F4002D7DD7; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=Ues9nfhwWoDqIf8ZQ34BXDuKswN0HuwHV+EKweuj97QVzr/HqC1mjvbruzfPqX85z80UZ+9K5GLmlRWvu8bZ9Esfhj3Wa85M6WwqghxTRPn3iVlLuNqQd5MZgE47Uy/mjeduO012siG8U54M0u5Jmy+x3lMWqNT6OFzDVrM6C9c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=NVIB4YZ2kDgX3qlQ+0A5ajMihiiTtAsgO51nyX8IOus=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HLI/igEawN3X0sfbemRZJ7UaxcdEPqwpJAvPrBJfzH2T/gcyvYY0e0VSyYYbnKJAQBKoeCYofW/oqN+jk3UUmj2h0cp04BHIWvq1owrcg/H3g/Adq2TpU1rjhsGMDPKG8Zyo6IFZFvZ1McBRxykvyCLBj3lYrXfNvuffrQKmWcU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lsl6nZXs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lsl6nZXs" Received: by smtp.kernel.org (Postfix) with ESMTPS id 46ADFC4DDE6; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=NVIB4YZ2kDgX3qlQ+0A5ajMihiiTtAsgO51nyX8IOus=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=lsl6nZXsNgATcI3jmXs23yepveXd5oGimMzXreiqX7FTUIyIpm7Q3iHUFGX5N7ODI zSJhA9Xavsd/pCjLWBrJPF8et7EzFZnqUp2GnshDhpBYzHzKsUJAAtMnbJS50i4NuX MOe8YoOey3AcRCSk42GNi/p0gQLtyREsFUolSwH/KlMAeccEtQpZnOtCHusOtOSAQq jCROPdORw5fz5jskYdTh+pHXCzkQbeDVYcSR1I84LE842SlJOjbtfOzXx7LNK/q+Ih t2AJJgSILt+IQiV4Kr+1M5/GZ4XSwrnMamAlfeGIeyZ5bfkLWEpawnwbAnankzHpqV 3qqkvbbkB1p6w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2341FC61DFF; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:09 -0700 Subject: [PATCH v12 08/45] KVM: Provide generic interface for checking memory private/shared status Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-8-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=3261; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=FbOIpdQVO+RwvRFKHPGy4BnPCrkZsXfvGDUTuDvfNzg=; b=gQ9Z+DSRv+BLdUurVmhuuUYQtb0izEpsgSvFLb1Fr/+8fxBISS4sS0q/umexz2Tl4hikiChn8 OqSEFW1Rlh+D9nERLQLR9zZUdBpXbj8Dk6OpWJlGXSDA6W1ca0RGYv0 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Introduce a generic kvm_is_private_gfn() interface using a static call to determine if a GFN is private. This allows the implementation for checking a GFN's private/shared status to be set at runtime. In preparation for choosing implementations between a guest_memfd lookup and the existing VM attribute lookup, rename the existing VM-attribute-based check to kvm_vm_is_private_gfn() to emphasize that it looks up VM attributes. Signed-off-by: Sean Christopherson Reviewed-by: Xiaoyao Li Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Reviewed-by: Binbin Wu Signed-off-by: Ackerley Tng --- include/linux/kvm_host.h | 14 ++++++++++++-- virt/kvm/kvm_main.c | 15 +++++++++++++++ 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9b97266f61ec8..3579feb631775 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2573,16 +2573,26 @@ bool kvm_arch_pre_set_vm_memory_attributes(struct k= vm *kvm, bool kvm_arch_post_set_vm_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); =20 -static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) +static inline bool kvm_vm_is_private_gfn(struct kvm *kvm, gfn_t gfn) { return kvm_get_vm_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIV= ATE; } +#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ + +#ifdef kvm_arch_has_private_mem +typedef bool (kvm_is_private_gfn_t)(struct kvm *kvm, gfn_t gfn); +DECLARE_STATIC_CALL(__kvm_is_private_gfn, kvm_is_private_gfn_t); + +static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) +{ + return static_call(__kvm_is_private_gfn)(kvm, gfn); +} #else static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) { return false; } -#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ +#endif /* kvm_arch_has_private_mem */ =20 #ifdef CONFIG_KVM_GUEST_MEMFD int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 91d5af7d60d0f..78278730b2ae4 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2628,6 +2628,20 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kv= m *kvm, } #endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ =20 +#ifdef kvm_arch_has_private_mem +DEFINE_STATIC_CALL_RET0(__kvm_is_private_gfn, kvm_is_private_gfn_t); +EXPORT_STATIC_CALL_GPL(__kvm_is_private_gfn); + +static void kvm_init_memory_attributes(void) +{ +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES + static_call_update(__kvm_is_private_gfn, kvm_vm_is_private_gfn); +#endif +} +#else +static void kvm_init_memory_attributes(void) { } +#endif + struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn) { return __gfn_to_memslot(kvm_memslots(kvm), gfn); @@ -6541,6 +6555,7 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align,= struct module *module) kvm_preempt_ops.sched_in =3D kvm_sched_in; kvm_preempt_ops.sched_out =3D kvm_sched_out; =20 + kvm_init_memory_attributes(); kvm_init_debug(); =20 r =3D kvm_vfio_ops_init(); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A45E32DF132; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=lw4MixwvTfJjA7g1XmW6r7D0yofQ73FLgHIaPgquVIKgmw2HxGYInBykX7rAn6alliu5v7jaZiZM7pq/kFeXUFpXYNtqWnhVGeUy5qUEslGHfTPXeYNX6gUsn18U8e0OfvidQUXVVKDLAJlRGUoNcdfW64RZKzsWkovfYw4wcKc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=Fli/DnBLQa0bHgyS0nPH35ihIsGgdhE8vQogj3yPsdk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FGjIQRZmD1h3Ga/6l4ppUAQ69UDDg3bUuI5VCzg1QnAXcmEDoTIkAtV97ZGMncTP5Tzqk91RqT0EG2SMESoWRcuRKCG/Zica2m2swIrliRs6L3wV7bP65DRRh/eWTmzl8O2FAmk66+Oo6+vBrmtHQ3gZoMszJzvkVLJfu//+sjg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fbGX/joD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fbGX/joD" Received: by smtp.kernel.org (Postfix) with ESMTPS id 5A134C4AF61; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=Fli/DnBLQa0bHgyS0nPH35ihIsGgdhE8vQogj3yPsdk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=fbGX/joDHj66klVr6YZzDrsFN+qzMM1kRbN4852/HbLwsIeGF7U1cGvb0uaqY5Z3J g2Vifm6ygULXaJESEtqgs2j+Am/cR+KR+IK5eczGm/IPxQRYBejGSDPv04zX9kP60z yfkkwuMx18hdP2NBjSOy3F4ZB7xXOuyJqNDNxvrszjnjCLXen6F1ZnGUmjS7UWtc8J 5Oxmu8xkZ0FRGFDzMFUjBw6i9U7ANRDOuMg6Ya7RQLnMC5tDNTz6rjNtwKZ8Y2oqLB ugw8O24hSnGCCJeYYGtEnHAzsCMSuQWAgtRP411h7W5Zml/vMUsLfL8+9OKLqbNLaN DnRGNIF1rLdrw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42B33C61DE4; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:10 -0700 Subject: [PATCH v12 09/45] KVM: guest_memfd: Stub in ability to enable in-place shared<=>private conversion Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-9-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=5719; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=fuPA5g+KPLXd1J5vplOVCQXEWYNrFEGb5t+IM2iiuDA=; b=Zl/GFK2YJOqfEO1qjinm3ljaqYpgc7Xmo3lnukUV3NCuYapl+/+wXFtGqMiNeGWakN0SxQX15 TjkhIIejtvfAdDTFm/ry6aZXV6coxoMu8iqP9u9A4q7YByz7s7W68ws X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Stub in global variable to enable in-place guest_memfd private<=3D>shared memory conversion, which will eventually be exposed to userspace via a module param, and wire up the __kvm_is_private_gfn() static call to the guest_memfd version when in-place conversion is enabled, i.e. when gmem is the sole authority on private vs. shared memory. Cc: Fuad Tabba Cc: Xiaoyao Li Signed-off-by: Sean Christopherson Co-developed-by: Ackerley Tng Signed-off-by: Ackerley Tng Reviewed-by: Binbin Wu Reviewed-by: David Hildenbrand (Arm) Reviewed-by: Fuad Tabba --- Documentation/virt/kvm/api.rst | 7 +++++++ include/linux/kvm_host.h | 6 ++++++ virt/kvm/guest_memfd.c | 26 ++++++++++++++++++++++++++ virt/kvm/kvm_main.c | 12 +++++++++++- 4 files changed, 50 insertions(+), 1 deletion(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index e0430cc750c9e..90a29424c54c8 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6382,6 +6382,11 @@ KVM_SET_USER_MEMORY_REGION2, e.g. shared memory can = be lazily mapped/allocated on-demand. =20 When mapping a gfn into the guest, KVM selects shared vs. private, i.e con= sumes +userspace_addr vs. guest_memfd, based on the state in guest_memfd, which i= s the +sole authority on private vs. shared memory. See :ref:`KVM_CREATE_GUEST_M= EMFD` +to find out more about the creation-time shared/private status. + +If in-place conversion is disabled, KVM selects shared vs. private, i.e co= nsumes userspace_addr vs. guest_memfd, based on the gfn's KVM_MEMORY_ATTRIBUTE_PR= IVATE state. At VM creation time, all memory is shared, i.e. the PRIVATE attrib= ute is '0' for all gfns. Userspace can control whether memory is shared/priva= te by @@ -6429,6 +6434,8 @@ the state of a gfn/page as needed. =20 The "flags" field is reserved for future extensions and must be '0'. =20 +.. _KVM_CREATE_GUEST_MEMFD: + 4.142 KVM_CREATE_GUEST_MEMFD ---------------------------- =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3579feb631775..b0110143d2ff8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2580,6 +2580,8 @@ static inline bool kvm_vm_is_private_gfn(struct kvm *= kvm, gfn_t gfn) #endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ =20 #ifdef kvm_arch_has_private_mem +extern bool gmem_in_place_conversion; + typedef bool (kvm_is_private_gfn_t)(struct kvm *kvm, gfn_t gfn); DECLARE_STATIC_CALL(__kvm_is_private_gfn, kvm_is_private_gfn_t); =20 @@ -2588,6 +2590,8 @@ static inline bool kvm_is_private_gfn(struct kvm *kvm= , gfn_t gfn) return static_call(__kvm_is_private_gfn)(kvm, gfn); } #else +#define gmem_in_place_conversion false + static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) { return false; @@ -2595,6 +2599,8 @@ static inline bool kvm_is_private_gfn(struct kvm *kvm= , gfn_t gfn) #endif /* kvm_arch_has_private_mem */ =20 #ifdef CONFIG_KVM_GUEST_MEMFD +bool kvm_gmem_is_private_gfn(struct kvm *kvm, gfn_t gfn); + int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, int *max_order); #else diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index ce5efcacda36e..cd966eef45b69 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -478,6 +478,32 @@ static int kvm_gmem_mmap(struct file *file, struct vm_= area_struct *vma) return 0; } =20 +bool kvm_gmem_is_private_gfn(struct kvm *kvm, gfn_t gfn) +{ + struct kvm_memory_slot *slot =3D gfn_to_memslot(kvm, gfn); + struct inode *inode; + + if (!slot) + return 0; + + CLASS(gmem_get_file, file)(slot); + if (!file) + return 0; + + inode =3D file_inode(file); + + /* + * Rely on the maple tree's internal RCU lock to ensure a stable result. + * This result can become stale as soon as the lock is dropped, so the + * caller _must_ protect consumption of private vs. shared either by + * holding guest_memfd's invalidate lock for the entire duration, or by + * checking mmu_invalidate_retry_gfn() under mmu_lock to serialize + * against concurrent attribute updates. + */ + return kvm_gmem_is_private_mem(inode, kvm_gmem_get_index(slot, gfn)); +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_is_private_gfn); + static struct file_operations kvm_gmem_fops =3D { .mmap =3D kvm_gmem_mmap, .open =3D generic_file_open, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 78278730b2ae4..46d2e123448c2 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -102,6 +102,10 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(halt_poll_ns_shrink); static bool __ro_after_init allow_unsafe_mappings; module_param(allow_unsafe_mappings, bool, 0444); =20 +#ifdef kvm_arch_has_private_mem +bool __ro_after_init gmem_in_place_conversion =3D false; +#endif + /* * Ordering of locks: * @@ -2423,6 +2427,9 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *k= vm, static u64 kvm_supported_vm_mem_attributes(struct kvm *kvm) { #ifdef kvm_arch_has_private_mem + if (gmem_in_place_conversion) + return 0; + if (!kvm || kvm_arch_has_private_mem(kvm)) return KVM_MEMORY_ATTRIBUTE_PRIVATE; #endif @@ -2634,8 +2641,11 @@ EXPORT_STATIC_CALL_GPL(__kvm_is_private_gfn); =20 static void kvm_init_memory_attributes(void) { + if (gmem_in_place_conversion) + static_call_update(__kvm_is_private_gfn, kvm_gmem_is_private_gfn); #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES - static_call_update(__kvm_is_private_gfn, kvm_vm_is_private_gfn); + else + static_call_update(__kvm_is_private_gfn, kvm_vm_is_private_gfn); #endif } #else --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C59F82E7179; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=J9dg8Mtap06wkHrYjWwDgwJld8jhRpY3VlMvD3MO/lZMmUX2UMyLRnA9tCG4u9jc9X/fnRFhnuV5dsNCcD82t2kTIil6sxks55vVZ+7FAkbaCyAseMkZ7ZSh/osuJ9TiEDMnqMdoH9sp8AtpDb3f9WoASaC51wxDaKqi3YvvO74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=DZpQ92XBOjBoM1b7sYZ3t5UrnywIOYsWYCcTY22qsc0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AvD2pvGxcGOlcRUgS+3QnHp1HfU98nElMF+SYSZ01mQ8Skz4cIASOP/7jzUKDu9PuDTTUW631WQVxs/oKTP2ge6Kpe9uCEKDwWaYpYL6O7JVQ3GmAEOoI9ilurif0E6IeafZ8dg8NWw33fhGlNkeNsScoBuauTKmGOgT5uwOKPg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ejh+TJUI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ejh+TJUI" Received: by smtp.kernel.org (Postfix) with ESMTPS id 8C688C4AF52; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=DZpQ92XBOjBoM1b7sYZ3t5UrnywIOYsWYCcTY22qsc0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Ejh+TJUIAmD4GchoslTswo/jrht6zz88x9ItWgmyd9LuevRCE1a0wSJeEFHK0GF4l Epd+5/TbrepFHtHaVDdQxnTi0x/HBmdPZpoGzTSPuCcP5QJUYkDL7gOGIgERoezzF5 w3XYrQMZoBLZVvNIhdE0UlVzGSc1XB6rhTlsi9AxwIgKhpEueo7tSWGq3iXpIL3NPw 4Sz0P43mfZffHtzqewT6KhZR7bIAUXQ35h+VHQ1G48bqg+sT7Vp5WYBzzN0CIFbr+Z AyGOzeiqto7HibazR7JWSeqTc4UZbFK8+K5OpNi4EckoFmwduN0x86Sn0KaB/NHxce ls8OE0I88TfLg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D83BC61DF1; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:11 -0700 Subject: [PATCH v12 10/45] KVM: Consolidate private memory and guest_memfd ifdeffery in kvm_host.h Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-10-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=2542; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=6Xy56OcFO1cFOCTRnCIpR40/ey3zdpWxICCe2ijHXsg=; b=Mp9gKeORouIVuaIPVdTu0NcWIIpf9aAxcyZKn6ShAp4Mz1JbXX5xFMCc5DPboZ9FoXuUQSGUf lZOGe6/A4FVAgRywtX2XRTeKaqWeGZUH6CwkTPZjKiWuiLdXu3/rg0t X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Move the kvm_arch_has_private_mem() stub and a few guest_memfd function definitions/declarations "down" in kvm_host.h to utilize existing #ifdefs, and so that related code is clustered together. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Xiaoyao Li Reviewed-by: Binbin Wu Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Reviewed-by: David Hildenbrand (Arm) Signed-off-by: Ackerley Tng --- include/linux/kvm_host.h | 37 ++++++++++++++++--------------------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index b0110143d2ff8..ab87effdd221f 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -722,27 +722,6 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm= _vcpu *vcpu) } #endif =20 -#ifndef kvm_arch_has_private_mem -static inline bool kvm_arch_has_private_mem(struct kvm *kvm) -{ - return false; -} -#endif - -#ifdef CONFIG_KVM_GUEST_MEMFD -bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm); - -static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm) -{ - u64 flags =3D GUEST_MEMFD_FLAG_MMAP; - - if (!kvm || kvm_arch_supports_gmem_init_shared(kvm)) - flags |=3D GUEST_MEMFD_FLAG_INIT_SHARED; - - return flags; -} -#endif - #ifndef kvm_arch_has_readonly_mem static inline bool kvm_arch_has_readonly_mem(struct kvm *kvm) { @@ -2592,6 +2571,11 @@ static inline bool kvm_is_private_gfn(struct kvm *kv= m, gfn_t gfn) #else #define gmem_in_place_conversion false =20 +static inline bool kvm_arch_has_private_mem(struct kvm *kvm) +{ + return false; +} + static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) { return false; @@ -2600,6 +2584,17 @@ static inline bool kvm_is_private_gfn(struct kvm *kv= m, gfn_t gfn) =20 #ifdef CONFIG_KVM_GUEST_MEMFD bool kvm_gmem_is_private_gfn(struct kvm *kvm, gfn_t gfn); +bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm); + +static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm) +{ + u64 flags =3D GUEST_MEMFD_FLAG_MMAP; + + if (!kvm || kvm_arch_supports_gmem_init_shared(kvm)) + flags |=3D GUEST_MEMFD_FLAG_INIT_SHARED; + + return flags; +} =20 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, int *max_order); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7B752EF67A; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; cv=none; b=MwjkFVaAJGH83xjpLuXp/apdeCt5LaxC1McPJWyesOpsxxXmYCnxmjiPJRxCvhTcyXjtWhBAimWIvzK1AgANK3Z3DRRXUEpUcY3jz8Fu6fiiapmTtz6hVrja4ZGqlhZJqu/NG7CB+5Ogh79wE7jS81AwOtldRG5QdvqFfGpjRWE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135920; c=relaxed/simple; bh=ARLoo7ngV37dIqzMdRwEWGmzq2H/OMebqzbn4gJysGk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HuYG6NdvK8ONN0luoTHa56k3y27qus0+uzwUXzxMO+dJ1FJtlz22KLMYSb3LnWG5lptGxa5eoUDtkQoWydTbG8VMdGoAEx0TLnFjVR0E8Q2KRV5gT4OMc+PBotoBXYXEdgArBg5iYDEqLywR+LSsAOZ6x6xU7F9s81qb1q6Chsw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=m6tryz0T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="m6tryz0T" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9833AC32781; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=ARLoo7ngV37dIqzMdRwEWGmzq2H/OMebqzbn4gJysGk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=m6tryz0TLerL/U8q0Y/S0LHoxleFiVZ6yt/yqDLekeGijGtgzzWHkrSItk/+h2Zsg VJI4TPBgaugYjroTn1o5ZztECLrSmfVKmbRZxvGSiI1hKENdxvhxIOfVjUDRHfHwvr 0/pjXsx5R6NYuECIxzcKHDJLD3iOJoDhelEs4z1WpSIOJdIIk+WEEVCWIDEofMpuAs 01UbLtikHAjMTfWhhzsNFrKDTFjKUUW0fKlMZz7MlgUumMOfIxirw3D6009dkDvzP3 GfrKq+9Y/91Hi/emf68/E7U91KLM9YXT5wL3mbzCYTe5Ivx2RhND5QDWCnZik5Ah0p vmlqUuyJsNlew== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79D57C61DE2; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:12 -0700 Subject: [PATCH v12 11/45] KVM: guest_memfd: Invalidate both SHARED and PRIVATE mappings for in-place conversions Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-11-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=3203; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=wwyziPople+SkEp/ySiFCSViS1Jnz5EzSDFKRRrvWlg=; b=Gg+nTA/eIzF8o4MMJR486GmGblCq886rCHl1mERVHrwwsNk5FGOc6c8F+wxYu3GEO7CBc7nRx W/TgAQ0rLwkD4vIstAaWR8LvwhpIyQJZK1xXuJYNgzO8xTgVvQ4sNkB X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson When removing one or more folios from a guest_memfd instance, invalidate both SHARED and PRIVATE mappings if in-place conversion is enabled, because stating the obvious, KVM needs to ensure that all mappings to the folio(s) are dropped. Opportunistically rename the helper to capture that it returns a filter for all gfns in anticipation of zapping only the previous mapping types on conversion. I.e. when doing in-place conversion to PRIVATE, only SHARED mappings need to be zapped (ignoring that KVM would ideally not invalidate ranges whose attributes aren't changing in the first place). Note, precisely zapping only the possible mapping types when in-place conversion is disabled is important for functional correctness, not just for performance. Specifically, if KVM zaps both when SHARED vs. PRIVATE is tracked per-VM, then a PUNCH_HOLE operation on a PRIVATE guest_memfd will incorrectly zap SHARED mappings that have nothing to do with that gmem instance (because they're mapped via a VMA, not a gmem fd). The incorrect over-zapping of SHARED memory that doesn't belong to the gmem fd requesting the zapping will be resolved in a later patch, where, if in-place conversion is enabled, KVM will use both shared and private memory from the guest_memfd. If both shared and private memory are from the guest_memfd, invalidation will only zap memory belonging to the given gmem instance. Signed-off-by: Sean Christopherson Signed-off-by: Ackerley Tng Reviewed-by: Binbin Wu Reviewed-by: David Hildenbrand (Arm) Reviewed-by: Fuad Tabba --- virt/kvm/guest_memfd.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index cd966eef45b69..0afe1468d2d9d 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -139,8 +139,11 @@ static struct folio *kvm_gmem_get_folio(struct inode *= inode, pgoff_t index) return folio; } =20 -static enum kvm_gfn_range_filter kvm_gmem_get_invalidate_filter(struct ino= de *inode) +static enum kvm_gfn_range_filter kvm_gmem_get_all_gfns_filter(struct inode= *inode) { + if (gmem_in_place_conversion) + return KVM_FILTER_SHARED | KVM_FILTER_PRIVATE; + if (GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED) return KVM_FILTER_SHARED; =20 @@ -191,11 +194,9 @@ static void __kvm_gmem_invalidate_start(struct gmem_fi= le *f, pgoff_t start, static void kvm_gmem_invalidate_start(struct inode *inode, pgoff_t start, pgoff_t end) { - enum kvm_gfn_range_filter attr_filter; + enum kvm_gfn_range_filter attr_filter =3D kvm_gmem_get_all_gfns_filter(in= ode); struct gmem_file *f; =20 - attr_filter =3D kvm_gmem_get_invalidate_filter(inode); - kvm_gmem_for_each_file(f, inode) __kvm_gmem_invalidate_start(f, start, end, attr_filter); } @@ -347,7 +348,7 @@ static int kvm_gmem_release(struct inode *inode, struct= file *file) * memory, as its lifetime is associated with the inode, not the file. */ __kvm_gmem_invalidate_start(f, 0, -1ul, - kvm_gmem_get_invalidate_filter(inode)); + kvm_gmem_get_all_gfns_filter(inode)); __kvm_gmem_invalidate_end(f, 0, -1ul); =20 list_del(&f->entry); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA1372F361E; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=sAKUYuVRaKycotpEIEzICzgrXyBje36wz+4HR9gqelfZ+xqCMagCic2UaVBP1tgjIk94+iPNcbfSqyrlQAtR7yba/9+ZcUFdqdEp7hprdus5SEdVKvLXXwXrcfAbj7lgpT8IGZPxcY7hOdWyHieAKEBNNCUnqf+uAgzhCrxfDDk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=pgbQeqwV6IRb5CiPCfNQynStgiWTbolys+BVEEK4H2w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WEyl4bKrUR/zLO3dQV2z1PobC0zi48IxP2nlxgX5+EjdcorXrpzFMoUVM7DFz7YKYSjSXNsdgfojr1ooZLFuU38h5X1WuOGyJNwVIJnBgkCOJXu1G9YEXsSvQuUnFOqVbttQtLILvFNW8g9W3tKEV0Y6cnFli7OO/HQ9Pz6ya/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mZSTa6ni; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mZSTa6ni" Received: by smtp.kernel.org (Postfix) with ESMTPS id B8328C4AF54; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=pgbQeqwV6IRb5CiPCfNQynStgiWTbolys+BVEEK4H2w=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=mZSTa6ni/SkaGctJC95sGPqw7Ik2tnPRymeXDDfMOLYPN72DoTj7xjt+asjczUfcc bppappQ9c9LUoY9EIPrTqJttmF6zojfgB2nzktvtf8rApIZv0SvZfihCbDJ7S2JB+U d2/XrY+rsOxIGTJBaYDtZG5LP6cIBB2pJ2qRa9UKxdruov3qKzP8mQef1N6WDnGvJq iquSBeuh+UxlIfCqyP9i9gNWfg2YJ2L3PtnNIH5ztsr+EKIfJsKR1SAlgX64MCzz1y Lj7ej/FBYV/ydqd5HKGBSe1XV3FH0KVEHGF0Fta3/DEHiovgEKvHNC77PH9HmfRubV e6ronKXVy4p8g== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 971A0C61DFF; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:13 -0700 Subject: [PATCH v12 12/45] KVM: guest_memfd: Always fault from guest_memfd if in-place conversion is enabled Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-12-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=4450; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=X0oHuYGMuEMc+mIS2D25NifHLJhyfzoCyNVUkFKAoCE=; b=EtQxw+QEVWJoZlS70xZzAwu5ZPYmJaQfBkoC/h5utIHwFISWqanhlhfA/RfJ33NDQYkw7Y4kT x0KHakJQKe3CW/InZ0fVH/Fv/TtLlWGt56gk9mRZgklUQUp8zSa/3yP X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng If a guest_memfd memslot is created but the guest_memfd does not have the GUEST_MEMFD_FLAG_MMAP set, KVM still fulfils guest faults by looking up the memslot's userspace_addr. Set KVM_MEMSLOT_GMEM_ONLY if in-place conversion is enabled so that the guest_memfd's memory will be used for both shared and private memory. With in-place conversion, guest_memfd will be the only backing memory for the memslot. No validation is performed to require userspace_addr to be a mapping from the associated guest_memfd because even after validation, userspace is free to remap something else at the provided userspace_addr. userspace_addr will still be used by functions like kvm_read_guest(), and if userspace_addr does not match up with the corresponding memory in the memslot's guest_memfd (whether userspace_addr points to the wrong offset or some non-guest_memfd memory, etc), that is a user error. Requiring both shared and private memory to come from the only associated guest_memfd simplifies invalidation in stage 2 page tables. On a PUNCH_HOLE operation on a guest_memfd, the invalidation is now guaranteed to be invalidating only memory mapped from the given guest_memfd. Suggested-by: Sean Christopherson Signed-off-by: Ackerley Tng Reviewed-by: Binbin Wu Reviewed-by: David Hildenbrand (Arm) Reviewed-by: Fuad Tabba --- Documentation/virt/kvm/api.rst | 22 ++++++++++++++-------- virt/kvm/guest_memfd.c | 2 +- 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 90a29424c54c8..668886f50024d 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6381,10 +6381,16 @@ mapping for userspace_addr is not required to be va= lid/populated at the time of KVM_SET_USER_MEMORY_REGION2, e.g. shared memory can be lazily mapped/alloc= ated on-demand. =20 -When mapping a gfn into the guest, KVM selects shared vs. private, i.e con= sumes -userspace_addr vs. guest_memfd, based on the state in guest_memfd, which i= s the -sole authority on private vs. shared memory. See :ref:`KVM_CREATE_GUEST_M= EMFD` -to find out more about the creation-time shared/private status. +When mapping a gfn into the guest, guest faults are always serviced from +guest_memfd regardless of whether memory is shared or private. KVM determ= ines +shared vs. private based on the state in guest_memfd, which is the sole +authority on private vs. shared memory. See :ref:`KVM_CREATE_GUEST_MEMFD`= to +find out more about the creation-time shared/private status. + +userspace_addr is expected to be the mmap()-ed address corresponding to the +right offset within the guest_memfd. Any mismatch between userspace_addr a= nd +guest_memfd is not validated and is a user error. userspace_addr is only u= sed +for host-side guest accesses such as kvm_read_guest(). =20 If in-place conversion is disabled, KVM selects shared vs. private, i.e co= nsumes userspace_addr vs. guest_memfd, based on the gfn's KVM_MEMORY_ATTRIBUTE_PR= IVATE @@ -6490,10 +6496,10 @@ specified via KVM_CREATE_GUEST_MEMFD. Currently de= fined flags: page tables. Private memory cannot. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D =20 -When the KVM MMU performs a PFN lookup to service a guest fault and the ba= cking -guest_memfd has the GUEST_MEMFD_FLAG_MMAP set, then the fault will always = be -consumed from guest_memfd, regardless of whether it is a shared or a priva= te -fault. +When the KVM MMU performs a PFN lookup to service a guest fault, the fault= will +always be consumed from guest_memfd, regardless of whether it is a shared = or a +private fault (unless in-place conversion is disabled and the backing +guest_memfd does not have the GUEST_MEMFD_FLAG_MMAP flag set). =20 See KVM_SET_USER_MEMORY_REGION2 for additional details. =20 diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 0afe1468d2d9d..e41802944756b 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -746,7 +746,7 @@ int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_sl= ot *slot, */ WRITE_ONCE(slot->gmem.file, file); slot->gmem.pgoff =3D start; - if (kvm_gmem_supports_mmap(inode)) + if (gmem_in_place_conversion || kvm_gmem_supports_mmap(inode)) slot->flags |=3D KVM_MEMSLOT_GMEM_ONLY; =20 xa_store_range(&f->bindings, start, end - 1, slot, GFP_KERNEL); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14E4C2F9D82; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=qs0DqDbkiftYw181C0oi4zFg5jAUdIu6xK+Hz7KETYkjDYXMENV9vI3m2zxJoLb1HnkkTYzf327z8zFkISXiFw7yU2mZ825xAmwYKMHmZQPoStZ19II4t3/ELWN+897Dn88c/AfLYbPTjM5PW/Hk8Uhp+BmaQuzKWyj+6gVDePU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=t9tIswHrdBmjog6KcGGiBEsXA8vzkM0vs3f9V0PMZDU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TteJB4SbBJQck2IX8OZ4V4ibD2eLsJDN1thyt0ZGgnYSz0tVPbr8v/p8YVepBB9smEgWS+GEPVal6XbvQk9dI8FhTS3kcUZzBLZtEp9CTKal5c46XJ/WMaJBklgiVJJGdCnqaypTGb6nWZ6nmKybzj3OmgHofVwXM1xq3DElaLg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rzXfk74h; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rzXfk74h" Received: by smtp.kernel.org (Postfix) with ESMTPS id C892EC4AF62; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135920; bh=t9tIswHrdBmjog6KcGGiBEsXA8vzkM0vs3f9V0PMZDU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=rzXfk74h/pMzdnZsp4qXd6RyV3Ok9XPvsQNi+4W/B+AzBGmJtInrcphLpd0g6YAq1 MefK2bo29SJYK0m/EdU2jnXDHtRQY0Ct6+sX/J5PDKkwKdlvGS8nwJ/XvMumBDz2yY s8iH50Ut/GyIFv5JGSwQLs1rgIZ2pLnqs0gosHqJvBHeQYihecsV7NFcueJOPL3DSW kvf0idjMt0kQUH30LHmpSj+zS/nMBs0xdTH+wMceqrmkEtC1e9qfp0PjHTpvoNFY1a 47vkJyTerD7QFCKS8eugNyamlzaiLAAld+Uy3su6+mnt79hnQxf5dFRRYlioNdxLdS vH9mUl3WD2emA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2300C61DFD; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:14 -0700 Subject: [PATCH v12 13/45] KVM: guest_memfd: Pass mapping type filter to invalidation helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-13-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=2917; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=ypeEDM+LfsQQ4QqwIgwW5NhyTvWeVmGP7oL9zB4ZTHc=; b=FpekWujCWkPT25P5EpG9jIkKIThptLZ8wxNDFM01ojRBI8RRpi94Rq41rKW7yMDZbiVZ8ew8A Bs4OUofPBSIBJTIvzAUjXDQsWuwD3ci2CM2wz2z6f5zwaW/gTMuvvM+ X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Accept the mapping type filter as a parameter in the invalidation start helper instead of querying it internally. This allows callers to specify which mappings (shared, private, or both) should be invalidated. In the next patch, the conversion process will use this new parameter to invalidate mappings only when they're different from the target state of the conversion, i.e. invalidate only shared mappings on a shared to private conversion and not both. No functional change intended. Signed-off-by: Ackerley Tng Reviewed-by: Binbin Wu Reviewed-by: David Hildenbrand (Arm) Reviewed-by: Fuad Tabba --- virt/kvm/guest_memfd.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index e41802944756b..e19de803149c2 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -192,13 +192,13 @@ static void __kvm_gmem_invalidate_start(struct gmem_f= ile *f, pgoff_t start, } =20 static void kvm_gmem_invalidate_start(struct inode *inode, pgoff_t start, - pgoff_t end) + pgoff_t end, + enum kvm_gfn_range_filter filter) { - enum kvm_gfn_range_filter attr_filter =3D kvm_gmem_get_all_gfns_filter(in= ode); struct gmem_file *f; =20 kvm_gmem_for_each_file(f, inode) - __kvm_gmem_invalidate_start(f, start, end, attr_filter); + __kvm_gmem_invalidate_start(f, start, end, filter); } =20 static void __kvm_gmem_invalidate_end(struct gmem_file *f, pgoff_t start, @@ -224,6 +224,7 @@ static void kvm_gmem_invalidate_end(struct inode *inode= , pgoff_t start, =20 static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t= len) { + enum kvm_gfn_range_filter filter =3D kvm_gmem_get_all_gfns_filter(inode); pgoff_t start =3D offset >> PAGE_SHIFT; pgoff_t end =3D (offset + len) >> PAGE_SHIFT; =20 @@ -233,7 +234,7 @@ static long kvm_gmem_punch_hole(struct inode *inode, lo= ff_t offset, loff_t len) */ filemap_invalidate_lock(inode->i_mapping); =20 - kvm_gmem_invalidate_start(inode, start, end); + kvm_gmem_invalidate_start(inode, start, end, filter); =20 truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1); =20 @@ -522,6 +523,8 @@ static int kvm_gmem_migrate_folio(struct address_space = *mapping, =20 static int kvm_gmem_error_folio(struct address_space *mapping, struct foli= o *folio) { + struct inode *inode =3D mapping->host; + enum kvm_gfn_range_filter filter; pgoff_t start, end; =20 filemap_invalidate_lock_shared(mapping); @@ -529,7 +532,8 @@ static int kvm_gmem_error_folio(struct address_space *m= apping, struct folio *fol start =3D folio->index; end =3D start + folio_nr_pages(folio); =20 - kvm_gmem_invalidate_start(mapping->host, start, end); + filter =3D kvm_gmem_get_all_gfns_filter(inode); + kvm_gmem_invalidate_start(inode, start, end, filter); =20 /* * Do not truncate the range, what action is taken in response to the --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DEA42FD1CA; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=uJQ5Vqqu0YPBUEAXSYiDV9aMUJv5149M/EEUnEIRBV0v4Cumpbj3h8Kyd7aW4LWKAdnUHMZWKlS70BqjifRIpdO0HKxENmTlPkpEeSy8nvrSq6JVL+fOnFdhsxCV7G5b4Qri2d6GJfGu/VJd4tizH0ScwE874IUGaV9rG+seS7E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=xOnBQXU90QN4x/dpBZ0xw8y0jZjsNuhoWApti5TQBLk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=erAL8bhiRdXeP9pj0oFc+YxRXnMEQQY0CUlRZ673nGjwylYE0WmAvrLgHqnr5F1M1N3Azn5mxd2Z57f3rw8dcAECrXYZGGtgij4y7Yx6DWZiCHOYa2lYpsLV3UdyHe1/NDEjO2tjgtaHKLxT1ixhCds96prv4IKfNKc6mESEFWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KiL/+3S6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KiL/+3S6" Received: by smtp.kernel.org (Postfix) with ESMTPS id E7AB1C32782; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=xOnBQXU90QN4x/dpBZ0xw8y0jZjsNuhoWApti5TQBLk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=KiL/+3S6vX7D0Lwe6hbS2bnsGWvXZt5e9WKUQ3Ahihd585U8YQ7OjWYlukXnCGJuM np/ooxFNba/ygsSjloSrL5WDSZ+2ymfoXDsBxEgQwWHWdIGieTGU+eETi8cd3WjFB+ y3SWDxCrH6HUHX3Ldpm7XHGtPhMdc8gRGCBYYa5hOZxNUoO6cH0VOoS3nopAokXCel dAl1nvvhc6v3DtIRDIhO1nbRcniLFOJoue57wurUIzyPCBXcKFHqWtDA9rcqHCIFdR P7/yhUZdmY6gK4JifR+IQ8l1mgGhMtzWlHeFcQdM/8PXNK//Mozy5eQKkSf9rFefKh //oJh9D3JGASw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD00EC61DE4; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:15 -0700 Subject: [PATCH v12 14/45] KVM: guest_memfd: Add base support for KVM_SET_MEMORY_ATTRIBUTES2 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-14-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=13772; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=dF7aSFOBL4CumilTb/gmaTnsPAJn/Lzwq7jAbv2B05A=; b=x8oDe0aLAoq2dmAVBQZWJFFXanBDL3eQYpQ6oRsciUd8W0OHcVwn9JeaRIH9Itu2ONM4EIvKo ZT1QHDzgeY0Bl5zivoQ22UGCsE/FHqI0K6TLjW4I9BVptv/3tt1L2XX X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add a new ioctl (and matching struct), KVM_SET_MEMORY_ATTRIBUTES2, using the same base ioctl number (0xd2), but with R/W semantics for the kernel instead of just read semantics. "Officially" documenting that KVM writes to the payload will allow KVM to support partial/incremental conversions, instead of all-or-nothing updates (which requires complex unwinding), by recording the failing offset if an error occurs. Opportunistically add a new struct as well, even though KVM could squeeze the error offset into "struct kvm_memory_attributes", as there's no cost to doing so in practice. Pad the struct with a pile of extra space to try and avoid ending up with "struct kvm_memory_attributes3" in the future. Use the same layout for the fields common to version 1 of the struct, e.g. to ease upgrading userspace, and to provide flexibility if KVM ever adds support for KVM_SET_MEMORY_ATTRIBUTES2 at VM scope. Introduce KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES to advertise the availability of the KVM_SET_MEMORY_ATTRIBUTES2 ioctl. Update the KVM API documentation to define the new ioctl and its behavior, and add the necessary UAPI definitions and capability checks. The process of setting memory attributes has a clear point of no return because, for CoCo VMs, zapping stage 2 page tables is a destructive operation. Unlike regular VMs, where re-faulting pages into the stage 2 page tables merely incurs a performance penalty, CoCo guests must (re-):accept pages after every fault. To preserve CoCo security guarantees, guests will not accept pages they did not explicitly request faults for. Consequently, during memory conversions, any operation that could cause the process to abort must be completed before the stage 2 page tables are zapped. Zap only the ranges that are not already in the requested state to avoid inadvertently destroying (CoCo) data. ARM CCA guests will try to mark the entire DRAM as private at boot. If there are no shared pages at all, the to-private conversion can be skipped, but the existence of a single shared page would require the conversion process to proceed, and if it proceeds, zapping both shared and private pages would destroy data and break the guest. Co-developed-by: Vishal Annapurve Signed-off-by: Vishal Annapurve Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Reviewed-by: Binbin Wu Suggested-by: Michael Roth Tested-by: Shivank Garg Suggested-by: Suzuki K Poulose Reviewed-by: Suzuki K Poulose Signed-off-by: Ackerley Tng --- Documentation/virt/kvm/api.rst | 71 +++++++++++++++++++++++- include/uapi/linux/kvm.h | 15 ++++++ virt/kvm/guest_memfd.c | 119 +++++++++++++++++++++++++++++++++++++= ++++ virt/kvm/kvm_main.c | 23 +++++--- 4 files changed, 219 insertions(+), 9 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 668886f50024d..4dbf452f6e809 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -117,7 +117,7 @@ description: x86 includes both i386 and x86_64. =20 Type: - system, vm, or vcpu. + system, vm, vcpu or guest_memfd. =20 Parameters: what parameters are accepted by the ioctl. @@ -6385,7 +6385,9 @@ When mapping a gfn into the guest, guest faults are a= lways serviced from guest_memfd regardless of whether memory is shared or private. KVM determ= ines shared vs. private based on the state in guest_memfd, which is the sole authority on private vs. shared memory. See :ref:`KVM_CREATE_GUEST_MEMFD`= to -find out more about the creation-time shared/private status. +find out more about the creation-time shared/private status. Userspace can +control whether memory is shared/private by toggling +KVM_MEMORY_ATTRIBUTE_PRIVATE via :ref:`KVM_SET_MEMORY_ATTRIBUTES2` as need= ed. =20 userspace_addr is expected to be the mmap()-ed address corresponding to the right offset within the guest_memfd. Any mismatch between userspace_addr a= nd @@ -6404,6 +6406,8 @@ S390: Returns -EINVAL if the VM has the KVM_VM_S390_UCONTROL flag set. Returns -EINVAL if called on a protected VM. =20 +.. _KVM_SET_MEMORY_ATTRIBUTES: + 4.141 KVM_SET_MEMORY_ATTRIBUTES ------------------------------- =20 @@ -6440,6 +6444,8 @@ the state of a gfn/page as needed. =20 The "flags" field is reserved for future extensions and must be '0'. =20 +See also: :ref:`KVM_SET_MEMORY_ATTRIBUTES2`. + .. _KVM_CREATE_GUEST_MEMFD: =20 4.142 KVM_CREATE_GUEST_MEMFD @@ -6599,6 +6605,67 @@ KVM_S390_KEYOP_SSKE Sets the storage key for the guest address ``guest_addr`` to the key specified in ``key``, returning the previous value in ``key``. =20 +.. _KVM_SET_MEMORY_ATTRIBUTES2: + +4.145 KVM_SET_MEMORY_ATTRIBUTES2 +--------------------------------- + +:Capability: KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES +:Architectures: all +:Type: guest_memfd ioctl +:Parameters: struct kvm_memory_attributes2 (in) +:Returns: 0 on success, <0 on error + +Errors: + + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + EINVAL The specified `offset` or `size` was invalid (e.g. not + page aligned, causes an overflow, or size is zero). + EFAULT The parameter address was invalid. + ENOMEM Ran out of memory trying to track private/shared state + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +KVM_SET_MEMORY_ATTRIBUTES2 is an extension to +KVM_SET_MEMORY_ATTRIBUTES that supports returning (writing) values to +userspace. The original (pre-extension) fields are shared with +KVM_SET_MEMORY_ATTRIBUTES identically. + +Attribute values are shared with KVM_SET_MEMORY_ATTRIBUTES. + +:: + + struct kvm_memory_attributes2 { + union { + __u64 address; + __u64 offset; + }; + __u64 size; + __u64 attributes; + __u64 flags; + __u64 reserved[12]; + }; + + #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) + +Set attributes for a range of offsets within a guest_memfd to +KVM_MEMORY_ATTRIBUTE_PRIVATE to limit the specified guest_memfd backed +memory range for guest use. Even if KVM_CAP_GUEST_MEMFD_MMAP is +supported, after a successful call to set +KVM_MEMORY_ATTRIBUTE_PRIVATE, the requested range will not be mappable +into host userspace and will only be mappable by the guest. + +To allow the range to be mappable into host userspace again, call +KVM_SET_MEMORY_ATTRIBUTES2 on the guest_memfd again with +KVM_MEMORY_ATTRIBUTE_PRIVATE unset. + +KVM does not directly manipulate the memory contents of pages during +attribute updates. However, the process of setting these attributes, +which includes operations such as unmapping pages from the host or +stage-2 page tables, may result in side effects on memory contents +that vary across different trusted firmware implementations. + +See also: :ref:`KVM_SET_MEMORY_ATTRIBUTES`. + 4.145 KVM_PPC_GET_COMPAT_CAPS ----------------------------- :Capability: KVM_CAP_PPC_COMPAT_CAPS diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index ac2d77d149635..ac371a50041c9 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -999,6 +999,7 @@ struct kvm_enable_cap { #define KVM_CAP_S390_HPAGE_2G 249 #define KVM_CAP_PPC_COMPAT_CAPS 250 #define KVM_CAP_ARM_PMU_V3_STRICT 251 +#define KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES 252 =20 struct kvm_irq_routing_irqchip { __u32 irqchip; @@ -1653,6 +1654,20 @@ struct kvm_memory_attributes { __u64 flags; }; =20 +/* Available with KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES */ +#define KVM_SET_MEMORY_ATTRIBUTES2 _IOWR(KVMIO, 0xd2, struct= kvm_memory_attributes2) + +struct kvm_memory_attributes2 { + union { + __u64 address; + __u64 offset; + }; + __u64 size; + __u64 attributes; + __u64 flags; + __u64 reserved[12]; +}; + #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) =20 #define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest= _memfd) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index e19de803149c2..803c7cdbbe0f6 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -506,11 +506,130 @@ bool kvm_gmem_is_private_gfn(struct kvm *kvm, gfn_t = gfn) } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gmem_is_private_gfn); =20 +/* + * Preallocate memory for attributes to be stored on a maple tree, pointed= to + * by mas. Adjacent ranges with attributes identical to the new attributes + * will be merged. Also sets mas's bounds up for storing attributes. + * + * This maintains the invariant that ranges with the same attributes will + * always be merged. + */ +static int kvm_gmem_mas_preallocate(struct ma_state *mas, u64 attributes, + pgoff_t start, size_t nr_pages) +{ + pgoff_t end =3D start + nr_pages; + pgoff_t last =3D end - 1; + void *entry; + + /* Try extending range. entry is NULL on overflow/wrap-around. */ + mas_set(mas, end); + entry =3D mas_find(mas, end); + if (entry && xa_to_value(entry) =3D=3D attributes) + last =3D mas->last; + + if (start > 0) { + mas_set(mas, start - 1); + entry =3D mas_find(mas, start - 1); + if (entry && xa_to_value(entry) =3D=3D attributes) + start =3D mas->index; + } + + mas_set_range(mas, start, last); + return mas_preallocate(mas, xa_mk_value(attributes), GFP_KERNEL); +} + +static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start, + size_t nr_pages, uint64_t attrs) +{ + bool to_private =3D attrs & KVM_MEMORY_ATTRIBUTE_PRIVATE; + struct address_space *mapping =3D inode->i_mapping; + struct gmem_inode *gi =3D GMEM_I(inode); + enum kvm_gfn_range_filter filter; + pgoff_t end =3D start + nr_pages; + struct maple_tree *mt; + struct ma_state mas; + int r; + + mt =3D &gi->attributes; + + filemap_invalidate_lock(mapping); + + mas_init(&mas, mt, start); + r =3D kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages); + if (r) + goto out; + + /* + * From this point on guest_memfd has performed necessary + * checks and can proceed to do guest-breaking changes. + */ + + filter =3D to_private ? KVM_FILTER_SHARED : KVM_FILTER_PRIVATE; + kvm_gmem_invalidate_start(inode, start, end, filter); + mas_store_prealloc(&mas, xa_mk_value(attrs)); + kvm_gmem_invalidate_end(inode, start, end); +out: + filemap_invalidate_unlock(mapping); + return r; +} + +static long kvm_gmem_set_attributes(struct file *file, void __user *argp) +{ + struct gmem_file *f =3D file->private_data; + struct inode *inode =3D file_inode(file); + struct kvm_memory_attributes2 attrs; + size_t nr_pages; + pgoff_t index; + int i; + + if (copy_from_user(&attrs, argp, sizeof(attrs))) + return -EFAULT; + + if (attrs.flags) + return -EINVAL; + for (i =3D 0; i < ARRAY_SIZE(attrs.reserved); i++) { + if (attrs.reserved[i]) + return -EINVAL; + } + if (!kvm_arch_has_private_mem(f->kvm)) + return -EINVAL; + if (attrs.attributes & ~KVM_MEMORY_ATTRIBUTE_PRIVATE) + return -EINVAL; + if (attrs.size =3D=3D 0 || attrs.offset + attrs.size < attrs.offset) + return -EINVAL; + if (!PAGE_ALIGNED(attrs.offset) || !PAGE_ALIGNED(attrs.size)) + return -EINVAL; + + if (attrs.offset >=3D i_size_read(inode) || + attrs.offset + attrs.size > i_size_read(inode)) + return -EINVAL; + + nr_pages =3D attrs.size >> PAGE_SHIFT; + index =3D attrs.offset >> PAGE_SHIFT; + return __kvm_gmem_set_attributes(inode, index, nr_pages, + attrs.attributes); +} + +static long kvm_gmem_ioctl(struct file *file, unsigned int ioctl, + unsigned long arg) +{ + switch (ioctl) { + case KVM_SET_MEMORY_ATTRIBUTES2: + if (!gmem_in_place_conversion) + return -ENOTTY; + + return kvm_gmem_set_attributes(file, (void __user *)arg); + default: + return -ENOTTY; + } +} + static struct file_operations kvm_gmem_fops =3D { .mmap =3D kvm_gmem_mmap, .open =3D generic_file_open, .release =3D kvm_gmem_release, .fallocate =3D kvm_gmem_fallocate, + .unlocked_ioctl =3D kvm_gmem_ioctl, }; =20 static int kvm_gmem_migrate_folio(struct address_space *mapping, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 46d2e123448c2..1ea8198821917 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2423,18 +2423,22 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm = *kvm, } #endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */ =20 +#ifdef kvm_arch_has_private_mem +static u64 kvm_supports_private_mem(struct kvm *kvm) +{ + return !kvm || kvm_arch_has_private_mem(kvm); +} +#else +#define kvm_supports_private_mem(kvm) false +#endif + #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static u64 kvm_supported_vm_mem_attributes(struct kvm *kvm) { -#ifdef kvm_arch_has_private_mem - if (gmem_in_place_conversion) + if (gmem_in_place_conversion || !kvm_supports_private_mem(kvm)) return 0; =20 - if (!kvm || kvm_arch_has_private_mem(kvm)) - return KVM_MEMORY_ATTRIBUTE_PRIVATE; -#endif - - return 0; + return KVM_MEMORY_ATTRIBUTE_PRIVATE; } =20 /* @@ -4976,6 +4980,11 @@ static int kvm_vm_ioctl_check_extension_generic(stru= ct kvm *kvm, long arg) return 1; case KVM_CAP_GUEST_MEMFD_FLAGS: return kvm_gmem_get_supported_flags(kvm); + case KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES: + if (!gmem_in_place_conversion || !kvm_supports_private_mem(kvm)) + return 0; + + return KVM_MEMORY_ATTRIBUTE_PRIVATE; #endif default: break; --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AFF9305E28; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=iO6ieAKmPl/EazGLc3kcZYjvBmVEDjJNAGkg4T+M/nhtjNB42kg1D2RYb8Sm33foIYK9S3XRXEMIoD2SF7XCEx1jqV1V7KasxtGeGRvU9Ll2YTkfdZCr1R34gpLluYULqMmcjGMMp3EAYRFMmrrrKWo7nADGlb0w8Ock0b/3vWo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=VIYQB25xERtKM4Wa+e3NDcaOtZsYVNAG2ONdWBE3P8E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RqmzIyjmR+UkpMFViV70+o7Yb0e6OLOjjPlXjZvRrXDRqj73MdlrlsYu/GmOFAN4JCwBfgwpOgLmMs0BKsXICgzStEKTRJzTiG5Kex28Z4Im58o4o41xT2VmrhJ45zTlcz3mPlKfvc8vpl57dmb70hUcjiwR4IBrreyQIaQdG1s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MEMuZDfO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MEMuZDfO" Received: by smtp.kernel.org (Postfix) with ESMTPS id 14F83C2BCC7; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=VIYQB25xERtKM4Wa+e3NDcaOtZsYVNAG2ONdWBE3P8E=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=MEMuZDfOwTSWPnf+wlSRXFqB86Z9bjDf5zwyUkHOaHs/wrFONn/Y1eIvJY6ajtKhw 0+wj9O9fwSRkWoWR9WLr+mnXTjyxSd2tYvcm9HAG2Q1QOmaT+r7Yde5VoTkroW85e6 yDqO2auS0AIf8xOu7AKjU9xhA1UGbQoBzVFwAoXPeGkHFYvdU3QEGLPCgMm8Yu4j43 9R6jVCnscU1Sy0dPxsoadgC98a3nJh0Oc9Q0nRMf5DTF9yDxuemYs3Y/2/bPM4YkJ9 ud8X8Wio2L3E1VrUSojnZvaGOZ0cTEs9vuQLvo9kVvtqhMM4M/Hk3+1s6J1SrmVdFu zovdwrI8y0bdA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E66ECC624A4; Mon, 31 Aug 2026 00:25:20 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:16 -0700 Subject: [PATCH v12 15/45] KVM: guest_memfd: Ensure pages are not in use before conversion Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-15-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=7741; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=IRpOCrAj0d7EfhfMDaQT+RCrQLlG814+PuqLTER1o3w=; b=GqaugOwXkdGbEDyigptaIR6ka07phQlgtzwZTeiexEsWbHhYiJ4yMDaJRkZNb3XXriqnCEa09 kOig7tkrQXiC1pjyghpHfRjaObkpdbU5aRO/Gdwzh2aLnCXKD9rkezx X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng When converting memory to private in guest_memfd, it is necessary to ensure that the pages are not currently being accessed by any other part of the kernel or userspace to avoid any current user writing to guest private memory. guest_memfd checks for any outstanding references to determine whether a page is still in use. The only expected references after unmapping the range requested for conversion are those that are held by guest_memfd itself. Update the kvm_memory_attributes2 structure to include an error_offset field. This allows KVM to report the exact offset where a conversion failed. If the safety check fails, return -EAGAIN and copy the error_offset back to userspace so that it can potentially retry the operation or handle the failure gracefully. Update documentation to document the error_offset field and the possible -EAGAIN error. Suggested-by: David Hildenbrand Co-developed-by: Vishal Annapurve Signed-off-by: Vishal Annapurve Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- Documentation/virt/kvm/api.rst | 19 +++++++++-- include/uapi/linux/kvm.h | 3 +- virt/kvm/guest_memfd.c | 77 ++++++++++++++++++++++++++++++++++++++= +--- 3 files changed, 91 insertions(+), 8 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 4dbf452f6e809..25cb15970f1ae 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6613,7 +6613,7 @@ KVM_S390_KEYOP_SSKE :Capability: KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES :Architectures: all :Type: guest_memfd ioctl -:Parameters: struct kvm_memory_attributes2 (in) +:Parameters: struct kvm_memory_attributes2 (in/out) :Returns: 0 on success, <0 on error =20 Errors: @@ -6622,6 +6622,8 @@ Errors: EINVAL The specified `offset` or `size` was invalid (e.g. not page aligned, causes an overflow, or size is zero). EFAULT The parameter address was invalid. + EAGAIN Some page within requested range had unexpected refcounts. The + offset of the page will be returned in `error_offset`. ENOMEM Ran out of memory trying to track private/shared state =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 @@ -6635,6 +6637,7 @@ Attribute values are shared with KVM_SET_MEMORY_ATTRI= BUTES. :: =20 struct kvm_memory_attributes2 { + /* in */ union { __u64 address; __u64 offset; @@ -6642,7 +6645,9 @@ Attribute values are shared with KVM_SET_MEMORY_ATTRI= BUTES. __u64 size; __u64 attributes; __u64 flags; - __u64 reserved[12]; + /* out */ + __u64 error_offset; + __u64 reserved[11]; }; =20 #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) @@ -6664,6 +6669,16 @@ which includes operations such as unmapping pages fr= om the host or stage-2 page tables, may result in side effects on memory contents that vary across different trusted firmware implementations. =20 +If this ioctl returns -EAGAIN, the offset of the page with unexpected +refcounts will be returned in ``error_offset``. This can occur if +there are transient refcounts on the pages, taken by other parts of +the kernel. + +Userspace is expected to figure out how to remove all known refcounts +on the shared pages, such as refcounts taken by get_user_pages(), and +try the ioctl again. A possible source of these long term refcounts is +if the guest_memfd memory was pinned in IOMMU page tables. + See also: :ref:`KVM_SET_MEMORY_ATTRIBUTES`. =20 4.145 KVM_PPC_GET_COMPAT_CAPS diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index ac371a50041c9..8dff2fc1972e9 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1665,7 +1665,8 @@ struct kvm_memory_attributes2 { __u64 size; __u64 attributes; __u64 flags; - __u64 reserved[12]; + __u64 error_offset; + __u64 reserved[11]; }; =20 #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 803c7cdbbe0f6..fe02c47c85fb5 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -538,8 +538,46 @@ static int kvm_gmem_mas_preallocate(struct ma_state *m= as, u64 attributes, return mas_preallocate(mas, xa_mk_value(attributes), GFP_KERNEL); } =20 +static bool kvm_gmem_has_outstanding_references(struct inode *inode, + pgoff_t start, size_t nr_pages, + pgoff_t *err_index) +{ + struct address_space *mapping =3D inode->i_mapping; + pgoff_t last =3D start + nr_pages - 1; + bool has_outstanding =3D false; + struct folio_batch fbatch; + pgoff_t next; + int i; + + folio_batch_init(&fbatch); + + next =3D start; + while (has_outstanding && filemap_get_folios(mapping, &next, last, &fbatc= h)) { + for (i =3D 0; i < folio_batch_count(&fbatch); ++i) { + struct folio *folio =3D fbatch.folios[i]; + + /* + * Outstanding references are anything other than those + * from the page cache, plus 1 temporary reference held + * by filemap_get_folios() in the folio batch. + */ + if (folio_ref_count(folio) !=3D folio_nr_pages(folio) + 1) { + has_outstanding =3D true; + *err_index =3D max(start, folio->index); + break; + } + } + + folio_batch_release(&fbatch); + cond_resched(); + } + + return has_outstanding; +} + static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start, - size_t nr_pages, uint64_t attrs) + size_t nr_pages, uint64_t attrs, + pgoff_t *err_index) { bool to_private =3D attrs & KVM_MEMORY_ATTRIBUTE_PRIVATE; struct address_space *mapping =3D inode->i_mapping; @@ -556,8 +594,28 @@ static int __kvm_gmem_set_attributes(struct inode *ino= de, pgoff_t start, =20 mas_init(&mas, mt, start); r =3D kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages); - if (r) + if (r) { + *err_index =3D start; goto out; + } + + if (to_private) { + /* + * Forcefully unmap the pages from all userspace page tables, + * and then verify there are no outstanding references, e.g. + * acquired via GUP or similar. Tell userspace to try again if + * there are outstanding references and hope that whatever has + * pinned the page will put its reference "soon". + */ + unmap_mapping_pages(mapping, start, nr_pages, false); + + if (kvm_gmem_has_outstanding_references(inode, start, nr_pages, + err_index)) { + mas_destroy(&mas); + r =3D -EAGAIN; + goto out; + } + } =20 /* * From this point on guest_memfd has performed necessary @@ -578,9 +636,10 @@ static long kvm_gmem_set_attributes(struct file *file,= void __user *argp) struct gmem_file *f =3D file->private_data; struct inode *inode =3D file_inode(file); struct kvm_memory_attributes2 attrs; + pgoff_t err_index; size_t nr_pages; pgoff_t index; - int i; + int i, r; =20 if (copy_from_user(&attrs, argp, sizeof(attrs))) return -EFAULT; @@ -606,8 +665,16 @@ static long kvm_gmem_set_attributes(struct file *file,= void __user *argp) =20 nr_pages =3D attrs.size >> PAGE_SHIFT; index =3D attrs.offset >> PAGE_SHIFT; - return __kvm_gmem_set_attributes(inode, index, nr_pages, - attrs.attributes); + r =3D __kvm_gmem_set_attributes(inode, index, nr_pages, attrs.attributes, + &err_index); + if (r) { + attrs.error_offset =3D ((uint64_t)err_index) << PAGE_SHIFT; + + if (copy_to_user(argp, &attrs, sizeof(attrs))) + return -EFAULT; + } + + return r; } =20 static long kvm_gmem_ioctl(struct file *file, unsigned int ioctl, --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 556D330BF70; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=SqykyjpRzsJN5E13hFLH1mDW31S1o6agrRGCgrHeMIkBsaQlbtT9WJIN9NnQPc8pVZH4DFVOFrsfQYtM8mV8DGl3nnj98iANQ7blgofAYKPfrG13qUne0IsVqKUa7GOpmxZ7GlWLbCfSUTkDKQClQet0Esi35W6GXD8Dh1wBjlM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=z59hdgGozA63z31zC23BZ4GXqCsjybzsO4sM4OB80Dg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dBb3Ff5Lc+E8xr7Itzeqs1cqq8beHBgJBe0P2Xls//hmOaMqDto7gjvQ7eWgJoJYEHISFTxIoIQVZXseT8QuW83s4Di5A+2whfnMuLXuPvhqSpiq0xmffBq4StZRlhB8owhF0qf5Vbv04I92xdIfTbHG2S3+8xvSn13lLA3aoL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DpR8VEQ0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DpR8VEQ0" Received: by smtp.kernel.org (Postfix) with ESMTPS id 24482C4DDE7; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=z59hdgGozA63z31zC23BZ4GXqCsjybzsO4sM4OB80Dg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=DpR8VEQ0UA58HaY31wEnwLZOCzp290pfHU61d3OSuj5SE4bgeUIjQGmPtrArEmfmT Bn6I0PLu8PiCZkzeqe5O9527n28QKyFJAW3rRC7f4xktgzT3I9G9qxFQKL078NBFBc iNqlPguHzpCvgruRgwpGAnMeY+flKzeXe/eftNDB6rsi0q3DM1uchmcCTOg99x+EG6 rq3WEyw0zzW6rsUFR+rFNNFuB3H2QQsmo6DqJXDA4RTp7TcQUeAMBXjHMxuBZLVhAu OxJfMImjWtWAzXOuiiJs9ewO9CND9s0UCeeXDRV5tROKHGVvFWiBuA51K5eRaGaYQ6 aQoICo1yoGMAw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09CD0C61DF1; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:17 -0700 Subject: [PATCH v12 16/45] KVM: guest_memfd: Call arch make_shared callback for to-shared conversion Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-16-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Fuad Tabba X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=5997; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=cwEhZDpHTNx7ou7qscSIiavupcBz2iS+igqE5zUMd8Q=; b=t6DynLmrFZFZfNeyDVzLpduvArXHO0s1vSOrUsIWpNPwO/0vT3Y8DyHNEr/kDVnMFXQfuCFtv SR6kCWPCwtjCJlL15dvtAK08r074OdOuuf9mOk9x9z7bVFOMfIzN+/a X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng When doing in-place conversion from PRIVATE to SHARED, immediately inform arch code of the conversion for all allocated pages/folios, e.g. so that arch code can put hardware metadata tables in the correct state. Eagerly updating the table for to SHARED conversions avoids having to implement on-demand updates, e.g. when faulting in host userspace mappings. Skip the entire flow if the arch doesn't implement conversion callbacks, as getting folios from the filemap is noticeably expensive, especially when converting large chunks of memory. Deliberately don't eagerly update the metadata table on conversions from SHARED to PRIVATE, because assigning a page to a VM (versus "returning" it to the host) requires the exact GFN associated with the page, i.e would require walking the memslot bindings. And because KVM *must* do on-demand metadata updates when getting a PFN for KVM-internal usage, as that's the only time a relevant memslot binding is guaranteed to exist. Note! Inform arch code of the conversion within the protection of the invalidation sequence, to ensure that any existing mappings are dropped before hardware is updated, and to ensure that new mappings can't be established until after the conversion is complete. Reviewed-by: Fuad Tabba Signed-off-by: Ackerley Tng --- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/x86.c | 5 +++++ include/linux/kvm_host.h | 1 + virt/kvm/guest_memfd.c | 42 ++++++++++++++++++++++++++++++++++= ++++ 5 files changed, 50 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index e213c9ae3e301..67b43c167045b 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -150,7 +150,7 @@ KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT KVM_X86_OP_OPTIONAL_RET0(gmem_make_private) #endif -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT) || defined(CONFIG_HAVE_KVM_= ARCH_GMEM_RECLAIM) KVM_X86_OP_OPTIONAL(gmem_make_shared) #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 744c1f6ff03ed..83e26ce45fb79 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1732,7 +1732,7 @@ struct kvm_x86_ops { int (*gmem_make_private)(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, kvm_pfn_t nr_pages); #endif -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT) || defined(CONFIG_HAVE_KVM_= ARCH_GMEM_RECLAIM) void (*gmem_make_shared)(kvm_pfn_t pfn, kvm_pfn_t nr_pages); #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2292249570314..75e03a2f79db2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10653,6 +10653,11 @@ int kvm_arch_gmem_make_private(struct kvm *kvm, gf= n_t gfn, kvm_pfn_t pfn, { return kvm_x86_call(gmem_make_private)(kvm, gfn, pfn, nr_pages); } + +void kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages) +{ + kvm_x86_call(gmem_make_shared)(pfn, nr_pages); +} #endif =20 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ab87effdd221f..485f18454eb45 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2610,6 +2610,7 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, =20 int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, kvm_pfn_t nr_pages); +void kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages); #ifndef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT #define kvm_arch_has_gmem_convert() false #endif diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index fe02c47c85fb5..d14a7024bdc7b 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -575,6 +575,43 @@ static bool kvm_gmem_has_outstanding_references(struct= inode *inode, return has_outstanding; } =20 +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +static void kvm_gmem_make_shared(struct inode *inode, pgoff_t start, pgoff= _t end) +{ + struct folio_batch fbatch; + pgoff_t next =3D start; + int i; + + folio_batch_init(&fbatch); + while (filemap_get_folios(inode->i_mapping, &next, end - 1, &fbatch)) { + for (i =3D 0; i < folio_batch_count(&fbatch); ++i) { + struct folio *folio =3D fbatch.folios[i]; + pgoff_t start_index, end_index; + kvm_pfn_t start_pfn; + kvm_pfn_t nr_pages; + + start_index =3D max(start, folio->index); + end_index =3D min(end, folio_next_index(folio)); + /* + * end_index is either in folio or points to + * the first page of the next folio. Hence, + * all pages in range [start_index, end_index) + * are contiguous. + */ + start_pfn =3D folio_file_pfn(folio, start_index); + nr_pages =3D end_index - start_index; + + kvm_arch_gmem_make_shared(start_pfn, nr_pages); + } + + folio_batch_release(&fbatch); + cond_resched(); + } +} +#else +static void kvm_gmem_make_shared(struct inode *inode, pgoff_t start, pgoff= _t end) {} +#endif + static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start, size_t nr_pages, uint64_t attrs, pgoff_t *err_index) @@ -624,7 +661,12 @@ static int __kvm_gmem_set_attributes(struct inode *ino= de, pgoff_t start, =20 filter =3D to_private ? KVM_FILTER_SHARED : KVM_FILTER_PRIVATE; kvm_gmem_invalidate_start(inode, start, end, filter); + + if (!to_private && kvm_arch_has_gmem_convert()) + kvm_gmem_make_shared(inode, start, end); + mas_store_prealloc(&mas, xa_mk_value(attrs)); + kvm_gmem_invalidate_end(inode, start, end); out: filemap_invalidate_unlock(mapping); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 793E130F533; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=VjEpQ/H528Rl87F7LNmie9RquYSOvykdwB5RJ7u513ywkcpqmJDRPj9SKnJKZRapJ+p7+aYj7fQr7A+/hUSN34Uscyth05HIAzFawiB7GenSZr1NQUXp8cda7EDVD02jD0CiTLbuyDJEL3XsAbbHvSXULQiINOXu4B8Ovhxv8u4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=0u7TejSHMad9bYL2p6sJmzMrRCXcgVnE5lch746jIL0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KV7vkG7xM8t7r2kWsQZCUUz2agw5MTSRGN/QFv1vBUwfSOU7dPfQTteiGrA3cNgcLsRFbVUpa/8djCJohLvTt6Jt8C7ElGZ7aRfUuchkwx3FO2/sXBKJm/PlAKQPD+l4RJdCKPex0oEAr550Pqiq/H0mX+oDWz/FZyBxPfiPdEA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q4XDKKI9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q4XDKKI9" Received: by smtp.kernel.org (Postfix) with ESMTPS id 5A0A7C2BD01; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=0u7TejSHMad9bYL2p6sJmzMrRCXcgVnE5lch746jIL0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Q4XDKKI970cDwrlLeOUyPXzL41fHU8ixKdjKvqGjaArmtm3P47QWIbAtRhaNOzqyb +1i/fJnnXYIf3u/mZ8lMtWzBTvlYBxs9+ErohCEzzYLLMjA/3yE66N1ASko4Gq81dG pndUl+FiJaAZRQTFsTP7SBd7DYFlV48y28KYOklloq6f51TTC+9J2A3EmaY6+s4AWv 60p1/q0rVQTJQn8RTpTCqKxMrJZc/MCnwzHODOapdLD+Fg3Vrg5zvFwPxLP7NMKIeN h1TH9DbipEWqk0soO5LzCrkaHmA6OmlViW/Ze8+jBEntG4wkOCmDMddn5bnkIfTXFY MXvuZUpRMzEQw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38934C61DE4; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:18 -0700 Subject: [PATCH v12 17/45] KVM: guest_memfd: Return early if range already has requested attributes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-17-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=1963; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=9VNi9pEuvRkZvdOHV26sDWiemmZK8/yVnQLh7gIYUDA=; b=CFxkPT3CEVpS6OYk5q/nJQbCjkiuG38jxjI0AEn8ybxQssVnDPdSR4CXvx5zB8nUH7yz3IHXZ FIdLJl85tJiBVhY6bcB+H02CGuqGFpIwiDvHRM0IWlde4lr2+MEuRAp X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Provide a function to check that a range has given attributes. Optimize setting memory attributes by returning early if all pages in the requested range already have the requested attributes. Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Reviewed-by: Binbin Wu Reviewed-by: Xiaoyao Li Signed-off-by: Ackerley Tng Reviewed-by: David Hildenbrand (Arm) --- virt/kvm/guest_memfd.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index d14a7024bdc7b..ac8e0c6d6e942 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -98,6 +98,24 @@ static bool kvm_gmem_is_shared_mem(struct inode *inode, = pgoff_t index) return !kvm_gmem_is_private_mem(inode, index); } =20 +static bool kvm_gmem_range_has_attributes(struct inode *inode, + pgoff_t index, size_t nr_pages, + u64 attributes) +{ + struct maple_tree *mt =3D &GMEM_I(inode)->attributes; + pgoff_t end =3D index + nr_pages - 1; + void *entry; + + lockdep_assert(mt_lock_is_held(mt)); + + mt_for_each(mt, entry, index, end) { + if (kvm_gmem_get_attributes(inode, entry) !=3D attributes) + return false; + } + + return true; +} + /* * Returns a locked folio on success. The caller is responsible for * setting the up-to-date flag before the memory is mapped into the guest. @@ -623,12 +641,15 @@ static int __kvm_gmem_set_attributes(struct inode *in= ode, pgoff_t start, pgoff_t end =3D start + nr_pages; struct maple_tree *mt; struct ma_state mas; - int r; + int r =3D 0; =20 mt =3D &gi->attributes; =20 filemap_invalidate_lock(mapping); =20 + if (kvm_gmem_range_has_attributes(inode, start, nr_pages, attrs)) + goto out; + mas_init(&mas, mt, start); r =3D kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages); if (r) { --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8899B30FF31; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=qTLCLA+t+MiJ8MDJnl4Spbs5fcO22np1KT5pgp4ujkcXfzkCv5uMMdn4+9oww+5icjG2KSslJxLtr+Jhazh32oGWGx4A+em+t0CgrjS4DOmJ9dCX0sjFpL8iV76LKL/oRlaRHUyCqyqEOCO2CiYjWu8Bvwd54QkUtW2knEj9Qrk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=INfl/U4MD7DiNVtE20rEnCF+KqGWKq4XKxW9ss03gIs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=emr8efGEK0YVISKVudmE/LZoYIOau5VisHdOHtBHkK8df0PSTIJH7hPPcHFOeaswEifStJfBjVXbcmJ4817j7UIiqxOYsktqEinePmouYhlxdtGA1v/TMjn2K1RcZmigNV6y040a4N4hFElH3FKjMnbKbOnW6qprfgPTCWxGAPE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=J7ixRXTW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J7ixRXTW" Received: by smtp.kernel.org (Postfix) with ESMTPS id 69A39C2BCFB; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=INfl/U4MD7DiNVtE20rEnCF+KqGWKq4XKxW9ss03gIs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=J7ixRXTWSc5y6i4hxgZ+vagyxK1c4T1Q5OxM7n49KwqCLSdHU+Q5P1825KMWSNCCl 6QzFq+JulXODd9QaYpj1JpQMbEE22BcGNDHQYBIRQX5CbCSKNoSn3V5xOjKiOcLHUp 6474IBrpI9oaHry4lMCko9TWcsafgUUhylnSP5nbH/kgvo0d3w8QwIvp9jbRpkZkVg lZlMeE2m1WBompIa00FCrC1XZeLeO1Io9MQFNwts96/g7IDNevknCIHRf2RZaB+R1H RA8MLBO7bcV4d+S1AUDPRg9bAbU37BgsXyZBO3ocN3AX4XQfLKe/Y8lWWZFV9i55PL LV61OtRr3BBkw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51E9BC61DFD; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:19 -0700 Subject: [PATCH v12 18/45] KVM: guest_memfd: Handle lru_add fbatch refcounts during conversion safety check Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-18-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Fuad Tabba X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=4735; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=Ty5BgQhIGN1xkSlbhdIOV4bbPZLbusnibdr1eTvGdlc=; b=91swEloDGQUbfYBsHAl/7syD81mtjIP2F0V0Ab45bJ3IEkR1Q5bwBtEL76LfwbU26GezrPCHL XT0wJWHs3EeBheGTp9WnqG2e2q+vGU7johvpQZYqq79+7IhHv7ZWqRu X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng A guest_memfd folio has no outstanding references if guest_memfd holds the only references on it. Any other references on the folio may indicate another user, and guest_memfd cannot convert it to private if there may be an existing host user. A folio will have outstanding references if it is present in a per-CPU lru_add fbatch. guest_memfd does not actually participate in LRU, but freshly-allocated folios are still added to the lru_add fbatch for batch LRU statistics processing. A folio may also have extra refcounts if it is on the mlock fbatch. These two known "usages" of the folio are handled by calling lru_cache_drain_for_folio, which drains both the lru_add and mlock fbatches. After draining, if the refcount is still elevated, then there are truly outstanding references. If the page may be dma pinned, DMA is using it and hence there are outstanding references. folio_maybe_dma_pinned() can have false positives, but that's only with a significant number of refcounts, at which point draining LRU is not going to move the needle - it can still be concluded that the folio has outstanding references. If the page is still mapped after guest_memfd tried to unmap it earlier in the conversion process, it also has outstanding references. Return true and exit early to avoid unnecessary draining in these 2 cases. Provide a drain status to only drain once ever while processing a batch of folios. Acked-by: Vlastimil Babka (SUSE) Suggested-by: David Hildenbrand Reviewed-by: Fuad Tabba Reviewed-by: Binbin Wu Signed-off-by: Ackerley Tng Reviewed-by: David Hildenbrand (Arm) --- mm/folio.c | 2 ++ virt/kvm/guest_memfd.c | 30 ++++++++++++++++++++++-------- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/mm/folio.c b/mm/folio.c index c02dcea9c03c2..50a6dbe55998e 100644 --- a/mm/folio.c +++ b/mm/folio.c @@ -33,6 +33,7 @@ #include #include #include +#include =20 #include "internal.h" #include "page_alloc.h" @@ -926,6 +927,7 @@ void lru_cache_drain_for_folio(const struct folio *foli= o, *drained =3D LRU_CACHE_DRAINED_ALL; } } +EXPORT_SYMBOL_FOR_KVM(lru_cache_drain_for_folio); =20 atomic_t lru_disable_count =3D ATOMIC_INIT(0); =20 diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index ac8e0c6d6e942..1fe935aaef36f 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -8,6 +8,7 @@ #include #include #include +#include =20 #include "kvm_mm.h" #include "guest_memfd.h" @@ -556,10 +557,28 @@ static int kvm_gmem_mas_preallocate(struct ma_state *= mas, u64 attributes, return mas_preallocate(mas, xa_mk_value(attributes), GFP_KERNEL); } =20 +static bool __folio_has_outstanding_references(struct folio *folio, + enum lru_cache_drained *drained) +{ + if (folio_maybe_dma_pinned(folio) || folio_mapped(folio)) + return true; + + /* 1 reference held by filemap_get_folios() in the folio batch. */ + lru_cache_drain_for_folio(folio, 1, drained); + + /* + * Outstanding references are anything other than those from the page + * cache, plus 1 temporary reference held by filemap_get_folios() in the + * folio batch. + */ + return folio_ref_count(folio) !=3D folio_nr_pages(folio) + 1; +} + static bool kvm_gmem_has_outstanding_references(struct inode *inode, pgoff_t start, size_t nr_pages, pgoff_t *err_index) { + enum lru_cache_drained drained =3D LRU_CACHE_NOT_DRAINED; struct address_space *mapping =3D inode->i_mapping; pgoff_t last =3D start + nr_pages - 1; bool has_outstanding =3D false; @@ -570,17 +589,12 @@ static bool kvm_gmem_has_outstanding_references(struc= t inode *inode, folio_batch_init(&fbatch); =20 next =3D start; - while (has_outstanding && filemap_get_folios(mapping, &next, last, &fbatc= h)) { + while (!has_outstanding && filemap_get_folios(mapping, &next, last, &fbat= ch)) { for (i =3D 0; i < folio_batch_count(&fbatch); ++i) { struct folio *folio =3D fbatch.folios[i]; =20 - /* - * Outstanding references are anything other than those - * from the page cache, plus 1 temporary reference held - * by filemap_get_folios() in the folio batch. - */ - if (folio_ref_count(folio) !=3D folio_nr_pages(folio) + 1) { - has_outstanding =3D true; + has_outstanding =3D __folio_has_outstanding_references(folio, &drained); + if (has_outstanding) { *err_index =3D max(start, folio->index); break; } --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A384F311C1D; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=JlgXF8dOczLfiB6Xfs+WBj4FgEJluJNBDVXjdxh3IKDHd399vXcYGYrfngcmRxtcV+9tXImc1EWrI/X3BBgv8S5CAjw9ldQcol8yw5rmD+0AvQjM42mLq39lmY/CYs4qhPTJW3MIQnFMkNVWiDwU1xClAic4k0aZ89tt7Bf7s6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=EXsOSFV1vU8DO2BOHPz/Ce8cakRIbhdqPxZFrapJwFs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=J43Y0wxIe5Jn7jE2jSefYoXuT17Na0gcW/vLSK8yIqRGmuQIs1CCwfxHBXKqO4N28jckMZ9TzQ5P+Xre1nH14vHpVfsFZdt8RvWhafwz1m4ji1G07azzY496CRnzJP4vr2ogFNfyJUlSdpnM+ZplrNba7/+Cd9N+8opZDTAx0Zw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R6Zn/gCf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R6Zn/gCf" Received: by smtp.kernel.org (Postfix) with ESMTPS id 82D06C2BCC7; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=EXsOSFV1vU8DO2BOHPz/Ce8cakRIbhdqPxZFrapJwFs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=R6Zn/gCf/axP6nr1BaengQ/HEwKUXk8Cq6jD0GwrDYmNy59fv6L18Ccf+xAvZbxd2 z53c5te2jZ88oxW7EYusMTkvM4Oz1lkt2N7Qt7QJE5VeQ3jwISbpaw/SD1jKsEsKrs EzgOlbKj0EQyJ+RU1TnYadGVrY/x+BwRtx6UMqTrCLLHi4EZpFyWj1afFRSWXu++M5 mtx4k8LGgMkC0M7p42WjHBkRnXIoVwLnLq/fVDRNOeR6KcRRQK7WrU4NGHNr1s/DUS ppr27tO+OozYMgdK6vqEcA3D+Vv/3EKbE6DSdwSAEI8DTa6dXZWMZKCDP5dd7qCCEk uo2qvO66eQIow== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CF2AC61DFF; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:20 -0700 Subject: [PATCH v12 19/45] KVM: guest_memfd: Zero page while getting pfn Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-19-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=3028; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=cUGg+sf4DufBRW7+H+nQruNnSYvplZ5+HPGnat0Z7N0=; b=iaUEix/boqWHf265FiFwo3RPtQTjSrXq4zSJrf1krBwQQG7RpimowPl9GrMH5wwiWvEnLTWM6 eoZpOCymVXPCpSEDnOsDrQzfSoTA27VtGyKj0i0A3pUg0T40KvmIX/V X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Move the folio initialization logic from kvm_gmem_get_pfn() into __kvm_gmem_get_pfn() to also zero pages if the page is to be used in kvm_gmem_populate(). With in-place conversion, the existing data in a guest_memfd page can be populated into guest memory through platform-specific ioctls. Without first zeroing the page obtained using __kvm_gmem_get_pfn(), it might contain uninitialized host memory, which would leak to the guest if the populate completes. guest_memfd pages are zeroed at most once in the page's entire lifetime with guest_memfd, and that is tracked using the uptodate flag. Zeroing the page in __kvm_gmem_get_pfn() is chosen over zeroing in kvm_gmem_get_folio() since other flows, such as a future write() syscall, can get a page, write to the page and then set page uptodate without zeroing. There may be some performance penalty due to redundant zeroing, but this would pale in comparison to the cost of actually assigning the page to the VM. This aligns with the concept of zeroing before first use - the other place where zeroing happens is in kvm_gmem_fault_user_mapping(). On populate failure, the page is not re-zeroed, since on SNP, if firmware rejects a CPUID page, the expected CPUID values provided by firmware are returned to userspace via page contents. More generally, page contents may be modified on populate failure. Don't mark the page uptodate again after populating, since the page would already be marked uptodate before the post_populate() call. Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Reviewed-by: Xiaoyao Li Reviewed-by: Binbin Wu Signed-off-by: Ackerley Tng Reviewed-by: David Hildenbrand (Arm) --- virt/kvm/guest_memfd.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 1fe935aaef36f..5db578fdce6af 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -1105,6 +1105,11 @@ static struct folio *__kvm_gmem_get_pfn(struct file = *file, return ERR_PTR(-EHWPOISON); } =20 + if (!folio_test_uptodate(folio)) { + clear_highpage(folio_page(folio, 0)); + folio_mark_uptodate(folio); + } + *pfn =3D folio_file_pfn(folio, index); if (max_order) *max_order =3D 0; @@ -1133,11 +1138,6 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_mem= ory_slot *slot, goto out; } =20 - if (!folio_test_uptodate(folio)) { - clear_highpage(folio_page(folio, 0)); - folio_mark_uptodate(folio); - } - if (kvm_arch_has_gmem_convert() && kvm_gmem_is_private_mem(file_inode(file), index)) r =3D kvm_arch_gmem_make_private(kvm, gfn, *pfn, @@ -1179,8 +1179,6 @@ static long __kvm_gmem_populate(struct kvm *kvm, stru= ct kvm_memory_slot *slot, } =20 ret =3D post_populate(kvm, gfn, pfn, src_page, opaque); - if (!ret) - folio_mark_uptodate(folio); =20 out_put_folio: folio_put(folio); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B625B317173; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=ASQ8HXZp9IKdOT8Qh/3S6f7Po27ty0b7MmBrVYeu+kHfPtRVU/5y9FYDAH7zEWbrHDPCwa4PngH6JuFVGg/fDbMZW2FoMIWkOsPdUnURJnL2slaWVRF0NcMEG5NuvE/XQXHu2BDOegpiqZUE8jlZhauEIx1sj8JrYa+1dOP531c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=I+b7lGuVoGz+swvsL7rzFlWNT2yN4ZfPLfGq8P0JE3c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=DgZeq5IrSN3XG/jA32rqhuRiQvF7Vn8EwN+v+fGB1NEkV8ygACdj0wBn9MN8lLIfp1tcsVt/Lu47/+pInPlfsyfDBEtRl1bahr29sYOZpAAFzzXnHSrb7EHWvyqX4CNWrnZgiUTJrTLdedbrQjYel/Yhy8+DP0iIOfVzW9V718g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Nj9sQJoq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Nj9sQJoq" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9C30DC32781; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=I+b7lGuVoGz+swvsL7rzFlWNT2yN4ZfPLfGq8P0JE3c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Nj9sQJoqAPH0H2Yl/kBKSHC4wJEQbhF/k/eQKR8YVNhkLabYa8ZbV7iEGoFE4zoAy mivPWY0LAI6h+61PLFMjO3hyDhkF6xb54oD3u/Yh3YEibteGkUS9GHGUSfFkcnvl5u /YbILdp9/gVWOtixolSLpfk60LouFI0p5WsxIJFj+iTfpa1nMTwVRU5wl84CpFAmxR iwafjSxSt4kPOUff8JIYs2G8ILnrN8cWbEr2bwvF2HnuLjqBwccZ5fLR3W543iUo5O eaZhWAgZNoF7drPA90u5RZT+blPY7SUXSbS9TaH6K9qN/dtg6Klh1h5pl+iWKsZd4t nsbCz+BxfSuuw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86861C61DF1; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:21 -0700 Subject: [PATCH v12 20/45] KVM: SEV: Make 'uaddr' parameter optional for KVM_SEV_SNP_LAUNCH_UPDATE Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-20-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=5596; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=wPi3PLQGrKXDeuAU5z9J7AGBTrQFd9B/lpdUMyrGDEE=; b=9k/bLEF3An1XBhsojLn0UeoSCr6PDB828IGyADWhdI3CcerN0zYcgRaFKrrd+pDMI6fEmwe7e Jh0JJIyXJ9DDWGXI4dES9EK0noyDEDN1Lyspx/TT+yisOfiYEXsItsu X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Michael Roth Make the source page for populating an SNP guest_memfd instance optional if in-place conversion/population is enabled. If KVM can convert the page in-place, then it's possible for guest memory to be initialized directly from userspace by mmap()'ing the guest_memfd and writing to it while the corresponding GPA ranges are in a 'shared' state, before converting them to the 'private' state expected by KVM_SEV_SNP_LAUNCH_UPDATE. Update the handling/documentation for KVM_SEV_SNP_LAUNCH_UPDATE to allow for 'uaddr' to be set to NULL when in-place conversion is enabled, which SNP_LAUNCH_UPDATE will then use to determine when it should/shouldn't copy in data from a separate memory location. Continue to enforce non-NULL when PRIVATE is tracked per-VM, not per-guest_memfd. Signed-off-by: Michael Roth [Moved condition to snp_launch_update] Signed-off-by: Sean Christopherson Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- Documentation/virt/kvm/x86/amd-memory-encryption.rst | 14 ++++++++++---- arch/x86/kvm/svm/sev.c | 11 ++++++----- virt/kvm/kvm_main.c | 1 + 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/Documentation/virt/kvm/x86/amd-memory-encryption.rst b/Documen= tation/virt/kvm/x86/amd-memory-encryption.rst index bd04a908a8dbd..5977fbe33b98c 100644 --- a/Documentation/virt/kvm/x86/amd-memory-encryption.rst +++ b/Documentation/virt/kvm/x86/amd-memory-encryption.rst @@ -503,7 +503,8 @@ secrets. =20 It is required that the GPA ranges initialized by this command have had the KVM_MEMORY_ATTRIBUTE_PRIVATE attribute set in advance. See the documentati= on -for KVM_SET_MEMORY_ATTRIBUTES for more details on this aspect. +for KVM_SET_MEMORY_ATTRIBUTES/KVM_SET_MEMORY_ATTRIBUTES2 for more details = on +this aspect. =20 Upon success, this command is not guaranteed to have processed the entire range requested. Instead, the ``gfn_start``, ``uaddr``, and ``len`` fields= of @@ -511,9 +512,14 @@ range requested. Instead, the ``gfn_start``, ``uaddr``= , and ``len`` fields of remaining range that has yet to be processed. The caller should continue calling this command until those fields indicate the entire range has been processed, e.g. ``len`` is 0, ``gfn_start`` is equal to the last GFN in the -range plus 1, and ``uaddr`` is the last byte of the userspace-provided sou= rce -buffer address plus 1. In the case where ``type`` is KVM_SEV_SNP_PAGE_TYPE= _ZERO, -``uaddr`` will be ignored completely. +range plus 1, and ``uaddr`` (if specified) is the last byte of the +userspace-provided source buffer address plus 1. + +In the case where ``type`` is KVM_SEV_SNP_PAGE_TYPE_ZERO, ``uaddr`` will be +ignored completely. For all other page types, ``uaddr`` is optional if in-= place +conversion is enabled (i.e. when the data had been written directly to +guest_memfd while the page was in the shared state) and is required if in-= place +conversion is disabled. =20 Parameters (in): struct kvm_sev_snp_launch_update =20 diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index dff684b88f153..63e8cfa9bf559 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -2400,7 +2400,7 @@ static int sev_gmem_post_populate(struct kvm *kvm, gf= n_t gfn, kvm_pfn_t pfn, */ if (ret && !snp_page_reclaim(kvm, pfn) && sev_populate_args->type =3D=3D KVM_SEV_SNP_PAGE_TYPE_CPUID && - sev_populate_args->fw_error =3D=3D SEV_RET_INVALID_PARAM) { + sev_populate_args->fw_error =3D=3D SEV_RET_INVALID_PARAM && src_page)= { void *src_vaddr =3D kmap_local_page(src_page); void *dst_vaddr =3D kmap_local_pfn(pfn); =20 @@ -2433,8 +2433,8 @@ static int snp_launch_update(struct kvm *kvm, struct = kvm_sev_cmd *argp) if (copy_from_user(¶ms, u64_to_user_ptr(argp->data), sizeof(params))) return -EFAULT; =20 - pr_debug("%s: GFN start 0x%llx length 0x%llx type %d flags %d\n", __func_= _, - params.gfn_start, params.len, params.type, params.flags); + pr_debug("%s: GFN start 0x%llx length 0x%llx type %d flags %d src %llx\n"= , __func__, + params.gfn_start, params.len, params.type, params.flags, params.uaddr); =20 if (!params.len || !PAGE_ALIGNED(params.len) || params.flags || (params.type !=3D KVM_SEV_SNP_PAGE_TYPE_NORMAL && @@ -2446,7 +2446,8 @@ static int snp_launch_update(struct kvm *kvm, struct = kvm_sev_cmd *argp) =20 if (params.type =3D=3D KVM_SEV_SNP_PAGE_TYPE_ZERO) src =3D NULL; - else if (!params.uaddr || !PAGE_ALIGNED(params.uaddr)) + else if ((!gmem_in_place_conversion && !params.uaddr) || + !PAGE_ALIGNED(params.uaddr)) return -EINVAL; else src =3D u64_to_user_ptr(params.uaddr); @@ -2493,7 +2494,7 @@ static int snp_launch_update(struct kvm *kvm, struct = kvm_sev_cmd *argp) =20 params.gfn_start +=3D count; params.len -=3D count * PAGE_SIZE; - if (params.type !=3D KVM_SEV_SNP_PAGE_TYPE_ZERO) + if (src && params.type !=3D KVM_SEV_SNP_PAGE_TYPE_ZERO) params.uaddr +=3D count * PAGE_SIZE; =20 if (copy_to_user(u64_to_user_ptr(argp->data), ¶ms, sizeof(params))) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 1ea8198821917..05c518c9b8078 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -104,6 +104,7 @@ module_param(allow_unsafe_mappings, bool, 0444); =20 #ifdef kvm_arch_has_private_mem bool __ro_after_init gmem_in_place_conversion =3D false; +EXPORT_SYMBOL_FOR_KVM_INTERNAL(gmem_in_place_conversion); #endif =20 /* --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D99B0322A1F; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; cv=none; b=ACiUKZsaOQgs4BbXtPTMinI0/eSvHq5pEfaay15CTW4WvOnqrJ/3WBArD02r1hQKs50JZSgDN7AJDgZD6dgaDOVLW+xnV4fElQiX4QZfKGj05B552OgIMtFtYjxrJO8kYwZtk/I/IObhuxR/Nz8qNDoM/o2UyF1Fq2Wv9y5Ac4g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135921; c=relaxed/simple; bh=8shTrhUaGv3fKB50otzvqy15VrpXV2k9eT7uzPpO87A=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qC0k3cU/AWuBJUO2k1yJQjYxCPkf3Ie30s87N2/P1s/2cpKTWFGvkplJ2U+DZYxwel+3Fyt9glIdWdSJxefnA5PUYCB1c9PAzjlsYciiIPSA4xnvznI5drGyDgD9vQMmP7fLqfaMwCB2zBqJPSKZgKRkzyPX0XtJnwsZE1T9RW0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=U2a+e3+F; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="U2a+e3+F" Received: by smtp.kernel.org (Postfix) with ESMTPS id BA28BC2BCFD; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=8shTrhUaGv3fKB50otzvqy15VrpXV2k9eT7uzPpO87A=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=U2a+e3+Fu3IkwA9ogIkqDjTokD5Ywx6OsEmQ2dtf62zdCoCUC+CShxrFeKOox7WVo QYUk1reCWg11G5xDMbfzGLtqejPe1f0T3f6GBI2dCzmzUczD9fDUf1nAb47E+FYffO GR17qgNZyuRjoN5W6tnFeLHKg4RsmT7oo1GhBtgDLaIc8xobHymV3xYEBxPs3UUy5P O8Z1WOItbP6G7q/T0LMvXLvPmPk2Cz4R30PJE7eNwvgiR3/3ujPgrTMUd/e0M4XUH7 j56lUxumT+3xRHANST2nTojscXUD/YxQgwAY6OEWsOmMSD7lyciIEhU3cOiwIdN8MW ZgIE95nqdWNcQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0B39C61DFD; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:22 -0700 Subject: [PATCH v12 21/45] KVM: TDX: Make source page optional for KVM_TDX_INIT_MEM_REGION Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-21-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=3571; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=njc9wPSBjD6tmKwt2PhlMiU1O1S+4eQ2KtODaEWL4xY=; b=xet/UHJxW/icBX2JrNtGGU28LtYGzpL8u5/A2zHQu7TLEnsVz5ggJjGJpERO2fdfgD7Fxr51C ip40S/BlqIICNfbuaM0UoGpRZF7btZdQpGjyeJFTBr9EgNvYJ9H3K1t X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Update tdx_gmem_post_populate() to handle cases where userspace requests "no source page". To handle "no source page", populate (perform TDH.MEM.PAGE.ADD) using memory in-place at the target PFN. Allow "no source page" only when gmem_in_place_conversion is enabled, because retroactively adding support for out-of-place conversion would mean requiring a userspace update for a feature that's being deprecated. Also, KVM supporting "no source page" without in-place conversion would effectively be an obscure zero-page optimization that relies on the page being zeroed when it is allocated by guest_memfd. Rejecting "no source page" without in-place conversion scenario is valuable for KVM developers since it helps newcomers understand what exactly is and isn't possible. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Shivank Garg Reviewed-by: Binbin Wu Tested-by: Yan Zhao Reviewed-by: Yan Zhao Reviewed-by: Xiaoyao Li Signed-off-by: Ackerley Tng --- Documentation/virt/kvm/x86/intel-tdx.rst | 4 ++++ arch/x86/kvm/vmx/tdx.c | 8 +++++--- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/Documentation/virt/kvm/x86/intel-tdx.rst b/Documentation/virt/= kvm/x86/intel-tdx.rst index 6a222e9d09541..d8d9409120e61 100644 --- a/Documentation/virt/kvm/x86/intel-tdx.rst +++ b/Documentation/virt/kvm/x86/intel-tdx.rst @@ -158,6 +158,10 @@ KVM_TDX_INIT_MEM_REGION Initialize @nr_pages TDX guest private memory starting from @gpa with user= space provided data from @source_addr. @source_addr must be PAGE_SIZE-aligned. =20 +If guest_memfd in-place conversion is enabled, pass 0 for @source_addr +to represent "no source page". A source page is required if in-place +conversion is not enabled or not supported. + Note, before calling this sub command, memory attribute of the range [gpa, gpa + nr_pages] needs to be private. Userspace can use KVM_SET_MEMORY_ATTRIBUTES to set the attribute. diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index b272c20586a74..7654a9f5fa128 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -3189,7 +3189,7 @@ static int tdx_gmem_post_populate(struct kvm *kvm, gf= n_t gfn, kvm_pfn_t pfn, if (KVM_BUG_ON(kvm_tdx->page_add_src, kvm)) return -EIO; =20 - kvm_tdx->page_add_src =3D src_page; + kvm_tdx->page_add_src =3D src_page ?: pfn_to_page(pfn); ret =3D kvm_tdp_mmu_map_private_pfn(arg->vcpu, gfn, pfn); kvm_tdx->page_add_src =3D NULL; =20 @@ -3235,7 +3235,8 @@ static int tdx_vcpu_init_mem_region(struct kvm_vcpu *= vcpu, struct kvm_tdx_cmd *c if (copy_from_user(®ion, u64_to_user_ptr(cmd->data), sizeof(region))) return -EFAULT; =20 - if (!PAGE_ALIGNED(region.source_addr) || !region.source_addr || + if (!PAGE_ALIGNED(region.source_addr) || + (!gmem_in_place_conversion && !region.source_addr) || !PAGE_ALIGNED(region.gpa) || !region.nr_pages || region.gpa + (region.nr_pages << PAGE_SHIFT) <=3D region.gpa || !vt_is_tdx_private_gpa(kvm, region.gpa) || @@ -3266,7 +3267,8 @@ static int tdx_vcpu_init_mem_region(struct kvm_vcpu *= vcpu, struct kvm_tdx_cmd *c break; } =20 - region.source_addr +=3D PAGE_SIZE; + if (region.source_addr) + region.source_addr +=3D PAGE_SIZE; region.gpa +=3D PAGE_SIZE; region.nr_pages--; =20 --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECEF232AAB5; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=dn00upj/iaBhinPAFP+laMkUqVWQQr/ZSTUhsCMJdLMxEfA1MYDfq3ACAyiANBSkEseJKVsb8fETgTNxZ4eh1IDxIYnoHvLg2wgcAKeDhyNvXxoBKM3m4mOSxaq68CUChij2WvarFQpKomBD1WXiQ2OmMg6+ew0uGv/6ziZ1Y/E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=NxJXoL+hsz4bVjsKWlEJSpWxkSslYL/LfQ9olC8zRh8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iiEyAPOoj6DzfobIowqhHNEabbKBvlAvkpCWbfTzJWVptrrlH26FZYVUYWTZfwGPnn4DMQ8rs2vzPt3jcXzy5a0v35yexmE0wNsFTElkQoHrcfppLdn91UpwFy2qZOXviOzGE1ZhhLzUDmKNSZqLL5jUkIxRsVE58wLFy4fQFIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ALKGmRAL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ALKGmRAL" Received: by smtp.kernel.org (Postfix) with ESMTPS id CF3FDC2BCC7; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=NxJXoL+hsz4bVjsKWlEJSpWxkSslYL/LfQ9olC8zRh8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=ALKGmRAL+Na46FxAZrk8zkS3oFn3FIKadNwnytZlryYVryhIdPqtHjGK2JrqnucJj zggqoWSjCFmxU/BX8cHOYaRbUI3TYrWvKUKzZFYUkAkDdezTGp9Rx7UNBQsEAOEuLq pmTPmMbPNPt0Y0SN2r7VqlcpOU0dnG2kg9eTaL/qjKXbbzIT27U7YyF5ht/+JlEA/D BuGctEQ7WZ5GPmMkSDKEjSTPSoNQJOvB+fxVbeynXshf4OtjMsJoi+15iUhEBklvfG 0x+3bUgIHvP/ptLb7zg8de5dCmpfmPJkYPhnGCdeHkMKIxlb7xPLvco+Fj98wYapyx jJtT7RIsI7OXQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7A27C61DE4; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:23 -0700 Subject: [PATCH v12 22/45] KVM: Move KVM_VM_MEMORY_ATTRIBUTES config definition to x86 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-22-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=2002; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=LaDn2CsI38OBZyS6stNAkFJkoZdaUfE8YC0Rnt/13Oo=; b=5ojShQdxW/ITFZ0NQyVogZ4ru4lAf3wNfSGMZhvme6tF3hgOkZs2drAVgwLc05VWwGcoPekF7 TjjTBYl+XCtCB93l/Nax9ptE7IrsyxW78rE0iJUz2HiE05VWEHJBTfT X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Bury KVM_VM_MEMORY_ATTRIBUTES in x86 to discourage other architectures from adding support for per-VM memory attributes, because tracking private vs. shared memory on a per-VM basis is now deprecated in favor of tracking on a per-guest_memfd basis, and while RWX memory attributes are on the horizon, they too are expected to be x86-only. This will also allow modifying KVM_VM_MEMORY_ATTRIBUTES to be user-selectable (in x86) without creating weirdness in KVM's Kconfigs. Now that guest_memfd supports in-place conversions, it's entirely possible to run x86 CoCo VMs without support for KVM_VM_MEMORY_ATTRIBUTES. Leave the code itself in common KVM so that it's trivial to undo this change if new per-VM attributes do come along. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Reviewed-by: Xiaoyao Li Reviewed-by: Binbin Wu Tested-by: Shivank Garg Reviewed-by: David Hildenbrand (Arm) Signed-off-by: Ackerley Tng --- arch/x86/kvm/Kconfig | 3 +++ virt/kvm/Kconfig | 3 --- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 2f8e1b5a8a72c..abb108886733a 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -80,6 +80,9 @@ config KVM_WERROR =20 If in doubt, say "N". =20 +config KVM_VM_MEMORY_ATTRIBUTES + bool + config KVM_SW_PROTECTED_VM bool "Enable support for KVM software-protected VMs" depends on EXPERT diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig index df575449d97b5..a0678ef8ee3f5 100644 --- a/virt/kvm/Kconfig +++ b/virt/kvm/Kconfig @@ -100,9 +100,6 @@ config KVM_ELIDE_TLB_FLUSH_IF_YOUNG config KVM_MMU_LOCKLESS_AGING bool =20 -config KVM_VM_MEMORY_ATTRIBUTES - bool - config KVM_GUEST_MEMFD select XARRAY_MULTI bool --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 135F0330650; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=I3mFD2KBRaYCKGM4OZOyxjPr3K6O5xKx5OU96JFzFTG6C0cZUxM+S2dfDCXOtDxP3ljIrTIt9tL1Fq6ST5CQueu/kvZOazl1uMIErokaFHI8SagsV1tqrXx1swNXUSa2V6fzNn1CJTIkJKuYW+tnfiQJPzhhxwOX2JUjKR66ezI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=Jk3hQh/dQvi3kinnKRipbnhcqE0PyTzUVBPn1RaGePE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RZna7q9YG+q2eU+VcR+qHHiVd0+ql2HjRTc1LH+Si0TMvORheDaqFVUVHqVRp/PqX15W3d5bKLsqdt3zKJOKxudVpGx8RLgIzRjVAj4Bwqb7lAei8XYJKM6X0jEH95gU7dPDDpFeYYsUF2qKGsVwfBKW9zbWiu0woigiXGJjMmQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PnmrHXLL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PnmrHXLL" Received: by smtp.kernel.org (Postfix) with ESMTPS id E4B9EC4AF17; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135921; bh=Jk3hQh/dQvi3kinnKRipbnhcqE0PyTzUVBPn1RaGePE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=PnmrHXLLZrFj0A8uJFO6L416kjg5RfTtcwSq1Yo5rGzZsMsoOee+4FtB0oJhPeWJi zD2mqIkJlr/3EdjHXXJsK/gAMS05pIsj87hAGer9dRbm89tnIXz8d1UwtjErpOcxPs gYwrhYJ9p1U4wzdfrJgy2VEqV+RaUNlLHQLobBEKgI6hXQh7gm0sn6aKKGuP5h26g1 atPZBTOwOYndZJjt8esVbp/1X2rxI9XhOhSgdi+K3qGO5n70V6kOnX1TbKIq8szVsd fL7v4Vp7x09PGz8tK9ByM/0CoZJ4BMCK80P+8rbA5y3gIor9FIP3h+TqdUGlcHxvcS AABm4iaIsIg8w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0A18C61DFF; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:24 -0700 Subject: [PATCH v12 23/45] KVM: Let userspace disable per-VM mem attributes, enable per-gmem attributes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-23-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=6333; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=zFhIN/HOt0ps9ZR1u6TEtlp5GXYBYggvUaVGRCTHweg=; b=GIyUxXbjoDLs6v3p+T1cBBqGmOVY5Ujjy4VII5X1UjcqGtJdO7WY7ymJj083LB/jiDvOH8+gL 2NSh9RmZfhQBYwzFlRqD71nNgRfgxC3FMO5IZobrmqCvvpxiPZKkmCj X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Allow the user to disable KVM_VM_MEMORY_ATTRIBUTES even when KVM supports PRIVATE and SHARED attributes, and expose gmem_in_place_conversion as a module parameter when per-VM attributes are supported. I.e. let userspace enable in-place PRIVATE<=3D>SHARED conversion of guest_memfd pages. Provide both a Kconfig option and a (conditional) module param so that deployments that use a custom kernel can fully disable per-VM tracking, while not forcing distros to ship two separate kernels in order to provide backwards compatibility for downstream users. Don't allow running VMs with mixed tracking for a given instance of KVM, i.e. disallow toggling the module param after KVM is loaded, as the extra complexity needed to handle per-VM behavior far outweighs any potential benefit. E.g. neither TDX nor SNP supports live migration, so in effect the requirement is that existing deployments that want to support both the old and the new models would need to tell their VMM which flavor of tracking to use. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg [Define module_param only if CONFIG_KVM_VM_MEMORY_ATTRIBUTES is enabled] Suggested-by: Xiaoyao Li Reviewed-by: Xiaoyao Li Co-developed-by: Ackerley Tng Signed-off-by: Ackerley Tng Acked-by: David Hildenbrand (Arm) --- Documentation/admin-guide/kernel-parameters.txt | 25 +++++++++++++++++++++= ++++ arch/x86/include/asm/kvm_host.h | 4 +++- arch/x86/kvm/Kconfig | 14 ++++++++++---- virt/kvm/kvm_main.c | 5 ++++- 4 files changed, 42 insertions(+), 6 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 68647ff4bdd24..b5719714458fb 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3156,6 +3156,31 @@ Kernel parameters kvm.enable_vmware_backdoor=3D[KVM] Support VMware backdoor PV interface. Default is false (don't support). =20 + kvm.gmem_in_place_conversion=3D + [KVM] Controls whether KVM enables in-place conversion + support for guest_memfd and tracks the private/shared + state of memory per guest_memfd instead of per VM. + + If enabled, KVM enables the KVM_SET_MEMORY_ATTRIBUTES2 + ioctl on guest_memfd file descriptors and disables the + legacy VM-scoped KVM_SET_MEMORY_ATTRIBUTES ioctl for + private memory state tracking. Only the + KVM_MEMORY_ATTRIBUTE_PRIVATE attribute moves to + per-guest_memfd tracking; other attributes remain + per-VM. + + This parameter toggles KVM's in-place conversion + capability support. Whether a VMM uses separate backends + or out-of-place memory management is determined by + userspace VMM design. + + Note, this parameter is only available when + CONFIG_KVM_VM_MEMORY_ATTRIBUTES=3Dy. When + CONFIG_KVM_VM_MEMORY_ATTRIBUTES is not set, in-place + conversion is unconditionally enabled. + + Default is Y (on). + kvm.nx_huge_pages=3D [KVM] Controls the software workaround for the X86_BUG_ITLB_MULTIHIT bug. diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 83e26ce45fb79..e840418427a1d 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1851,7 +1851,9 @@ enum kvm_intr_type { ((vcpu) && (vcpu)->arch.handling_intr_from_guest && \ (!!in_nmi() =3D=3D ((vcpu)->arch.handling_intr_from_guest =3D=3D KVM_HAN= DLING_NMI))) =20 -#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES +#if defined(CONFIG_KVM_SW_PROTECTED_VM) || \ + defined(CONFIG_KVM_INTEL_TDX) || \ + defined(CONFIG_KVM_AMD_SEV) #define kvm_arch_has_private_mem(kvm) ((kvm)->arch.has_private_mem) #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index abb108886733a..2c3c22aeafa54 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -81,13 +81,21 @@ config KVM_WERROR If in doubt, say "N". =20 config KVM_VM_MEMORY_ATTRIBUTES - bool + bool "Enable per-VM PRIVATE vs. SHARED attributes (for CoCo VMs)" + depends on KVM_SW_PROTECTED_VM || KVM_INTEL_TDX || KVM_AMD_SEV + help + Enable support for tracking PRIVATE vs. SHARED memory using per-VM + memory attributes. Using per-VM attributes is deprecated in favor of + tracking PRIVATE state in guest_memfd. Select this if you need to run + CoCo VMs using a VMM that doesn't support guest_memfd memory + attributes. + + If unsure, say N. =20 config KVM_SW_PROTECTED_VM bool "Enable support for KVM software-protected VMs" depends on EXPERT depends on KVM_X86 && X86_64 - select KVM_VM_MEMORY_ATTRIBUTES help Enable support for KVM software-protected VMs. Currently, software- protected VMs are purely a development and testing vehicle for @@ -138,7 +146,6 @@ config KVM_INTEL_TDX bool "Intel Trust Domain Extensions (TDX) support" default y depends on INTEL_TDX_HOST - select KVM_VM_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_POPULATE help Provides support for launching Intel Trust Domain Extensions (TDX) @@ -162,7 +169,6 @@ config KVM_AMD_SEV depends on KVM_AMD && X86_64 depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=3Dy && CRYPTO_DEV_CCP_DD=3Dm) select ARCH_HAS_CC_PLATFORM - select KVM_VM_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 05c518c9b8078..929fd3e1a01e6 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -103,7 +103,10 @@ static bool __ro_after_init allow_unsafe_mappings; module_param(allow_unsafe_mappings, bool, 0444); =20 #ifdef kvm_arch_has_private_mem -bool __ro_after_init gmem_in_place_conversion =3D false; +bool __ro_after_init gmem_in_place_conversion =3D !IS_ENABLED(CONFIG_KVM_V= M_MEMORY_ATTRIBUTES); +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES +module_param(gmem_in_place_conversion, bool, 0444); +#endif EXPORT_SYMBOL_FOR_KVM_INTERNAL(gmem_in_place_conversion); #endif =20 --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25C362E738D; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=Ju9j/cvoHxvvtr52cl1qTUlAQ9NGj6gIYQ5lY138RYtl0ebXSsVhTMogWvFDNJAGgiU5QH3PXerTQuWVMeZ0bCR3rGMp6sK2ACrgBrnjN5ITj0GukWOIHL6Uf5LWyvRD+jGwDN+xHIacNvWbeqLl73AicxiIhRIQznIomnd+CMA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=9EkpEqxMLjQNDYpL+7i928KW0BKp0BAiSqRXLiWPp6s=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OVIs8u23K8O6imclLLJvFFs6lON16BUb56EzT8doLY86WImagluuxvWLyZQH5jRT/a0tBYKm8SOLtJuZWHH3Lk2E0DABz8ZQyOhgD2BzHPXhtpz9ONoDlLf3z1yhMCCTiHLSsE+zNIevOqtctnLtzPqgr1QGwgDojxsh1MCmC4A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gyYzRbzX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gyYzRbzX" Received: by smtp.kernel.org (Postfix) with ESMTPS id 09B95C2BCFD; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=9EkpEqxMLjQNDYpL+7i928KW0BKp0BAiSqRXLiWPp6s=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=gyYzRbzXGlkG/aUpiIys2LbQmRsW7tP50wUDEsyt9RjxvJ64sTjxOT3hh/k8Hdy4w /fwFxfHMH7fRxj90PD7xEba/JSh55g2ZM4d14gJ/TSaGDaT0pImU/S/v7Xv/qJo/Xb tu/MCelKnAxVZlN0r7KRn54+V+a0/6fi2RFkmt4av+ChvYzGOJKjvHcWe8xsdI9h+Q XDMxxlXdtvgro814Qmu1PWjZ5th3gM/KubMPWZ/mXTJzFDLXnNAGALHc7jQoRPJb35 Hk1JTxKCTvRb0/erM0C1KWQGBKeUbtdQgAdTGiIDqdW3guhLOitVVrEZUwIBOaWrxo d37knLUInc+IQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8926C61DF1; Mon, 31 Aug 2026 00:25:21 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:25 -0700 Subject: [PATCH v12 24/45] KVM: guest_memfd: Enable INIT_SHARED on guest_memfd for x86 Coco VMs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-24-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng , Xiaoyao Li X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=1693; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=3+Z8h6K2ZEH0/i+xD8ty2YuUR3PhzVzKd8N9HPdoh4c=; b=kFzzNs3iUa2auFaXzj1DJYK15zRovrlFdOgPxqF+B0zfKR4KcuuLFX1O1DbwmGlWyFFcc13bR c62upXEB2l4AoL7J7gxH3T2810e7NU8NfEcZrGUtWkWt3/4kWC/EnPJ X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Now that guest_memfd supports tracking private vs. shared within gmem itself, allow userspace to specify INIT_SHARED on a guest_memfd instance for x86 Confidential Computing (CoCo) VMs, so long as in-place conversion is enabled, i.e. when it's actually possible for a guest_memfd instance to contain shared memory. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Reviewed-by: Xiaoyao Li Tested-by: Shivank Garg Reviewed-by: Binbin Wu Signed-off-by: Ackerley Tng --- arch/x86/kvm/x86.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 75e03a2f79db2..c081e4b9b25de 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10637,14 +10637,15 @@ bool kvm_arch_no_poll(struct kvm_vcpu *vcpu) } =20 #ifdef CONFIG_KVM_GUEST_MEMFD -/* - * KVM doesn't yet support initializing guest_memfd memory as shared for V= Ms - * with private memory (the private vs. shared tracking needs to be moved = into - * guest_memfd). - */ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm) { - return !kvm_arch_has_private_mem(kvm); + /* + * INIT_SHARED is supported if in-place conversion is enabled, or if + * the VM doesn't support private memory. If the VM has private memory + * and in-place conversion is disabled, then guest_memfd can _only_ be + * used for private memory. + */ + return gmem_in_place_conversion || !kvm_arch_has_private_mem(kvm); } =20 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41DC02EEE60; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=ZuvDwUH+L8iUA909dl9fjmxsrMZrpIST0VsVHqvoOfWPMkiG7q2kNUe4BYWKCv00iyHf4XSwh3fzps4crwriqU3COrOYDdMHPaCOsZFRaw2sVEIgNLrn6lGxcOzd4PeqMoZt/uKkwRROoGy5ErmB+fydrPNS4QffwTMauwv1Euk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=wJ0HFZ639XCyCJUINstQYMk/aJO14np7PyBQNhcLW4c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YWbG128XDzlB0bJnjyAK2U85Db3YxI9y7dwh/sgagt/QgfoUw7bczYmsqv4EV1qjVntlJVg5b8YetcTFogah6UVlGsX5olgFgks3PjP9+YA3I15hrPEAHLCXhbVuWRe74yjIPzU9seq1mfeONmlVf34NF6R2vTuNg7UFJfCQ1ls= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KMKT7i65; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KMKT7i65" Received: by smtp.kernel.org (Postfix) with ESMTPS id 23065C32782; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=wJ0HFZ639XCyCJUINstQYMk/aJO14np7PyBQNhcLW4c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=KMKT7i65k/WsFlmyDROAB+p2TxV5AT2YqWmFO8F/kkNy+psOVgqFQSuMVDQHUCcAE w0nRpVO0OuvNRxZrMB3QmECirZwuZ3H+MC4v0aobw5RjZLjg+Bv/g5qhF7Zh/cU5t3 UIgreUwECiomF/jxbrlJG+RRoY/xRBph+8P9LUa6CIUjFE+LlrNf10vRtCO5XDw3xd EvbMRfxV1gdAxI5Fpvediv6wX3A3jMwyXM8emEwHhQalNJmBTvSa3HDwDSZGayOozW PTMwxI0E81BxWaAJe9HYOVGjm9hS0y+21MLEWR/pD+I3v4RXbJTsV4IpguLH/6JQUD YKpwUpDff+pew== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CC78C61DE2; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:26 -0700 Subject: [PATCH v12 25/45] KVM: selftests: Create gmem fd before "regular" fd when adding memslot Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-25-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=2916; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=FzqOYqjoRqio5Tf5GMzyqkhVO+XY7bfbFLlkq3H6LaQ=; b=JKi9QkjIbWgmEZutF4Q114HPVnpKdnnCJthVjkvj3BeW4MpXotZ/Lfvs9fsNMGSQfuVR1qof0 Ju2ph/EWKJQAF3XVOFrleCZghv14OwSSWO0BT/UqKjcOhAk++RFTpts X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson When adding a memslot associated with a guest_memfd instance, create/dup the guest_memfd before creating the "normal" backing file. This will allow dup'ing the gmem fd as the normal fd when guest_memfd supports mmap(), i.e. to make guest_memfd the _only_ backing source for the memslot. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/lib/kvm_util.c | 45 +++++++++++++++-----------= ---- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index 9ddc047d5c275..3413d45e34faf 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1077,6 +1077,29 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backi= ng_src_type src_type, if (alignment > 1) region->mmap_size +=3D alignment; =20 + if (flags & KVM_MEM_GUEST_MEMFD) { + if (guest_memfd < 0) { + u32 guest_memfd_flags =3D 0; + + TEST_ASSERT(!guest_memfd_offset, + "Offset must be zero when creating new guest_memfd"); + guest_memfd =3D vm_create_guest_memfd(vm, mem_size, guest_memfd_flags); + } else { + /* + * Install a unique fd for each memslot so that the fd + * can be closed when the region is deleted without + * needing to track if the fd is owned by the framework + * or by the caller. + */ + guest_memfd =3D kvm_dup(guest_memfd); + } + + region->region.guest_memfd =3D guest_memfd; + region->region.guest_memfd_offset =3D guest_memfd_offset; + } else { + region->region.guest_memfd =3D -1; + } + region->fd =3D -1; if (backing_src_is_shared(src_type)) region->fd =3D kvm_memfd_alloc(region->mmap_size, @@ -1106,28 +1129,6 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backi= ng_src_type src_type, =20 region->backing_src_type =3D src_type; =20 - if (flags & KVM_MEM_GUEST_MEMFD) { - if (guest_memfd < 0) { - u32 guest_memfd_flags =3D 0; - TEST_ASSERT(!guest_memfd_offset, - "Offset must be zero when creating new guest_memfd"); - guest_memfd =3D vm_create_guest_memfd(vm, mem_size, guest_memfd_flags); - } else { - /* - * Install a unique fd for each memslot so that the fd - * can be closed when the region is deleted without - * needing to track if the fd is owned by the framework - * or by the caller. - */ - guest_memfd =3D kvm_dup(guest_memfd); - } - - region->region.guest_memfd =3D guest_memfd; - region->region.guest_memfd_offset =3D guest_memfd_offset; - } else { - region->region.guest_memfd =3D -1; - } - region->unused_phy_pages =3D sparsebit_alloc(); if (vm_arch_has_protected_memory(vm)) region->protected_phy_pages =3D sparsebit_alloc(); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5641D3382C7; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=j1HztZF7sXQPXsmDI5qLba7b3JOCSEN1h3LHaemjJfHVzvnvPG2mHenwJNxZLfnyupgEtYtL8x/764RmbRnO15nrD56eepmVlXemlWbTpX3EkytXNn9gJzYiGL1zFGgr64to4N7idBmC75H8dEFO+AiWSVLWjEmQJo/wEcssgXY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=M3bGP8eN2EkZz+TH6F9ZZ7lnZVDiPkWGy0QRvi56dQk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AVhr0LLGfNf7C6gVBvlulxCmHpMvGtKWLI6mt5Q2+UO7km3EsH8wMORpGoWdKISQhsglWtfnE7jOXEOMnksaRNWpg2OruPUbv99QMyQ1tM99dMd2EBExZ5r4fe+Ida4XHjRjT0t3FK9FB/F+NLSlqcbGEPVFKnYYcOUEHh407cI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bSe4BTN/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bSe4BTN/" Received: by smtp.kernel.org (Postfix) with ESMTPS id 3AB80C2BCFC; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=M3bGP8eN2EkZz+TH6F9ZZ7lnZVDiPkWGy0QRvi56dQk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=bSe4BTN/UUC6U8zw2icNXCdGw6WWu+oDjPpgVDbsEgXIt1DkfwIZL8QxDdjh5gjgI 2gxsvm8rK6ud1BrYk87CnxAcyzN/QvNTy/S+zLvmXXDqDJEiC/+jvacDkNQo3J1x49 Hi5fhAuQLFgly7WHMAU0qBwDfy0XSWp0JjTGfBN9Aki9uGrrcwVDtM7/XvpEQYO+pt lKWGKSX4mJ8fPWkAMdebho7fhH3qD+xzhHm4EUxKLVg+MHGhIX5R8VFwUQQT11zE7z LJsWvzBV3lI5rf62K7tN0lIatAQHjfb03o+vpMrm3TW2VqmHb+fsm1TEb6Ne8z8u4z Gq/frVhKC4ptQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2465EC61DE4; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:27 -0700 Subject: [PATCH v12 26/45] KVM: selftests: Rename guest_memfd{,_offset} to gmem_{fd,offset} Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-26-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135915; l=5047; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=CnemoPfuFdaUvSMt9CmQCu9n5JepQ0i6fFu3iUOfF5I=; b=ImO7KsNUAOvVm4KzayHLnGzn1IPUTQrDIoBwwQmwHLMBTZ4gPEZFQmcuIh7SHUEbdn8uyQ4H2 6IUvtQf8AROAHmfLddoKJqpsR3tPXdFyG52/+m+vjDnE4tf3wKqsLW/ X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Rename local variables and function parameters for the guest memory file descriptor and its offset to use a "gmem_" prefix instead of "guest_memfd_". No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/kvm_util.h | 6 +++--- tools/testing/selftests/kvm/lib/kvm_util.c | 26 +++++++++++++---------= ---- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index 5a7a455b53875..cf49a84de02e8 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -690,17 +690,17 @@ int __vm_set_user_memory_region(struct kvm_vm *vm, u3= 2 slot, u32 flags, gpa_t gpa, u64 size, void *hva); void vm_set_user_memory_region2(struct kvm_vm *vm, u32 slot, u32 flags, gpa_t gpa, u64 size, void *hva, - u32 guest_memfd, u64 guest_memfd_offset); + u32 gmem_fd, u64 gmem_offset); int __vm_set_user_memory_region2(struct kvm_vm *vm, u32 slot, u32 flags, gpa_t gpa, u64 size, void *hva, - u32 guest_memfd, u64 guest_memfd_offset); + u32 gmem_fd, u64 gmem_offset); =20 void vm_userspace_mem_region_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type, gpa_t gpa, u32 slot, u64 npages, u32 flags); void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type, gpa_t gpa, u32 slot, u64 npages, u32 flags, - int guest_memfd_fd, u64 guest_memfd_offset); + int gmem_fd, u64 gmem_offset); =20 #ifndef vm_arch_has_protected_memory static inline bool vm_arch_has_protected_memory(struct kvm_vm *vm) diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index 3413d45e34faf..af501a27211a2 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -970,7 +970,7 @@ void vm_set_user_memory_region(struct kvm_vm *vm, u32 s= lot, u32 flags, =20 int __vm_set_user_memory_region2(struct kvm_vm *vm, u32 slot, u32 flags, gpa_t gpa, u64 size, void *hva, - u32 guest_memfd, u64 guest_memfd_offset) + u32 gmem_fd, u64 gmem_offset) { struct kvm_userspace_memory_region2 region =3D { .slot =3D slot, @@ -978,8 +978,8 @@ int __vm_set_user_memory_region2(struct kvm_vm *vm, u32= slot, u32 flags, .guest_phys_addr =3D gpa, .memory_size =3D size, .userspace_addr =3D (uintptr_t)hva, - .guest_memfd =3D guest_memfd, - .guest_memfd_offset =3D guest_memfd_offset, + .guest_memfd =3D gmem_fd, + .guest_memfd_offset =3D gmem_offset, }; =20 TEST_REQUIRE_SET_USER_MEMORY_REGION2(); @@ -989,10 +989,10 @@ int __vm_set_user_memory_region2(struct kvm_vm *vm, u= 32 slot, u32 flags, =20 void vm_set_user_memory_region2(struct kvm_vm *vm, u32 slot, u32 flags, gpa_t gpa, u64 size, void *hva, - u32 guest_memfd, u64 guest_memfd_offset) + u32 gmem_fd, u64 gmem_offset) { int ret =3D __vm_set_user_memory_region2(vm, slot, flags, gpa, size, hva, - guest_memfd, guest_memfd_offset); + gmem_fd, gmem_offset); =20 TEST_ASSERT(!ret, "KVM_SET_USER_MEMORY_REGION2 failed, errno =3D %d (%s)", errno, strerror(errno)); @@ -1002,7 +1002,7 @@ void vm_set_user_memory_region2(struct kvm_vm *vm, u3= 2 slot, u32 flags, /* FIXME: This thing needs to be ripped apart and rewritten. */ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type, gpa_t gpa, u32 slot, u64 npages, u32 flags, - int guest_memfd, u64 guest_memfd_offset) + int gmem_fd, u64 gmem_offset) { int ret; struct userspace_mem_region *region; @@ -1078,12 +1078,12 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_back= ing_src_type src_type, region->mmap_size +=3D alignment; =20 if (flags & KVM_MEM_GUEST_MEMFD) { - if (guest_memfd < 0) { - u32 guest_memfd_flags =3D 0; + if (gmem_fd < 0) { + u32 gmem_flags =3D 0; =20 - TEST_ASSERT(!guest_memfd_offset, + TEST_ASSERT(!gmem_offset, "Offset must be zero when creating new guest_memfd"); - guest_memfd =3D vm_create_guest_memfd(vm, mem_size, guest_memfd_flags); + gmem_fd =3D vm_create_guest_memfd(vm, mem_size, gmem_flags); } else { /* * Install a unique fd for each memslot so that the fd @@ -1091,11 +1091,11 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_back= ing_src_type src_type, * needing to track if the fd is owned by the framework * or by the caller. */ - guest_memfd =3D kvm_dup(guest_memfd); + gmem_fd =3D kvm_dup(gmem_fd); } =20 - region->region.guest_memfd =3D guest_memfd; - region->region.guest_memfd_offset =3D guest_memfd_offset; + region->region.guest_memfd =3D gmem_fd; + region->region.guest_memfd_offset =3D gmem_offset; } else { region->region.guest_memfd =3D -1; } --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C85F339365; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=qoKRKzXfiI0PitVp7O3EIFhdAvQHn9blC/ubKDLdw+4PAJ4vYmZYOK+3JkOjGU6f6WYWIRd3nFfpBREevIWAF3TQ3kJn9T1m0EcTX1IMulJYAFqjJMrqIcnaPX/Q364MNt4tVpA/3ksXjumkhPRJQ4Rl8Vo2+pXyJ6FY42OEHcQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=nzr/nMuFBpeaHfjIOVqkEt0JBSSlqJx2fuhse27sC1I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kkQ82CQrz69ijCof2e6bZvn/p4oIMnAfSUsxTpE06jeQKLSjQcEWgb/LeKjOQvwTbmS+jWQIjDKIk8z6fJUVzLgqnwapiKITr+2WRjsJyCXziEl5A8rCw3iDK19nHpLC5L+EWxEnfszGzAERKpSyTuqs5oNqkIZ4NK6m7G5R4gA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=grC1APFp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="grC1APFp" Received: by smtp.kernel.org (Postfix) with ESMTPS id 51737C2BCFF; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=nzr/nMuFBpeaHfjIOVqkEt0JBSSlqJx2fuhse27sC1I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=grC1APFppg1541K6Gewfgh682BN+CaAeQGbgvnZGlxcYf9VyVOp/514NYuivE4h59 FFS12YGmQeX6DdDAuUw7316qTtgUmGbDMGCiEAyDoAMZelNaca7oT5f1kpZN7Y/y30 ikgPIVudCLCldL0NSmFN+cG44NblspY4HkWOSsQQNaohxYiw8HulfRUNpyL4o/W12f VJWTbfB//hQInyB25r5sY26l3579NoT+klApVo1SrN/xAxkI3BP67aFEl3PNLNkVcr QXyYZ4O505N4qPxuc5CBHDPOqhAv8+yne6AIGWaps5SE62/s3dPHW+MzEriTu9pltA 1qawbSbT7tWSA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A3A7C61DFD; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:28 -0700 Subject: [PATCH v12 27/45] KVM: selftests: Add support for mmap() on guest_memfd in core library Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-27-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=6306; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=2P7KrHKxXOiNhlI6jqhEEHAIx4SiQuNaqnM7Bico8Qo=; b=zEkZyZjkCLnZniKkFQ8JynPpcxqBk+K4Eh+wcYnvb51HcMGlBd/4QCkwuB5hR29VEwvqOsWzT GiLsBGlxAWbAkqLjUZ/Sg0fSnWvW1sKFtpqp2r1gByCYwHvNIxtI+6p X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Accept gmem_flags in vm_mem_add() to be able to create a guest_memfd within vm_mem_add(). When vm_mem_add() is used to set up a guest_memfd for a memslot, set up the provided (or created) gmem_fd as the fd for the user memory region. This makes it available to be mmap()-ed from just like fds from other memory sources. For guest_memfds, mmap() using gmem_offset instead of 0 all the time. Always use MAP_SHARED if mmap-ing from guest_memfd instead of reading flag from the configured src_type, which doesn't include guest_memfd. Add a kvm_slot_to_fd() helper to provide convenient access to the file descriptor of a memslot. Update existing callers of vm_mem_add() to pass 0 for gmem_flags to preserve existing behavior. Signed-off-by: Sean Christopherson [mmap() with gmem_offset, and always with MAP_SHARED.] Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/kvm_util.h | 7 +++++- tools/testing/selftests/kvm/lib/kvm_util.c | 28 ++++++++++++------= ---- .../kvm/x86/private_mem_conversions_test.c | 2 +- 3 files changed, 23 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index cf49a84de02e8..a091c495442a1 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -700,7 +700,7 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm, gpa_t gpa, u32 slot, u64 npages, u32 flags); void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type, gpa_t gpa, u32 slot, u64 npages, u32 flags, - int gmem_fd, u64 gmem_offset); + int gmem_fd, u64 gmem_offset, u64 gmem_flags); =20 #ifndef vm_arch_has_protected_memory static inline bool vm_arch_has_protected_memory(struct kvm_vm *vm) @@ -732,6 +732,11 @@ void *addr_gva2hva(struct kvm_vm *vm, gva_t gva); gpa_t addr_hva2gpa(struct kvm_vm *vm, void *hva); void *addr_gpa2alias(struct kvm_vm *vm, gpa_t gpa); =20 +static inline int kvm_slot_to_fd(struct kvm_vm *vm, u32 slot) +{ + return memslot2region(vm, slot)->fd; +} + #ifndef vcpu_arch_put_guest #define vcpu_arch_put_guest(mem, val) do { (mem) =3D (val); } while (0) #endif diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index af501a27211a2..08d0bbcd421ea 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1002,12 +1002,14 @@ void vm_set_user_memory_region2(struct kvm_vm *vm, = u32 slot, u32 flags, /* FIXME: This thing needs to be ripped apart and rewritten. */ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type, gpa_t gpa, u32 slot, u64 npages, u32 flags, - int gmem_fd, u64 gmem_offset) + int gmem_fd, u64 gmem_offset, u64 gmem_flags) { int ret; struct userspace_mem_region *region; size_t backing_src_pagesz =3D get_backing_src_pagesz(src_type); + int mmap_flags =3D vm_mem_backing_src_alias(src_type)->flag; size_t mem_size =3D npages * vm->page_size; + off_t mmap_offset =3D 0; size_t alignment =3D 1; =20 TEST_REQUIRE_SET_USER_MEMORY_REGION2(); @@ -1079,8 +1081,6 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backin= g_src_type src_type, =20 if (flags & KVM_MEM_GUEST_MEMFD) { if (gmem_fd < 0) { - u32 gmem_flags =3D 0; - TEST_ASSERT(!gmem_offset, "Offset must be zero when creating new guest_memfd"); gmem_fd =3D vm_create_guest_memfd(vm, mem_size, gmem_flags); @@ -1101,13 +1101,17 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_back= ing_src_type src_type, } =20 region->fd =3D -1; - if (backing_src_is_shared(src_type)) + if (flags & KVM_MEM_GUEST_MEMFD && gmem_flags & GUEST_MEMFD_FLAG_MMAP) { + region->fd =3D kvm_dup(gmem_fd); + mmap_flags =3D MAP_SHARED; + mmap_offset =3D gmem_offset; + } else if (backing_src_is_shared(src_type)) { region->fd =3D kvm_memfd_alloc(region->mmap_size, src_type =3D=3D VM_MEM_SRC_SHARED_HUGETLB); + } =20 - region->mmap_start =3D kvm_mmap(region->mmap_size, PROT_READ | PROT_WRITE, - vm_mem_backing_src_alias(src_type)->flag, - region->fd); + region->mmap_start =3D __kvm_mmap(region->mmap_size, PROT_READ | PROT_WRI= TE, + mmap_flags, region->fd, mmap_offset); =20 TEST_ASSERT(!is_backing_src_hugetlb(src_type) || region->mmap_start =3D=3D align_ptr_up(region->mmap_start, backing_s= rc_pagesz), @@ -1153,10 +1157,10 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_back= ing_src_type src_type, =20 /* If shared memory, create an alias. */ if (region->fd >=3D 0) { - region->mmap_alias =3D kvm_mmap(region->mmap_size, - PROT_READ | PROT_WRITE, - vm_mem_backing_src_alias(src_type)->flag, - region->fd); + region->mmap_alias =3D __kvm_mmap(region->mmap_size, + PROT_READ | PROT_WRITE, + mmap_flags, region->fd, + mmap_offset); =20 /* Align host alias address */ region->host_alias =3D align_ptr_up(region->mmap_alias, alignment); @@ -1167,7 +1171,7 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type, gpa_t gpa, u32 slot, u64 npages, u32 flags) { - vm_mem_add(vm, src_type, gpa, slot, npages, flags, -1, 0); + vm_mem_add(vm, src_type, gpa, slot, npages, flags, -1, 0, 0); } =20 /* diff --git a/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c= b/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c index 3aa0673f79f33..f3de2a4ab0551 100644 --- a/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c @@ -399,7 +399,7 @@ static void test_mem_conversions(enum vm_mem_backing_sr= c_type src_type, u32 nr_v for (i =3D 0; i < nr_memslots; i++) vm_mem_add(vm, src_type, BASE_DATA_GPA + slot_size * i, BASE_DATA_SLOT + i, slot_size / vm->page_size, - KVM_MEM_GUEST_MEMFD, memfd, slot_size * i); + KVM_MEM_GUEST_MEMFD, memfd, slot_size * i, 0); =20 for (i =3D 0; i < nr_vcpus; i++) { gpa_t gpa =3D BASE_DATA_GPA + i * per_cpu_size; --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86A3331355D; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=J44XhUkcvyP39PBovRggfWNfAegTVDWfi2ESgKAr7zBaykXvW7qO/P8HDRC/dL7ktPpq/nUFV8QvIwbxmOBEOkPYD4gwvHKX77+0vqk0nwm+IaNkjTm4upnS2uUxUkTRp0YYYxkX9Z2ffOE9pVdqUe2O7LgJJzV/4WToWD1OOeU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=a4uO0W/pGoQMRdSfY0gjr7ckE3bbyWjhBY1ZCgow1bY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=NzfPJEo7MC0IRQq/EqfdaIYydMpVojQWERPz/WezeHKOZzla7CG7msZyEP2NC6tUOnzvTEIqNYWZSm/atF7FxrP50Q2ETbXitZW5jcKd1VbSYhtIUVYIU2ur5Sp0+CcUql4J9lMrvAuTwdfMKdM1byM7wbPKMncfYxlJxu7bAoI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FlvNkwZe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FlvNkwZe" Received: by smtp.kernel.org (Postfix) with ESMTPS id 6B1DEC2BCFB; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=a4uO0W/pGoQMRdSfY0gjr7ckE3bbyWjhBY1ZCgow1bY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=FlvNkwZe/kou0Z6nlaSUIG9Njf8BCS1B2LllbA2DrlBxTwLPlV3/EpND8Y61zkBuO OQbdKuqgkuR5Wr1xzBSfBQNKmDCY82e6zLamcuqPf62MLdxVggB4+fkyDwljrnPSdw auyURwTzi0gmvVddR9rfGUFoDpG0j4rkUIZQt2Mka1+UcJeMEzqbuPjeuhEGIxPkqU IMHNi1JWIRbhcg3eiT9ccmcIR2fhqSrudIxWHypa2SoAUM5/U2YnmyfL5HFg2yuRpC qMi80c8BasGORuQJ2Xzds2UFs7xPSLf5nGxNEFAXj3QwPCOTNVtOkGEbZRN35QZp16 XNdd6ExYUgVZQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51FDDC61DF1; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:29 -0700 Subject: [PATCH v12 28/45] KVM: selftests: Add selftests global for guest memory attributes capability Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-28-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=2380; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=dHYrti6WIKjQ1dgvMw4UrVKAOHIaicLWAJZOdnwzNRo=; b=KzmA7WFfRXK+LXae5PyPGG+GqegEtvr1HPmpjZT4XD1quEgpK87wCsvWyJBbU8jr8Cn7M8q4B Hdxr/I/Cz+sCpDZSxn3ftblb0bY/0WQRM5qX//OjyEOF3venXcUNAhm X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Add a global variable, kvm_has_gmem_attributes, to make the result of checking for KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES available to all tests. kvm_has_gmem_attributes is true if guest_memfd tracks memory attributes, as opposed to VM-level tracking. This global variable is synced to the guest for testing convenience, to avoid introducing subtle bugs when host/guest state is desynced. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/test_util.h | 2 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/tools/testing/selftests/kvm/include/test_util.h b/tools/testin= g/selftests/kvm/include/test_util.h index a6a3e16578959..d5825211f2727 100644 --- a/tools/testing/selftests/kvm/include/test_util.h +++ b/tools/testing/selftests/kvm/include/test_util.h @@ -116,6 +116,8 @@ struct kvm_random_state { extern u32 kvm_random_seed; extern struct kvm_random_state kvm_rng; =20 +extern bool kvm_has_gmem_attributes; + struct kvm_random_state new_kvm_random_state(u32 seed); u32 kvm_random_u32(struct kvm_random_state *state); =20 diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index 08d0bbcd421ea..cd468b2bb8343 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -32,6 +32,8 @@ static void kvm_seed_rng(u32 seed) kvm_rng =3D new_kvm_random_state(kvm_random_seed); } =20 +bool kvm_has_gmem_attributes; + static size_t vcpu_mmap_sz(void); =20 int __open_path_or_exit(const char *path, int flags, const char *enoent_he= lp) @@ -527,6 +529,7 @@ struct kvm_vm *__vm_create(struct vm_shape shape, u32 n= r_runnable_vcpus, kvm_seed_rng(kvm_random_seed); =20 sync_global_to_guest(vm, kvm_rng); + sync_global_to_guest(vm, kvm_has_gmem_attributes); =20 kvm_arch_vm_post_create(vm, nr_runnable_vcpus); =20 @@ -2311,6 +2314,8 @@ void __attribute((constructor)) kvm_selftest_init(voi= d) srandom(time(0)); kvm_seed_rng(random()); =20 + kvm_has_gmem_attributes =3D kvm_has_cap(KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIB= UTES); + kvm_selftest_arch_init(); } =20 --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0B1D33F582; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=ZWmnGfyjUuA9ZED97v2dKnOLFHesAcwQj6QlPu0S6VIoZg9Yifwn91IWTysGJ0uq6LrzLqSxsIuIklBwFioUoO3qMA1J9dJuQy5XeWARi3AiXq8RCOr9LCNTT/uECdtPQv6pARngA+OK1ZGHbK697AsJnB8q9xL/AptyGE1cmmA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=Un65aX2hliMBp9F+laOBU+7N5M3Zse1cuzM3vKCbt0E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YsBYawOgZUb0RdL4f3xkzkvZgCJrd2mgV9kLRBatiSNMBsWhlm8VGZ45oNHPpoMMiX32UrBz4Wh48sRrPM4d+Mbr9RHKOQMJnV9M0nWmxEX/AEnK5CriQsP0M7GRAF/hFZj/6s0yO4FYtExXBqkaVsOqwvtehtLjJecNIdvLrdY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fBvVPBqz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fBvVPBqz" Received: by smtp.kernel.org (Postfix) with ESMTPS id 83557C2BCF5; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=Un65aX2hliMBp9F+laOBU+7N5M3Zse1cuzM3vKCbt0E=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=fBvVPBqzFLSplmlBFpA9l87l812gcH4JvkdqznS5UPRtUqBhVYUV5uefA3KxqCmjn 2YAd1GXaI+jQA5hrmIlAT2xY6elwmZESqLaZSlwCOT1dSH+Y3kYEk+jVjJRxQxUNpb CjFNz8sJInEFDoLK0jgdXvqPU/pHaz75m5zkrddNUE+NhggBghenl6+CdDD7ElSNTb PRjRul3TtvmEG12mdW6ixhKcs48NqaAGcTreDWbv7m0nfDUAp/FqGtMbEco9xVP+a5 w/K7l9Tq1Gz01WX3qysSK/BYABYCNK15oQc/yaF39wokGTxMtqfIjq18M6d8bmdlwJ 74ZevgOgXz+9w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B6C6C61DE2; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:30 -0700 Subject: [PATCH v12 29/45] KVM: selftests: Add helpers for calling ioctls on guest_memfd Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-29-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=5371; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=kfZyKqtPosaaYAmiKmAsB+X294/S7Udll9mbrIpe02M=; b=hmZmO272ZRshRNeKpv2i+PZq14uO3q710ThyELXjUP0dtVdPH1eKjNxLk4f6ocXZLAVgdSVGO zfJBxhecQ86BzBD20OOafwTF8PBgzhmORW6dZlxq81ll1kyMZQiNl4x X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Add helper functions to kvm_util.h to support calling ioctls, specifically KVM_SET_MEMORY_ATTRIBUTES2, on a guest_memfd file descriptor. Introduce gmem_ioctl() and __gmem_ioctl() macros, modeled after the existing vm_ioctl() helpers, to provide a standard way to call ioctls on a guest_memfd. Add gmem_set_memory_attributes() and its derivatives (gmem_set_private(), gmem_set_shared()) to set memory attributes on a guest_memfd region. Also provide "__" variants that return the ioctl error code instead of aborting the test. These helpers will be used by upcoming guest_memfd tests. To avoid code duplication, factor out the check for supported memory attributes into a new macro, TEST_ASSERT_SUPPORTED_ATTRIBUTES, and use it in both the existing vm_set_memory_attributes() and the new gmem_set_memory_attributes() helpers. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/kvm_util.h | 94 ++++++++++++++++++++++= +--- 1 file changed, 86 insertions(+), 8 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index a091c495442a1..575032e9791d2 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -392,6 +392,16 @@ static __always_inline void static_assert_is_vcpu(stru= ct kvm_vcpu *vcpu) { } __TEST_ASSERT_VM_VCPU_IOCTL(!ret, #cmd, ret, (vcpu)->vm); \ }) =20 +#define __gmem_ioctl(gmem_fd, cmd, arg) \ + kvm_do_ioctl(gmem_fd, cmd, arg) + +#define gmem_ioctl(gmem_fd, cmd, arg) \ +({ \ + int ret =3D __gmem_ioctl(gmem_fd, cmd, arg); \ + \ + TEST_ASSERT(!ret, __KVM_IOCTL_ERROR(#cmd, ret)); \ +}) + /* * Looks up and returns the value corresponding to the capability * (KVM_CAP_*) given by cap. @@ -418,8 +428,16 @@ static inline void vm_enable_cap(struct kvm_vm *vm, u3= 2 cap, u64 arg0) vm_ioctl(vm, KVM_ENABLE_CAP, &enable_cap); } =20 +/* + * KVM_SET_MEMORY_ATTRIBUTES{,2} overwrites _all_ attributes. These + * flows need significant enhancements to support multiple attributes. + */ +#define TEST_ASSERT_SUPPORTED_ATTRIBUTES(attributes) \ + TEST_ASSERT(!(attributes) || (attributes) =3D=3D KVM_MEMORY_ATTRIBUTE_PRI= VATE, \ + "Update me to support multiple attributes!") + static inline void vm_set_memory_attributes(struct kvm_vm *vm, gpa_t gpa, - u64 size, u64 attributes) + size_t size, u64 attributes) { struct kvm_memory_attributes attr =3D { .attributes =3D attributes, @@ -428,17 +446,11 @@ static inline void vm_set_memory_attributes(struct kv= m_vm *vm, gpa_t gpa, .flags =3D 0, }; =20 - /* - * KVM_SET_MEMORY_ATTRIBUTES overwrites _all_ attributes. These flows - * need significant enhancements to support multiple attributes. - */ - TEST_ASSERT(!attributes || attributes =3D=3D KVM_MEMORY_ATTRIBUTE_PRIVATE, - "Update me to support multiple attributes!"); + TEST_ASSERT_SUPPORTED_ATTRIBUTES(attributes); =20 vm_ioctl(vm, KVM_SET_MEMORY_ATTRIBUTES, &attr); } =20 - static inline void vm_mem_set_private(struct kvm_vm *vm, gpa_t gpa, u64 size) { @@ -451,6 +463,72 @@ static inline void vm_mem_set_shared(struct kvm_vm *vm= , gpa_t gpa, vm_set_memory_attributes(vm, gpa, size, 0); } =20 +static inline int __gmem_set_memory_attributes(int fd, u64 offset, + size_t size, u64 attributes, + u64 *error_offset) +{ + struct kvm_memory_attributes2 attr =3D { + .attributes =3D attributes, + .offset =3D offset, + .size =3D size, + .flags =3D 0, + .error_offset =3D 0, + }; + int r; + + r =3D __gmem_ioctl(fd, KVM_SET_MEMORY_ATTRIBUTES2, &attr); + + /* Copy error_offset regardless of r so caller can check. */ + if (error_offset) + *error_offset =3D attr.error_offset; + + return r; +} + +static inline int __gmem_set_private(int fd, u64 offset, size_t size, + u64 *error_offset) +{ + return __gmem_set_memory_attributes(fd, offset, size, + KVM_MEMORY_ATTRIBUTE_PRIVATE, + error_offset); +} + +static inline int __gmem_set_shared(int fd, u64 offset, size_t size, + u64 *error_offset) +{ + return __gmem_set_memory_attributes(fd, offset, size, 0, + error_offset); +} + +static inline void gmem_set_memory_attributes(int fd, u64 offset, + size_t size, u64 attributes) +{ + struct kvm_memory_attributes2 attr =3D { + .attributes =3D attributes, + .offset =3D offset, + .size =3D size, + .flags =3D 0, + }; + + TEST_ASSERT_SUPPORTED_ATTRIBUTES(attributes); + + __TEST_REQUIRE(kvm_check_cap(KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES) > 0, + "No valid attributes for guest_memfd ioctl!"); + + gmem_ioctl(fd, KVM_SET_MEMORY_ATTRIBUTES2, &attr); +} + +static inline void gmem_set_private(int fd, u64 offset, size_t size) +{ + gmem_set_memory_attributes(fd, offset, size, + KVM_MEMORY_ATTRIBUTE_PRIVATE); +} + +static inline void gmem_set_shared(int fd, u64 offset, size_t size) +{ + gmem_set_memory_attributes(fd, offset, size, 0); +} + void vm_guest_mem_fallocate(struct kvm_vm *vm, gpa_t gpa, u64 size, bool punch_hole); =20 --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C94BD33FE33; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; cv=none; b=Nj35UmgT6VdoknCCZkQxWW5BY6f1IPlxHOq+X5tKZj0OIt0NXUQmTeIMFh35AfwNhF/p1PIRGHpRYYEzzSO4hmEvVcpPn+Gk+E2V9E8RmhQXAyOeZNw4K7SRj0gATl+rigmAV1+Osh8VrT5xbo8FvDzSotgnXTo2oFZ3zJbQB0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135922; c=relaxed/simple; bh=+6vgxUOfb7LgpoJyJR4WM6IO1ok7zPbKRnYFI46ufEc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lApKUAEJ17yoDvdiCN1/bksJrqZQjdUfU1Q10P4oAVlmRsyTMPkdtVnj3LJjwNefzaknNZFyNKY37B8LqXNQotE7LMt5Kh0xYLHrcBvWLlp+mV3o9ryxgx/2uwbEBVrZ6ffe+lQ4M+1sDWNe+/ZLZRYZ3C3Egjefwwry3fUXhOk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ReHkkk0L; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ReHkkk0L" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9E412C4AF14; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=+6vgxUOfb7LgpoJyJR4WM6IO1ok7zPbKRnYFI46ufEc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=ReHkkk0LsW+IRJHyxkir11ki90prIqp5j6b6rQ7m4kjhqRUdEzeJVhDkd2BKfq6W6 xOdEhCJhCQb0h5+0VHyHnOv1vfNcb3QhihGMV7UWoT59hK0PhHgvWaFM7VpBsedBah zenOq470Mrt6TvtIlJ5Y4Ibf2bFuTc+pQBUuYfTlse4rci+IS0fxoTfPPJCYoRBDu3 Kw5b+J1kFYTiIHZYMq3IwWp3WK/oByzO0Bna8nB5pOVrrNXvhFY0zVASmNy4EtLrZE 5AhrbOr2Mcg47JEkPt3wnmSNwRGidGhQ7V0JWBHL5f7pg+HMuw4s8MV/iJzkkdrgR7 oRqaa5hT31QPw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86A1CC61DE4; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:31 -0700 Subject: [PATCH v12 30/45] KVM: selftests: Test basic single-page conversion flow Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-30-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=7821; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=pi+KgU8vksVssztsxAyhslBtRnND4GPyLTIyP95FxlQ=; b=I2oo2mz+877YVRqR8yLs5FnL4bNFZkVNVq67qV7zIBp3I8hHZxD6VylOOQGXQ41pMcbZv4mOM 9iXcE7pf4KJDuLJ9xiQU3gIneYTLVj7gq5HRHNVcXknrZt5fle9insV X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add a selftest for the guest_memfd memory attribute conversion ioctls. The test starts the guest_memfd as all-private (the default state), and verifies the basic flow of converting a single page to shared and then back to private. Add infrastructure that supports extensions to other conversion flow tests. This infrastructure will be used in upcoming patches for other conversion tests. Add test as an x86-specific test since guest_memfd's testing vehicle (KVM_X86_SW_PROTECTED_VM) is x86-specific. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- v10: Updated __GMEM_CONVERSION_TEST macro to avoid potential macro argument reuse --- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../kvm/x86/guest_memfd_conversions_test.c | 200 +++++++++++++++++= ++++ 2 files changed, 201 insertions(+) diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selft= ests/kvm/Makefile.kvm index 96bab7002d39e..239bc61ea384f 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -155,6 +155,7 @@ TEST_GEN_PROGS_x86 +=3D x86/max_vcpuid_cap_test TEST_GEN_PROGS_x86 +=3D x86/triple_fault_event_test TEST_GEN_PROGS_x86 +=3D x86/recalc_apic_map_test TEST_GEN_PROGS_x86 +=3D x86/aperfmperf_test +TEST_GEN_PROGS_x86 +=3D x86/guest_memfd_conversions_test TEST_GEN_PROGS_x86 +=3D access_tracking_perf_test TEST_GEN_PROGS_x86 +=3D coalesced_io_test TEST_GEN_PROGS_x86 +=3D dirty_log_perf_test diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c new file mode 100644 index 0000000000000..77f95d49349ce --- /dev/null +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -0,0 +1,200 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2024, Google LLC. + */ +#include +#include + +#include +#include +#include + +#include "kvm_util.h" +#include "kselftest_harness.h" +#include "test_util.h" +#include "ucall_common.h" + +FIXTURE(gmem_conversions) { + struct kvm_vcpu *vcpu; + int gmem_fd; + /* HVA of the first byte of the memory mmap()-ed from gmem_fd. */ + char *mem; +}; + +typedef FIXTURE_DATA(gmem_conversions) test_data_t; + +FIXTURE_SETUP(gmem_conversions) { } + +static size_t page_size; + +static void guest_do_rmw(void); +#define GUEST_MEMFD_SHARING_TEST_GVA 0x90000000ULL + +/* + * Defer setup until the individual test is invoked so that tests can spec= ify + * the number of pages and flags for the guest_memfd instance. + */ +static void gmem_conversions_do_setup(test_data_t *t, int nr_pages, + int gmem_flags) +{ + const struct vm_shape shape =3D { + .mode =3D VM_MODE_DEFAULT, + .type =3D KVM_X86_SW_PROTECTED_VM, + }; + /* + * Use high GPA above APIC_DEFAULT_PHYS_BASE to avoid clashing with + * APIC_DEFAULT_PHYS_BASE. + */ + const gpa_t gpa =3D SZ_4G; + const u32 slot =3D 1; + struct kvm_vm *vm; + + vm =3D __vm_create_shape_with_one_vcpu(shape, &t->vcpu, nr_pages, guest_d= o_rmw); + + vm_mem_add(vm, VM_MEM_SRC_SHMEM, gpa, slot, nr_pages, + KVM_MEM_GUEST_MEMFD, -1, 0, gmem_flags); + + t->gmem_fd =3D kvm_slot_to_fd(vm, slot); + t->mem =3D addr_gpa2hva(vm, gpa); + virt_map(vm, GUEST_MEMFD_SHARING_TEST_GVA, gpa, nr_pages); +} + +static void gmem_conversions_do_teardown(test_data_t *t) +{ + /* No need to close gmem_fd, it's owned by the VM structure. */ + kvm_vm_free(t->vcpu->vm); +} + +FIXTURE_TEARDOWN(gmem_conversions) +{ + gmem_conversions_do_teardown(self); +} + +/* + * In these test definition macros, __nr_pages and nr_pages is used to set= up + * the total number of pages in the guest_memfd under test. This will be + * available in the test definitions as nr_pages. + */ + +#define __GMEM_CONVERSION_TEST(test, __nr_pages, flags) \ +static void __gmem_conversions_##test(test_data_t *t, int nr_pages); \ + \ +TEST_F(gmem_conversions, test) \ +{ \ + const int nr =3D (__nr_pages); \ + \ + gmem_conversions_do_setup(self, nr, flags); \ + __gmem_conversions_##test(self, nr); \ +} \ +static void __gmem_conversions_##test(test_data_t *t, int nr_pages) \ + +#define GMEM_CONVERSION_TEST(test, __nr_pages, flags) \ + __GMEM_CONVERSION_TEST(test, __nr_pages, (flags) | GUEST_MEMFD_FLAG_MMAP) + +#define __GMEM_CONVERSION_TEST_INIT_PRIVATE(test, __nr_pages) \ + GMEM_CONVERSION_TEST(test, __nr_pages, 0) + +#define GMEM_CONVERSION_TEST_INIT_PRIVATE(test) \ + __GMEM_CONVERSION_TEST_INIT_PRIVATE(test, 1) + +struct guest_check_data { + void *mem; + char expected_val; + char write_val; +}; +static struct guest_check_data guest_data; + +static void guest_do_rmw(void) +{ + for (;;) { + char *mem =3D READ_ONCE(guest_data.mem); + + GUEST_ASSERT_EQ(READ_ONCE(*mem), READ_ONCE(guest_data.expected_val)); + WRITE_ONCE(*mem, READ_ONCE(guest_data.write_val)); + + GUEST_SYNC(0); + } +} + +static void run_guest_do_rmw(struct kvm_vcpu *vcpu, u64 pgoff, + char expected_val, char write_val) +{ + struct ucall uc; + int r; + + guest_data.mem =3D (void *)GUEST_MEMFD_SHARING_TEST_GVA + pgoff * page_si= ze; + guest_data.expected_val =3D expected_val; + guest_data.write_val =3D write_val; + sync_global_to_guest(vcpu->vm, guest_data); + + do { + r =3D __vcpu_run(vcpu); + } while (r =3D=3D -1 && errno =3D=3D EINTR); + + TEST_ASSERT_EQ(r, 0); + + switch (get_ucall(vcpu, &uc)) { + case UCALL_ABORT: + REPORT_GUEST_ASSERT(uc); + case UCALL_SYNC: + break; + default: + TEST_FAIL("Unexpected ucall %lu", uc.cmd); + } +} + +static void host_do_rmw(char *mem, u64 pgoff, char expected_val, + char write_val) +{ + TEST_ASSERT_EQ(READ_ONCE(mem[pgoff * page_size]), expected_val); + WRITE_ONCE(mem[pgoff * page_size], write_val); +} + +static void test_private(test_data_t *t, u64 pgoff, char starting_val, + char write_val) +{ + TEST_EXPECT_SIGBUS(WRITE_ONCE(t->mem[pgoff * page_size], write_val)); + run_guest_do_rmw(t->vcpu, pgoff, starting_val, write_val); + TEST_EXPECT_SIGBUS(READ_ONCE(t->mem[pgoff * page_size])); +} + +static void test_convert_to_private(test_data_t *t, u64 pgoff, + char starting_val, char write_val) +{ + gmem_set_private(t->gmem_fd, pgoff * page_size, page_size); + test_private(t, pgoff, starting_val, write_val); +} + +static void test_shared(test_data_t *t, u64 pgoff, char starting_val, + char host_write_val, char write_val) +{ + host_do_rmw(t->mem, pgoff, starting_val, host_write_val); + run_guest_do_rmw(t->vcpu, pgoff, host_write_val, write_val); + TEST_ASSERT_EQ(READ_ONCE(t->mem[pgoff * page_size]), write_val); +} + +static void test_convert_to_shared(test_data_t *t, u64 pgoff, + char starting_val, char host_write_val, + char write_val) +{ + gmem_set_shared(t->gmem_fd, pgoff * page_size, page_size); + test_shared(t, pgoff, starting_val, host_write_val, write_val); +} + +GMEM_CONVERSION_TEST_INIT_PRIVATE(init_private) +{ + test_private(t, 0, 0, 'A'); + test_convert_to_shared(t, 0, 'A', 'B', 'C'); + test_convert_to_private(t, 0, 'C', 'E'); +} + +int main(int argc, char *argv[]) +{ + TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); + TEST_REQUIRE(kvm_check_cap(KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES) & + KVM_MEMORY_ATTRIBUTE_PRIVATE); + + page_size =3D getpagesize(); + + return test_harness_run(argc, argv); +} --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F013534389F; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=Xga5l8/vhYlPDQT8QzBFLcjBztkxXGYQrE+i46i7P9ClWyPAUyzI3o1AWXpmjuYwSrfOHCfecn7OidqADkKidkxQcMuWPFOz4qKU2xwDiwT77Qi4TUth+fdkHkj9E8BZu2gLkr1wXXuICJsusSGT4qvbePWXV8e4FWAcnnflP38= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=aioaJipFuIL/IiIdVQFffSKoyIBWlO7RRAfxqr7t6Iw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=IX+bNAoyLisO8KstUi18fik2igNU4XLjF70VZZY2jHOjym8XX4LuviNSsteKs/gjfeznZFTFV/T9aOp2XxPiuR24v1fyBH9wjR5lFa7fE+YUKOl437pVl7lEiBjegbAGFKkJuFQcg/5xxL1XS7c3TmF/6c18pg21YsKlir0l8Gc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bwCr2oa7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bwCr2oa7" Received: by smtp.kernel.org (Postfix) with ESMTPS id C3C18C4AF62; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=aioaJipFuIL/IiIdVQFffSKoyIBWlO7RRAfxqr7t6Iw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=bwCr2oa7MjlpgIMpV5IMn9nVmbWDuUJe7pUJAIw28LiCU8XTplP9coJMkzRy/HYLo CXcANtrVxycZ1OndPF3O1tgaoMPmf35RbkSYRIkmUsXbZ0aG2Y05pMxQLkpAvb4qLL YUBYsTraIXqKXm+MFYsAxYwUZWalf8QihktRBhcfOOezNC478boEf/7Wq7cXR46pFz glCQq8LjvVNB3LPOpay7qLZeAqSavDaus9VfvomlFK6TqcKWecUUr7egUzX3soHeON Te3G9X7umUDRyuwE0zLCQUiscYX8GnS3dCu9SRys8mVENhd6KHIkt0PoS2suL3KzkC vhHTBvBRXoQ0A== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FBAEC61DFD; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:32 -0700 Subject: [PATCH v12 31/45] KVM: selftests: Test conversion flow when INIT_SHARED Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-31-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=1820; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=xYw2zR0sil2DMulPFsHrm+Fn54xO5rySrIonHJxlHdY=; b=biTv8DxdIPMmqKcaPDRSer+ZFsb/kdS9KwcKlME5qJy2Ary24lqI1vc+9urmZBiMg6+4afZzv nQji3gwfMF3DpmlvEhLSIM3IyNefcJ6PMRkzuv4bmBVy8WRbZwgmZPS X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add a test case to verify that conversions between private and shared memory work correctly when the memory is initially created as shared. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- .../selftests/kvm/x86/guest_memfd_conversions_test.c | 13 +++++++++= ++++ 1 file changed, 13 insertions(+) diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c index 77f95d49349ce..4c20d3d8e9349 100644 --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -97,6 +97,12 @@ static void __gmem_conversions_##test(test_data_t *t, in= t nr_pages) \ #define GMEM_CONVERSION_TEST_INIT_PRIVATE(test) \ __GMEM_CONVERSION_TEST_INIT_PRIVATE(test, 1) =20 +#define __GMEM_CONVERSION_TEST_INIT_SHARED(test, __nr_pages) \ + GMEM_CONVERSION_TEST(test, __nr_pages, GUEST_MEMFD_FLAG_INIT_SHARED) + +#define GMEM_CONVERSION_TEST_INIT_SHARED(test) \ + __GMEM_CONVERSION_TEST_INIT_SHARED(test, 1) + struct guest_check_data { void *mem; char expected_val; @@ -188,6 +194,13 @@ GMEM_CONVERSION_TEST_INIT_PRIVATE(init_private) test_convert_to_private(t, 0, 'C', 'E'); } =20 +GMEM_CONVERSION_TEST_INIT_SHARED(init_shared) +{ + test_shared(t, 0, 0, 'A', 'B'); + test_convert_to_private(t, 0, 'B', 'C'); + test_convert_to_shared(t, 0, 'C', 'D', 'E'); +} + int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EB0F345CAA; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=cn4W/asJFhdbvaukcQA5seVisS1X7lgzt3A3wL1XSp1cUpGEjE7psAuhj/mFEpsRvSj/LaQUoBtUakvGqQhpcB/Ng2F3ZAaNYHbU1zIhWSsHUbjB15cjInisjuEdxynRfNYJuscqJ9FI49HKKRocMJR1gNFPdHtNcVGxzoXT94g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=we+RUoIj4s6zn8J2yaxtxIVwUnD24bYSVQYPl3eVvFU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QzDP4vHfc8x6fTh4FTL4O3W6F5Lz3Myn28EAB3sgoHza0u5dzKnOB7pe4RvVWL1ez4XyqnNKOjPYctygC9R0Tl/mVW1jOpYnrAtpKRXMNQlEYfZYRr/H3JCfLg4uAdKtnMsf7rsy2PEVNEchh8HKnCsTstm/lWZDxOJfrZE3PrQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tU7qgQve; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tU7qgQve" Received: by smtp.kernel.org (Postfix) with ESMTPS id D5A82C2BCFB; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135922; bh=we+RUoIj4s6zn8J2yaxtxIVwUnD24bYSVQYPl3eVvFU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=tU7qgQve50FJJ2Q+r5lcw2n48KTibdcbTz2w1ad715l1gYNDO7LNZ7oZNImd4nhCV 5d7F5tYXrvcSXCaBCR/Hs185H8TMEOXY93WP9EivXDTZl8CwCBFo2ex8+6HCRXQklA X4QmnXwqUBe2NTC1i95JWGrnA4CKYG3LG6f8X3fCnDcqM1wX6VZbmXvoTrDVCxCI+O wbergiQ+FVnBgV0zLXsdu2L/IUBQXBuO6F7g/HOSG6IpmPEH6FEr4Ya3GLA1Y0qhFp wbhiOEv7pFF+Vvt6PCawKwNid4y5ZEFs7UGExh2vJJXYoYtCd/xxRrdoV73P/ndIOn z6CAfGCyOsypg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B872DC61DFF; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:33 -0700 Subject: [PATCH v12 32/45] KVM: selftests: Test conversion precision in guest_memfd Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-32-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=4642; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=jtZjcg6wGzKgK/2cSXFW0G1Fk1Ysj4xXn5RYOoutiT8=; b=BWwpnaO/ppzNsjzvl6ryVPzJ03QyAOXNIIsUPy/1AXg1xCUTlEBaIbIDj9LIeAA4ek7JCR54N F39XwjjeJTHAq4pMBh/eM4Vin5uAcNuNCNrcmko4haKka/WT/Xm2OKA X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng The existing guest_memfd conversion tests only use single-page memory regions. This provides no coverage for multi-page guest_memfd objects, specifically whether KVM correctly handles the page index for conversion operations. An incorrect implementation could, for example, always operate on the first page regardless of the index provided. Add a new test case to verify that conversions between private and shared memory correctly target the specified page within a multi-page guest_memfd. This test also verifies the precision of memory conversions by converting a single page and then iterating through all other pages to ensure they remain in their original state. To support this test, add a new GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED macro that handles setting up and tearing down the VM for each page iteration. The teardown logic is adjusted to prevent a double-free in this new scenario. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- v10: Update macro GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED to avoid potential macro argument reuse --- .../kvm/x86/guest_memfd_conversions_test.c | 67 ++++++++++++++++++= ++++ 1 file changed, 67 insertions(+) diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c index 4c20d3d8e9349..a8d89df53e46d 100644 --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -61,8 +61,13 @@ static void gmem_conversions_do_setup(test_data_t *t, in= t nr_pages, =20 static void gmem_conversions_do_teardown(test_data_t *t) { + /* Use NULL to avoid second free in FIXTURE_TEARDOWN (multipage tests). */ + if (!t->vcpu) + return; + /* No need to close gmem_fd, it's owned by the VM structure. */ kvm_vm_free(t->vcpu->vm); + t->vcpu =3D NULL; } =20 FIXTURE_TEARDOWN(gmem_conversions) @@ -103,6 +108,30 @@ static void __gmem_conversions_##test(test_data_t *t, = int nr_pages) \ #define GMEM_CONVERSION_TEST_INIT_SHARED(test) \ __GMEM_CONVERSION_TEST_INIT_SHARED(test, 1) =20 +/* + * Repeats test over nr_pages in a guest_memfd of size nr_pages, providing= each + * test iteration with test_page, the index of the page under test in + * guest_memfd. test_page takes values 0..(nr_pages - 1) inclusive. + */ +#define GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED(test, __nr_pages) \ +static void __gmem_conversions_multipage_##test(test_data_t *t, int nr_pag= es, \ + const int test_page); \ + \ +TEST_F(gmem_conversions, test) \ +{ \ + const u64 flags =3D GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_INIT_SHARED;= \ + const int nr =3D (__nr_pages); \ + int i; \ + \ + for (i =3D 0; i < nr; ++i) { \ + gmem_conversions_do_setup(self, nr, flags); \ + __gmem_conversions_multipage_##test(self, nr, i); \ + gmem_conversions_do_teardown(self); \ + } \ +} \ +static void __gmem_conversions_multipage_##test(test_data_t *t, int nr_pag= es, \ + const int test_page) + struct guest_check_data { void *mem; char expected_val; @@ -201,6 +230,44 @@ GMEM_CONVERSION_TEST_INIT_SHARED(init_shared) test_convert_to_shared(t, 0, 'C', 'D', 'E'); } =20 +GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED(indexing, 4) +{ + int i; + + /* Get a char that varies with both i and n. */ +#define combine(x, n) (((x) << 4) + (n)) +#define i_(n) (combine(i, n)) +#define t_(n) (combine(test_page, n)) + + /* + * Start with the highest index, to catch any errors when, perhaps, the + * first page is returned even for the last index. + */ + for (i =3D nr_pages - 1; i >=3D 0; --i) + test_shared(t, i, 0, i_(0), i_(2)); + + test_convert_to_private(t, test_page, t_(2), t_(3)); + + for (i =3D 0; i < nr_pages; ++i) { + if (i =3D=3D test_page) + test_private(t, test_page, t_(3), t_(4)); + else + test_shared(t, i, i_(2), i_(3), i_(4)); + } + + test_convert_to_shared(t, test_page, t_(4), t_(5), t_(6)); + + for (i =3D 0; i < nr_pages; ++i) { + char expected =3D i =3D=3D test_page ? t_(6) : i_(4); + + test_shared(t, i, expected, i_(7), i_(8)); + } + +#undef t_ +#undef i_ +#undef combine +} + int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BA4E345ED0; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=cu3sYPaAgduTPi10lyF7qIPE5wv9biVjdY1xukpuNHCWn5nR6r3uT7B7zKeS9uhHnlwu09GotTchK6SXsudIPNDYfZJbN7qQw8dF7zr5cczXLlQKeyJ4cW1JCd7g8cHls4lOVY6Qw98NinPiDdogA5LgETLl7s6Bsx19WiGzwdk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=Ohg00gp82JlQ1LRKDAbUvlgWiGhAXe+0uLM4Cvj5Moc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=l+KI5j7lWpCVbNnBvWt0yso9F8a3nMdKuJM/gIlfroCPmnT3UKlc0/+6JiqJN/amV5z3ofImzuGEl+1PrHxh4QsZw7onf6Hd7trVneUdPN0IayYvFjVnj7ejALcBZwxME0drFiCzWkK8IN0eullg7aZY/94pkGx30IjQZLGVjdI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IvVf4DTX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IvVf4DTX" Received: by smtp.kernel.org (Postfix) with ESMTPS id F254CC2BCF5; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=Ohg00gp82JlQ1LRKDAbUvlgWiGhAXe+0uLM4Cvj5Moc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=IvVf4DTXCK/sRrZP11cGRQmf093Be3Ab/egCpVqrqnT3+6E2YiQrhMR9W040tD12z Z+9tDaBEABaSlDBG9r48ZFjv8v/11jw7hVOkq5uc4mjao5gu1bII34MZtrhdqsEAaq WG5bvyVLVRDl72ZFUBTzvXOnDamPAUceTx4kdN4AVJXwpdA18IByiOs7Zb09gh/I6y wmTnO/IS+MB7/f6ssZLQs72Zn1byrZFitIv5m1ayisoWr0y4tKVGFBf4BKmrkvX/g/ 98xuyrOXGu+6NOR+lNY6zQAL3QfVUR0pRSESY/nDSsSDLAepqMhP4BDopsgqAV6l0w EWhzbAoRaiENQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D46AAC61DE2; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:34 -0700 Subject: [PATCH v12 33/45] KVM: selftests: Test conversion before allocation Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-33-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=1811; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=a2z3tw2jbxfSjKhbx7xOpPsEYazJcgX/b0bAut8/tlE=; b=1diaXiAQiDZHo8GGbJSRZADZP2BnxiXUNSNdhvxA9SOoFSS7qJN/joh4qkuCEdJSeNSQuMZID NA4XQacd0AoAj0lKcrfLVuAtJ43ml1DuXWeXTDX5t32XEtHB8qpP56e X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add two test cases to the guest_memfd conversions selftest to cover the scenario where a conversion is requested before any memory has been allocated in the guest_memfd region. The KVM_SET_MEMORY_ATTRIBUTES2 ioctl can be called on a memory region at any time. If the guest had not yet faulted in any pages for that region, the kernel must record the conversion request and apply the requested state when the pages are eventually allocated. The new tests cover both conversion directions. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- .../selftests/kvm/x86/guest_memfd_conversions_test.c | 14 ++++++++++= ++++ 1 file changed, 14 insertions(+) diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c index a8d89df53e46d..f46f1530a94b9 100644 --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -268,6 +268,20 @@ GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED(indexing, 4) #undef combine } =20 +/* + * Test that even if there are no folios yet, conversion requests are reco= rded + * in guest_memfd. + */ +GMEM_CONVERSION_TEST_INIT_SHARED(before_allocation_shared) +{ + test_convert_to_private(t, 0, 0, 'A'); +} + +GMEM_CONVERSION_TEST_INIT_PRIVATE(before_allocation_private) +{ + test_convert_to_shared(t, 0, 0, 'A', 'B'); +} + int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53215351C06; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=ScDKrCTWhLBZxS6RdflrjdLlZCo65z7/y6XSD99uNikZPy6gCz+ZjJGcyYPTY19YK5RqExsALzuYWFr0e6jzZGUryn9IsbT4QTzo2B5EE7BV7y4nx5RYMY7kM2TF3xxn2eH3kt3IXKD96crW0J7XHEdG7bdt8bZsvYMB0ihyAqI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=OvAkVPkagVMy5Xn4F+rMZ//usPS9bakNam/vvgs9Y5E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UZZuUwIwaaywRMpll3GcXz5/PGYC7eLYgwQuQ4oby333E7sFsi6GTaiJ7/SIF+rZVcR3eRAzVxPGdr8vAumW9RipzIZi8n/H5Gsonuts7kWPn8Q+eXTtOxuN1EUTQmvov5PwDzk7WQrEVaIKatOJVXigiZWhSVpNpww94rK5Mx4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IicruLaT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IicruLaT" Received: by smtp.kernel.org (Postfix) with ESMTPS id 0FDD8C4AF11; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=OvAkVPkagVMy5Xn4F+rMZ//usPS9bakNam/vvgs9Y5E=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=IicruLaTlmeS+wCZzY/WJRg1JvHKeWI8OmKsHGYsfw9mA4DTlbfJb8mE9o9fF07VZ S0oVpvVhIy9ckEBjapEMf3HuxdmsOUSTgyAdGdrF+o4rE/C8MozNrzZLVwA5NpiuoJ bmrfAo3JQ6E6uIipPiUYdl2KRNB9tvUSKQxpzm8f+FizNIZNrNVzdl6i9nc9GE0PfG 3+umJfL7+ah/R3tTDIdPFsdZOxNwZxCCcfQqT28gPEVeCRcg40N5irIOgiTWfR+khh kROiepQaOz6L/g+7TWbsb2SuTOIuXA8v+gl98baVGSE7V4Pk3J0efrM9KOYurdCkTx jDaXMIPIXx6YA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB532C61DFD; Mon, 31 Aug 2026 00:25:22 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:35 -0700 Subject: [PATCH v12 34/45] KVM: selftests: Convert with allocated folios in different layouts Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-34-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=2317; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=+hxkHJeDQpRciSqIWpnQc1pHp7KNzfkACeixqLSu/Gs=; b=MM8eGbO3rz6hZnoKnOSzc+DYeMRhE+qHY4HlBE7BG1z1ck2Nf9cx3lLleHcoZ6ZRBd4kAO3jW Ew8OekNZZ5uAkR8MQ03Leo2tjTLDL8mcbW1k60Q8Q6aAWTP84uCGjZ0 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add a guest_memfd selftest to verify that memory conversions work correctly with allocated folios in different layouts. By iterating through which pages are initially faulted, the test covers various layouts of contiguous allocated and unallocated regions, exercising conversion with different range layouts. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- .../kvm/x86/guest_memfd_conversions_test.c | 30 ++++++++++++++++++= ++++ 1 file changed, 30 insertions(+) diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c index f46f1530a94b9..acdff2d154046 100644 --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -282,6 +282,36 @@ GMEM_CONVERSION_TEST_INIT_PRIVATE(before_allocation_pr= ivate) test_convert_to_shared(t, 0, 0, 'A', 'B'); } =20 +/* + * Test that when some of the folios in the conversion range are allocated, + * conversion requests are handled correctly in guest_memfd. Vary the ran= ges + * allocated before conversion, using test_page, to cover various layouts = of + * contiguous allocated and unallocated regions. + */ +GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED(unallocated_folios, 8) +{ + const int second_page_to_fault =3D 4; + int i; + + /* + * Fault 2 of the pages to test filemap range operations except when + * test_page =3D=3D second_page_to_fault. + */ + host_do_rmw(t->mem, test_page, 0, 'A'); + if (test_page !=3D second_page_to_fault) + host_do_rmw(t->mem, second_page_to_fault, 0, 'A'); + + gmem_set_private(t->gmem_fd, 0, nr_pages * page_size); + for (i =3D 0; i < nr_pages; ++i) { + char expected =3D (i =3D=3D test_page || i =3D=3D second_page_to_fault) = ? 'A' : 0; + + test_private(t, i, expected, 'B'); + } + + for (i =3D 0; i < nr_pages; ++i) + test_convert_to_shared(t, i, 'B', 'C', 'D'); +} + int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 794A53537F8; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=UX2iF+j1dV1U1o84Vw1fNDC3ReAHJSNGO2x62IMIFCPL3VdjdEnedGY2WEQXt/yRX0DBKxMTJBxxXg9IEaxz2+6VbwwWQU8J2q3Ec/4i3IYCQLt+T3YoXnUSXiVh4mlcUNahBaLLSgcrvNX/ojOEXaD/zIU4OpXsbdhy6i4yF3s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=xJxv/ExQUAEvw7T0RAQG38FUrG9mFWRgWPRG7f5xFX4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OVg8bZ/uKtFHkB7M//+8m0PEQFIActhHjSMrVdZzl0f307lJ5fRJOdKBD0r7mOkfVY3tPD/cf9AYkvqXErxadcRxzxR1+2+6o6faBZeED3VQvzWnd2FyvESO5G6FLOw6SIU7bhc7TgvojEVrsnNcWbF5os/7n+wh+zZxh3bcKlA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LKiRnc4Q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LKiRnc4Q" Received: by smtp.kernel.org (Postfix) with ESMTPS id 32C49C4AF64; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=xJxv/ExQUAEvw7T0RAQG38FUrG9mFWRgWPRG7f5xFX4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=LKiRnc4QKxQ40mY/5ieaFKg89gjw3u53hXQZzCyfxjPIGCU5oxrVSGCRG8U08s1iy UsXe4Vx1/kVtYyvmeIefrLfAilisshQpKP9/EZbKS6Gsk+uaj4qkn6aThd0p5XVbuK V1inqUA4xGFHFXV6u5MsRI+oUi5c81lwc8MisQEBqnohsc6QMeMTCpkO861mDhXLfW RwqY4PFBtuZv95/JfDUov/ws382t9f2h1mjHFLix7Bnqx7M7HuTMs9+Uz3p+52OhVB HIkdPkOsPo+wvKw+fxpLNXfMSxnm+wfsn7BMSQqjlURInckNXFEJs3rxvHQcvXRc6f /hsd9fIjXRo/g== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ECADC61DE4; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:36 -0700 Subject: [PATCH v12 35/45] KVM: selftests: Test that truncation does not change shared/private status Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-35-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=2066; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=tNExLNWLFDn50j0RlKFtcHpdRLOf4T07FWqRSpyOYNY=; b=YUToXWIB44Po+fdLbmu0EdGYzg3DMaTns2uQZw0fL0Iq2D4zUBQHSEZYILSpCQ07agH3oEiZH gxvtPHsPUXzBhBletXy8wcGKYoBB0+IOFC/BYas0faHs9bAX/B1Cgz4 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add a test to verify that deallocating a page in a guest memfd region via fallocate() with FALLOC_FL_PUNCH_HOLE does not alter the shared or private status of the corresponding memory range. When a page backing a guest memfd mapping is deallocated, e.g., by punching a hole or truncating the file, and then subsequently faulted back in, the new page must inherit the correct shared/private status tracked by guest_memfd. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- .../selftests/kvm/x86/guest_memfd_conversions_test.c | 14 ++++++++++= ++++ 1 file changed, 14 insertions(+) diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c index acdff2d154046..b7a54d2a90928 100644 --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -10,6 +10,7 @@ #include =20 #include "kvm_util.h" +#include "kvm_syscalls.h" #include "kselftest_harness.h" #include "test_util.h" #include "ucall_common.h" @@ -312,6 +313,19 @@ GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED(unallocated= _folios, 8) test_convert_to_shared(t, i, 'B', 'C', 'D'); } =20 +/* Truncation should not affect shared/private status. */ +GMEM_CONVERSION_TEST_INIT_SHARED(truncate) +{ + host_do_rmw(t->mem, 0, 0, 'A'); + kvm_fallocate(t->gmem_fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, 0, = page_size); + host_do_rmw(t->mem, 0, 0, 'A'); + + test_convert_to_private(t, 0, 'A', 'B'); + + kvm_fallocate(t->gmem_fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, 0, = page_size); + test_private(t, 0, 0, 'A'); +} + int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8782A353A91; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=pzzhH+nL26A1Bo0sv12lxBN76h5YTCxVqTayqyr//35qaCX8nu9epdht5nBOvuFoEbvdW/SOOfbOjLicXkAW9yZ+qAbAaAy6btTqZi+rsGdaJP4OiJVSmHzM11t9b6D+xqBtmHdcD40oahhVFAqY7zyoTxRoQh8SKKoXewcEdAQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=XlXlwtPHpVsAHVmS0wCTOzfJJ8Ren1/HCkQ1nzWHDlg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=t10uzn+/fn9IoQ6OHTk3STXTS5sWOt/lmDCyTd7WSxCevYsHvsBuP0lNx7lRXFSWjTvnpDygGLm8dXPQecGvZmDyKe1o9oz01fEn3f7vD46xFM3EmkdhCp2OIoxhYEkvIUH4c4nrC6NIlRDZ/Y8XgUAGGzbRtRG7yGmv/b6kNKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=u9IJ5iK/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="u9IJ5iK/" Received: by smtp.kernel.org (Postfix) with ESMTPS id 5A60DC2BD04; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=XlXlwtPHpVsAHVmS0wCTOzfJJ8Ren1/HCkQ1nzWHDlg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=u9IJ5iK/6z+oTOLKVBCiBM1jcEUKxQGSbYxCnOSuLnNIHilNS7dP59fFkbGBOaMGx kfcOimvxRq6yEPDEvRMHKtQrp1c5fcAxn222JmD2i9xkvqnmY1X6BbnT3ZggsAKjv+ WyWPflkh6Lq7ZSu0Z7Iudcuiy6F/kdgTBcnOS+7+ZvJMV7JaHJ3krWZm9h+hbbjWNJ fAaelo77ccCMb8Y/Dojp5oWfk2bYovKXhsV98r0CTVSgci6VwWl2p48KNuQFli4Vas WchKwNuUplpS5NpZ6FBR2wQEPA3H7tu9qtfWtqaiiYD00i0LeO+EoV9wcsrd2DvEr+ h7+ov7sGwC63w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FABDC61DE2; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:37 -0700 Subject: [PATCH v12 36/45] KVM: selftests: Test that shared/private status is consistent across processes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-36-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=5042; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=eSll1fxb3h9LF8Yf3q9CpNkCTF4Ydzl3TlGQL7Ph+7s=; b=a39svXm+jG0d08eS29xYIfcHadhb0Rsn115zbUmWat1bqwzQXYbDtRwIpu+8p+IQlZLDehNZp 958MlnX5Tw/A6TcC3FJLKB5t6/4/NYhZHNn1Bzru3gDtGYVNFpwCtfs X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Add a test to verify that a guest_memfd's shared/private status is consistent across processes, and that any shared pages previously mapped in any process are unmapped from all processes. The test forks a child process after creating the shared guest_memfd region so that the second process exists alongside the main process for the entire test. The processes then take turns to access memory to check that the shared/private status is consistent across processes. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Co-developed-by: Ackerley Tng Signed-off-by: Ackerley Tng --- .../kvm/x86/guest_memfd_conversions_test.c | 118 +++++++++++++++++= ++++ 1 file changed, 118 insertions(+) diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c index b7a54d2a90928..a972a09c5c6e5 100644 --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -2,7 +2,9 @@ /* * Copyright (c) 2024, Google LLC. */ +#include #include +#include #include =20 #include @@ -326,6 +328,122 @@ GMEM_CONVERSION_TEST_INIT_SHARED(truncate) test_private(t, 0, 0, 'A'); } =20 +/* Test that shared/private memory protections work and are seen from any = process. */ +GMEM_CONVERSION_TEST_INIT_SHARED(forked_accesses) +{ + enum test_state { + STATE_INIT, + STATE_CHECK_SHARED, + STATE_DONE_CHECKING_SHARED, + STATE_CHECK_PRIVATE, + STATE_DONE_CHECKING_PRIVATE, + }; + + struct sync_state { + pthread_mutex_t mutex; + pthread_cond_t cond; + enum test_state step; + } *sync; + + pthread_mutexattr_t mattr; + pthread_condattr_t cattr; + pid_t child_pid, parent_pid; + int status; + + sync =3D kvm_mmap(sizeof(*sync), PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_ANONYMOUS, -1); + + pthread_mutexattr_init(&mattr); + pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED); + pthread_mutex_init(&sync->mutex, &mattr); + pthread_mutexattr_destroy(&mattr); + + pthread_condattr_init(&cattr); + pthread_condattr_setpshared(&cattr, PTHREAD_PROCESS_SHARED); + pthread_cond_init(&sync->cond, &cattr); + pthread_condattr_destroy(&cattr); + + sync->step =3D STATE_INIT; + +#define TEST_STATE_AWAIT(__state) \ + do { \ + pthread_mutex_lock(&sync->mutex); \ + while (sync->step !=3D (__state)) { \ + struct timespec ts, stop; \ + int ret; \ + \ + clock_gettime(CLOCK_REALTIME, &ts); \ + stop =3D timespec_add_ns(ts, 100 * 1000000UL); \ + \ + ret =3D pthread_cond_timedwait(&sync->cond, &sync->mutex, &stop); \ + if (ret =3D=3D ETIMEDOUT) { \ + bool alive =3D (child_pid =3D=3D 0) ? \ + (getppid() =3D=3D parent_pid) : \ + (waitpid(child_pid, NULL, WNOHANG) =3D=3D 0); \ + TEST_ASSERT(alive, "Other process exited prematurely"); \ + } else { \ + TEST_ASSERT(!ret, "pthread_cond_timedwait failed"); \ + } \ + } \ + pthread_mutex_unlock(&sync->mutex); \ + } while (0) + +#define TEST_STATE_SET(__state) \ + do { \ + pthread_mutex_lock(&sync->mutex); \ + sync->step =3D (__state); \ + pthread_cond_broadcast(&sync->cond); \ + pthread_mutex_unlock(&sync->mutex); \ + } while (0) + + parent_pid =3D getpid(); + child_pid =3D fork(); + TEST_ASSERT(child_pid !=3D -1, "fork failed"); + + if (child_pid =3D=3D 0) { + const char inconsequential =3D 0xdd; + + TEST_STATE_AWAIT(STATE_CHECK_SHARED); + + /* + * This maps the pages into the child process as well, and tests + * that the conversion process will unmap the guest_memfd memory + * from all processes. + */ + host_do_rmw(t->mem, 0, 0xB, 0xC); + + TEST_STATE_SET(STATE_DONE_CHECKING_SHARED); + TEST_STATE_AWAIT(STATE_CHECK_PRIVATE); + + TEST_EXPECT_SIGBUS(READ_ONCE(t->mem[0])); + TEST_EXPECT_SIGBUS(WRITE_ONCE(t->mem[0], inconsequential)); + + TEST_STATE_SET(STATE_DONE_CHECKING_PRIVATE); + exit(0); + } + + test_shared(t, 0, 0, 0xA, 0xB); + + TEST_STATE_SET(STATE_CHECK_SHARED); + TEST_STATE_AWAIT(STATE_DONE_CHECKING_SHARED); + + test_convert_to_private(t, 0, 0xC, 0xD); + + TEST_STATE_SET(STATE_CHECK_PRIVATE); + TEST_STATE_AWAIT(STATE_DONE_CHECKING_PRIVATE); + + TEST_ASSERT_EQ(waitpid(child_pid, &status, 0), child_pid); + TEST_ASSERT(WIFEXITED(status) && WEXITSTATUS(status) =3D=3D 0, + "Child exited with unexpected status"); + + pthread_mutex_destroy(&sync->mutex); + pthread_cond_destroy(&sync->cond); + kvm_munmap(sync, sizeof(*sync)); + +#undef TEST_STATE_SET +#undef TEST_STATE_AWAIT +} + int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91D3F3546E5; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=tG0FRQYUXyOqZ8Xh4La1ldT3Gi1nfce3E6Sxjb8pCHMQysBgUeXj8sIP8ih0FsxoNupgGyhjVX4FnvsXrJDOAtM5tVshy7shVBMcJtwFuelK8xlHR22ZxuhgDFOICs8F6Y3bUlGe7YLy4wANbCDJeFKjQlSotqOnlN+eyvzTFz0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=L5YUbh7/6Cud6VplfoNsOCErNzDbpKzPkXGdi+gnbdU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=V5ydDSVSnMeRUnkWpRrsH/wZVf7iRvs7xRHQknE6A2H1h2DORFTqM+NZmmnw+Scdik3oTev+fjg3Q0SCYnRUVFcWuXa3HDkg8by4GApd8u26Md7bfr+P4w/hoA4cWIIMpbCiIjTu27jAMgdSlD6TRTpQh57XRmFakiOuPK7H8Tk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bPTQ+t1u; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bPTQ+t1u" Received: by smtp.kernel.org (Postfix) with ESMTPS id 6F903C2BCC7; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=L5YUbh7/6Cud6VplfoNsOCErNzDbpKzPkXGdi+gnbdU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=bPTQ+t1u1KJjMEQTfYicl1vmRbZYLA+VOfQu93BZwsXWbldiwg6PNm30qHe0W10K0 XtCa9z6HDK8A5k1txAJpJL58AlHLyAbmczeQ/IpK9XpD6lIr2kzcHefew4QTc5BA88 iMDUhwUuI2EL9SEMLGlKffIEEJth7aLvjv2DGcPxEIdq8mDIK4UyFpvM6ZpyWqMrys OvbqhqLr7XXtMTr5UFO7JxdaMUp1M+lcM5bMovKgq/lgGcTFj9M0vUGLOiA6ESK0m7 sKIoxphgrKi/QDY1hKD5ebCMyEvYt2uuX6DixiqQZQMIxUZyiKfE6ZyzfJv+zI1FQH JSTusVRdWlHdQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55FB9C624A5; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:38 -0700 Subject: [PATCH v12 37/45] KVM: selftests: Add helpers to pin pages with CONFIG_GUP_TEST Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-37-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=2738; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=kIKJvxj0NxcBPACHPOvZ7iTswFnwraQaYm+ccyn/5BU=; b=rGCmlwIzORjmkoJ/SJOyYoV95Ir/p5lCBR7+rCAJ4+Z6RA7bgx6OE8YI2QhKLVx8Bf7niBqyg E+q8HPCVsK9A9d7RM04XRCGr5I32p1B80Sh8qYZ73BetOSxApsH5i8U X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add helper functions to allow KVM selftests to pin memory using CONFIG_GUP_TEST. This is useful for creating test scenarios where some page has an increased refcount, such as when testing guest_memfd in-place conversion. The helpers open /sys/kernel/debug/gup_test and invoke the PIN_LONGTERM_TEST_START and PIN_LONGTERM_TEST_STOP ioctls. Since this functionality depends on the kernel being built with CONFIG_GUP_TEST, provide stub implementations that trigger a test failure if the configuration is missing. Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/kvm_util.h | 3 +++ tools/testing/selftests/kvm/lib/kvm_util.c | 25 ++++++++++++++++++++++= +++ 2 files changed, 28 insertions(+) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index 575032e9791d2..3cfe243d7c8b0 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -1206,6 +1206,9 @@ static inline int pin_self_to_any_cpu(void) return pin_task_to_any_cpu(pthread_self()); } =20 +void pin_pages(void *vaddr, uint64_t size); +void unpin_pages(void); + void kvm_print_vcpu_pinning_help(void); void kvm_parse_vcpu_pinning(const char *pcpus_string, u32 vcpu_to_pcpu[], int nr_vcpus); diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index cd468b2bb8343..b1456ed76e27e 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -19,6 +19,8 @@ #include #include =20 +#include "../../../../../mm/gup_test.h" + #define KVM_UTIL_MIN_PFN 2 =20 u32 kvm_random_seed; @@ -645,6 +647,29 @@ int __pin_task_to_cpu(pthread_t task, int cpu) return pthread_setaffinity_np(task, sizeof(cpuset), &cpuset); } =20 +static int gup_test_fd =3D -1; + +void pin_pages(void *vaddr, uint64_t size) +{ + const struct pin_longterm_test args =3D { + .addr =3D (uint64_t)vaddr, + .size =3D size, + .flags =3D PIN_LONGTERM_TEST_FLAG_USE_WRITE, + }; + + gup_test_fd =3D __open_path_or_exit("/sys/kernel/debug/gup_test", O_RDWR, + "Is CONFIG_GUP_TEST enabled?"); + + TEST_ASSERT_EQ(ioctl(gup_test_fd, PIN_LONGTERM_TEST_START, &args), 0); +} + +void unpin_pages(void) +{ + TEST_ASSERT_EQ(ioctl(gup_test_fd, PIN_LONGTERM_TEST_STOP), 0); + + kvm_free_fd(gup_test_fd); +} + static u32 parse_pcpu(const char *cpu_str, const cpu_set_t *allowed_mask) { u32 pcpu =3D atoi_non_negative("CPU number", cpu_str); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB03B348C50; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=QBjzNhw0lWbMtLDhHtGbjX5TD12qrthf7z+6krW6kuLfAr1DutgaxDFqC9l5ps9hE7OMwZDY5aLyjH0O98kwXCoO4Ouiy+gJ0rn+yfq9KtOhpAKEaD/eeB7hj7eYPUKAgjin88LZUZ4ocTjsTzbz2hVOIqUjKF3MlC+yVf2WHP0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=O2ZOrSXnl5WNaoxX9Izw0ZuqjNjEJYMBQDu9NefBE4Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Qm7AVl8OEb4vMhOXIdwMYocrJxxbtFI8GYWLIAyR5LZby0ngMBUJaHujXblaCWOnlfL8jePrWRJ+/K9RoiO53Xt7ESyoIKYfaGq6Cf1OzknGsIJWn1+fAcEyHX9Hsr5gMniNTq2Lawr2XDNhHOPoCvNcqs1mp/Ub6sTNuuh3nyA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VMgK7Pod; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VMgK7Pod" Received: by smtp.kernel.org (Postfix) with ESMTPS id 8C311C4AF0E; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=O2ZOrSXnl5WNaoxX9Izw0ZuqjNjEJYMBQDu9NefBE4Y=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=VMgK7PodQ4QUM3aZ/tkqaqvCeR1qILhFNUs90KQSNnouXI2p7Z7VtvfyD11A2V10l 02KvmNS6INKTPvVBKEb6+aN0TWqoqYsLWQ6VixcHsGxR8Kq8u5ySUS4Fk3ImQJ+BL5 N38jzh8BZgOlkeTM3qmzhE3Nszg2NVfTWPfww5ITdf1lLG0++zBSfyBao5wYUk0B0G eRIeKfuPbFl7+cXlxzQT81WZ/z6Us7fAVoPfJCgXZjCzemRWKNvji8h+JryXaDdVVC kBY0gSSsl/zvyLqzTjvW14EtlDSYmR5BN46xH28GKcFbRGyU47Pmx/B/Dzi+YIJPLk oTqkRfqDeIGVQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E0ABC61DFD; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:39 -0700 Subject: [PATCH v12 38/45] KVM: selftests: Test conversion with elevated page refcount Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-38-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=3125; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=bdq0ORBWbWV6e+1Lwjwn03V1TzR8SfWSd/UMLobub+U=; b=NnP1BZbu0N94uPpRBk5CPXpMs+ccxDq0jjhSkfmKYKGqoRwJkG0Eihwb83HjkVPFK2YhDypLb uOdQzcWiUHlA8CmSmJnSj3ENMc7Qo9hZ3K2a+vP+fDkSUwboGAXmkC+ X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Add a selftest to verify that converting a shared guest_memfd page to a private page fails if the page has an elevated reference count. When KVM converts a shared page to a private one, it expects the page to have a reference count equal to the reference counts taken by the filemap. If another kernel subsystem holds a reference to the page, the conversion must be aborted. The test asserts that both bulk and single-page conversion attempts correctly fail with EAGAIN for the pinned page. After the page is unpinned, the test verifies that subsequent conversions succeed. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- .../kvm/x86/guest_memfd_conversions_test.c | 56 ++++++++++++++++++= ++++ 1 file changed, 56 insertions(+) diff --git a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c= b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c index a972a09c5c6e5..e1c84df0f0596 100644 --- a/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/guest_memfd_conversions_test.c @@ -444,6 +444,62 @@ GMEM_CONVERSION_TEST_INIT_SHARED(forked_accesses) #undef TEST_STATE_AWAIT } =20 +static void test_convert_to_private_fails(test_data_t *t, u64 pgoff, + size_t nr_pages, + u64 expected_error_offset) +{ + /* +1 to make it anything but expected_error_offset. */ + u64 error_offset =3D expected_error_offset + 1; + u64 offset =3D pgoff * page_size; + int ret; + + do { + ret =3D __gmem_set_private(t->gmem_fd, offset, + nr_pages * page_size, &error_offset); + } while (ret =3D=3D -1 && errno =3D=3D EINTR); + TEST_ASSERT(ret =3D=3D -1 && errno =3D=3D EAGAIN, + "Wanted EAGAIN on page %lu, got %d (ret =3D %d)", pgoff, + errno, ret); + TEST_ASSERT_EQ(error_offset, expected_error_offset); +} + +GMEM_CONVERSION_MULTIPAGE_TEST_INIT_SHARED(elevated_refcount, 4) +{ + int i; + + pin_pages(t->mem + test_page * page_size, page_size); + + for (i =3D 0; i < nr_pages; i++) + test_shared(t, i, 0, 'A', 'B'); + + /* + * Converting in bulk should fail as long any page in the range has + * unexpected refcounts. + */ + test_convert_to_private_fails(t, 0, nr_pages, test_page * page_size); + + for (i =3D 0; i < nr_pages; i++) { + /* + * Converting page-wise should also fail as long any page in the + * range has unexpected refcounts. + */ + if (i =3D=3D test_page) + test_convert_to_private_fails(t, i, 1, test_page * page_size); + else + test_convert_to_private(t, i, 'B', 'C'); + } + + unpin_pages(); + + gmem_set_private(t->gmem_fd, 0, nr_pages * page_size); + + for (i =3D 0; i < nr_pages; i++) { + char expected =3D i =3D=3D test_page ? 'B' : 'C'; + + test_private(t, i, expected, 'D'); + } +} + int main(int argc, char *argv[]) { TEST_REQUIRE(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(KVM_X86_SW_PROTECTED_V= M)); --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD43028315D; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=f48fHxMNjUPqIUNafKijFikSkph2cbn1hgJSCki5k8m21ti+joGSnBBnbsmH+SJyYA1KoDOs6wufHhkpQfufmbLRIgVZmLd53hmhvFZVLCptJQvTSWEN/zMAWjzizVloq0Xjs+2HhmFGgU+f45E/epW4LIF/nwvhFyRUE9m2PMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=MBsdJBcO4TQvHjj9pDczibnuw9xpxWQnfVZGWTJVKUg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=X6gsvteom7SRq/dr7N9gVZ8mHDhQE2qyT8vFp/LYn0YRvdlQa3lYjocjQ7eNORqC9pmKNbr52dIjpQPNhSn0KH0alLQye5DlPWh//kPXoBuXlg1ZEEAc8Lujz9xQutQVM4O8pG7/bvUwOmbKHBTXsKWZqo9j8I+vfFpKWne38Bk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TbZGfcdq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TbZGfcdq" Received: by smtp.kernel.org (Postfix) with ESMTPS id A56CBC2BD05; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=MBsdJBcO4TQvHjj9pDczibnuw9xpxWQnfVZGWTJVKUg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=TbZGfcdq7kzK5CCsa3AykZxdq6sTTAfupmtjkchE+NV99+G/oXHy/z32qCaBG7yil 9WrCpttCdJesYEqjoYscTa5LPn6hBJ9lJIvghrBPbP0LTL1lSKIhn0dRgaZ5oPHzlm NtHTPWAFdbkEyOSvLgkSxbm1/0JEvdxU82qzQ5QaH5WZpr3D34nFvw6VKaSJ4SV2zg m5V4ti0tiMJM3cLO1P89RafdUv96KHCQ5ZPh3NMtWFGAdGY2VHr1pg6ixUI15GdHoz lklc7TXC7dPcSjRY38X+LQvhpoUXxXjmwsyauWIyZ+Bj9Ogs5qgKvlcG06xLwK6Mbc pNF1/5ioaidzA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84BD6C61DE4; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:40 -0700 Subject: [PATCH v12 39/45] KVM: selftests: Reset shared memory after hole-punching Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-39-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=2579; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=uV08q4RDK3FyngpumGt+e56rGp5WLQvKfEyZx0mVgVY=; b=iT+PS0xDjfvKp4QdlflKy+7dtweWMwhTAxNwE8dsJVeEO94OnCnUEaqmPrG+y6GLz0nP34akA 1N1NalAM+U6DNWnLRM30OqjSc6w6dw8krqKqAhRtjiHKWDOuBdx1it5 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng private_mem_conversions_test used to reset the shared memory that was used for the test to an initial pattern at the end of each test iteration. Then, it would punch out the pages, which would zero memory. Without in-place conversion, the resetting would write shared memory, and hole-punching will zero private memory, hence resetting the test to the state at the beginning of the for loop. With in-place conversion, resetting writes memory as shared, and hole-punching zeroes the same physical memory, hence undoing the reset done before the hole punch. Move the resetting after the hole-punching, and reset the entire PER_CPU_DATA_SIZE instead of just the tested range. With in-place conversion, this zeroes and then resets the same physical memory. Without in-place conversion, the private memory is zeroed, and the shared memory is reset to init_p. This is sufficient since at each test stage, the memory is assumed to start as shared, and private memory is always assumed to start zeroed. Conversion zeroes memory, so the future test stages will work as expected. Fixes: 43f623f350ce1 ("KVM: selftests: Add x86-only selftest for private me= mory conversions") Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/x86/private_mem_conversions_test.c | 9 ++++++-= -- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c= b/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c index f3de2a4ab0551..cf50e9a332c5b 100644 --- a/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c @@ -202,15 +202,18 @@ static void guest_test_explicit_conversion(u64 base_g= pa, bool do_fallocate) guest_sync_shared(gpa, size, p3, p4); memcmp_g(gpa, p4, size); =20 - /* Reset the shared memory back to the initial pattern. */ - memset((void *)gpa, init_p, size); - /* * Free (via PUNCH_HOLE) *all* private memory so that the next * iteration starts from a clean slate, e.g. with respect to * whether or not there are pages/folios in guest_mem. */ guest_map_shared(base_gpa, PER_CPU_DATA_SIZE, true); + + /* + * Hole-punching above zeroed private memory. Reset shared + * memory in preparation for the next GUEST_STAGE. + */ + memset((void *)base_gpa, init_p, PER_CPU_DATA_SIZE); } } =20 --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA8DA357A4A; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; cv=none; b=XIa2kacywd1BUbyEEGm2mLLceDIz+8clZQLkhtnedwFMHpSugjSdX5BQ98Qoenmsj91Gh0cOkl1HGfMhQ+nQLLesMGLwF5yJ3aKV1ztqQq6+uwGEf5rHjz7uGooIS1/prDoIQ817Qht8ifRUIvMt/GmGhgmApKkVyhY2v0bcO50= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135923; c=relaxed/simple; bh=tbu5H941T3WFmTkcFYkjGccJoW2TPAMlO8ekt00YDMs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=R7UqODSwkaK/1lr+3pbiD42CQ0mZQoekKvm+NvhpKbkYWzNx/Q2ftd/Ub6UlvmsEMcepiILl8ISuuG6FqBKH1mzdHNvQXXMboKSmCjiNX0sa8FOeUb6b+j447speO/DaIgy53cHzdXiQ+nzuEHDdVewy8Ze3QSpotH4Kl9Hs+7E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qLdd0WYG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qLdd0WYG" Received: by smtp.kernel.org (Postfix) with ESMTPS id B5432C2BCFF; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=tbu5H941T3WFmTkcFYkjGccJoW2TPAMlO8ekt00YDMs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=qLdd0WYG07wxjmoTmvbspYTXLe89JNFssEMi+/rkuZGYIRdzdKg7RubQxPC5eEDUl xWPTTAwNH4hVWxT3FqNU/Vbec3rXeaie2lyXwjrK+h4G9GHlzAc5jCWlFMnGfzYFP0 xRBQZUgqZqIdMbazPBh5dLiCtF3oFuLW1uPBIX7DgzhRL0FGgCACrjgUBx+J1jrhJU A5dwzkyx6yZuROIt8d5vl2jcvZdQ0M3L3hgfECB+WmB+Iq6k7EUx2kh2men14JXQEu Ejed8kqKvgMVxdfNM3soweAU5SAeUNYmfa5/nhl0TN563MUzz87i6EEZf477JX/bMa c+KgOuJdDlLtQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B894C624C2; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:41 -0700 Subject: [PATCH v12 40/45] KVM: selftests: Provide function to look up guest_memfd details from gpa Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-40-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=3994; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=aODtj3bXiyKjP4uoEZrBjbprOQXjj5kFxo+MyuHmpxU=; b=814LT748FADHWK7t5YnPGMVd2fhYxt4cbFtAGUXwoXEic+bZE3wurSvaMhHeph8EyNcGn/JJt Y4jxXfT+FFPDaK7q3wrjhxRlMZ2eNvl2G3nr6Qflsl9ATYEbd1JM9x1 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Introduce a new helper, kvm_gpa_to_guest_memfd(), to find the guest_memfd-related details of a memory region that contains a given guest physical address (GPA). The function returns the file descriptor for the memfd, the offset into the file that corresponds to the GPA, and the number of bytes remaining in the region from that GPA. kvm_gpa_to_guest_memfd() was factored out from vm_guest_mem_fallocate(); refactor vm_guest_mem_fallocate() to use the new helper. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/kvm_util.h | 3 +++ tools/testing/selftests/kvm/lib/kvm_util.c | 37 ++++++++++++++++------= ---- 2 files changed, 26 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index 3cfe243d7c8b0..795c9eb311071 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -428,6 +428,9 @@ static inline void vm_enable_cap(struct kvm_vm *vm, u32= cap, u64 arg0) vm_ioctl(vm, KVM_ENABLE_CAP, &enable_cap); } =20 +int kvm_gpa_to_guest_memfd(struct kvm_vm *vm, gpa_t gpa, off_t *fd_offset, + size_t *nr_bytes); + /* * KVM_SET_MEMORY_ATTRIBUTES{,2} overwrites _all_ attributes. These * flows need significant enhancements to support multiple attributes. diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index b1456ed76e27e..1188d0a6c0a5f 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1332,27 +1332,20 @@ void vm_guest_mem_fallocate(struct kvm_vm *vm, u64 = base, u64 size, bool punch_hole) { const int mode =3D FALLOC_FL_KEEP_SIZE | (punch_hole ? FALLOC_FL_PUNCH_HO= LE : 0); - struct userspace_mem_region *region; u64 end =3D base + size; - gpa_t gpa, len; off_t fd_offset; - int ret; + int fd, ret; + size_t len; + gpa_t gpa; =20 for (gpa =3D base; gpa < end; gpa +=3D len) { - u64 offset; - - region =3D userspace_mem_region_find(vm, gpa, gpa); - TEST_ASSERT(region && region->region.flags & KVM_MEM_GUEST_MEMFD, - "Private memory region not found for GPA 0x%lx", gpa); + fd =3D kvm_gpa_to_guest_memfd(vm, gpa, &fd_offset, &len); + len =3D min(end - gpa, len); =20 - offset =3D gpa - region->region.guest_phys_addr; - fd_offset =3D region->region.guest_memfd_offset + offset; - len =3D min_t(u64, end - gpa, region->region.memory_size - offset); - - ret =3D fallocate(region->region.guest_memfd, mode, fd_offset, len); + ret =3D fallocate(fd, mode, fd_offset, len); TEST_ASSERT(!ret, "fallocate() failed to %s at %lx (len =3D %lu), fd =3D= %d, mode =3D %x, offset =3D %lx", punch_hole ? "punch hole" : "allocate", gpa, len, - region->region.guest_memfd, mode, fd_offset); + fd, mode, fd_offset); } } =20 @@ -1689,6 +1682,22 @@ void *addr_gpa2alias(struct kvm_vm *vm, gpa_t gpa) return (void *) ((uintptr_t) region->host_alias + offset); } =20 +int kvm_gpa_to_guest_memfd(struct kvm_vm *vm, gpa_t gpa, off_t *fd_offset, + size_t *nr_bytes) +{ + struct userspace_mem_region *region; + gpa_t gpa_offset; + + region =3D userspace_mem_region_find(vm, gpa, gpa); + TEST_ASSERT(region && region->region.flags & KVM_MEM_GUEST_MEMFD, + "guest_memfd memory region not found for GPA 0x%lx", gpa); + + gpa_offset =3D gpa - region->region.guest_phys_addr; + *fd_offset =3D region->region.guest_memfd_offset + gpa_offset; + *nr_bytes =3D region->region.memory_size - gpa_offset; + return region->region.guest_memfd; +} + /* Create an interrupt controller chip for the specified VM. */ void vm_create_irqchip(struct kvm_vm *vm) { --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E8DB35BDA4; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; cv=none; b=g6UmKqOeqeDdEmST6o4JLwyuwoUfMJQE3aD6YBSMe9cSTUJE28NuJfZ9U/Uwb6gggU6G9yXnD6EMuacq31cMzDy8KcgWTkLCTvy+c+s9wYejg+gx169O1bk/uECBWkgQg8yn9bSl2K5CboL3sn8mQpcDwTr5wSoltj717y48Cvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; c=relaxed/simple; bh=cexXrOxEwxsSXxGvJ8HyqJerH22y1Xrx1Bofk7vx/tU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=riyoU5Nvsb5M/suxvbBoyVHp1DCTO+zziCnScbUCWsuPIUFya8dyjDGoGPCPRyUDf33HWbmlD71ivHpwZDB5YCNrbM0eEYzJ6g1Fh/wbpvJjfVyj5GU/QlDLbiTh0cR05rKEpSDSmoPE0wFebGgcFpza/VOvYoGJMo7K9juiEww= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ADeR/KWz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ADeR/KWz" Received: by smtp.kernel.org (Postfix) with ESMTPS id D0996C4AF19; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135923; bh=cexXrOxEwxsSXxGvJ8HyqJerH22y1Xrx1Bofk7vx/tU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=ADeR/KWzFtQYq/A2ltpAZVzNPLuodNbtsTW+K9BdPSJniSvOclkfR8cm1U8C+0pIA HqZzhYfWs4UmI9e6cXuvVUgTjb2LO5QSlqEKYxASDJlsiCGMIhmcj4iYo1/YITSucM K5xoN29Rk4skCnenJW9VG8YBx3cm8z13Ju/dVqI5LSJwcTwIKksTORmKu+2MPC0TzT Whwv8pCY904jnOYn0q0SmjlxQHRkp9QRpDoIeD5Cy+l2XoO6l8zZza7JgukigwVySr MjsX0RrG3D2VQ0rs2b2L8PNyZmejc6fFIIi5TDBLAi+oiGG20nlb4VxWVIfgh3+S4T QrtZlkis+LKaw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4967C61DE2; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:42 -0700 Subject: [PATCH v12 41/45] KVM: selftests: Provide common function to set memory attributes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-41-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=2853; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=oZL5zDgCEKFycuVBPQBHjILKCAfxW/6ACiO1HnvozsQ=; b=1CjiFY3VuzGw0xAenTUSTcXrsv6CaFEldIqZtdOHaRPqeX00Q0helBtVFwlfAJBh8N2vzV6j7 0GQXLuLZAYrAGSUh790JSJenF9H5v9AwVpAW6qLxEJf5KPAHFld+9PJ X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Introduce vm_mem_set_memory_attributes(), which handles setting of memory attributes for a range of guest physical addresses, regardless of whether the attributes should be set via guest_memfd or via the memory attributes at the VM level. Refactor existing vm_mem_set_{shared,private} functions to use the new function. Opportunistically update the size parameter to use size_t instead of u64. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Co-developed-by: Ackerley Tng Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/kvm_util.h | 46 +++++++++++++++++++---= ---- 1 file changed, 34 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index 795c9eb311071..777fa3dbf88d6 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -454,18 +454,6 @@ static inline void vm_set_memory_attributes(struct kvm= _vm *vm, gpa_t gpa, vm_ioctl(vm, KVM_SET_MEMORY_ATTRIBUTES, &attr); } =20 -static inline void vm_mem_set_private(struct kvm_vm *vm, gpa_t gpa, - u64 size) -{ - vm_set_memory_attributes(vm, gpa, size, KVM_MEMORY_ATTRIBUTE_PRIVATE); -} - -static inline void vm_mem_set_shared(struct kvm_vm *vm, gpa_t gpa, - u64 size) -{ - vm_set_memory_attributes(vm, gpa, size, 0); -} - static inline int __gmem_set_memory_attributes(int fd, u64 offset, size_t size, u64 attributes, u64 *error_offset) @@ -532,6 +520,40 @@ static inline void gmem_set_shared(int fd, u64 offset,= size_t size) gmem_set_memory_attributes(fd, offset, size, 0); } =20 +static inline void vm_mem_set_memory_attributes(struct kvm_vm *vm, gpa_t g= pa, + size_t size, u64 attrs) +{ + if (kvm_has_gmem_attributes) { + gpa_t end =3D gpa + size; + off_t fd_offset; + gpa_t addr; + size_t len; + int fd; + + for (addr =3D gpa; addr < end; addr +=3D len) { + fd =3D kvm_gpa_to_guest_memfd(vm, addr, &fd_offset, &len); + len =3D min(end - addr, len); + + gmem_set_memory_attributes(fd, fd_offset, len, attrs); + } + } else { + vm_set_memory_attributes(vm, gpa, size, attrs); + } +} + +static inline void vm_mem_set_private(struct kvm_vm *vm, gpa_t gpa, + size_t size) +{ + vm_mem_set_memory_attributes(vm, gpa, size, + KVM_MEMORY_ATTRIBUTE_PRIVATE); +} + +static inline void vm_mem_set_shared(struct kvm_vm *vm, gpa_t gpa, + size_t size) +{ + vm_mem_set_memory_attributes(vm, gpa, size, 0); +} + void vm_guest_mem_fallocate(struct kvm_vm *vm, gpa_t gpa, u64 size, bool punch_hole); =20 --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20FCC35BDDB; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; cv=none; b=Y/F+GpgFO6huNGIsGQ1vqTGeBZR2GZbUwoF6udv/D4z22P9gdLyAgqnjJ7bkYhNzhFRGl03u0o3dm7wJhvFIqPyx0qUnHnxcIxfgYqEhsAnYp40jqBydQCI75UyGEdkJyZF51NIJg1fhO7wkfY0IMel52YuYJr+jnK4HFFXyQ4I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; c=relaxed/simple; bh=fWo2Z0U3qWprNJYqj9czwlh+dvNvCpwaJIaom6CG1/8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OaXUHCTPG5G2ShGH1sPPwibk3GdB+dTI8SCEA2X+j8Uk7OunKYjlfZ30XOp0p1idvoUuQCzQ0BXpC/DQj+dt8VIhqWUmpi7RWw68ZQ0gLfx+rMkVIpuD6EXlwot7fG8+ZHVGlA+alMuHxOiucEXgT1oNGFf6Kmh1YVkAgakGc7w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ik78keNE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ik78keNE" Received: by smtp.kernel.org (Postfix) with ESMTPS id F3795C4AF1C; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135924; bh=fWo2Z0U3qWprNJYqj9czwlh+dvNvCpwaJIaom6CG1/8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Ik78keNEnmqSHqq/TYhu54FvsEboBX79IL4E9g54NCkfXby7T880vv9Dt8sX28jDp 5vvTTJsOjyhiBGE+K1ekAfgybTAJ8suCjOGiKijObGtT1sX4Q2QO8z/gvhLL6EJibc 9L0HqB8EA+BHmwJd2eoK0RMwgN2Gbv0EFt5FZYDEtdNKK02Ss6kY22w9AW4z1OfnoU mtkY6nSvVFiql+7iXS/j5vmzoei/Bv96CSci145QpYn9EaM/uQXexyfCGju4ADXdiv iRP3+70GHroXX7JEF7dxd1V9nYDfYmepDnGmF4osDusI+fdWAF6DuxkdhG47gWpmXX W9bcEjtW3kQvw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D013BC624A5; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:43 -0700 Subject: [PATCH v12 42/45] KVM: selftests: Make TEST_EXPECT_SIGBUS thread-safe Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-42-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=5394; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=NFeQPx2A8iYZ1avesMkwS1pPuZuvzIa7AvT5Kppv2Bw=; b=vyn6pMBj4ik/HeRaYFWmV98z3MzA/IB0spOi+dmmSl9FUVZmmtZ3HtkBeVSFPeOZLT8jdN3SK ZOreYmwcuulCyIkkNEipvynQArx6xuFm4TSUU8DdP3VviWYcDyV0JEA X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng The TEST_EXPECT_SIGBUS macro is not thread-safe as it uses a global sigjmp_buf and installs a global SIGBUS signal handler. If multiple threads execute the macro concurrently, they will race on installing the signal handler and stomp on other threads' jump buffers, leading to incorrect test behavior. Make TEST_EXPECT_SIGBUS thread-safe with the following changes: Share the KVM tests' global signal handler. sigaction() applies to all threads; without sharing a global signal handler, one thread may have removed the signal handler that another thread added, hence leading to unexpected signals. The alternative of layering signal handlers was considered, but calling sigaction() within TEST_EXPECT_SIGBUS() necessarily creates a race. To avoid adding new setup and teardown routines to do sigaction() and keep usage of TEST_EXPECT_SIGBUS() simple, share the KVM tests' global signal handler. Opportunistically rename report_unexpected_signal to catchall_signal_handler. To continue to only expect SIGBUS within specific regions of code, use a thread-specific variable, expecting_sigbus, to replace installing and removing signal handlers. Make the execution environment for the thread, sigjmp_buf, a thread-specific variable. As part of TEST_EXPECT_SIGBUS(), assert the prerequisite for this setup, that the current signal handler is the catchall_signal_handler. Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/include/test_util.h | 32 +++++++++++++--------= ---- tools/testing/selftests/kvm/lib/kvm_util.c | 18 ++++++++++---- tools/testing/selftests/kvm/lib/test_util.c | 7 ------ 3 files changed, 30 insertions(+), 27 deletions(-) diff --git a/tools/testing/selftests/kvm/include/test_util.h b/tools/testin= g/selftests/kvm/include/test_util.h index d5825211f2727..75e98ec14107f 100644 --- a/tools/testing/selftests/kvm/include/test_util.h +++ b/tools/testing/selftests/kvm/include/test_util.h @@ -83,21 +83,23 @@ do { \ __builtin_unreachable(); \ } while (0) =20 -extern sigjmp_buf expect_sigbus_jmpbuf; -void expect_sigbus_handler(int signum); - -#define TEST_EXPECT_SIGBUS(action) \ -do { \ - struct sigaction sa_old, sa_new =3D { \ - .sa_handler =3D expect_sigbus_handler, \ - }; \ - \ - sigaction(SIGBUS, &sa_new, &sa_old); \ - if (sigsetjmp(expect_sigbus_jmpbuf, 1) =3D=3D 0) { \ - action; \ - TEST_FAIL("'%s' should have triggered SIGBUS", #action); \ - } \ - sigaction(SIGBUS, &sa_old, NULL); \ +extern __thread sigjmp_buf expect_sigbus_jmpbuf; +extern __thread volatile sig_atomic_t expecting_sigbus; +void catchall_signal_handler(int signum); + +#define TEST_EXPECT_SIGBUS(action) \ +do { \ + struct sigaction __sa =3D {}; \ + \ + TEST_ASSERT_EQ(sigaction(SIGBUS, NULL, &__sa), 0); \ + TEST_ASSERT_EQ(__sa.sa_handler, &catchall_signal_handler); \ + \ + expecting_sigbus =3D true; \ + if (sigsetjmp(expect_sigbus_jmpbuf, 1) =3D=3D 0) { \ + action; \ + TEST_FAIL("'%s' should have triggered SIGBUS", #action);\ + } \ + expecting_sigbus =3D false; \ } while (0) =20 size_t parse_size(const char *size); diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index 1188d0a6c0a5f..cac5624e99774 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -2316,13 +2316,20 @@ __weak void kvm_selftest_arch_init(void) { } =20 -static void report_unexpected_signal(int signum) +__thread sigjmp_buf expect_sigbus_jmpbuf; +__thread volatile sig_atomic_t expecting_sigbus; + +void catchall_signal_handler(int signum) { + switch (signum) { + case SIGBUS: { + if (expecting_sigbus) + siglongjmp(expect_sigbus_jmpbuf, 1); + + TEST_FAIL("Unexpected SIGBUS (%d)\n", signum); + } #define KVM_CASE_SIGNUM(sig) \ case sig: TEST_FAIL("Unexpected " #sig " (%d)\n", signum) - - switch (signum) { - KVM_CASE_SIGNUM(SIGBUS); KVM_CASE_SIGNUM(SIGSEGV); KVM_CASE_SIGNUM(SIGILL); KVM_CASE_SIGNUM(SIGFPE); @@ -2334,12 +2341,13 @@ static void report_unexpected_signal(int signum) void __attribute((constructor)) kvm_selftest_init(void) { struct sigaction sig_sa =3D { - .sa_handler =3D report_unexpected_signal, + .sa_handler =3D catchall_signal_handler, }; =20 /* Tell stdout not to buffer its content. */ setbuf(stdout, NULL); =20 + expecting_sigbus =3D false; sigaction(SIGBUS, &sig_sa, NULL); sigaction(SIGSEGV, &sig_sa, NULL); sigaction(SIGILL, &sig_sa, NULL); diff --git a/tools/testing/selftests/kvm/lib/test_util.c b/tools/testing/se= lftests/kvm/lib/test_util.c index 4dff72f6bd346..89d6d1b572959 100644 --- a/tools/testing/selftests/kvm/lib/test_util.c +++ b/tools/testing/selftests/kvm/lib/test_util.c @@ -19,13 +19,6 @@ #include "test_util.h" #include "kvm_syscalls.h" =20 -sigjmp_buf expect_sigbus_jmpbuf; - -void __attribute__((used)) expect_sigbus_handler(int signum) -{ - siglongjmp(expect_sigbus_jmpbuf, 1); -} - /* * Random number generator that is usable from guest code. This is the * Park-Miller LCG using standard constants. --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A94035C6A3; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; cv=none; b=rdE2LuX7WtBMdISMP+zDR7oVTvdGaBv5c51TQhNk2lq34eVM0j+r+wgsV0yC0/3iZBA2KKRKoRJiEtGYuhHXVkYlI1izTT1Lm4HbUaPlAk6D+gxMSso1FWW01w45p1aniKEox1lBD7HhE7KSU2QZ9fTiJbYbWi+CHcrDPeWkYrs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; c=relaxed/simple; bh=EqWswed0nqsU3A77F7jDzGh2B7dnfbU8JSq2ruxvWuw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YpudwLANymAFvRHj8rVqG6zkelPqD9jW72YAXLCeVrl9sycyauAViAo9KpQYBjOLbclsmdm32l31IR3tiV+NEQVcDteW1Ak8rZWPCWID1cHko3yzbnus6EsOUR/VIX6hP20bfvhWy8T4DiQW0Wy1opcaYB7Z/wuKx+m/4Uiqxr8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LklT/8W4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LklT/8W4" Received: by smtp.kernel.org (Postfix) with ESMTPS id 13FA5C32781; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135924; bh=EqWswed0nqsU3A77F7jDzGh2B7dnfbU8JSq2ruxvWuw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=LklT/8W4BGSEFQwQTMNOCM3QYeAHhZIlYKxDRjT1AhUogmb+ISrsWVpdzhYHqF9R/ JZdXEOb9ngsSnvTE57UJD4kurmIWFMQ8UaUg4Qi8untiWU2PCQ4aFn73MtUirEcVQc tzGHhA4ZXr3nzKDtB3MYEkQMCrX7uyp0DeGsQ72/7G5diykYW40T1DE1BaArvkpvfx AfHvxIMRsnCvJlkQ1ZcFJIVACuFUo00qVI/eNOYH0B9wi4x5HCDcykDq56gVYaME3q Ai+hUCIv4JAFh0Fg3wHBJWmbhlwiykIVeFXF78IBaiaaVy+2zQ+DFfhoPCSob3wBVu qSDH0FctXaJCg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E92C5C61DFD; Mon, 31 Aug 2026 00:25:23 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:44 -0700 Subject: [PATCH v12 43/45] KVM: selftests: Set up page size and alignment independently for guest_memfd Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-43-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=5603; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=ue816NztnupM7oJGebeqg3yOQdtD8BkHUYR0RLajx5I=; b=vOOiAnrvKW8tbEziRbwXAwLwCE2Sy8pStAetzWwVfHvGSPc7qrMG14CZg0vFAv9jCJqpGdIAK 11hU6b2VsHpDc3yFixThGKKACuQ1aJl2isX8QdGkIRHSHupBkqI1pAV X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Currently, vm_mem_add derives the backing source page size, alignment padding, and mmap size from the backing source type upfront before checking if guest_memfd is being mmapped. With shared memory also mmap()-ed from guest_memfd, the alignment of the mmap-ed address needs to respect guest_memfd's backing page size. Refactor the backing store setup to configure the backing source page size, alignment, mmap flags, and mmap offset directly for guest_memfd when it is mmapped, ignoring the backing source type. Skip hugepage validation and anonymous memory madvise calls when mmapping from guest_memfd, since those are not applicable when mmapping guest_memfd. Signed-off-by: Ackerley Tng --- tools/testing/selftests/kvm/lib/kvm_util.c | 74 ++++++++++++++++++--------= ---- 1 file changed, 45 insertions(+), 29 deletions(-) diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index cac5624e99774..3f559f207408f 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1034,11 +1034,12 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_back= ing_src_type src_type, { int ret; struct userspace_mem_region *region; - size_t backing_src_pagesz =3D get_backing_src_pagesz(src_type); - int mmap_flags =3D vm_mem_backing_src_alias(src_type)->flag; size_t mem_size =3D npages * vm->page_size; - off_t mmap_offset =3D 0; - size_t alignment =3D 1; + size_t backing_src_pagesz; + off_t mmap_offset; + bool is_gmem_mmap; + size_t alignment; + int mmap_flags; =20 TEST_REQUIRE_SET_USER_MEMORY_REGION2(); =20 @@ -1090,19 +1091,31 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_back= ing_src_type src_type, /* Allocate and initialize new mem region structure. */ region =3D calloc(1, sizeof(*region)); TEST_ASSERT(region !=3D NULL, "Insufficient Memory"); - region->mmap_size =3D mem_size; =20 - /* - * When using THP mmap is not guaranteed to returned a hugepage aligned - * address so we have to pad the mmap. Padding is not needed for HugeTLB - * because mmap will always return an address aligned to the HugeTLB - * page size. - */ - if (src_type =3D=3D VM_MEM_SRC_ANONYMOUS_THP) - alignment =3D max(backing_src_pagesz, alignment); + is_gmem_mmap =3D (flags & KVM_MEM_GUEST_MEMFD) && + (gmem_flags & GUEST_MEMFD_FLAG_MMAP); + + if (is_gmem_mmap) { + backing_src_pagesz =3D getpagesize(); + alignment =3D 1; + mmap_flags =3D MAP_SHARED; + mmap_offset =3D gmem_offset; + } else { + backing_src_pagesz =3D get_backing_src_pagesz(src_type); + /* + * When using THP mmap is not guaranteed to returned a hugepage aligned + * address so we have to pad the mmap. Padding is not needed for HugeTLB + * because mmap will always return an address aligned to the HugeTLB + * page size. + */ + alignment =3D src_type =3D=3D VM_MEM_SRC_ANONYMOUS_THP ? backing_src_pag= esz : 1; + mmap_flags =3D vm_mem_backing_src_alias(src_type)->flag; + mmap_offset =3D 0; + } =20 TEST_ASSERT_EQ(gpa, align_up(gpa, backing_src_pagesz)); =20 + region->mmap_size =3D mem_size; /* Add enough memory to align up if necessary */ if (alignment > 1) region->mmap_size +=3D alignment; @@ -1129,10 +1142,8 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backi= ng_src_type src_type, } =20 region->fd =3D -1; - if (flags & KVM_MEM_GUEST_MEMFD && gmem_flags & GUEST_MEMFD_FLAG_MMAP) { + if (is_gmem_mmap) { region->fd =3D kvm_dup(gmem_fd); - mmap_flags =3D MAP_SHARED; - mmap_offset =3D gmem_offset; } else if (backing_src_is_shared(src_type)) { region->fd =3D kvm_memfd_alloc(region->mmap_size, src_type =3D=3D VM_MEM_SRC_SHARED_HUGETLB); @@ -1141,22 +1152,27 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_back= ing_src_type src_type, region->mmap_start =3D __kvm_mmap(region->mmap_size, PROT_READ | PROT_WRI= TE, mmap_flags, region->fd, mmap_offset); =20 - TEST_ASSERT(!is_backing_src_hugetlb(src_type) || - region->mmap_start =3D=3D align_ptr_up(region->mmap_start, backing_s= rc_pagesz), - "mmap_start %p is not aligned to HugeTLB page size 0x%lx", - region->mmap_start, backing_src_pagesz); - /* Align host address */ region->host_mem =3D align_ptr_up(region->mmap_start, alignment); =20 - /* As needed perform madvise */ - if ((src_type =3D=3D VM_MEM_SRC_ANONYMOUS || - src_type =3D=3D VM_MEM_SRC_ANONYMOUS_THP) && thp_configured()) { - ret =3D madvise(region->host_mem, mem_size, - src_type =3D=3D VM_MEM_SRC_ANONYMOUS ? MADV_NOHUGEPAGE : MADV_HUG= EPAGE); - TEST_ASSERT(ret =3D=3D 0, "madvise failed, addr: %p length: 0x%lx src_ty= pe: %s", - region->host_mem, mem_size, - vm_mem_backing_src_alias(src_type)->name); + if (!is_gmem_mmap) { + TEST_ASSERT(!is_backing_src_hugetlb(src_type) || + region->mmap_start =3D=3D + align_ptr_up(region->mmap_start, backing_src_pagesz), + "mmap_start %p is not aligned to HugeTLB page size 0x%lx", + region->mmap_start, backing_src_pagesz); + + /* As needed perform madvise */ + if ((src_type =3D=3D VM_MEM_SRC_ANONYMOUS || + src_type =3D=3D VM_MEM_SRC_ANONYMOUS_THP) && thp_configured()) { + int advice =3D src_type =3D=3D VM_MEM_SRC_ANONYMOUS ? + MADV_NOHUGEPAGE : MADV_HUGEPAGE; + + ret =3D madvise(region->host_mem, mem_size, advice); + TEST_ASSERT(ret =3D=3D 0, "madvise failed, addr: %p length: 0x%lx src_t= ype: %s", + region->host_mem, mem_size, + vm_mem_backing_src_alias(src_type)->name); + } } =20 region->backing_src_type =3D src_type; --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BFD435E55D; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; cv=none; b=Ndojjmgih9PKWmo6m7dkYNswKeLwvbuujm5YHepIBM/Ku9XnkN7ncwn25zHylFtrCYIiXXhuFihQNnEHDxbfRwN93JURt7wsxHAPAgnMIVcP8jsgb1w5p3bPf4fachT7ZwdBH+Pw7GMtx6Svu/h/yMebt2cWtfj9NqUVOg9D43I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; c=relaxed/simple; bh=L1SvzQQsKRUvOZ0fQVdIdVUEFR5k64uVcH/NuUMnWr4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PSJZGOyWBOsIYeGMe1uYAVAv74uBRV0r1f/944D538IkhTekWU1AeYa7d8jhFULB20ahwxhwbD8jxLzii0rd7wnPjzNexqr7xkp5KgSl7M8Y840mKm+gialKXvYCyMkBN4gM2aTUBO89dm6uSAR1C0bcVczCCoTBDV+b9dTvD5M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=O/5NBW6O; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="O/5NBW6O" Received: by smtp.kernel.org (Postfix) with ESMTPS id 2CB53C2BCF5; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135924; bh=L1SvzQQsKRUvOZ0fQVdIdVUEFR5k64uVcH/NuUMnWr4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=O/5NBW6OI201GlvVp10lsHH14kGYEqgCVacehiFDMnVFB0S/+QISvCCMgW5J7NISs M5dTwBDLWrF2IAOLExZGYZDrRmzvgcbBow1fEAn+tmL3jrUoRBHQ/+153Pg2ZNkim4 /Gd5RX8JS86jbiXhlEBAy2lPMiv94sYO2BPRXFgWNgyQmX6UPSftMJPq0vI4Ri/fy3 d1CH7KR04eOiQ2yIAsOI6GajGrRt4sNTBv9LiBT06ZbFUuUlG9hfkjVk9F36gKi2e1 CiG3O000yZiN5XrtSasyHBxzLNN6V9QShpqA/7vPvFHbUVtrzveSYCmoJO6RipGyvp mCYeEvhB/pOJQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CF04C61DE4; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:45 -0700 Subject: [PATCH v12 44/45] KVM: selftests: Update private_mem_conversions_test for in-place conversions Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-44-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=5680; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=1X1DA4tCAHh53esF+mJakIrdUfA/0k+CNqzh025SQ+I=; b=q2PZoPyAMrhu+u+7JvbSLoJN9cFAttmsbb84heBDBlERga8e9Hybf0q4RUT8pEXtUMNU1LnQn xooi06cBSXdCr8UzLSYDo4mk68lH7DL7w3WrvB1d1g3KSsm0AjYYexF X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng Update private_mem_conversions_test for in-place conversions. In-place conversions support is detected in selftests with kvm_has_gmem_attributes. With in-place conversions, specifying userspace_addr from some other memory provider that is not the guest_memfd associated with the memslot is a user error, since KVM will only use both shared and private memory from the guest_memfd. Hence, when kvm_has_gmem_attributes, only test in-place conversions with single backing, where guest_memfd provides both shared and private memory. For single backing, guest_memfd must be created with GUEST_MEMFD_FLAG_MMAP. Initialize the guest_memfd as shared to align with how memory would default to shared when shared/private state was tracked at the VM level (the test expects this, it was written for non-in-place-conversions). When handling a hypercall to set attributes, use vm_mem_set_memory_attributes() to send the ioctl to the guest_memfd instead of the VM. When testing in-place conversions (single-backing), don't allow the user to configure src_type, since src_type will be ignored. Don't use src_type to determine alignment for rounding up per-cpu test memory size, since guest_memfd's backing page size is always the system PAGE_SIZE. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Ackerley Tng --- .../kvm/x86/private_mem_conversions_test.c | 57 +++++++++++++++++-= ---- 1 file changed, 44 insertions(+), 13 deletions(-) diff --git a/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c= b/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c index cf50e9a332c5b..33b610383693b 100644 --- a/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c +++ b/tools/testing/selftests/kvm/x86/private_mem_conversions_test.c @@ -306,9 +306,12 @@ static void handle_exit_hypercall(struct kvm_vcpu *vcp= u) if (do_fallocate) vm_guest_mem_fallocate(vm, gpa, size, map_shared); =20 - if (set_attributes) - vm_set_memory_attributes(vm, gpa, size, - map_shared ? 0 : KVM_MEMORY_ATTRIBUTE_PRIVATE); + if (set_attributes) { + u64 attrs =3D map_shared ? 0 : KVM_MEMORY_ATTRIBUTE_PRIVATE; + + vm_mem_set_memory_attributes(vm, gpa, size, attrs); + } + run->hypercall.ret =3D 0; } =20 @@ -352,8 +355,21 @@ static void *__test_mem_conversions(void *__vcpu) size_t nr_bytes =3D min_t(size_t, vm->page_size, size - i); u8 *hva =3D addr_gpa2hva(vm, gpa + i); =20 - /* In all cases, the host should observe the shared data. */ - memcmp_h(hva, gpa + i, uc.args[3], nr_bytes); + if (kvm_has_gmem_attributes && + uc.args[0] =3D=3D SYNC_PRIVATE) { + TEST_EXPECT_SIGBUS(READ_ONCE(*hva)); + } else { + /* + * If not testing in-place conversion + * (dual backing), the host should + * always observe shared data, since the + * shared memory is a separate page. + * + * For SYNC_SHARED, test that the host + * can see shared memory. + */ + memcmp_h(hva, gpa + i, uc.args[3], nr_bytes); + } =20 /* For shared, write the new pattern to guest memory. */ if (uc.args[0] =3D=3D SYNC_SHARED) @@ -369,20 +385,29 @@ static void *__test_mem_conversions(void *__vcpu) } } =20 +/* Align each vCPU's chunk of memory naturally to the size of the backing = store. */ +static size_t compute_per_cpu_size(enum vm_mem_backing_src_type src_type) +{ + size_t alignment; + + if (kvm_has_gmem_attributes) + alignment =3D getpagesize(); + else + alignment =3D get_backing_src_pagesz(src_type); + + return align_up(PER_CPU_DATA_SIZE, max_t(size_t, SZ_2M, alignment)); +} + static void test_mem_conversions(enum vm_mem_backing_src_type src_type, u3= 2 nr_vcpus, u32 nr_memslots) { - /* - * Allocate enough memory so that each vCPU's chunk of memory can be - * naturally aligned with respect to the size of the backing store. - */ - const size_t alignment =3D max_t(size_t, SZ_2M, get_backing_src_pagesz(sr= c_type)); - const size_t per_cpu_size =3D align_up(PER_CPU_DATA_SIZE, alignment); + const size_t per_cpu_size =3D compute_per_cpu_size(src_type); const size_t memfd_size =3D per_cpu_size * nr_vcpus; const size_t slot_size =3D memfd_size / nr_memslots; struct kvm_vcpu *vcpus[KVM_MAX_VCPUS]; pthread_t threads[KVM_MAX_VCPUS]; struct kvm_vm *vm; + u64 gmem_flags; int memfd, i; =20 const struct vm_shape shape =3D { @@ -397,12 +422,16 @@ static void test_mem_conversions(enum vm_mem_backing_= src_type src_type, u32 nr_v =20 vm_enable_cap(vm, KVM_CAP_EXIT_HYPERCALL, (1 << KVM_HC_MAP_GPA_RANGE)); =20 - memfd =3D vm_create_guest_memfd(vm, memfd_size, 0); + gmem_flags =3D 0; + if (kvm_has_gmem_attributes) + gmem_flags =3D GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_INIT_SHARED; + + memfd =3D vm_create_guest_memfd(vm, memfd_size, gmem_flags); =20 for (i =3D 0; i < nr_memslots; i++) vm_mem_add(vm, src_type, BASE_DATA_GPA + slot_size * i, BASE_DATA_SLOT + i, slot_size / vm->page_size, - KVM_MEM_GUEST_MEMFD, memfd, slot_size * i, 0); + KVM_MEM_GUEST_MEMFD, memfd, slot_size * i, gmem_flags); =20 for (i =3D 0; i < nr_vcpus; i++) { gpa_t gpa =3D BASE_DATA_GPA + i * per_cpu_size; @@ -462,6 +491,8 @@ int main(int argc, char *argv[]) while ((opt =3D getopt(argc, argv, "hm:s:n:")) !=3D -1) { switch (opt) { case 's': + TEST_ASSERT(!kvm_has_gmem_attributes, + "src_type is only configurable when testing without in-place conve= rsion"); src_type =3D parse_backing_src_type(optarg); break; case 'n': --=20 2.55.0.897.gb25b4bd76c-goog From nobody Sat Sep 26 20:29:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 640CB35E931; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; cv=none; b=RjBkRb2dePlFp9eS9qPnk/DByP4ZoHm2UGSbDDLWwPGEuUXkBIGX6KH07NfU/Gds+7oL9dc3bnxYA87WBghZlonXhtgQbAoRXHJNFmAK+RBjyAUCXrvt5gftm8mT8/XwyD78hpkbpO4/yi3JfZUb1wdEpMtDNKmWb0lq/8CLaOk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788135924; c=relaxed/simple; bh=lUde8iYPK4GJUXtsVUXDi0w9BjDZRsLdMY2F9VQx/eQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=V7N5FRDWPih9oNux2cdhKDP2kqBI1DjlgQ8ZV0PFSNujDZZwpzJS2pFFAaA2ZZrDQNVHJVjvWS/X9MBMRNgCH6sPJmoPmLLvh6DBRvbrk8h3F04RbnVzmjz6jzSPSHGLUvg17IcPHFiH5NoYQ2xFOSs/50l9cmZo+jQsUFF1ycI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QDL1/U5G; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QDL1/U5G" Received: by smtp.kernel.org (Postfix) with ESMTPS id 3E56DC2BCC7; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1788135924; bh=lUde8iYPK4GJUXtsVUXDi0w9BjDZRsLdMY2F9VQx/eQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=QDL1/U5Gi2tI2Qsasc8gRi1lHT6KFbEmiOS2zkGQiMMd9NtAzAMilbTUB3CKXz5zx ttpag6CJtyLtVicvWYq/9LIjc1XNwVhRvwJ5dIyLS8/waaX0jF3F73MpmmyHU6rfbF q8XZb9UvWYE/5bAWG+W/Wp+5HDew4a4hKCfdXKCcdpuqGecuhW+AM0m39q3uFnO0Nd 884bzQI+Q8B6qJLtlZndcQGLmdG3yNSZTfTWyeGeO9ss5Ck9GV/IY4K4XZHUvFpj2y wNMuF1AVUEzTEf0r06Ku3D787meq/w8H4jMg9F8FVYbwDGIB4cVNzMjOXu9q9YkydU DvKIGry0Tt90Q== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23D49C61DE2; Mon, 31 Aug 2026 00:25:24 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Sun, 30 Aug 2026 17:25:46 -0700 Subject: [PATCH v12 45/45] KVM: selftests: Update private memory exits test to work with per-gmem attributes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260830-gmem-inplace-conversion-v12-45-85e5fd25252a@google.com> References: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> In-Reply-To: <20260830-gmem-inplace-conversion-v12-0-85e5fd25252a@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , John Hubbard , Peter Xu , tarunsahu@google.com, Randy Dunlap , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , Fuad Tabba , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1788135916; l=3875; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=mmy51FSAm74sXNx3NluUXGdNtIyrfc0yrVpZYstCmaY=; b=x1tSYHkMiZA15KG15MvuWidKGKIO9ZIkKkpYPRyas8DXVI2Iq/AGOOCZ5f2nE5+/RixYjQCnq p1pUwVfslNYAIHRIX9zYPGrQXJ1fNDZmZcdduy/XQlbcBp8CmqVZs0+ X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Sean Christopherson Skip setting memory to private in the private memory exits test when using per-gmem memory attributes, as memory is initialized to private by default for guest_memfd, and using vm_mem_set_private() on a guest_memfd instance requires creating guest_memfd with GUEST_MEMFD_FLAG_MMAP (which is totally doable, but would need to be conditional and is ultimately unnecessary). Expect an emulated MMIO instead of a memory fault exit when attributes are per-gmem, as deleting the memslot effectively drops the private status, i.e. the GPA becomes shared and thus supports emulated MMIO. Skip the "memslot not private" test entirely, as private vs. shared state for x86 software-protected VMs comes from the memory attributes themselves, and so when doing in-place conversions there can never be a disconnect between the expected and actual states. Signed-off-by: Sean Christopherson Reviewed-by: Fuad Tabba Tested-by: Shivank Garg Signed-off-by: Ackerley Tng --- .../selftests/kvm/x86/private_mem_kvm_exits_test.c | 36 ++++++++++++++++++= ---- 1 file changed, 30 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c b= /tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c index e6d16e9d61200..f7228e0f1ac61 100644 --- a/tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c +++ b/tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c @@ -62,8 +62,9 @@ static void test_private_access_memslot_deleted(void) =20 virt_map(vm, EXITS_TEST_GVA, EXITS_TEST_GPA, EXITS_TEST_NPAGES); =20 - /* Request to access page privately */ - vm_mem_set_private(vm, EXITS_TEST_GPA, EXITS_TEST_SIZE); + /* Request to access page privately. */ + if (!kvm_has_gmem_attributes) + vm_mem_set_private(vm, EXITS_TEST_GPA, EXITS_TEST_SIZE); =20 kvm_pthread_create(&vm_thread, NULL, (pthread_fn_t)run_vcpu_get_exit_reason, (void *)vcpu); @@ -73,10 +74,26 @@ static void test_private_access_memslot_deleted(void) kvm_pthread_join(vm_thread, &thread_return); exit_reason =3D (u32)(u64)thread_return; =20 - TEST_ASSERT_EQ(exit_reason, KVM_EXIT_MEMORY_FAULT); - TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, KVM_MEMORY_EXIT_FLAG_PRIVAT= E); - TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, EXITS_TEST_GPA); - TEST_ASSERT_EQ(vcpu->run->memory_fault.size, EXITS_TEST_SIZE); + /* + * If attributes are tracked per-gmem, deleting the memslot that points + * at the gmem instance effectively makes the memory shared, and so the + * read should trigger emulated MMIO. + * + * If attributes are tracked per-VM, deleting the memslot shouldn't + * affect the private attribute, and so KVM should generate a memory + * fault exit (emulated MMIO on private GPAs is disallowed). + */ + if (kvm_has_gmem_attributes) { + TEST_ASSERT_EQ(exit_reason, KVM_EXIT_MMIO); + TEST_ASSERT_EQ(vcpu->run->mmio.phys_addr, EXITS_TEST_GPA); + TEST_ASSERT_EQ(vcpu->run->mmio.len, sizeof(u64)); + TEST_ASSERT_EQ(vcpu->run->mmio.is_write, false); + } else { + TEST_ASSERT_EQ(exit_reason, KVM_EXIT_MEMORY_FAULT); + TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, KVM_MEMORY_EXIT_FLAG_PRIVA= TE); + TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, EXITS_TEST_GPA); + TEST_ASSERT_EQ(vcpu->run->memory_fault.size, EXITS_TEST_SIZE); + } =20 kvm_vm_free(vm); } @@ -87,6 +104,13 @@ static void test_private_access_memslot_not_private(voi= d) struct kvm_vcpu *vcpu; u32 exit_reason; =20 + /* + * Accessing non-private memory as private with a software-protected VM + * isn't possible when doing in-place conversions. + */ + if (kvm_has_gmem_attributes) + return; + vm =3D vm_create_shape_with_one_vcpu(protected_vm_shape, &vcpu, guest_repeatedly_read); =20 --=20 2.55.0.897.gb25b4bd76c-goog