From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7D56EB64DC for ; Wed, 28 Jun 2023 22:44:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231546AbjF1WoC (ORCPT ); Wed, 28 Jun 2023 18:44:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231441AbjF1WnW (ORCPT ); Wed, 28 Jun 2023 18:43:22 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 273491FDC; Wed, 28 Jun 2023 15:43:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992202; x=1719528202; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1Sq/+JnMtm+xsGNEP3FqZg9J60yyeR0JKdJ3AsGMaUY=; b=iWpH/MEHn+fwxEBu/2m43EkK7wksKJAzUuiCWtT0YcyWsVGKg331EEgR A3/m5Vy8TbCRZj8fVvRCYf4dbo3tuxpqCrADifbBu0p4i5zXjxfZrvVQU VWzB0u8+5xCy+gEA65/nazuynGFCw6/lRaKofvot2sWITIR3UX52VdwnH VMJlUsdrf4d9f/bfU0BHLPtrzK/n9iDd+xEVLWzAkxxsSbS2aCEFKQKeD JHEHbnTRySWp3nET8/VpEVh/e5igs2+bwzbRxtxvMR50WyZc/xarj1/Au Xu1bR8fxrD6TTLrubRhbMkpJdDpY6u2l54HIedmM9ZyTsi06i8TJj+/ZE g==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699120" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699120" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830299980" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830299980" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:20 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 01/11] KVM: selftests: Fix test_add_overlapping_private_memory_regions() Date: Wed, 28 Jun 2023 15:43:00 -0700 Message-Id: <4557c452b8ed5a44a22b566db00045991b92fbc6.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata The last test in test_add_overlapping_private_memory_regions() doesn't use overlapping regions resulting in the failure. When the region is overlaps with the existing ones, the error code is EEXIST instead of EINVAL. Pass the overlapping region, and check if the errno is EEXIST. Fixes: bdb645960cb5 ("KVM: selftests: Expand set_memory_region_test to vali= date guest_memfd()") Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - no change Changes v1 -> v2: - no change --- .../selftests/kvm/set_memory_region_test.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/set_memory_region_test.c b/tools/t= esting/selftests/kvm/set_memory_region_test.c index f46841843300..ea7da324c4d6 100644 --- a/tools/testing/selftests/kvm/set_memory_region_test.c +++ b/tools/testing/selftests/kvm/set_memory_region_test.c @@ -432,6 +432,7 @@ static void test_add_overlapping_private_memory_regions= (void) { struct kvm_vm *vm; int memfd; + int r; =20 pr_info("Testing ADD of overlapping KVM_MEM_PRIVATE memory regions\n"); =20 @@ -453,8 +454,19 @@ static void test_add_overlapping_private_memory_region= s(void) vm_set_user_memory_region2(vm, MEM_REGION_SLOT, KVM_MEM_PRIVATE, MEM_REGION_GPA, 0, NULL, -1, 0); =20 - test_invalid_guest_memfd(vm, memfd, MEM_REGION_SIZE, - "Overlapping guest_memfd() bindings should fail"); + r =3D __vm_set_user_memory_region2(vm, MEM_REGION_SLOT, KVM_MEM_PRIVATE, + MEM_REGION_GPA * 2 - MEM_REGION_SIZE, + MEM_REGION_SIZE * 2, + 0, memfd, 0); + TEST_ASSERT(r =3D=3D -1 && errno =3D=3D EEXIST, "%s", + "Overlapping guest_memfd() bindings should fail"); + + r =3D __vm_set_user_memory_region2(vm, MEM_REGION_SLOT, KVM_MEM_PRIVATE, + MEM_REGION_GPA * 2 + MEM_REGION_SIZE, + MEM_REGION_SIZE * 2, + 0, memfd, 0); + TEST_ASSERT(r =3D=3D -1 && errno =3D=3D EEXIST, "%s", + "Overlapping guest_memfd() bindings should fail"); =20 close(memfd); kvm_vm_free(vm); --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B08FEB64DC for ; Wed, 28 Jun 2023 22:43:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231615AbjF1Wnr (ORCPT ); Wed, 28 Jun 2023 18:43:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231443AbjF1WnX (ORCPT ); Wed, 28 Jun 2023 18:43:23 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8B5B213C; Wed, 28 Jun 2023 15:43:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992202; x=1719528202; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QjzjGI4HpC0gb3kGtq3uL/qndabYhUtInoPjaSCgPgg=; b=f82L7Wjff/dnkVUYF6U5Yrsz0DAT1PWwG9TGL5jdtlmrBpipKrZ5HjoC 91HycWsmFRdvPQ+glY7cwo786/mtyfnSBI+8RUPa9AcCY2bYLzz3aZUak +vip3ONcSJLsIHHxB1BouaR5dPNyu73v6CA+waHtto67PI6qhoTag+SaH LC4OhVb5sE75s7FPLMYXQULVtLz7NGy6K2eqEhfnLhb0BlUMQ6T410CM4 VwLucr7ZRtdDOWfNMSzHv4qN8aqcQ4tXXdeEzZTJRkgBFo/EPDMGDaz+b Zji59nCEUkxsGPFJI+YYO+Rsk1TNSV/JR+Z4MUMJmjfutcer3m1B+Cpws w==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699122" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699122" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830299984" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830299984" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:20 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 02/11] KVM: selftests: Fix guest_memfd() Date: Wed, 28 Jun 2023 15:43:01 -0700 Message-Id: <4095debe0b61fd7de1884ef82d671d30277a9251.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata Some test cases should succeed. Check !ret instead of ret. Fixes: 36eedd5b91e3 ("KVM: selftests: Add basic selftest for guest_memfd()") Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - no change Changes v1 -> v2: - no change --- tools/testing/selftests/kvm/guest_memfd_test.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing= /selftests/kvm/guest_memfd_test.c index 3b6532b833b2..f3b99c1e5464 100644 --- a/tools/testing/selftests/kvm/guest_memfd_test.c +++ b/tools/testing/selftests/kvm/guest_memfd_test.c @@ -72,11 +72,11 @@ static void test_fallocate(int fd, size_t page_size, si= ze_t total_size) =20 ret =3D fallocate(fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, total_size, page_size); - TEST_ASSERT(ret, "fallocate(PUNCH_HOLE) at total_size should be fine (no-= op)"); + TEST_ASSERT(!ret, "fallocate(PUNCH_HOLE) at total_size should be fine (no= -op)"); =20 ret =3D fallocate(fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, total_size + page_size, page_size); - TEST_ASSERT(ret, "fallocate(PUNCH_HOLE) after total_size should be fine (= no-op)"); + TEST_ASSERT(!ret, "fallocate(PUNCH_HOLE) after total_size should be fine = (no-op)"); =20 ret =3D fallocate(fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, page_size, page_size - 1); --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C393EB64D7 for ; Wed, 28 Jun 2023 22:44:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232050AbjF1WoH (ORCPT ); Wed, 28 Jun 2023 18:44:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231450AbjF1WnY (ORCPT ); Wed, 28 Jun 2023 18:43:24 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D3C926BB; Wed, 28 Jun 2023 15:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992203; x=1719528203; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CfrGuoqJ0Z2MzAHPaiTO3KDD2FUDhhBeXfYraoPJAuA=; b=S5ly2IyEDPK4kI2v2fIiYyDjnNPuHv1IQBzrVp56XNb/+bTA6lcK+PnD VYXow+arV9TyDktnZ0G5sVSZbqMg2PCAj+rdh24oqgKSrNn2S4Pp2xLAi tu1Hz5GQv0Lfy/pk+oMWCxDaHn2NiFCGApLqwkGpjWxoPwI4wxjvX8SaE 2oZTppdGa52yohOwYDIGKjwjahQBZxdNCdejqgkPfeZE1UvD0Lrqyz4/s J10ssJExMQQenIqdAiQsAqof4NQ//5Tegj28AEHchD1KIobwi6iY7K8dq LnuZOJOsC/JRHomxapuRr6K8Z2GiyU/mCtZD3U9gKvzK7nPigXEzXq4Bk g==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699129" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699129" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830299989" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830299989" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:21 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 03/11] KVM: selftests: x86: typo in private_mem_conversions_test.c Date: Wed, 28 Jun 2023 15:43:02 -0700 Message-Id: <643ea41e42fe03e2233100069f76a023fb2e0685.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata Fix typo in the comment in private_mem_conversions_test.c. Fixes: a0f5f8c91180 ("KVM: selftests: x86: Add selftest for private memory = conversions") Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - Newly added --- .../testing/selftests/kvm/x86_64/private_mem_conversions_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/x86_64/private_mem_conversions_tes= t.c b/tools/testing/selftests/kvm/x86_64/private_mem_conversions_test.c index 9b9dd197a260..de6fdb129c68 100644 --- a/tools/testing/selftests/kvm/x86_64/private_mem_conversions_test.c +++ b/tools/testing/selftests/kvm/x86_64/private_mem_conversions_test.c @@ -227,7 +227,7 @@ static void *__test_mem_conversions(void *__vcpu) /* In all cases, the host should observe the shared data. */ memcmp_h(hva, uc.args[3], size); =20 - /* For shared, write the new patter to guest memory. */ + /* For shared, write the new pattern to guest memory. */ if (uc.args[0] =3D=3D SYNC_SHARED) memset(hva, uc.args[4], size); break; --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96F3CEB64D7 for ; Wed, 28 Jun 2023 22:44:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232096AbjF1WoU (ORCPT ); Wed, 28 Jun 2023 18:44:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229610AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C1892704; Wed, 28 Jun 2023 15:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992203; x=1719528203; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jWafnjK1vMbps3ygDg0zyljZpP2h9t350GzjKyOqVDk=; b=SrqraJa1Cts979EPRTFgyP3P0LU4gYDAOV42S1tzS/yon9Zh+vI+xkDk zwUN2NWk7yDSSM3229462ztaVzT4OJbpyV2XX8sz1b/K2TivByPeqZom2 YMLAz1H6IOGP7u5+5F2qchfoxbTcMAJTxIqOqJEp3hwtVc7+yAjy30RW4 POtzpbPGJyaqHazN/9P9phTXNZXxpLG6xxnhxXEG/b8EZibO8PDfB/H7u BhsM30SBwB85gqjq8qsIq9ULAAEk5+4vNNmlVu76idumZikr6Mk9OX67d TGqATiVVpDjCxX0801VBspBZwpWmlXCzs3NXhKgR7NZZ1KVgaEftXEArt Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699138" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699138" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830299992" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830299992" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:21 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 04/11] KVM: x86: Add is_vm_type_supported callback Date: Wed, 28 Jun 2023 15:43:03 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata For TDX, allow the backend can override the supported vm type. Add KVM_X86_TDX_VM to reserve the bit. Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - no change - didn't bother to rename KVM_X86_PROTECTED_VM to KVM_X86_SW_PROTECTED_VM Changes v1 -> v2 - no change --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/svm/svm.c | 7 +++++++ arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/x86.c | 10 +++++++++- arch/x86/kvm/x86.h | 2 ++ 7 files changed, 27 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index 13bc212cd4bc..c0143906fe6d 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -20,6 +20,7 @@ KVM_X86_OP(hardware_disable) KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 8ae131dc645d..3ca93e75041f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1543,6 +1543,7 @@ struct kvm_x86_ops { bool (*has_emulated_msr)(struct kvm *kvm, u32 index); void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); =20 + bool (*is_vm_type_supported)(unsigned long vm_type); unsigned int vm_size; int (*vm_init)(struct kvm *kvm); void (*vm_destroy)(struct kvm *kvm); diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kv= m.h index 6afbfbb32d56..53d382b3b423 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -561,5 +561,6 @@ struct kvm_pmu_event_filter { =20 #define KVM_X86_DEFAULT_VM 0 #define KVM_X86_PROTECTED_VM 1 +#define KVM_X86_TDX_VM 2 =20 #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index eb308c9994f9..e9ed8729f63b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4756,6 +4756,12 @@ static void svm_vm_destroy(struct kvm *kvm) sev_vm_destroy(kvm); } =20 +static bool svm_is_vm_type_supported(unsigned long type) +{ + /* FIXME: Check if CPU is capable of SEV. */ + return __kvm_is_vm_type_supported(type); +} + static int svm_vm_init(struct kvm *kvm) { if (!pause_filter_count || !pause_filter_thresh) @@ -4784,6 +4790,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata =3D { .vcpu_free =3D svm_vcpu_free, .vcpu_reset =3D svm_vcpu_reset, =20 + .is_vm_type_supported =3D svm_is_vm_type_supported, .vm_size =3D sizeof(struct kvm_svm), .vm_init =3D svm_vm_init, .vm_destroy =3D svm_vm_destroy, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 44fb619803b8..b5394ba8cb9c 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7469,6 +7469,11 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu) return err; } =20 +static bool vmx_is_vm_type_supported(unsigned long type) +{ + return __kvm_is_vm_type_supported(type); +} + #define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible.= See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/h= w-vuln/l1tf.html for details.\n" #define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation d= isabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/d= oc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" =20 @@ -8138,6 +8143,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata =3D { .hardware_disable =3D vmx_hardware_disable, .has_emulated_msr =3D vmx_has_emulated_msr, =20 + .is_vm_type_supported =3D vmx_is_vm_type_supported, .vm_size =3D sizeof(struct kvm_vmx), .vm_init =3D vmx_vm_init, .vm_destroy =3D vmx_vm_destroy, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c9e1c9369be2..b5f865f39a00 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4418,12 +4418,18 @@ static int kvm_ioctl_get_supported_hv_cpuid(struct = kvm_vcpu *vcpu, return 0; } =20 -static bool kvm_is_vm_type_supported(unsigned long type) +bool __kvm_is_vm_type_supported(unsigned long type) { return type =3D=3D KVM_X86_DEFAULT_VM || (type =3D=3D KVM_X86_PROTECTED_VM && IS_ENABLED(CONFIG_KVM_PROTECTED_VM) && tdp_enabled); } +EXPORT_SYMBOL_GPL(__kvm_is_vm_type_supported); + +static bool kvm_is_vm_type_supported(unsigned long type) +{ + return static_call(kvm_x86_is_vm_type_supported)(type); +} =20 int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) { @@ -4618,6 +4624,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, lon= g ext) r =3D BIT(KVM_X86_DEFAULT_VM); if (kvm_is_vm_type_supported(KVM_X86_PROTECTED_VM)) r |=3D BIT(KVM_X86_PROTECTED_VM); + if (kvm_is_vm_type_supported(KVM_X86_TDX_VM)) + r |=3D BIT(KVM_X86_TDX_VM); break; default: break; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index c544602d07a3..7d5aa8f0571a 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -9,6 +9,8 @@ #include "kvm_cache_regs.h" #include "kvm_emulate.h" =20 +bool __kvm_is_vm_type_supported(unsigned long type); + struct kvm_caps { /* control of guest tsc rate supported? */ bool has_tsc_control; --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD661EB64D7 for ; Wed, 28 Jun 2023 22:44:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232329AbjF1Woy (ORCPT ); Wed, 28 Jun 2023 18:44:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231464AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75D7F2708; Wed, 28 Jun 2023 15:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992203; x=1719528203; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZUG6Bm+xxuoCBKu5kXKP2G7VsQHqLtGPxdo6Qzaf3vQ=; b=dbhNSZad+Qxiw8CPdkFAp1wrqnD7uXqx/vacBQO6zrIf1Q1SHdSEicFd T/s4njxUx+OAzDRSRu803pzLcSlgygrQmWP8rBbe0OUjiHFxo04iBhwIv ZTQKrtP6I4ilG0o4sYIGyTwH+7SqDhdlEwKxuYOSEDY0xKVgSE9XdXqyU vqgxXtowC5Gui+TfoNHoynP+crkQY2y+BvfSrTltceLu179zEcqudN0X2 7HmNCyCAxNQuxYxaD3jOuFwjP38H++G8HeF7pVj6jhfIZOrcc6mD9OxUs lzGhs2xAVrPcNzrA+FvLz+/BTeUOg33pp5vsxkXAGJsa69l+YtEm3aL1D A==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699144" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699144" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830299996" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830299996" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:22 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 05/11] KVM: x86/mmu: Pass around full 64-bit error code for the KVM page fault Date: Wed, 28 Jun 2023 15:43:04 -0700 Message-Id: <219ac3c013d093a3b15ed8419536d10e7838c386.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata Because the full 64-bit error code, or more info about the fault, for the KVM page fault will be needed for protected VM, TDX and SEV-SNP, update kvm_mmu_do_page_fault() to accept the 64-bit value so it can pass it to the callbacks. The upper 32 bits of error code are discarded at kvm_mmu_page_fault() by lower_32_bits(). Now it's passed down as full 64 bits. Currently two hardware defined bits, PFERR_GUEST_FINAL_MASK and PFERR_GUEST_PAGE_MASK, and one software defined bit, PFERR_IMPLICIT_ACCESS, is defined. PFERR_IMPLICIT_ACCESS: commit 4f4aa80e3b88 ("KVM: X86: Handle implicit supervisor access with SMAP= ") introduced a software defined bit PFERR_IMPLICIT_ACCESS at bit 48 to indicate implicit access for SMAP with instruction emulator. Concretely emulator_read_std() and emulator_write_std() set the bit. permission_fault() checks the bit as smap implicit access. The vendor page fault handler shouldn't pass the bit to kvm_mmu_page_fault(). PFERR_GUEST_FINAL_MASK and PFERR_GUEST_PAGE_MASK: commit 147277540bbc ("kvm: svm: Add support for additional SVM NPF error co= des") introduced them to optimize the nested page fault handling. Other code path doesn't use the bits. Those two bits can be safely passed down without functionality change. The accesses of fault->error_code are as follows - FNAME(page_fault): PFERR_IMPLICIT_ACCESS shouldn't be passed down. PFERR_GUEST_FINAL_MASK and PFERR_GUEST_PAGE_MASK aren't used. - kvm_mmu_page_fault(): explicit mask with PFERR_RSVD_MASK, and PFERR_NESTED_GUEST_PAGE is used outside of the masking upper 32 bits. - mmutrace: change u32 -> u64 - pgprintk(): change %x -> %llx No functional change is intended. This is a preparation to pass on more info with page fault error code. Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - Make depends on a patch to clear PFERR_IMPLICIT_ACCESS - drop clearing the upper 32 bit, instead just pass whole 64 bits - update commit message to mention about PFERR_IMPLICIT_ACCESS and PFERR_NESTED_GUEST_PAGE Changes v1 -> v2: - no change --- arch/x86/kvm/mmu/mmu.c | 5 ++--- arch/x86/kvm/mmu/mmu_internal.h | 4 ++-- arch/x86/kvm/mmu/mmutrace.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 4 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index dc2b9a2f717c..b8ba7f11c3cb 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4510,7 +4510,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault static int nonpaging_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { - pgprintk("%s: gva %lx error %x\n", __func__, fault->addr, fault->error_co= de); + pgprintk("%s: gva %llx error %llx\n", __func__, fault->addr, fault->error= _code); =20 /* This path builds a PAE pagetable, we can map 2mb pages at maximum. */ fault->max_level =3D PG_LEVEL_2M; @@ -5820,8 +5820,7 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu= , gpa_t cr2_or_gpa, u64 err } =20 if (r =3D=3D RET_PF_INVALID) { - r =3D kvm_mmu_do_page_fault(vcpu, cr2_or_gpa, - lower_32_bits(error_code), false, + r =3D kvm_mmu_do_page_fault(vcpu, cr2_or_gpa, error_code, false, &emulation_type); if (KVM_BUG_ON(r =3D=3D RET_PF_INVALID, vcpu->kvm)) return -EIO; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index f1786698ae00..7f9ec1e5b136 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -191,7 +191,7 @@ static inline bool is_nx_huge_page_enabled(struct kvm *= kvm) struct kvm_page_fault { /* arguments to kvm_mmu_do_page_fault. */ const gpa_t addr; - const u32 error_code; + const u64 error_code; const bool prefetch; =20 /* Derived from error_code. */ @@ -283,7 +283,7 @@ enum { }; =20 static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_o= r_gpa, - u32 err, bool prefetch, int *emulation_type) + u64 err, bool prefetch, int *emulation_type) { struct kvm_page_fault fault =3D { .addr =3D cr2_or_gpa, diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index 2d7555381955..2e77883c92f6 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -261,7 +261,7 @@ TRACE_EVENT( TP_STRUCT__entry( __field(int, vcpu_id) __field(gpa_t, cr2_or_gpa) - __field(u32, error_code) + __field(u64, error_code) __field(u64 *, sptep) __field(u64, old_spte) __field(u64, new_spte) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 0662e0278e70..42d48b1ec7b3 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -758,7 +758,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault struct guest_walker walker; int r; =20 - pgprintk("%s: addr %lx err %x\n", __func__, fault->addr, fault->error_cod= e); + pgprintk("%s: addr %llx err %llx\n", __func__, fault->addr, fault->error_= code); WARN_ON_ONCE(fault->is_tdp); =20 /* --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 091FCEB64D7 for ; Wed, 28 Jun 2023 22:44:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231766AbjF1Wos (ORCPT ); Wed, 28 Jun 2023 18:44:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231453AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A79D2713; Wed, 28 Jun 2023 15:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992204; x=1719528204; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JlGf3qio/GSOic4b+mRRSLRlEPSFXzbwOCeBFvvFIr0=; b=nuixus/BNp34WxSRvhuRqOmzf30X47bSRy2gedoEjC6GjCI1y7N21sQh 5qPXD1MXi327uJZjVP28QIjduSTFxbHCO+13NjAiXsmJRFZ54JUsgMAAG 988nLjcC8+M3DkqzuNMD+mUTVp6yVsJUV0EUeW4bQqYmwQdoXdx412Yud mwFaXEHrfJYchHv7BRAfDclWU5cf8OMFTxZnIpbVfNILIdB6eCWFkcQrZ /fegwMOy9YXklCHMIrpsmd8XgooPs3Hso8PHxYB6LEszg6D9xj39zUbv5 Dlnf6dtCzKhA6yPeYuEnc1i95KqGEgF+5Ja7f6TpZpFkcauRo7ldxyBF7 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699150" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699150" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830299999" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830299999" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:22 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 06/11] KVM: x86: Introduce PFERR_GUEST_ENC_MASK to indicate fault is private Date: Wed, 28 Jun 2023 15:43:05 -0700 Message-Id: <253b10a7bfcb7f9ff7911bc5a4b9971aebf0c4b9.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata Add two PFERR codes to designate that the page fault is private and that it requires looking up memory attributes. The vendor kvm page fault handler should set PFERR_GUEST_ENC_MASK bit based on their fault information. It may or may not use the hardware value directly or parse the hardware value to set the bit. For KVM_X86_PROTECTED_VM, ask memory attributes for the fault privateness. Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - Revive PFERR_GUEST_ENC_MASK - rename struct kvm_page_fault::is_private =3D> private - Add check KVM_X86_PROTECTED_VM Changes v1 -> v2: - Introduced fault type and replaced is_private with fault_type. - Add kvm_get_fault_type() to encapsulate the difference. --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu/mmu.c | 14 +++++++++----- arch/x86/kvm/mmu/mmu_internal.h | 16 ++++++++++++++-- 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 3ca93e75041f..831bfd1e719a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -255,6 +255,7 @@ enum x86_intercept_stage; #define PFERR_SGX_BIT 15 #define PFERR_GUEST_FINAL_BIT 32 #define PFERR_GUEST_PAGE_BIT 33 +#define PFERR_GUEST_ENC_BIT 34 #define PFERR_IMPLICIT_ACCESS_BIT 48 =20 #define PFERR_PRESENT_MASK BIT(PFERR_PRESENT_BIT) @@ -266,6 +267,7 @@ enum x86_intercept_stage; #define PFERR_SGX_MASK BIT(PFERR_SGX_BIT) #define PFERR_GUEST_FINAL_MASK BIT_ULL(PFERR_GUEST_FINAL_BIT) #define PFERR_GUEST_PAGE_MASK BIT_ULL(PFERR_GUEST_PAGE_BIT) +#define PFERR_GUEST_ENC_MASK BIT_ULL(PFERR_GUEST_ENC_BIT) #define PFERR_IMPLICIT_ACCESS BIT_ULL(PFERR_IMPLICIT_ACCESS_BIT) =20 #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK | \ diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b8ba7f11c3cb..464c70b35383 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3228,7 +3228,7 @@ void kvm_mmu_hugepage_adjust(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault */ fault->req_level =3D __kvm_mmu_max_mapping_level(vcpu->kvm, slot, fault->gfn, fault->max_level, - fault->is_private); + fault->private); if (fault->req_level =3D=3D PG_LEVEL_4K || fault->huge_page_disallowed) return; =20 @@ -4328,7 +4328,7 @@ static int kvm_do_memory_fault_exit(struct kvm_vcpu *= vcpu, struct kvm_page_fault *fault) { vcpu->run->exit_reason =3D KVM_EXIT_MEMORY_FAULT; - if (fault->is_private) + if (fault->private) vcpu->run->memory.flags =3D KVM_MEMORY_EXIT_FLAG_PRIVATE; else vcpu->run->memory.flags =3D 0; @@ -4386,10 +4386,14 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu,= struct kvm_page_fault *fault return RET_PF_EMULATE; } =20 - if (fault->is_private !=3D kvm_mem_is_private(vcpu->kvm, fault->gfn)) - return kvm_do_memory_fault_exit(vcpu, fault); + if (fault->private !=3D kvm_mem_is_private(vcpu->kvm, fault->gfn)) { + if (vcpu->kvm->arch.vm_type =3D=3D KVM_X86_PROTECTED_VM) + return RET_PF_RETRY; + else + return kvm_do_memory_fault_exit(vcpu, fault); + } =20 - if (fault->is_private) + if (fault->private) return kvm_faultin_pfn_private(vcpu, fault); =20 async =3D false; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 7f9ec1e5b136..a6e45b39ca90 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -200,10 +200,10 @@ struct kvm_page_fault { const bool present; const bool rsvd; const bool user; + const bool private; =20 /* Derived from mmu and global state. */ const bool is_tdp; - const bool is_private; const bool nx_huge_page_workaround_enabled; =20 /* @@ -282,6 +282,18 @@ enum { RET_PF_SPURIOUS, }; =20 +static inline bool kvm_is_fault_private(struct kvm *kvm, gpa_t gpa, u64 er= ror_code) +{ + /* + * This is racy with mmu_seq. If we hit a race, it would result in a + * spurious KVM_EXIT_MEMORY_FAULT. + */ + if (kvm->arch.vm_type =3D=3D KVM_X86_PROTECTED_VM) + return kvm_mem_is_private(kvm, gpa_to_gfn(gpa)); + + return error_code & PFERR_GUEST_ENC_MASK; +} + static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_o= r_gpa, u64 err, bool prefetch, int *emulation_type) { @@ -293,6 +305,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa, .present =3D err & PFERR_PRESENT_MASK, .rsvd =3D err & PFERR_RSVD_MASK, .user =3D err & PFERR_USER_MASK, + .private =3D kvm_is_fault_private(vcpu->kvm, cr2_or_gpa, err), .prefetch =3D prefetch, .is_tdp =3D likely(vcpu->arch.mmu->page_fault =3D=3D kvm_tdp_page_fault), .nx_huge_page_workaround_enabled =3D @@ -301,7 +314,6 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa, .max_level =3D KVM_MAX_HUGEPAGE_LEVEL, .req_level =3D PG_LEVEL_4K, .goal_level =3D PG_LEVEL_4K, - .is_private =3D kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT), }; int r; =20 --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89EB3EB64D7 for ; Wed, 28 Jun 2023 22:45:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232445AbjF1WpO (ORCPT ); Wed, 28 Jun 2023 18:45:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231494AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A1F426B9; Wed, 28 Jun 2023 15:43:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992205; x=1719528205; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0Hrv6P8Eo/E7Tn5rAa++Nq6hcFfaCBIgIk5SxiRD7OA=; b=eRfiLNKBUfUwsZblap5yqmhbZed3m0AsjOA7g3GqIfdht4yRyxlMOfPH q1GFhi+EcWeaqmhlfTC0x8kSooUyMe1wC/9/5RNWrUiP9wq3AhZpSlTCO x7NIbcIde3NT6FaRj+LGa5BjoOuiw5n0WosEzjnw9E2M/OaY1aBm8OPfz LVEB+uCDBz6rUsruOn3Grdt+jK+waICl9VptTRLBTA8OpqbXOxjjcJY8P VV/PQuP4H8Vg+3jAvo5SttHhp3yCBJ/Z9AyN62wx9bH/97IU8C4zdKesN TTpwvPsRL19h4GBBrTKxe0quwOsGL4zZbg1tb5Ac1S7UH+iMi4ETAaro6 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699159" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699159" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830300005" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830300005" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:23 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao , Brijesh Singh , Ashish Kalra Subject: [RFC PATCH v3 07/11] KVM: x86: Export the kvm_zap_gfn_range() for the SNP use Date: Wed, 28 Jun 2023 15:43:06 -0700 Message-Id: <93778ff0e10657491f0a2906a251f8f68b774a15.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Brijesh Singh While resolving the RMP page fault, there may be cases where the page level between the RMP entry and TDP does not match and the 2M RMP entry must be split into 4K RMP entries. Or a 2M TDP page need to be broken into multiple of 4K pages. To keep the RMP and TDP page level in sync, zap the gfn range after splitting the pages in the RMP entry. The zap should force the TDP to gets rebuilt with the new page level. Signed-off-by: Brijesh Singh Signed-off-by: Ashish Kalra Signed-off-by: Michael Roth Link: https://lore.kernel.org/r/20230612042559.375660-39-michael.roth@amd.c= om --- Changes v2 -> v3: - Newly added --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 2 -- arch/x86/kvm/mmu/mmu.c | 1 + 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 831bfd1e719a..bdf507797c73 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1842,6 +1842,8 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm, void kvm_mmu_zap_all(struct kvm *kvm); void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen); void kvm_mmu_change_mmu_pages(struct kvm *kvm, unsigned long kvm_nr_mmu_pa= ges); +void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end); + =20 int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3); =20 diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 92d5a1924fc1..963c734642f6 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -235,8 +235,6 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu= , struct kvm_mmu *mmu, return -(u32)fault & errcode; } =20 -void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end); - int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu); =20 int kvm_mmu_post_init_vm(struct kvm *kvm); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 464c70b35383..5a80ec49bdcd 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -6727,6 +6727,7 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *= kvm, =20 return need_tlb_flush; } +EXPORT_SYMBOL_GPL(kvm_zap_gfn_range); =20 static void kvm_rmap_zap_collapsible_sptes(struct kvm *kvm, const struct kvm_memory_slot *slot) --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4629EB64D7 for ; Wed, 28 Jun 2023 22:45:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231450AbjF1Wo6 (ORCPT ); Wed, 28 Jun 2023 18:44:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231493AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1B3F2718; Wed, 28 Jun 2023 15:43:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992205; x=1719528205; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6umfemwMrYz7WsXi1NlQUufCvU391Sg+THAWamuc+xA=; b=iApbV70icuAbdT/5dIrjojSKpNPuTyJJ4R2J4ZJR2jDJdWjiR53zQhI1 lTT3XZqi4AERoQmOB8DTTDZR0T2+JwkBrGaQZmFqr0p62yNt7fnnCqBuJ cRkDkAduMul4IdKXFE0GO8zLDKKQD1sUwZROeJHwdxra7tq1jFtzS66Xx 6iKQhbpI2Q55XhMUSpv/figvEtRgUUUoezCJ3KfllsfZedk/7j/8nP+t4 X/7durnmWT3F2RijVbnSgAPwwOLF0CX6VYCXIC4sms9bTGHAbTH2R68gl er00vDvW+k4OcyfcGYmEi76O2UOcbXymrZLgIYr6qPtR2ElZZgLvjsXjP Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699165" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699165" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830300010" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830300010" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:24 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 08/11] KVM: Fix set_mem_attr ioctl when error case Date: Wed, 28 Jun 2023 15:43:07 -0700 Message-Id: <358fb191b3690a5cbc2c985d3ffc67224df11cf3.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata kvm_vm_ioctl_set_mem_attributes() discarded an error code of xa_err() unconditionally. If an error occurred at the beginning, return error. Fixes: 3779c214835b ("KVM: Introduce per-page memory attributes") Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - Newly added --- virt/kvm/kvm_main.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 422d49634c56..fdef56f85174 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2423,6 +2423,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm= *kvm, gfn_t start, end; unsigned long i; void *entry; + int err =3D 0; =20 /* flags is currently not used. */ if (attrs->flags) @@ -2447,14 +2448,17 @@ static int kvm_vm_ioctl_set_mem_attributes(struct k= vm *kvm, KVM_MMU_UNLOCK(kvm); =20 for (i =3D start; i < end; i++) { - if (xa_err(xa_store(&kvm->mem_attr_array, i, entry, - GFP_KERNEL_ACCOUNT))) + err =3D xa_err(xa_store(&kvm->mem_attr_array, i, entry, + GFP_KERNEL_ACCOUNT)); + if (err) break; } =20 KVM_MMU_LOCK(kvm); - if (i > start) + if (i > start) { + err =3D 0; kvm_mem_attrs_changed(kvm, attrs->attributes, start, i); + } kvm_mmu_invalidate_end(kvm); KVM_MMU_UNLOCK(kvm); =20 @@ -2463,7 +2467,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm= *kvm, attrs->address =3D i << PAGE_SHIFT; attrs->size =3D (end - i) << PAGE_SHIFT; =20 - return 0; + return err; } #endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ =20 --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 797F7EB64D7 for ; Wed, 28 Jun 2023 22:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232388AbjF1WpD (ORCPT ); Wed, 28 Jun 2023 18:45:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231496AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DE8A271B; Wed, 28 Jun 2023 15:43:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992206; x=1719528206; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0O4z6SfvuczhQYYUrNCDtvfPr0mE8UJ5jvjA0cCcdRQ=; b=dQa30bG9+xY3raAv/AHb5Ao7LfD771dQKfGFceHzVMgaGvahasKzIqvP 56SHkiAy6REVb43sjzZRIfNN/vwJ2ZDMXaeqVspUoldiKBYb5cx6jsVyk yhpuEv2gZTzn3spyXHB69jDxEaCX6xz0SLeNv7fPqRtO4CBJDVIi7BjLj 9hbV4M0WAZXvHIO9AAS3X5ZIrwEoRp+0CUA4//qoTx22zfIu+6pL7qzOt d9XIyrAElDnBhpCSpmF2KU8zQRhFZL4uS1PwroUEG0Qd3BjKdDCk6ZWGh CGrrQeld7H87WnKLn9DvMQj4kbCc0hWXz4K8fGGPAxpG3QOgsa5BTpVNg g==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699172" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699172" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830300014" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830300014" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:24 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 09/11] KVM: Add new members to struct kvm_gfn_range to operate on Date: Wed, 28 Jun 2023 15:43:08 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Isaku Yamahata Add new members to strut kvm_gfn_range to indicate which mapping (private-vs-shared) to operate on. only_private and only_shared. Update mmu notifier, set memory attributes ioctl or KVM gmem callback to initialize them. It was premature for set_memory_attributes ioctl to call kvm_unmap_gfn_range(). Instead, let kvm_arch_ste_memory_attributes() handle it and add a new x86 vendor callback to react to memory attribute change. [1] - If it's from the mmu notifier, zap shared pages only - If it's from the KVM gmem, zap private pages only - If setting memory attributes, vendor callback checks new attributes and make decisions. SNP would do nothing and handle it later with gmem callback TDX callback would do as follows. When it converts pages to shared, zap private pages only. When it converts pages to private, zap shared pages only. TDX needs to know which mapping to operate on. Shared-EPT vs. Secure-EPT. The following sequence to convert the GPA to private doesn't work for TDX because the page can already be private. 1) Update memory attributes to private in memory attributes xarray 2) Zap the GPA range irrespective of private-or-shared. Even if the page is already private, zap the entry. 3) EPT violation on the GPA 4) Populate the GPA as private The page is zeroed, and the guest has to accept the page again. In step 2, TDX wants to zap only shared pages and skip private ones. [1] https://lore.kernel.org/all/ZJX0hk+KpQP0KUyB@google.com/ Suggested-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- Changes v2 -> v3: - Drop the KVM_GFN_RANGE flags - Updated struct kvm_gfn_range - Change kvm_arch_set_memory_attributes() to return bool for flush - Added set_memory_attributes x86 op for vendor backends - Refined commit message to describe TDX care concretely Changes v1 -> v2: - consolidate KVM_GFN_RANGE_FLAGS_GMEM_{PUNCH_HOLE, RELEASE} into KVM_GFN_RANGE_FLAGS_GMEM. - Update the commit message to describe TDX more. Drop SEV_SNP. --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 20 +++++++++++++++----- include/linux/kvm_host.h | 13 ++++++++----- virt/kvm/guest_mem.c | 2 ++ virt/kvm/kvm_main.c | 12 ++++++------ 6 files changed, 35 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index c0143906fe6d..fc65374a8bad 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -125,6 +125,7 @@ KVM_X86_OP_OPTIONAL(mem_enc_unregister_region) KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from) KVM_X86_OP_OPTIONAL(vm_move_enc_context_from) KVM_X86_OP_OPTIONAL(guest_memory_reclaimed) +KVM_X86_OP_OPTIONAL_RET0(set_memory_attributes) KVM_X86_OP(get_msr_feature) KVM_X86_OP(can_emulate_instruction) KVM_X86_OP(apic_init_signal_blocked) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index bdf507797c73..a4af4175034b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1534,6 +1534,8 @@ static inline u16 kvm_lapic_irq_dest_mode(bool dest_m= ode_logical) return dest_mode_logical ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL; } =20 +struct kvm_gfn_range; + struct kvm_x86_ops { const char *name; =20 @@ -1716,6 +1718,7 @@ struct kvm_x86_ops { int (*vm_copy_enc_context_from)(struct kvm *kvm, unsigned int source_fd); int (*vm_move_enc_context_from)(struct kvm *kvm, unsigned int source_fd); void (*guest_memory_reclaimed)(struct kvm *kvm); + bool (*set_memory_attributes)(struct kvm *kvm, struct kvm_gfn_range *rang= e); =20 int (*get_msr_feature)(struct kvm_msr_entry *entry); =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5a80ec49bdcd..3795f447603c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7327,11 +7327,13 @@ static bool hugepage_has_attrs(struct kvm *kvm, str= uct kvm_memory_slot *slot, return true; } =20 -void kvm_arch_set_memory_attributes(struct kvm *kvm, - struct kvm_memory_slot *slot, - unsigned long attrs, - gfn_t start, gfn_t end) +bool kvm_arch_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range = *range) { + struct kvm_memory_slot *slot =3D range->slot; + unsigned long attrs =3D range->attributes; + gfn_t start =3D range->start; + gfn_t end =3D range->end; + bool flush =3D false; int level; =20 lockdep_assert_held_write(&kvm->mmu_lock); @@ -7342,7 +7344,13 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm, * the slot if the slot will never consume the PRIVATE attribute. */ if (!kvm_slot_can_be_private(slot)) - return; + return flush; + + if (kvm->arch.vm_type =3D=3D KVM_X86_PROTECTED_VM && + !kvm_x86_ops.set_memory_attributes) + flush =3D kvm_unmap_gfn_range(kvm, range); + else + flush =3D static_call(kvm_x86_set_memory_attributes)(kvm, range); =20 /* * The sequence matters here: upper levels consume the result of lower @@ -7388,6 +7396,8 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm, hugepage_set_mixed(slot, gfn, level); } } + + return flush; } =20 void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 1a47cedae8a1..5ca0c8ee4292 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -260,7 +260,13 @@ struct kvm_gfn_range { struct kvm_memory_slot *slot; gfn_t start; gfn_t end; - pte_t pte; + union { + unsigned long attributes; + pte_t pte; + unsigned long callback_arg; /* needs a better name */ + }; + bool only_private; + bool only_shared; bool may_block; }; bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); @@ -2323,10 +2329,7 @@ static inline unsigned long kvm_get_memory_attribute= s(struct kvm *kvm, gfn_t gfn return xa_to_value(xa_load(&kvm->mem_attr_array, gfn)); } =20 -void kvm_arch_set_memory_attributes(struct kvm *kvm, - struct kvm_memory_slot *slot, - unsigned long attrs, - gfn_t start, gfn_t end); +bool kvm_arch_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range = *range); =20 static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) { diff --git a/virt/kvm/guest_mem.c b/virt/kvm/guest_mem.c index cdf2d84683c8..63ac006db7ee 100644 --- a/virt/kvm/guest_mem.c +++ b/virt/kvm/guest_mem.c @@ -117,6 +117,8 @@ static void kvm_gmem_invalidate_begin(struct kvm *kvm, = struct kvm_gmem *gmem, .end =3D slot->base_gfn + index_end - slot->gmem.index, .slot =3D slot, .pte =3D __pte(0), + .only_private =3D true, + .only_shared =3D false, .may_block =3D true, }; =20 diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index fdef56f85174..c9bc0a6c9973 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -604,6 +604,8 @@ static __always_inline int __kvm_handle_hva_range(struc= t kvm *kvm, * the second or later invocation of the handler). */ gfn_range.pte =3D range->pte; + gfn_range.only_private =3D false; + gfn_range.only_shared =3D true; gfn_range.may_block =3D range->may_block; =20 /* @@ -2391,7 +2393,9 @@ static void kvm_mem_attrs_changed(struct kvm *kvm, un= signed long attrs, bool flush =3D false; int i; =20 - gfn_range.pte =3D __pte(0); + gfn_range.attributes =3D attrs; + gfn_range.only_private =3D false; + gfn_range.only_shared =3D false; gfn_range.may_block =3D true; =20 for (i =3D 0; i < kvm_arch_nr_memslot_as_ids(kvm); i++) { @@ -2405,11 +2409,7 @@ static void kvm_mem_attrs_changed(struct kvm *kvm, u= nsigned long attrs, continue; gfn_range.slot =3D slot; =20 - flush |=3D kvm_unmap_gfn_range(kvm, &gfn_range); - - kvm_arch_set_memory_attributes(kvm, slot, attrs, - gfn_range.start, - gfn_range.end); + flush |=3D kvm_arch_set_memory_attributes(kvm, &gfn_range); } } =20 --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0C18EB64D7 for ; Wed, 28 Jun 2023 22:44:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232259AbjF1Wom (ORCPT ); Wed, 28 Jun 2023 18:44:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231481AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B2F62721; Wed, 28 Jun 2023 15:43:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992206; x=1719528206; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e3EqarE0usjS78tG7qfYkQZ0emCI9c7Ncg6aVMj8zgU=; b=DIaRZB++TQtuwrhzvaAiw3dvSM5SNhCG+LkWGfKbrCa+sNa1UQBwpUNg CCbsFI7JuSUvtIuwCm8rKDi36cC2CJi3oiN4kO1UUVN6e6b5MPNB5ctRH dO/1l82eORGscb7/QjLnLUQbhf1BjWX+oQ7fNI65T6RxxTeTd2k1n0xj4 DHlTz42KXo9LBZ4+IB0SdX8wB416WGFpVPHSUI56MZOwTMer8tf5YLadE OnKd9bIf27BD4hQCp1o1ZOvUT+GXIpuBhHPJ1rj7v8/YsBBWdC1fHOwVQ xje4rKmM9i/FSeQV+zPi3vEvlUJpVjjrKAm48v8AFaGzL02KuIortg3wR w==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699182" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699182" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830300023" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830300023" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:25 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 10/11] KVM: x86: Add gmem hook for initializing private memory Date: Wed, 28 Jun 2023 15:43:09 -0700 Message-Id: <9256891e95298b0a26d4276e8d4afe8c28304767.1687991811.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Michael Roth All gmem pages are expected to be 'private' as defined by a particular arch/platform. Platforms like SEV-SNP require additional operations to move these pages into a private state, so implement a hook that can be used to prepare this memory prior to mapping it into a guest. In the case of SEV-SNP, whether or not a 2MB page can be mapped via a 2MB mapping in the guest's nested page table depends on whether or not any subpages within the range have already been initialized as private in the RMP table, so this hook will also be used by the KVM MMU to clamp the maximum mapping size accordingly. Signed-off-by: Michael Roth Link: https://lore.kernel.org/r/20230612042559.375660-2-michael.roth@amd.com --- Changes v2 -> v3: - Newly added --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 11 ++++++++++- 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index fc65374a8bad..cce8621e3216 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -135,6 +135,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(gmem_prepare) =20 #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index a4af4175034b..653f208979cf 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1738,6 +1738,9 @@ struct kvm_x86_ops { * Returns vCPU specific APICv inhibit reasons */ unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu); + + int (*gmem_prepare)(struct kvm *kvm, struct kvm_memory_slot *slot, + kvm_pfn_t pfn, gfn_t gfn, u8 *max_level); }; =20 struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 3795f447603c..fdd89cd8f68e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4341,6 +4341,7 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *v= cpu, struct kvm_page_fault *fault) { int order, r; + u8 max_level; =20 if (!kvm_slot_can_be_private(fault->slot)) return kvm_do_memory_fault_exit(vcpu, fault); @@ -4349,7 +4350,15 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *= vcpu, if (r) return r; =20 - fault->max_level =3D min(kvm_max_level_for_order(order), fault->max_level= ); + max_level =3D kvm_max_level_for_order(order); + r =3D static_call(kvm_x86_gmem_prepare)(vcpu->kvm, fault->slot, fault->pf= n, + fault->gfn, &max_level); + if (r) { + kvm_release_pfn_clean(fault->pfn); + return r; + } + + fault->max_level =3D min(max_level, fault->max_level); fault->map_writable =3D !(fault->slot->flags & KVM_MEM_READONLY); return RET_PF_CONTINUE; } --=20 2.25.1 From nobody Sat Feb 7 18:15:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42494EB64DA for ; Wed, 28 Jun 2023 22:45:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232507AbjF1WpW (ORCPT ); Wed, 28 Jun 2023 18:45:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231497AbjF1Wn1 (ORCPT ); Wed, 28 Jun 2023 18:43:27 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D93DF2728; Wed, 28 Jun 2023 15:43:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687992206; x=1719528206; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R6aeeOVtCVl/aF4rr9VOp5md60UJagkTMuBGcdbekMM=; b=QOCMc1NmYINHU7HXGIdpdYk6l9m6WRlH8hbbzwzS/RUpFaMqc45O9WeT u3vIyxtvJ+FuqTkOJF9wrBovBww+ODBibSgapAvHrHgcV0xri33plpoqn ObvcdBNUzDdZKtv0JLJ67dTfytaNLwgOYACiQGDuuDHQ68m3BIn6KY4L1 DcPuj+o+FYHnRKH0o97+f4Jm+MnTdKB/Xfi7GdQWeEwySc3/PMBAABGPZ BBB8PGGjWOuFEmGMM4LAubXkVRplUXx0CLuFao0vO+RDn2cdmd5CclaXt bSXMkPk8Z7aSq89WNozP4Qo1LpdPleneg7pXExeJn0PukPZ61Q8fypJtp Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="392699190" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="392699190" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10755"; a="830300029" X-IronPort-AV: E=Sophos;i="6.01,166,1684825200"; d="scan'208";a="830300029" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2023 15:43:25 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Kai Huang , Zhi Wang , chen.bo@intel.com, linux-coco@lists.linux.dev, Chao Peng , Ackerley Tng , Vishal Annapurve , Michael Roth , Yuan Yao Subject: [RFC PATCH v3 11/11] KVM: x86: Add gmem hook for invalidating private memory Date: Wed, 28 Jun 2023 15:43:10 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Michael Roth TODO: add a CONFIG option that can be to completely skip arch invalidation loop and avoid __weak references for arch/platforms that don't need an additional invalidation hook. In some cases, like with SEV-SNP, guest memory needs to be updated in a platform-specific manner before it can be safely freed back to the host. Add hooks to wire up handling of this sort when freeing memory in response to FALLOC_FL_PUNCH_HOLE operations. Also issue invalidations of all allocated pages when releasing the gmem file so that the pages are not left in an unusable state when they get freed back to the host. Signed-off-by: Michael Roth Link: https://lore.kernel.org/r/20230612042559.375660-3-michael.roth@amd.com --- Changes v2 -> v3: - Newly added --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 6 ++++ include/linux/kvm_host.h | 3 ++ virt/kvm/guest_mem.c | 48 ++++++++++++++++++++++++++++-- 5 files changed, 57 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index cce8621e3216..a864a2093002 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -136,6 +136,7 @@ KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL_RET0(gmem_prepare) +KVM_X86_OP_OPTIONAL(gmem_invalidate) =20 #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 653f208979cf..a91d17fa2fe8 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1741,6 +1741,7 @@ struct kvm_x86_ops { =20 int (*gmem_prepare)(struct kvm *kvm, struct kvm_memory_slot *slot, kvm_pfn_t pfn, gfn_t gfn, u8 *max_level); + void (*gmem_invalidate)(struct kvm *kvm, kvm_pfn_t start, kvm_pfn_t end); }; =20 struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b5f865f39a00..e722ace8150d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -13260,6 +13260,12 @@ bool kvm_arch_no_poll(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_arch_no_poll); =20 +#ifdef CONFIG_KVM_PRIVATE_MEM +void kvm_arch_gmem_invalidate(struct kvm *kvm, kvm_pfn_t start, kvm_pfn_t = end) +{ + static_call_cond(kvm_x86_gmem_invalidate)(kvm, start, end); +} +#endif =20 int kvm_spec_ctrl_test_value(u64 value) { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 5ca0c8ee4292..cfd98572d8be 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2346,6 +2346,7 @@ static inline bool kvm_mem_is_private(struct kvm *kvm= , gfn_t gfn) #ifdef CONFIG_KVM_PRIVATE_MEM int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, int *order); +void kvm_arch_gmem_invalidate(struct kvm *kvm, kvm_pfn_t start, kvm_pfn_t = end); #else static inline int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, @@ -2354,6 +2355,8 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, KVM_BUG_ON(1, kvm); return -EIO; } + +void kvm_arch_gmem_invalidate(struct kvm *kvm, kvm_pfn_t start, kvm_pfn_t = end) { } #endif /* CONFIG_KVM_PRIVATE_MEM */ =20 #endif diff --git a/virt/kvm/guest_mem.c b/virt/kvm/guest_mem.c index 63ac006db7ee..5b8e11760d28 100644 --- a/virt/kvm/guest_mem.c +++ b/virt/kvm/guest_mem.c @@ -142,16 +142,58 @@ static void kvm_gmem_invalidate_end(struct kvm *kvm, = struct kvm_gmem *gmem, KVM_MMU_UNLOCK(kvm); } =20 +void __weak kvm_arch_gmem_invalidate(struct kvm *kvm, kvm_pfn_t start, kvm= _pfn_t end) +{ +} + +/* Handle arch-specific hooks needed before releasing guarded pages. */ +static void kvm_gmem_issue_arch_invalidate(struct kvm *kvm, struct file *f= ile, + pgoff_t start, pgoff_t end) +{ + pgoff_t file_end =3D i_size_read(file_inode(file)) >> PAGE_SHIFT; + pgoff_t index =3D start; + + end =3D min(end, file_end); + + while (index < end) { + struct folio *folio; + unsigned int order; + struct page *page; + kvm_pfn_t pfn; + + folio =3D __filemap_get_folio(file->f_mapping, index, + FGP_LOCK, 0); + if (!folio) { + index++; + continue; + } + + page =3D folio_file_page(folio, index); + pfn =3D page_to_pfn(page); + order =3D folio_order(folio); + + kvm_arch_gmem_invalidate(kvm, pfn, pfn + min((1ul << order), end - index= )); + + index =3D folio_next_index(folio); + folio_unlock(folio); + folio_put(folio); + + cond_resched(); + } +} + static long kvm_gmem_punch_hole(struct file *file, loff_t offset, loff_t l= en) { struct kvm_gmem *gmem =3D file->private_data; - pgoff_t start =3D offset >> PAGE_SHIFT; - pgoff_t end =3D (offset + len) >> PAGE_SHIFT; struct kvm *kvm =3D gmem->kvm; + pgoff_t start, end; =20 if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len)) return 0; =20 + start =3D offset >> PAGE_SHIFT; + end =3D (offset + len) >> PAGE_SHIFT; + /* * Bindings must stable across invalidation to ensure the start+end * are balanced. @@ -160,6 +202,7 @@ static long kvm_gmem_punch_hole(struct file *file, loff= _t offset, loff_t len) =20 kvm_gmem_invalidate_begin(kvm, gmem, start, end); =20 + kvm_gmem_issue_arch_invalidate(kvm, file, start, end); truncate_inode_pages_range(file->f_mapping, offset, offset + len - 1); =20 kvm_gmem_invalidate_end(kvm, gmem, start, end); @@ -266,6 +309,7 @@ static int kvm_gmem_release(struct inode *inode, struct= file *file) * pointed at this file. */ kvm_gmem_invalidate_begin(kvm, gmem, 0, -1ul); + kvm_gmem_issue_arch_invalidate(gmem->kvm, file, 0, -1ul); truncate_inode_pages_final(file->f_mapping); kvm_gmem_invalidate_end(kvm, gmem, 0, -1ul); =20 --=20 2.25.1