From nobody Mon Sep 29 21:06:45 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61C60C7EE29 for ; Thu, 25 May 2023 22:59:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241255AbjEYW7f (ORCPT ); Thu, 25 May 2023 18:59:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233071AbjEYW71 (ORCPT ); Thu, 25 May 2023 18:59:27 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7F86E6; Thu, 25 May 2023 15:59:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685055566; x=1716591566; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nR46GYpJqRolBfY/YICB4Ng3v+nWMwewiHqSaXESjks=; b=liPXCuMirnk0bH/EmfXb5qArW5TU7apsIrhtih0AIclpK05DQ2OzELM3 Jp2EC+5zYV2cgrMzt8ABL1RB6TqKf5JFU10A8cS5rlQlaSiBVOlGS3DBp VgMlR6aYwpzl3DYipgv/wSe2z1A0O07lO0cUPIy8jT9PaXId1nMl7TTEg CewNyiOCi4ur4OhsXE7KMuPLVj3CBoSCPLeKhzyM+Nd8fXZT8CBdKpeeM ZCcnrQBRc/83tcVlrp/AgdtPc4cp/BrB85BEco2ZgHIceH8C2IexYGVzX CULTM+y2MLzsGZELz3YyIixOtxji39oxIsBiPgsQqBrosArkUvBtkjVH7 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10721"; a="356434731" X-IronPort-AV: E=Sophos;i="6.00,192,1681196400"; d="scan'208";a="356434731" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 May 2023 15:59:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10721"; a="849363625" X-IronPort-AV: E=Sophos;i="6.00,192,1681196400"; d="scan'208";a="849363625" Received: from daubin-mobl.amr.corp.intel.com (HELO box.shutemov.name) ([10.249.39.107]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 May 2023 15:59:00 -0700 Received: by box.shutemov.name (Postfix, from userid 1000) id BA29810DC16; Fri, 26 May 2023 01:58:57 +0300 (+03) From: "Kirill A. Shutemov" To: dave.hansen@intel.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de Cc: decui@microsoft.com, rick.p.edgecombe@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, seanjc@google.com, thomas.lendacky@amd.com, x86@kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" , stable@vger.kernel.org Subject: [PATCH 1/2] x86/mm: Allow guest.enc_status_change_prepare() to fail Date: Fri, 26 May 2023 01:58:46 +0300 Message-Id: <20230525225847.28592-2-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230525225847.28592-1-kirill.shutemov@linux.intel.com> References: <20230525225847.28592-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" TDX code is going to provide guest.enc_status_change_prepare() that is able to fail. Add a way to return an error from the callback. While there, fix enc_status_change_finish_noop(). It is defined as always-fail now which doesn't make sense for noop. Signed-off-by: Kirill A. Shutemov Cc: stable@vger.kernel.org --- arch/x86/include/asm/x86_init.h | 2 +- arch/x86/kernel/x86_init.c | 4 ++-- arch/x86/mm/mem_encrypt_amd.c | 4 +++- arch/x86/mm/pat/set_memory.c | 3 ++- 4 files changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_ini= t.h index 88085f369ff6..1ca9701917c5 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -150,7 +150,7 @@ struct x86_init_acpi { * @enc_cache_flush_required Returns true if a cache flush is needed befor= e changing page encryption status */ struct x86_guest { - void (*enc_status_change_prepare)(unsigned long vaddr, int npages, bool e= nc); + bool (*enc_status_change_prepare)(unsigned long vaddr, int npages, bool e= nc); bool (*enc_status_change_finish)(unsigned long vaddr, int npages, bool en= c); bool (*enc_tlb_flush_required)(bool enc); bool (*enc_cache_flush_required)(void); diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index d82f4fa2f1bf..64664311ac2b 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -130,8 +130,8 @@ struct x86_cpuinit_ops x86_cpuinit =3D { =20 static void default_nmi_init(void) { }; =20 -static void enc_status_change_prepare_noop(unsigned long vaddr, int npages= , bool enc) { } -static bool enc_status_change_finish_noop(unsigned long vaddr, int npages,= bool enc) { return false; } +static bool enc_status_change_prepare_noop(unsigned long vaddr, int npages= , bool enc) { return true; } +static bool enc_status_change_finish_noop(unsigned long vaddr, int npages,= bool enc) { return true; } static bool enc_tlb_flush_required_noop(bool enc) { return false; } static bool enc_cache_flush_required_noop(void) { return false; } static bool is_private_mmio_noop(u64 addr) {return false; } diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c index e0b51c09109f..4f95c449a406 100644 --- a/arch/x86/mm/mem_encrypt_amd.c +++ b/arch/x86/mm/mem_encrypt_amd.c @@ -319,7 +319,7 @@ static void enc_dec_hypercall(unsigned long vaddr, int = npages, bool enc) #endif } =20 -static void amd_enc_status_change_prepare(unsigned long vaddr, int npages,= bool enc) +static bool amd_enc_status_change_prepare(unsigned long vaddr, int npages,= bool enc) { /* * To maintain the security guarantees of SEV-SNP guests, make sure @@ -327,6 +327,8 @@ static void amd_enc_status_change_prepare(unsigned long= vaddr, int npages, bool */ if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP) && !enc) snp_set_memory_shared(vaddr, npages); + + return true; } =20 /* Return true unconditionally: return value doesn't matter for the SEV si= de */ diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 7159cf787613..b8f48ebe753c 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -2151,7 +2151,8 @@ static int __set_memory_enc_pgtable(unsigned long add= r, int numpages, bool enc) cpa_flush(&cpa, x86_platform.guest.enc_cache_flush_required()); =20 /* Notify hypervisor that we are about to set/clr encryption attribute. */ - x86_platform.guest.enc_status_change_prepare(addr, numpages, enc); + if (!x86_platform.guest.enc_status_change_prepare(addr, numpages, enc)) + return -EIO; =20 ret =3D __change_page_attr_set_clr(&cpa, 1); =20 --=20 2.39.3 From nobody Mon Sep 29 21:06:45 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F0EEC7EE29 for ; Thu, 25 May 2023 23:00:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229765AbjEYXAi (ORCPT ); Thu, 25 May 2023 19:00:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231964AbjEYXAe (ORCPT ); Thu, 25 May 2023 19:00:34 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5602C1A4; Thu, 25 May 2023 16:00:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685055629; x=1716591629; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=h2+VFs78zM5hSi1cfMqUT06G+rYbm2kFes9haAPQVuw=; b=cqRUXBBjb63SJJmn8igmdSehy07ho/a9Mt41eqBCGUbE0iUxOQP+xZNZ U2veR+y53Jom6HLWQxQeBdhvIq2i+fraaORiAVrBAqmGVtsIzo7k7POci hIMXXIbKrJbcY5fxGEik4BZCEg78RmGsxbWFJXDo8NEfcOoARrfz7cNrW z/m8G2ETAOsHi4xgXvW/TiLOHb3UP7SXg3WRIJypwK8AGkaLk07maMvG8 KfT+V9mr7HMACAOyGRwITP4r1kriC3IbB35IYc5xr/3hfEXO2e4wJFjMm hsngfhtrpd88k20/OERjXd23lJ0yEHtHWjkCncSyRjJRwUXnf1uzW/Tp4 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10721"; a="440404189" X-IronPort-AV: E=Sophos;i="6.00,192,1681196400"; d="scan'208";a="440404189" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 May 2023 15:59:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10721"; a="737995684" X-IronPort-AV: E=Sophos;i="6.00,192,1681196400"; d="scan'208";a="737995684" Received: from daubin-mobl.amr.corp.intel.com (HELO box.shutemov.name) ([10.249.39.107]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 May 2023 15:59:00 -0700 Received: by box.shutemov.name (Postfix, from userid 1000) id C2DA710DC17; Fri, 26 May 2023 01:58:57 +0300 (+03) From: "Kirill A. Shutemov" To: dave.hansen@intel.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de Cc: decui@microsoft.com, rick.p.edgecombe@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, seanjc@google.com, thomas.lendacky@amd.com, x86@kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" , stable@vger.kernel.org Subject: [PATCH 2/2] x86/tdx: Fix race between set_memory_encrypted() and load_unaligned_zeropad() Date: Fri, 26 May 2023 01:58:47 +0300 Message-Id: <20230525225847.28592-3-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230525225847.28592-1-kirill.shutemov@linux.intel.com> References: <20230525225847.28592-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Touching privately mapped GPA that is not properly converted to private with MapGPA and accepted leads to unrecoverable exit to VMM. load_unaligned_zeropad() can touch memory that is not owned by the caller, but just happened to next after the owned memory. This load_unaligned_zeropad() behaviour makes it important when kernel asks VMM to convert a GPA from shared to private or back. Kernel must never have a page mapped into direct mapping (and aliases) as private when the GPA is already converted to shared or when GPA is not yet converted to private. guest.enc_status_change_prepare() called before adjusting direct mapping and therefore it is responsible for converting the memory to private. guest.enc_tlb_flush_required() called after adjusting direct mapping and it converts the memory to shared. It is okay to have a shared mapping of memory that is not converted properly. handle_mmio() knows how to deal with load_unaligned_zeropad() stepping on it. Signed-off-by: Kirill A. Shutemov Fixes: 7dbde7631629 ("x86/mm/cpa: Add support for TDX shared memory") Cc: stable@vger.kernel.org --- arch/x86/coco/tdx/tdx.c | 56 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 53 insertions(+), 3 deletions(-) diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index e146b599260f..84525df750d4 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -840,6 +840,30 @@ static bool tdx_enc_status_changed(unsigned long vaddr= , int numpages, bool enc) return true; } =20 +static bool tdx_enc_status_change_prepare(unsigned long vaddr, int numpage= s, + bool enc) +{ + /* + * Only handle shared->private conversion here. + * See the comment in tdx_early_init(). + */ + if (enc) + return tdx_enc_status_changed(vaddr, numpages, enc); + return true; +} + +static bool tdx_enc_status_change_finish(unsigned long vaddr, int numpages, + bool enc) +{ + /* + * Only handle private->shared conversion here. + * See the comment in tdx_early_init(). + */ + if (!enc) + return tdx_enc_status_changed(vaddr, numpages, enc); + return true; +} + void __init tdx_early_init(void) { u64 cc_mask; @@ -867,9 +891,35 @@ void __init tdx_early_init(void) */ physical_mask &=3D cc_mask - 1; =20 - x86_platform.guest.enc_cache_flush_required =3D tdx_cache_flush_required; - x86_platform.guest.enc_tlb_flush_required =3D tdx_tlb_flush_required; - x86_platform.guest.enc_status_change_finish =3D tdx_enc_status_changed; + /* + * Touching privately mapped GPA that is not properly converted to + * private with MapGPA and accepted leads to unrecoverable exit + * to VMM. + * + * load_unaligned_zeropad() can touch memory that is not owned by + * the caller, but just happened to next after the owned memory. + * This load_unaligned_zeropad() behaviour makes it important when + * kernel asks VMM to convert a GPA from shared to private or back. + * Kernel must never have a page mapped into direct mapping (and + * aliases) as private when the GPA is already converted to shared or + * when GPA is not yet converted to private. + * + * guest.enc_status_change_prepare() called before adjusting direct + * mapping and therefore it is responsible for converting the memory + * to private. + * + * guest.enc_tlb_flush_required() called after adjusting direct mapping + * and it converts the memory to shared. + * + * It is okay to have a shared mapping of memory that is not converted + * properly. handle_mmio() knows how to deal with load_unaligned_zeropad() + * stepping on it. + */ + x86_platform.guest.enc_status_change_prepare =3D tdx_enc_status_change_pr= epare; + x86_platform.guest.enc_status_change_finish =3D tdx_enc_status_change_fi= nish; + + x86_platform.guest.enc_cache_flush_required =3D tdx_cache_flush_required; + x86_platform.guest.enc_tlb_flush_required =3D tdx_tlb_flush_required; =20 pr_info("Guest detected\n"); } --=20 2.39.3