From nobody Thu Jun 25 05:47:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A4F9C433F5 for ; Tue, 22 Feb 2022 18:58:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235143AbiBVS6e (ORCPT ); Tue, 22 Feb 2022 13:58:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235153AbiBVS60 (ORCPT ); Tue, 22 Feb 2022 13:58:26 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E876ADE2E9 for ; Tue, 22 Feb 2022 10:58:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645556280; x=1677092280; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mnfE3Ie1oXRHjkG/pTiqBvf/XO47rcs2La5n3K55+Bg=; b=TvHcIwRQD0aJh9IfvQHN1kzKAoS8b3J3di+5sZS9XSRqmbb5lNPW1Hzt knvx5/UWK+aAwC7tB5PTyNfBf9cUAg8wjIk93Bxr25sLbkyuo+OGrqZsu ST2OxRHNNZJGDPNfRKmvYguwE+Re6CJxFMheOLq7o7CqbgK/TCyNpPUHv XWUdjyd5KL542MXazm6ugCM28JyXmPQ/FYih2+SSzkNdrG5PLQmmX7ZEK HZLDH9aONnavRKb2FXsfkGfYlDnL0hZqegPVfYdQGgZhKaSS2apN78VoE +XMJxI6b6ZMHOQqwLxjWYikXXSVSDNIkXKD9xWhzR5NL41UW84XEK8Hzc A==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="315013545" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="315013545" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 10:57:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="639012249" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga004.jf.intel.com with ESMTP; 22 Feb 2022 10:57:34 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 42A5142; Tue, 22 Feb 2022 20:57:51 +0200 (EET) From: "Kirill A. Shutemov" To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@intel.com, luto@kernel.org, peterz@infradead.org Cc: sathyanarayanan.kuppuswamy@linux.intel.com, aarcange@redhat.com, ak@linux.intel.com, dan.j.williams@intel.com, david@redhat.com, hpa@zytor.com, jmattson@google.com, seanjc@google.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, x86@kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Wei Liu , Dexuan Cui , Tianyu Lan Subject: [PATCH 1/4] x86/hyperv: Add missing ARCH_HAS_CC_PLATFORM dependency Date: Tue, 22 Feb 2022 21:57:37 +0300 Message-Id: <20220222185740.26228-2-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> References: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" On x86, cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) supposes to return true for HyperV if isolation is supported. But it only does it if the kernel is compiled with AMD_MEM_ENCRYPT enabled. It happens due to missed ARCH_HAS_CC_PLATFORM dependency. Without ARCH_HAS_CC_PLATFORM enabled, cc_platform_has() always returns false. Signed-off-by: Kirill A. Shutemov Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Cc: Dexuan Cui Cc: Tianyu Lan Reviewed-by: Tom Lendacky --- drivers/hv/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig index 0747a8f1fcee..424d3f0751dc 100644 --- a/drivers/hv/Kconfig +++ b/drivers/hv/Kconfig @@ -8,6 +8,7 @@ config HYPERV || (ARM64 && !CPU_BIG_ENDIAN)) select PARAVIRT select X86_HV_CALLBACK_VECTOR if X86 + select ARCH_HAS_CC_PLATFORM if x86 select VMAP_PFN help Select this option to run Linux as a Hyper-V client operating --=20 2.34.1 From nobody Thu Jun 25 05:47:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E9CEC433F5 for ; Tue, 22 Feb 2022 18:58:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235161AbiBVS6a (ORCPT ); Tue, 22 Feb 2022 13:58:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234970AbiBVS6Z (ORCPT ); Tue, 22 Feb 2022 13:58:25 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D665D205D for ; Tue, 22 Feb 2022 10:57:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645556279; x=1677092279; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yfB7BEiFcDeCEeGFve1GIQOD1WyQln7kKbaZCHmlqwQ=; b=l6p6qt+2fOMCvrA4BXIGPJTsZTVhvb2oObbdDHJyl07aVyw/+rAIXl65 UChJ7ObfK9v7XS9crt/tdKp9ifEh+DeUtgkDkzqFp+/+V/TbyH8lKAWZ4 QY9pm+HvkDQZToLtF0kWvFLx3pUppwLIfXBGV4EHoO37wrc33fZPeWexc DdvvQ+ZUoMB+uMDgyTs4gOEOOjyRUjOnT+10z+ftsUEONr/SgwQtDOsWe hXSoNXME8NrfBcz1lZAuPgSqXJ7fKqg3+y+xwta2JyF7s/qfiUxP0RrDF BOQk8EdcngdU8xCPZXuLs7MTTyh+tpSnM/N+9F4LzCfO04pojkUVldhCj A==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="315013536" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="315013536" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 10:57:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="543044104" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga007.fm.intel.com with ESMTP; 22 Feb 2022 10:57:34 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 5176B1ED; Tue, 22 Feb 2022 20:57:51 +0200 (EET) From: "Kirill A. Shutemov" To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@intel.com, luto@kernel.org, peterz@infradead.org Cc: sathyanarayanan.kuppuswamy@linux.intel.com, aarcange@redhat.com, ak@linux.intel.com, dan.j.williams@intel.com, david@redhat.com, hpa@zytor.com, jmattson@google.com, seanjc@google.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, x86@kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCH 2/4] x86: Rename cc_platform.c to arch/x86/coco/core.c Date: Tue, 22 Feb 2022 21:57:38 +0300 Message-Id: <20220222185740.26228-3-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> References: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Move cc_platform.c to the newly created arch/x86/coco. The directly is going to be a home space for code related to confidential computing. Intel TDX code will land here. AMD SEV code will also eventually be moved there. Signed-off-by: Kirill A. Shutemov Reviewed-by: Tom Lendacky --- arch/x86/Kbuild | 2 ++ arch/x86/coco/Makefile | 6 ++++++ arch/x86/{kernel/cc_platform.c =3D> coco/core.c} | 0 arch/x86/kernel/Makefile | 5 ----- 4 files changed, 8 insertions(+), 5 deletions(-) create mode 100644 arch/x86/coco/Makefile rename arch/x86/{kernel/cc_platform.c =3D> coco/core.c} (100%) diff --git a/arch/x86/Kbuild b/arch/x86/Kbuild index f384cb1a4f7a..5a83da703e87 100644 --- a/arch/x86/Kbuild +++ b/arch/x86/Kbuild @@ -1,4 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_ARCH_HAS_CC_PLATFORM) +=3D coco/ + obj-y +=3D entry/ =20 obj-$(CONFIG_PERF_EVENTS) +=3D events/ diff --git a/arch/x86/coco/Makefile b/arch/x86/coco/Makefile new file mode 100644 index 000000000000..c1ead00017a7 --- /dev/null +++ b/arch/x86/coco/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS_REMOVE_core.o =3D -pg +KASAN_SANITIZE_core.o :=3D n +CFLAGS_core.o +=3D -fno-stack-protector + +obj-y +=3D core.o diff --git a/arch/x86/kernel/cc_platform.c b/arch/x86/coco/core.c similarity index 100% rename from arch/x86/kernel/cc_platform.c rename to arch/x86/coco/core.c diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 6aef9ee28a39..6462e3dd98f4 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -21,7 +21,6 @@ CFLAGS_REMOVE_ftrace.o =3D -pg CFLAGS_REMOVE_early_printk.o =3D -pg CFLAGS_REMOVE_head64.o =3D -pg CFLAGS_REMOVE_sev.o =3D -pg -CFLAGS_REMOVE_cc_platform.o =3D -pg endif =20 KASAN_SANITIZE_head$(BITS).o :=3D n @@ -30,7 +29,6 @@ KASAN_SANITIZE_dumpstack_$(BITS).o :=3D n KASAN_SANITIZE_stacktrace.o :=3D n KASAN_SANITIZE_paravirt.o :=3D n KASAN_SANITIZE_sev.o :=3D n -KASAN_SANITIZE_cc_platform.o :=3D n =20 # With some compiler versions the generated code results in boot hangs, ca= used # by several compilation units. To be safe, disable all instrumentation. @@ -49,7 +47,6 @@ endif KCOV_INSTRUMENT :=3D n =20 CFLAGS_head$(BITS).o +=3D -fno-stack-protector -CFLAGS_cc_platform.o +=3D -fno-stack-protector =20 CFLAGS_irq.o :=3D -I $(srctree)/$(src)/../include/asm/trace =20 @@ -151,8 +148,6 @@ obj-$(CONFIG_UNWINDER_GUESS) +=3D unwind_guess.o =20 obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D sev.o =20 -obj-$(CONFIG_ARCH_HAS_CC_PLATFORM) +=3D cc_platform.o - ### # 64 bit specific files ifeq ($(CONFIG_X86_64),y) --=20 2.34.1 From nobody Thu Jun 25 05:47:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91EB6C433F5 for ; Tue, 22 Feb 2022 18:57:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235144AbiBVS6J (ORCPT ); Tue, 22 Feb 2022 13:58:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234906AbiBVS6F (ORCPT ); Tue, 22 Feb 2022 13:58:05 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0E9EBD2DB for ; Tue, 22 Feb 2022 10:57:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645556259; x=1677092259; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iqtVod6gdoeQwYMqOAiSSDILP+NKKWVVo+oPlRWZhX0=; b=FeXB8dh0gj28eIBxzeGkDP5DntUoK6cWlGlIQM0x0hxCt8KtEadbmNwg to/uMuXN1iWSvDJl/w8oN1L0MOXWHmto32env0uLwCqhyKEyI/KnjSiqI Iyv1hbVQGFL1O38lsbfhLTyiH+Vg8QG4//9Pn/gs+ewu/SbplWRh3yiQw 96kFll7RyTc+02/5DR6GIN59k+oqOO8SA1/GNdFTfXeDbRbJzsJduZ5n/ AAhIbH9lmFUhloFzUKtt+4fYgmCQEupPTILP9ATbVAnkuC/qST65UHy7U 7X+EtSuSK588lOCnKAyvph9XcrVNnRho9ao+nwZL9ZFh/okueDMtvy3xE w==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="239169227" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="239169227" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 10:57:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="637120618" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga002.fm.intel.com with ESMTP; 22 Feb 2022 10:57:34 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 5C0B32FC; Tue, 22 Feb 2022 20:57:51 +0200 (EET) From: "Kirill A. Shutemov" To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@intel.com, luto@kernel.org, peterz@infradead.org Cc: sathyanarayanan.kuppuswamy@linux.intel.com, aarcange@redhat.com, ak@linux.intel.com, dan.j.williams@intel.com, david@redhat.com, hpa@zytor.com, jmattson@google.com, seanjc@google.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, x86@kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCH 3/4] x86/coco: Explicitly declare type of confidential computing platform Date: Tue, 22 Feb 2022 21:57:39 +0300 Message-Id: <20220222185740.26228-4-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> References: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Kernel derives type of confidential computing platform from sme_me_mask value and hv_is_isolation_supported(). This detection process will be more complicated as more platforms get added. Declare confidential computing vendor explicitly via cc_set_vendor(). Signed-off-by: Kirill A. Shutemov Reviewed-by: Tom Lendacky --- arch/x86/coco/core.c | 29 +++++++++++++++++------------ arch/x86/include/asm/coco.h | 14 ++++++++++++++ arch/x86/kernel/cpu/mshyperv.c | 3 +++ arch/x86/mm/mem_encrypt_identity.c | 11 +++++++---- 4 files changed, 41 insertions(+), 16 deletions(-) create mode 100644 arch/x86/include/asm/coco.h diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c index 6a6ffcd978f6..476dcd198af5 100644 --- a/arch/x86/coco/core.c +++ b/arch/x86/coco/core.c @@ -9,18 +9,15 @@ =20 #include #include -#include =20 -#include +#include #include =20 -static bool __maybe_unused intel_cc_platform_has(enum cc_attr attr) +static enum cc_vendor vendor __ro_after_init; + +static bool intel_cc_platform_has(enum cc_attr attr) { -#ifdef CONFIG_INTEL_TDX_GUEST - return false; -#else return false; -#endif } =20 /* @@ -74,12 +71,20 @@ static bool hyperv_cc_platform_has(enum cc_attr attr) =20 bool cc_platform_has(enum cc_attr attr) { - if (sme_me_mask) + switch (vendor) { + case CC_VENDOR_AMD: return amd_cc_platform_has(attr); - - if (hv_is_isolation_supported()) + case CC_VENDOR_INTEL: + return intel_cc_platform_has(attr); + case CC_VENDOR_HYPERV: return hyperv_cc_platform_has(attr); - - return false; + default: + return false; + } } EXPORT_SYMBOL_GPL(cc_platform_has); + +__init void cc_set_vendor(enum cc_vendor v) +{ + vendor =3D v; +} diff --git a/arch/x86/include/asm/coco.h b/arch/x86/include/asm/coco.h new file mode 100644 index 000000000000..e49f9ddb6ae6 --- /dev/null +++ b/arch/x86/include/asm/coco.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_COCO_H +#define _ASM_X86_COCO_H + +enum cc_vendor { + CC_VENDOR_NONE, + CC_VENDOR_AMD, + CC_VENDOR_HYPERV, + CC_VENDOR_INTEL, +}; + +void cc_set_vendor(enum cc_vendor v); + +#endif /* _ASM_X86_COCO_H */ diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 5a99f993e639..c1c0123859b9 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -33,6 +33,7 @@ #include #include #include +#include =20 /* Is Linux running as the root partition? */ bool hv_root_partition; @@ -344,6 +345,8 @@ static void __init ms_hyperv_init_platform(void) */ swiotlb_force =3D SWIOTLB_FORCE; #endif + if (hv_get_isolation_type() !=3D HV_ISOLATION_TYPE_NONE) + cc_set_vendor(CC_VENDOR_HYPERV); } =20 if (hv_max_functions_eax >=3D HYPERV_CPUID_NESTED_FEATURES) { diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_i= dentity.c index 3f0abb403340..06314ae3998e 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -44,6 +44,7 @@ #include #include #include +#include =20 #include "mm_internal.h" =20 @@ -565,8 +566,7 @@ void __init sme_enable(struct boot_params *bp) } else { /* SEV state cannot be controlled by a command line option */ sme_me_mask =3D me_mask; - physical_mask &=3D ~sme_me_mask; - return; + goto out; } =20 /* @@ -600,6 +600,9 @@ void __init sme_enable(struct boot_params *bp) sme_me_mask =3D 0; else sme_me_mask =3D active_by_default ? me_mask : 0; - - physical_mask &=3D ~sme_me_mask; +out: + if (sme_me_mask) { + physical_mask &=3D ~sme_me_mask; + cc_set_vendor(CC_VENDOR_AMD); + } } --=20 2.34.1 From nobody Thu Jun 25 05:47:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59C54C433EF for ; Tue, 22 Feb 2022 18:57:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235120AbiBVS6H (ORCPT ); Tue, 22 Feb 2022 13:58:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234867AbiBVS6F (ORCPT ); Tue, 22 Feb 2022 13:58:05 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A111BBE3E for ; Tue, 22 Feb 2022 10:57:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645556259; x=1677092259; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AuodqAdtsYft9lvEIxVYtPnZkgg8T2osK8Sb22u/TqE=; b=CyqtsY+KCpLwGoRvlBSDLr0pvMbhbTcryU4XxdJg9FlocgLpy9lZc/5v aPUH1YAaXfv8kgsS2js547URg6R5ApAwVobm3VeprqmWCJADanRs60nZ1 cVu9oLFSBGhY0zVsWYZLWGEo+2wnR/kJPj/dBdhmRvNirTnrfKdEUwpXV UXN6pE6Z7Wflw/VwKfL2e6BR4LvXFHjn9tdBVkmpiB7WY4HB9IWlwgsSE x4x0T9w+WzQGlXkwUhirj4Uy/3YT69uWV+XKZ7rJBRSY3JdyY2mZNaaoe 3gTg4IfnbqtEz0aEmmO017kUBFlVEqs4qfozrbaboMbYNNBvi4S5yG9JG Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="232406837" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="232406837" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 10:57:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="591415361" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga008.fm.intel.com with ESMTP; 22 Feb 2022 10:57:34 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 69DE840D; Tue, 22 Feb 2022 20:57:51 +0200 (EET) From: "Kirill A. Shutemov" To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@intel.com, luto@kernel.org, peterz@infradead.org Cc: sathyanarayanan.kuppuswamy@linux.intel.com, aarcange@redhat.com, ak@linux.intel.com, dan.j.williams@intel.com, david@redhat.com, hpa@zytor.com, jmattson@google.com, seanjc@google.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, x86@kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCH 4/4] x86/coco: Add API to handle encryption mask Date: Tue, 22 Feb 2022 21:57:40 +0300 Message-Id: <20220222185740.26228-5-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> References: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" AMD SME/SEV uses a bit in the page table entries to indicate that the page is encrypted and not accessible to the VMM. TDX uses a similar approach, but the polarity of the mask is opposite to AMD: if the bit is set the page is accessible to VMM. Provide vendor-neutral API to deal with the mask: cc_mkenc() and cc_mkdec() modify given address to make it encrypted/decrypted. It can be applied to phys_addr_t, pgprotval_t or page table entry value. pgprot_encrypted() and pgprot_decrypted() reimplemented using new helpers. The implementation will be extended to cover TDX. pgprot_decrypted() is used by drivers (i915, virtio_gpu, vfio). cc_mkdec() called by pgprot_decrypted(). Export cc_mkdec(). Signed-off-by: Kirill A. Shutemov Reviewed-by: Tom Lendacky --- arch/x86/coco/core.c | 27 +++++++++++++++++++++++++++ arch/x86/include/asm/coco.h | 18 ++++++++++++++++++ arch/x86/include/asm/pgtable.h | 13 +++++++------ arch/x86/mm/mem_encrypt_identity.c | 1 + arch/x86/mm/pat/set_memory.c | 5 +++-- 5 files changed, 56 insertions(+), 8 deletions(-) diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c index 476dcd198af5..fc1365dd927e 100644 --- a/arch/x86/coco/core.c +++ b/arch/x86/coco/core.c @@ -14,6 +14,7 @@ #include =20 static enum cc_vendor vendor __ro_after_init; +static u64 cc_mask __ro_after_init; =20 static bool intel_cc_platform_has(enum cc_attr attr) { @@ -84,7 +85,33 @@ bool cc_platform_has(enum cc_attr attr) } EXPORT_SYMBOL_GPL(cc_platform_has); =20 +u64 cc_mkenc(u64 val) +{ + switch (vendor) { + case CC_VENDOR_AMD: + return val | cc_mask; + default: + return val; + } +} + +u64 cc_mkdec(u64 val) +{ + switch (vendor) { + case CC_VENDOR_AMD: + return val & ~cc_mask; + default: + return val; + } +} +EXPORT_SYMBOL_GPL(cc_mkdec); + __init void cc_set_vendor(enum cc_vendor v) { vendor =3D v; } + +__init void cc_set_mask(u64 mask) +{ + cc_mask =3D mask; +} diff --git a/arch/x86/include/asm/coco.h b/arch/x86/include/asm/coco.h index e49f9ddb6ae6..3d98c3a60d34 100644 --- a/arch/x86/include/asm/coco.h +++ b/arch/x86/include/asm/coco.h @@ -2,6 +2,8 @@ #ifndef _ASM_X86_COCO_H #define _ASM_X86_COCO_H =20 +#include + enum cc_vendor { CC_VENDOR_NONE, CC_VENDOR_AMD, @@ -10,5 +12,21 @@ enum cc_vendor { }; =20 void cc_set_vendor(enum cc_vendor v); +void cc_set_mask(u64 mask); + +#ifdef CONFIG_ARCH_HAS_CC_PLATFORM +u64 cc_mkenc(u64 val); +u64 cc_mkdec(u64 val); +#else +static inline u64 cc_mkenc(u64 val) +{ + return val; +} + +static inline u64 cc_mkdec(u64 val) +{ + return val; +} +#endif =20 #endif /* _ASM_X86_COCO_H */ diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 8a9432fb3802..62ab07e24aef 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -15,17 +15,12 @@ cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))) \ : (prot)) =20 -/* - * Macros to add or remove encryption attribute - */ -#define pgprot_encrypted(prot) __pgprot(__sme_set(pgprot_val(prot))) -#define pgprot_decrypted(prot) __pgprot(__sme_clr(pgprot_val(prot))) - #ifndef __ASSEMBLY__ #include #include #include #include +#include #include #include =20 @@ -38,6 +33,12 @@ void ptdump_walk_pgd_level_debugfs(struct seq_file *m, s= truct mm_struct *mm, void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); =20 +/* + * Macros to add or remove encryption attribute + */ +#define pgprot_encrypted(prot) __pgprot(cc_mkenc(pgprot_val(prot))) +#define pgprot_decrypted(prot) __pgprot(cc_mkdec(pgprot_val(prot))) + #ifdef CONFIG_DEBUG_WX #define debug_checkwx() ptdump_walk_pgd_level_checkwx() #define debug_checkwx_user() ptdump_walk_user_pgd_level_checkwx() diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_i= dentity.c index 06314ae3998e..b43bc24d2bb6 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -604,5 +604,6 @@ void __init sme_enable(struct boot_params *bp) if (sme_me_mask) { physical_mask &=3D ~sme_me_mask; cc_set_vendor(CC_VENDOR_AMD); + cc_set_mask(sme_me_mask); } } diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index b4072115c8ef..af77dbfd143c 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1990,6 +1990,7 @@ int set_memory_global(unsigned long addr, int numpage= s) static int __set_memory_enc_pgtable(unsigned long addr, int numpages, bool= enc) { struct cpa_data cpa; + pgprot_t empty =3D __pgprot(0); int ret; =20 /* Should not be working on unaligned addresses */ @@ -1999,8 +2000,8 @@ static int __set_memory_enc_pgtable(unsigned long add= r, int numpages, bool enc) memset(&cpa, 0, sizeof(cpa)); cpa.vaddr =3D &addr; cpa.numpages =3D numpages; - cpa.mask_set =3D enc ? __pgprot(_PAGE_ENC) : __pgprot(0); - cpa.mask_clr =3D enc ? __pgprot(0) : __pgprot(_PAGE_ENC); + cpa.mask_set =3D enc ? pgprot_encrypted(empty) : pgprot_decrypted(empty); + cpa.mask_clr =3D enc ? pgprot_decrypted(empty) : pgprot_encrypted(empty); cpa.pgd =3D init_mm.pgd; =20 /* Must avoid aliasing mappings in the highmem code */ --=20 2.34.1 From nobody Thu Jun 25 05:47:51 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0661CC433EF for ; Wed, 23 Feb 2022 04:35:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237587AbiBWEgT (ORCPT ); Tue, 22 Feb 2022 23:36:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230322AbiBWEgR (ORCPT ); Tue, 22 Feb 2022 23:36:17 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2070.outbound.protection.outlook.com [40.107.243.70]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C02B51321; Tue, 22 Feb 2022 20:35:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hWnQNcPkc4KFBE1LAJ3yPgkVvWzUmOKvlHNvMzO1I1qtlLLU9YS0pD6GIzRcIdYK1evPJvIfngP1UROMq0GxQcn0WK4PKnNt5NjXcCA+Kj0rZ5uFFyr/ktcAhoT/lCE6bsLBiKSFRP+yJgSLrIkb9XNdRXwoTW2ofEUjzFz1PcO1WdZZzXkZxkzxLVXyMN3AtDdentjdexFpu8vjRUzV3ltcuL+kECweAUrPoHIvM4hu0Sl6b0bdvdwvzlcjrhXI5j2SCmacI5FGZohvzaNl4QTwZqqOQCRibI1auys1Z+1UgyJlm5aTlaLeRYp4knONgbEHwsxRlxcafkFSbGVp1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9U5rD0hb1Z8Ln2RGlpgoLf7gHNc6ts0MWLIZEKws//M=; b=RJCDJY96x5741cSfeNgNVCo3Z34R2D844AcWkmmJO2SERYO/iJr3hDN5f/PTVG6lLPTKQbc6MXtMk6yyvexDcgLC6OzTr6DaPStiWgdf1SyNb29HlhI5LvkRdaSro24HZh1+sDv37nI/bDkFGnMWvc+Rbd7bs+UqirlBSw4b7EMeFx5taxqXWGga8CP3iS7eA7B7pcvL0hF/4Zo9Q1MVkcdBe3XeC+2YYC5Duq0Nd1DcYhtn8oUJDE159i2e4rVGJlM8A7rDA4hMesp6v3JvXwHios7bqQ1lA/e9N29N95Myw0W/+/WK6OHTh6w19oDrxlRnGeg6Vu4i+kiGlJZKKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9U5rD0hb1Z8Ln2RGlpgoLf7gHNc6ts0MWLIZEKws//M=; b=PcKKm2ufLQwemof50mZOu3LXJrv77Qcy1W9vNU3ed3LFVeuTnzu1LOxC8HMMjsOVLGG4cDBq3/lepjR524ePlp3fs3EkFTQXKj5zm0OWVQneuuZPIN8dnbJn8qHk7X1SfAnssy5vxHStI8F5yILlCrOF6Az5Ehy+FLctFo6EwDQ= Received: from DM6PR14CA0049.namprd14.prod.outlook.com (2603:10b6:5:18f::26) by CY4PR12MB1480.namprd12.prod.outlook.com (2603:10b6:910:f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.24; Wed, 23 Feb 2022 04:35:47 +0000 Received: from DM6NAM11FT006.eop-nam11.prod.protection.outlook.com (2603:10b6:5:18f:cafe::45) by DM6PR14CA0049.outlook.office365.com (2603:10b6:5:18f::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21 via Frontend Transport; Wed, 23 Feb 2022 04:35:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT006.mail.protection.outlook.com (10.13.173.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5017.22 via Frontend Transport; Wed, 23 Feb 2022 04:35:46 +0000 Received: from sbrijesh-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Tue, 22 Feb 2022 22:35:44 -0600 From: Brijesh Singh To: , , , CC: Thomas Gleixner , Ingo Molnar , Joerg Roedel , Tom Lendacky , "H. Peter Anvin" , Paolo Bonzini , "Sean Christopherson" , Andy Lutomirski , "Dave Hansen" , Peter Gonda , "Peter Zijlstra" , David Rientjes , Borislav Petkov , Michael Roth , "Kirill A . Shutemov" , Andi Kleen , Brijesh Singh Subject: [PATCH] x86/mm/cpa: Generalize __set_memory_enc_pgtable() Date: Tue, 22 Feb 2022 22:35:28 -0600 Message-ID: <20220223043528.2093214-1-brijesh.singh@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> References: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 90a0bf09-cd3b-470e-65c1-08d9f685f601 X-MS-TrafficTypeDiagnostic: CY4PR12MB1480:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +/IBxH6JMWh6jr29k7hPjzuPI+IDOuAqn8QIbM4JAZHgfLkS+JRGjnLM4/+4XYs1p+TdCSdETxxxUbBvPgJqHHne40ys8xCUiwiHZCxFrxoVZr55Q7SKo1fMXDpQip0DYB04+QGwX2zmAv7GAcKPhwcbjdHTavJbSjE7sf0F3lvCzGqbkhMP3EYHvCM9kwmoTWNsfBI6bewMXI+2JMoXfBDrfdl4e5ehARI2pI7UjTs40ITUReclgH8xkDkw6icsAl4oVF/aSuVKqQUIPZZ6w54gLuEkjTNHQCl2LXDy4riu9wUkcRtRbeVVqu/vHUwGYyPtY3EH6koahpwZ9JiFD96QN8CZdVG2vrvZeUq9ycfxCVcTuYR2n9K4ZUd4WNQSdcCVKBeLPS/00fDyZJTbAsnaWEanehM/MRb4ARUSQw1tAZAxfC8aJJEojqChbvoBoFmQlVKr/O5WZLjMRVYqkoPRTs/1SvRqPpEo7WDvK3amiCcP3b0uK8v2zYRjs+kyj6WR+9QrD+BpPEpmxKu1dBgwcaS0Cd8kPVUxLVpQasoecPtct8CwBsnfK0OqxhRF8FA6KHScXCXTJf6JY5Ap9PmZwF5n6vNM9GDs6mBB7m3/3k61MLSWQQK8+GM4jSOvO5jfG2Eb0XtzJTNUk5oOivzGaZcaV6kq7SbPClnywWBcoss0iK5naDCqRxsDntAhbg/WgCIDdId8P6AI1zsEb5fyLNslrrDSgcE9ZtwalYC6O7Tpo5hs2ja/iMQIDdZJxgi+X5md1LF6L8wmh4/5oYvPbF2K24o3FY9mXkvqTAU= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(26005)(2906002)(4326008)(70206006)(70586007)(44832011)(356005)(81166007)(1076003)(7416002)(8936002)(8676002)(54906003)(2616005)(186003)(110136005)(316002)(36756003)(5660300002)(40460700003)(426003)(86362001)(47076005)(36860700001)(7696005)(83380400001)(82310400004)(336012)(966005)(16526019)(6666004)(508600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2022 04:35:46.6912 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 90a0bf09-cd3b-470e-65c1-08d9f685f601 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT006.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1480 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The kernel provides infrastructure to set or clear the encryption mask from the pages for AMD SEV, but TDX requires few tweaks. - TDX and SEV have different requirements to the cache and tlb flushing. - TDX has own routine to notify VMM about page encryption status change. Modify __set_memory_enc_pgtable() and make it flexible enough to cover both AMD SEV and Intel TDX. The AMD-specific behavior is isolated in callback under x86_platform.cc. TDX will provide own version of the callbacks. Signed-off-by: Brijesh Singh Reviewed-by: Tom Lendacky --- Depends on Krill's CC cleanup https://lore.kernel.org/all/20220222185740.26228-1-kirill.shutemov@linux.in= tel.com/ arch/x86/include/asm/set_memory.h | 1 - arch/x86/include/asm/x86_init.h | 21 +++++++++ arch/x86/mm/mem_encrypt_amd.c | 75 ++++++++++++++++++++++--------- arch/x86/mm/pat/set_memory.c | 20 +++++---- 4 files changed, 85 insertions(+), 32 deletions(-) diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_m= emory.h index ff0f2d90338a..ce8dd215f5b3 100644 --- a/arch/x86/include/asm/set_memory.h +++ b/arch/x86/include/asm/set_memory.h @@ -84,7 +84,6 @@ int set_pages_rw(struct page *page, int numpages); int set_direct_map_invalid_noflush(struct page *page); int set_direct_map_default_noflush(struct page *page); bool kernel_page_present(struct page *page); -void notify_range_enc_status_changed(unsigned long vaddr, int npages, bool= enc); =20 extern int kernel_set_to_readonly; =20 diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_ini= t.h index 22b7412c08f6..dce92e2cb9e1 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -141,6 +141,26 @@ struct x86_init_acpi { void (*reduced_hw_early_init)(void); }; =20 +/** + * struct x86_cc_runtime - Functions used by misc guest incarnations like = SEV, TDX, etc. + * + * @enc_status_change_prepare Notify HV before the encryption status of a = range + * is changed. + * + * @enc_status_change_finish Notify HV after the encryption status of a ra= nge + * is changed. + * + * @enc_tlb_flush_required Flush the TLB before changing the encryption st= atus. + * + * @enc_cache_flush_required Flush the caches before changing the encrypti= on status. + */ +struct x86_cc_runtime { + void (*enc_status_change_prepare)(unsigned long vaddr, int npages, bool e= nc); + void (*enc_status_change_finish)(unsigned long vaddr, int npages, bool en= c); + bool (*enc_tlb_flush_required)(bool enc); + bool (*enc_cache_flush_required)(void); +}; + /** * struct x86_init_ops - functions for platform specific setup * @@ -287,6 +307,7 @@ struct x86_platform_ops { struct x86_legacy_features legacy; void (*set_legacy_features)(void); struct x86_hyper_runtime hyper; + const struct x86_cc_runtime *cc; }; =20 struct x86_apic_ops { diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c index 2b2d018ea345..22b86af5edf1 100644 --- a/arch/x86/mm/mem_encrypt_amd.c +++ b/arch/x86/mm/mem_encrypt_amd.c @@ -177,25 +177,6 @@ void __init sme_map_bootdata(char *real_mode_data) __sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, true); } =20 -void __init sme_early_init(void) -{ - unsigned int i; - - if (!sme_me_mask) - return; - - early_pmd_flags =3D __sme_set(early_pmd_flags); - - __supported_pte_mask =3D __sme_set(__supported_pte_mask); - - /* Update the protection map with memory encryption mask */ - for (i =3D 0; i < ARRAY_SIZE(protection_map); i++) - protection_map[i] =3D pgprot_encrypted(protection_map[i]); - - if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) - swiotlb_force =3D SWIOTLB_FORCE; -} - void __init sev_setup_arch(void) { phys_addr_t total_mem =3D memblock_phys_mem_size(); @@ -256,7 +237,17 @@ static unsigned long pg_level_to_pfn(int level, pte_t = *kpte, pgprot_t *ret_prot) return pfn; } =20 -void notify_range_enc_status_changed(unsigned long vaddr, int npages, bool= enc) +static bool amd_enc_tlb_flush_required(bool enc) +{ + return true; +} + +static bool amd_enc_cache_flush_required(void) +{ + return !this_cpu_has(X86_FEATURE_SME_COHERENT); +} + +static void enc_dec_hypercall(unsigned long vaddr, int npages, bool enc) { #ifdef CONFIG_PARAVIRT unsigned long sz =3D npages << PAGE_SHIFT; @@ -287,6 +278,18 @@ void notify_range_enc_status_changed(unsigned long vad= dr, int npages, bool enc) #endif } =20 +static void amd_enc_status_change_prepare(unsigned long vaddr, int npages,= bool enc) +{ +} + +static void amd_enc_status_change_finish(unsigned long vaddr, int npages, = bool enc) +{ + if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) + return; + + enc_dec_hypercall(vaddr, npages, enc); +} + static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc) { pgprot_t old_prot, new_prot; @@ -392,7 +395,7 @@ static int __init early_set_memory_enc_dec(unsigned lon= g vaddr, =20 ret =3D 0; =20 - notify_range_enc_status_changed(start, PAGE_ALIGN(size) >> PAGE_SHIFT, en= c); + early_set_mem_enc_dec_hypercall(start, PAGE_ALIGN(size) >> PAGE_SHIFT, en= c); out: __flush_tlb_all(); return ret; @@ -410,7 +413,35 @@ int __init early_set_memory_encrypted(unsigned long va= ddr, unsigned long size) =20 void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npage= s, bool enc) { - notify_range_enc_status_changed(vaddr, npages, enc); + enc_dec_hypercall(vaddr, npages, enc); +} + +static const struct x86_cc_runtime amd_cc_runtime =3D { + .enc_status_change_prepare =3D amd_enc_status_change_prepare, + .enc_status_change_finish =3D amd_enc_status_change_finish, + .enc_tlb_flush_required =3D amd_enc_tlb_flush_required, + .enc_cache_flush_required =3D amd_enc_cache_flush_required, +}; + +void __init sme_early_init(void) +{ + unsigned int i; + + if (!sme_me_mask) + return; + + early_pmd_flags =3D __sme_set(early_pmd_flags); + + __supported_pte_mask =3D __sme_set(__supported_pte_mask); + + /* Update the protection map with memory encryption mask */ + for (i =3D 0; i < ARRAY_SIZE(protection_map); i++) + protection_map[i] =3D pgprot_encrypted(protection_map[i]); + + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) + swiotlb_force =3D SWIOTLB_FORCE; + + x86_platform.cc =3D &amd_cc_runtime; } =20 void __init mem_encrypt_free_decrypted_mem(void) diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index af77dbfd143c..4de2a7509039 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1997,6 +1997,8 @@ static int __set_memory_enc_pgtable(unsigned long add= r, int numpages, bool enc) if (WARN_ONCE(addr & ~PAGE_MASK, "misaligned address: %#lx\n", addr)) addr &=3D PAGE_MASK; =20 + BUG_ON(!x86_platform.cc); + memset(&cpa, 0, sizeof(cpa)); cpa.vaddr =3D &addr; cpa.numpages =3D numpages; @@ -2008,10 +2010,12 @@ static int __set_memory_enc_pgtable(unsigned long a= ddr, int numpages, bool enc) kmap_flush_unused(); vm_unmap_aliases(); =20 - /* - * Before changing the encryption attribute, we need to flush caches. - */ - cpa_flush(&cpa, !this_cpu_has(X86_FEATURE_SME_COHERENT)); + /* Flush the caches as needed before changing the encryption attribute. */ + if (x86_platform.cc->enc_tlb_flush_required(enc)) + cpa_flush(&cpa, x86_platform.cc->enc_cache_flush_required()); + + /* Notify hypervisor that we are about to set/clr encryption attribute. */ + x86_platform.cc->enc_status_change_prepare(addr, numpages, enc); =20 ret =3D __change_page_attr_set_clr(&cpa, 1); =20 @@ -2024,11 +2028,9 @@ static int __set_memory_enc_pgtable(unsigned long ad= dr, int numpages, bool enc) */ cpa_flush(&cpa, 0); =20 - /* - * Notify hypervisor that a given memory range is mapped encrypted - * or decrypted. - */ - notify_range_enc_status_changed(addr, numpages, enc); + /* Notify hypervisor that we have successfully set/clr encryption attribu= te. */ + if (!ret) + x86_platform.cc->enc_status_change_finish(addr, numpages, enc); =20 return ret; } --=20 2.25.1