From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 751D3C001B1 for ; Tue, 27 Jun 2023 03:29:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230171AbjF0D3D (ORCPT ); Mon, 26 Jun 2023 23:29:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229727AbjF0D2S (ORCPT ); Mon, 26 Jun 2023 23:28:18 -0400 Received: from mail-ot1-x32b.google.com (mail-ot1-x32b.google.com [IPv6:2607:f8b0:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04A3D2106; Mon, 26 Jun 2023 20:22:54 -0700 (PDT) Received: by mail-ot1-x32b.google.com with SMTP id 46e09a7af769-6b74faaac3bso1294473a34.1; Mon, 26 Jun 2023 20:22:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836173; x=1690428173; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PE11yTMAExZ2ZoYVCcn2Tq5He2Vs69778pFMKMEukgk=; b=L/IebApgWhFXHgPqF3us42fzqDADcFJR4vXUnyhDeL+wsoamYd/aEpG2bPbCG1XevS fraPdupxXbgIchbLfBM00pEIOjfm84OFptvDPRrLcSJtNOatnAJqnXKWF213wICpwa4F XD9mK4KbsvwY3v5nthP1o84FxOS9aS9vt/mIZaxDDcsMWhm4GBxUmYrWm/3TW97zQ6/M lA06WcxUQtIseHh/WjUjwvtE6iXBWs5w0oTPHCQXLlRIwmVdp984G+RUxhEZQgBc2qus qSlCmdCmLr7ZV5tDFgQU0EERQXKBW4O6Ss+agZJ36A7vObM6IPd4NwmYv6yrGNDUp5bN M1Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836173; x=1690428173; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PE11yTMAExZ2ZoYVCcn2Tq5He2Vs69778pFMKMEukgk=; b=fWR/nWxqQI6DY3iTxh1NiaJB9hKA3jKOnhkDdG9igoCLMwm/jLSf9vHXBfWMkOa/KW 0fANLvKHIF7h6lxkiqYiT8QFiv7TDvx1NiWFW+6/Yt+2N6FLnaQiWo20zf83jQzkBt1p svoWgJ/HocaYgzHLxe4abkAWQCTqjZLSlqeTTiSz5psOfSR8q0g4zJYKHw93Pn3X0+hU tdqGW+b1HflGomLWKCZW+Q8C3YOPsrEBMZ59vlZ2ZfheTScA/hRsN10kNIpVo+21yrIr u0SnynbI1gpHyJHFeF8GHyDWF4YfcccXYel2K1hQXM9iH7b2sFHS5D94OwOBIPPOUClV bhzw== X-Gm-Message-State: AC+VfDwBWTjACfReW83vG6ycY3WKlGOZA/Kt1C181fV6R6/56jn6slHu 7VeeCA2GzdEghxcxTYhJnZo= X-Google-Smtp-Source: ACHHUZ79+MrhScxTTNGAmqrE/xOgycwWaeRWadcQYj3WROzVjuuQ9Yk4HBBqB4EzDr8QBu1G/sBHuA== X-Received: by 2002:a05:6808:2022:b0:3a1:b47d:9296 with SMTP id q34-20020a056808202200b003a1b47d9296mr13366617oiw.17.1687836173245; Mon, 26 Jun 2023 20:22:53 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.22.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:22:52 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 1/9] x86/hyperv: Add sev-snp enlightened guest static key Date: Mon, 26 Jun 2023 23:22:39 -0400 Message-Id: <20230627032248.2170007-2-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan Introduce static key isolation_type_en_snp for enlightened sev-snp guest check. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/ivm.c | 11 +++++++++++ arch/x86/include/asm/mshyperv.h | 3 +++ arch/x86/kernel/cpu/mshyperv.c | 9 +++++++-- drivers/hv/hv_common.c | 6 ++++++ include/asm-generic/mshyperv.h | 12 +++++++++--- 5 files changed, 36 insertions(+), 5 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index cc92388b7a99..5d3ee3124e00 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -409,3 +409,14 @@ bool hv_isolation_type_snp(void) { return static_branch_unlikely(&isolation_type_snp); } + +DEFINE_STATIC_KEY_FALSE(isolation_type_en_snp); +/* + * hv_isolation_type_en_snp - Check system runs in the AMD SEV-SNP based + * isolation enlightened VM. + */ +bool hv_isolation_type_en_snp(void) +{ + return static_branch_unlikely(&isolation_type_en_snp); +} + diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyper= v.h index 49bb4f2bd300..31c476f4e656 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -26,6 +26,7 @@ union hv_ghcb; =20 DECLARE_STATIC_KEY_FALSE(isolation_type_snp); +DECLARE_STATIC_KEY_FALSE(isolation_type_en_snp); =20 typedef int (*hyperv_fill_flush_list_func)( struct hv_guest_mapping_flush_list *flush, @@ -45,6 +46,8 @@ extern void *hv_hypercall_pg; =20 extern u64 hv_current_partition_id; =20 +extern bool hv_isolation_type_en_snp(void); + extern union hv_ghcb * __percpu *hv_ghcb_pg; =20 int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index c7969e806c64..5398fb2f4d39 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -402,8 +402,12 @@ static void __init ms_hyperv_init_platform(void) pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n", ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); =20 - if (hv_get_isolation_type() =3D=3D HV_ISOLATION_TYPE_SNP) + + if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + static_branch_enable(&isolation_type_en_snp); + } else if (hv_get_isolation_type() =3D=3D HV_ISOLATION_TYPE_SNP) { static_branch_enable(&isolation_type_snp); + } } =20 if (hv_max_functions_eax >=3D HYPERV_CPUID_NESTED_FEATURES) { @@ -473,7 +477,8 @@ static void __init ms_hyperv_init_platform(void) =20 #if IS_ENABLED(CONFIG_HYPERV) if ((hv_get_isolation_type() =3D=3D HV_ISOLATION_TYPE_VBS) || - (hv_get_isolation_type() =3D=3D HV_ISOLATION_TYPE_SNP)) + ((hv_get_isolation_type() =3D=3D HV_ISOLATION_TYPE_SNP) && + ms_hyperv.paravisor_present)) hv_vtom_init(); /* * Setup the hook to get control post apic initialization. diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index 542a1d53b303..4b4aa53c34c2 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -502,6 +502,12 @@ bool __weak hv_isolation_type_snp(void) } EXPORT_SYMBOL_GPL(hv_isolation_type_snp); =20 +bool __weak hv_isolation_type_en_snp(void) +{ + return false; +} +EXPORT_SYMBOL_GPL(hv_isolation_type_en_snp); + void __weak hv_setup_vmbus_handler(void (*handler)(void)) { } diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index 402a8c1c202d..6b5c41f90398 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -36,15 +36,21 @@ struct ms_hyperv_info { u32 nested_features; u32 max_vp_index; u32 max_lp_index; - u32 isolation_config_a; + union { + u32 isolation_config_a; + struct { + u32 paravisor_present : 1; + u32 reserved_a1 : 31; + }; + }; union { u32 isolation_config_b; struct { u32 cvm_type : 4; - u32 reserved1 : 1; + u32 reserved_b1 : 1; u32 shared_gpa_boundary_active : 1; u32 shared_gpa_boundary_bits : 6; - u32 reserved2 : 20; + u32 reserved_b2 : 20; }; }; u64 shared_gpa_boundary; --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFA0AEB64DC for ; Tue, 27 Jun 2023 03:29:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230223AbjF0D3I (ORCPT ); Mon, 26 Jun 2023 23:29:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229738AbjF0D2U (ORCPT ); Mon, 26 Jun 2023 23:28:20 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B2BF2107; Mon, 26 Jun 2023 20:22:55 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-544c0d768b9so3289183a12.0; Mon, 26 Jun 2023 20:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836175; x=1690428175; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V1z877s9cdWqHPRWW2XoomxLF4Ko1N0KieO7s3qiybM=; b=QAabkL3Ofzm9XDTI/IWFugYTFhTKChrsC0ueRyh4N6Olfut7c8UBUesPAqNxSrg0ee bCAlVUcJ7r7ceeC0x4le+45nMSHUdDm/vPJuy9Slh2uFzcc3LTea076asMc+ZyQ8PBz+ iRdRWZO58T1CUIaYKbLP+iZJwQJETM42r256fnZ/qJHyE3ZbkmZU2bdJ++Gz5DDsb1/d 2qCEPs9M6LJD9QiSs58f31BX3F/yoghTFbp5pETDlnXDYJusFH5D9bgH5GCxZHkQ9GWx 6vh355jexv4Jncp9qChheDeEPKOWFNFyWWUY87vpdCj3CcfHSWtTL5IGGw8D4WKPzjgk zBPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836175; x=1690428175; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V1z877s9cdWqHPRWW2XoomxLF4Ko1N0KieO7s3qiybM=; b=hEXaccSZMlChT5XyY9RFrgUD8hRVfs+2gEkz/dFPwNqzxAUyYCPAtaBgP3YtuWajSn hUKC5MacNRMqWPyfN4C5hiCvLrq4gfpPx6vxxuKWEdnVP/nN2nFJHeP8SWbN0okK12W8 FQLU5jpw7BYl6V/iPw/IxvderXBeYq3ysSgqAJkFAvJEze7AVPEP/MwS3e6WEN2patso tAUITaAoFI4d+qOVtz/kU9c1zkQ2yXS8clcFp/Bhr3akpP6LYLK/LNp1wQJ5dZcdt/68 C8kOOcwsbfXDwXwZQSJnEd4ZKJDIAsps6qD6iXM5dMb4hLECQ3i1xJVbUeUrEhv/6bOD aweQ== X-Gm-Message-State: AC+VfDxinsnLmnIbyd001rfdZlrmeancI/2SmM5nQLKoX2O2lWytzWNB rommwM9DMCWTwR11KxSXfYasq75cLeBGpQ== X-Google-Smtp-Source: ACHHUZ7iTDnZ5NE1qKX2TZOw6G9hJyTpCMSyOiyqniONvH4HTaMyPh3bzrvV4qXmyrvrApT1rXHCLQ== X-Received: by 2002:a17:90a:550:b0:258:b651:4f80 with SMTP id h16-20020a17090a055000b00258b6514f80mr28854208pjf.36.1687836174790; Mon, 26 Jun 2023 20:22:54 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.22.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:22:54 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 2/9] x86/hyperv: Set Virtual Trust Level in VMBus init message Date: Mon, 26 Jun 2023 23:22:40 -0400 Message-Id: <20230627032248.2170007-3-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan SEV-SNP guest provides vtl(Virtual Trust Level) and get it from Hyper-V hvcall via register hvcall HVCALL_ GET_VP_REGISTERS. During initialization of VMBus, vtl needs to be set in the VMBus init message. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/hv_init.c | 36 ++++++++++++++++++++++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 7 ++++++ drivers/hv/connection.c | 1 + include/asm-generic/mshyperv.h | 1 + include/linux/hyperv.h | 4 ++-- 5 files changed, 47 insertions(+), 2 deletions(-) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 6c04b52f139b..1ba367a9686e 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -378,6 +378,40 @@ static void __init hv_get_partition_id(void) local_irq_restore(flags); } =20 +static u8 __init get_vtl(void) +{ + u64 control =3D HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS; + struct hv_get_vp_registers_input *input; + struct hv_get_vp_registers_output *output; + u64 vtl =3D 0; + u64 ret; + unsigned long flags; + + local_irq_save(flags); + input =3D *this_cpu_ptr(hyperv_pcpu_input_arg); + output =3D (struct hv_get_vp_registers_output *)input; + if (!input) { + local_irq_restore(flags); + goto done; + } + + memset(input, 0, struct_size(input, element, 1)); + input->header.partitionid =3D HV_PARTITION_ID_SELF; + input->header.vpindex =3D HV_VP_INDEX_SELF; + input->header.inputvtl =3D 0; + input->element[0].name0 =3D HV_X64_REGISTER_VSM_VP_STATUS; + + ret =3D hv_do_hypercall(control, input, output); + if (hv_result_success(ret)) + vtl =3D output->as64.low & HV_X64_VTL_MASK; + else + pr_err("Hyper-V: failed to get VTL! %lld", ret); + local_irq_restore(flags); + +done: + return vtl; +} + /* * This function is to be invoked early in the boot sequence after the * hypervisor has been detected. @@ -506,6 +540,8 @@ void __init hyperv_init(void) /* Query the VMs extended capability once, so that it can be cached. */ hv_query_ext_cap(0); =20 + /* Find the VTL */ + ms_hyperv.vtl =3D get_vtl(); return; =20 clean_guest_os_id: diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hype= rv-tlfs.h index cea95dcd27c2..4bf0b315b0ce 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -301,6 +301,13 @@ enum hv_isolation_type { #define HV_X64_MSR_TIME_REF_COUNT HV_REGISTER_TIME_REF_COUNT #define HV_X64_MSR_REFERENCE_TSC HV_REGISTER_REFERENCE_TSC =20 +/* + * Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and + * there is not associated MSR address. + */ +#define HV_X64_REGISTER_VSM_VP_STATUS 0x000D0003 +#define HV_X64_VTL_MASK GENMASK(3, 0) + /* Hyper-V memory host visibility */ enum hv_mem_host_visibility { VMBUS_PAGE_NOT_VISIBLE =3D 0, diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c index 5978e9dbc286..02b54f85dc60 100644 --- a/drivers/hv/connection.c +++ b/drivers/hv/connection.c @@ -98,6 +98,7 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo = *msginfo, u32 version) */ if (version >=3D VERSION_WIN10_V5) { msg->msg_sint =3D VMBUS_MESSAGE_SINT; + msg->msg_vtl =3D ms_hyperv.vtl; vmbus_connection.msg_conn_id =3D VMBUS_MESSAGE_CONNECTION_ID_4; } else { msg->interrupt_page =3D virt_to_phys(vmbus_connection.int_page); diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index 6b5c41f90398..f73a044ecaa7 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -54,6 +54,7 @@ struct ms_hyperv_info { }; }; u64 shared_gpa_boundary; + u8 vtl; }; extern struct ms_hyperv_info ms_hyperv; extern bool hv_nested; diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h index bfbc37ce223b..1f2bfec4abde 100644 --- a/include/linux/hyperv.h +++ b/include/linux/hyperv.h @@ -665,8 +665,8 @@ struct vmbus_channel_initiate_contact { u64 interrupt_page; struct { u8 msg_sint; - u8 padding1[3]; - u32 padding2; + u8 msg_vtl; + u8 reserved[6]; }; }; u64 monitor_page1; --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAC74EB64DD for ; Tue, 27 Jun 2023 03:29:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230247AbjF0D31 (ORCPT ); Mon, 26 Jun 2023 23:29:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229501AbjF0D2U (ORCPT ); Mon, 26 Jun 2023 23:28:20 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6CB92109; Mon, 26 Jun 2023 20:22:56 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-54fbcfe65caso3376673a12.1; Mon, 26 Jun 2023 20:22:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836176; x=1690428176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JlZUHIVX+pLdqy3tdQrgw8rCrC0s3HRsxT6pRIRlNIE=; b=DB0Z17X6Lpl4T2saaSH+rXYHxawtqP5dN2yw0s7VS0VB99q80xDbUEkg4AgWUSp0iK sPVnb26Pf7zvgT1qOIje7gXwd9JGrquTTCLwmr1qc4b0XvCZeLqkW0RxOWmROKrCBLJv bFaIxteNjNfoQV+A1fRL4dRRSDDQek8OkHjYVk3Eo9Wc6SVqgrBxZQUZG9euvk/ny+7l WbxOANGYP0nWiPQUFKsGMieNM85vcXr1QUmyRxiUxOmeFp78gwNtVDI9fGw77CskuUkl aLNuWxAwMaME/ttN6S1Xz+r+0sb1/mRHSCu6TmAh3isumIwCo34SySN5cYPvb4zQi6LA cEqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836176; x=1690428176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JlZUHIVX+pLdqy3tdQrgw8rCrC0s3HRsxT6pRIRlNIE=; b=ZZbzzynPRMvUEUd8zFu/YB71QU8ChsCajxw80Xx0xK6vueHrAQafLK4/J0Jjr+Cbhs f6eUgGP2dol3LZ54IPKwJAAVgsIFeLmOfBx+mPxjhWScJGFQkXXm022BpXNHdgbIhWQ+ cilifdiVTeBSu5Jv5+v4CgBW6s5V5l6DHYuTbYz6H2/SiiCXWhxpn0dfeMCB44zBmb0Y I6rwoe5M2VkyLHFSemAst3t6TibgtTDd4xH43mVz+sXnWfPbYo3/hf/OTLQZfT8zD/HR sIPjsLZWe809XHlnM152cyh4kEixH4EhPhszvJuYHjOSWxjolBlr4NekIeCQ1Ky8/CJa zNQw== X-Gm-Message-State: AC+VfDw9FCxcDd+KAdmnqZyRY3ScGAflOjC3heHi3YCQBUMjuV9U2Gx9 WZ7C8TNxoTfRdzflk+ZNGZc= X-Google-Smtp-Source: ACHHUZ77gvLwNNqt8tVeeNeOyVQr0v27R+P+nvuNlP7G9bF/btL8Y8w4fKE7CJZlnG6+st9m8HiJFA== X-Received: by 2002:a17:90a:4e:b0:261:a75:928d with SMTP id 14-20020a17090a004e00b002610a75928dmr22135434pjb.18.1687836176324; Mon, 26 Jun 2023 20:22:56 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.22.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:22:55 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 3/9] x86/hyperv: Mark Hyper-V vp assist page unencrypted in SEV-SNP enlightened guest Date: Mon, 26 Jun 2023 23:22:41 -0400 Message-Id: <20230627032248.2170007-4-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan hv vp assist page needs to be shared between SEV-SNP guest and Hyper-V. So mark the page unencrypted in the SEV-SNP guest. Signed-off-by: Tianyu Lan Reviewed-by: Michael Kelley --- arch/x86/hyperv/hv_init.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 1ba367a9686e..b004370d3b01 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -106,8 +107,21 @@ static int hv_cpu_init(unsigned int cpu) * in hv_cpu_die(), otherwise a CPU may not be stopped in the * case of CPU offlining and the VM will hang. */ - if (!*hvp) + if (!*hvp) { *hvp =3D __vmalloc(PAGE_SIZE, GFP_KERNEL | __GFP_ZERO); + + /* + * Hyper-V should never specify a VM that is a Confidential + * VM and also running in the root partition. Root partition + * is blocked to run in Confidential VM. So only decrypt assist + * page in non-root partition here. + */ + if (*hvp && hv_isolation_type_en_snp()) { + WARN_ON_ONCE(set_memory_decrypted((unsigned long)(*hvp), 1)); + memset(*hvp, 0, PAGE_SIZE); + } + } + if (*hvp) msr.pfn =3D vmalloc_to_pfn(*hvp); =20 --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FD97EB64DC for ; Tue, 27 Jun 2023 03:29:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229763AbjF0D3Q (ORCPT ); Mon, 26 Jun 2023 23:29:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229742AbjF0D2U (ORCPT ); Mon, 26 Jun 2023 23:28:20 -0400 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 675942108; Mon, 26 Jun 2023 20:22:58 -0700 (PDT) Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-25ec175b86bso2944637a91.1; Mon, 26 Jun 2023 20:22:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836178; x=1690428178; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fCJn92OFxJwU4IHGtLWMi2EWhzMeBfc+oQquYxaIu1M=; b=JbFLkeTaVejKuX3CeuGv44gh+qkdefxVqrHJOKNTDrxVRruZIyb0M4GfupBU7MRgwl 1QfqHrNFWnAlAX2c5vvQb8CfQ5RUudBYAnrzYxc3UQXsEwEMgi0RUWDwHVy0VABtOKzJ gbPlb/iif6JFZDpEhAtRTb9N5Tt2d0HR+RfKjSqDxf+JnVvRBjWqbhaOBBLlJ71VrfPk 6AdarjmdqyR9xzmP60NECzv9U31LlPl+oShpHXyuo+sD0sAEmJdm9kH1mmMLbZ/7dlwa TITD2U9Vn+cNuhwpLsKNN9W75FiETCk5KH5caVuvoPwkLasWTxo6KJ0l3dhCITCJhW8h DpyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836178; x=1690428178; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fCJn92OFxJwU4IHGtLWMi2EWhzMeBfc+oQquYxaIu1M=; b=bxyf/xcAFvF2H/2yffRdBnGLEff95l/d5hoPYUzA9wENdHJgqt53FbXJy6YyNuYZnZ 1yIEzbArOapenrPdNRvjof+0ki9XJdHRaUmYhYT6PUXDL/EHTpux5GFJqyhM49xbv/TZ EVrOOn0eDwvS9Qk5c+IYNys3QdPf0qlRfPpDh0uZt6VQqNWXtuRih9pJ7/smmTKyApFh L2kmqnIBU+SCAcKz+E+a/MBqxsAcp6/esLZX/Tj8yiGpAZYCssK108Ktjd1aYd832iAQ 2TCE+o+UF/6F5pAuaic/sHUwJ9SEHOzqP0B7sI4DWv3zhisQvxtE74B5ffvc5bTLlPPz /XEw== X-Gm-Message-State: AC+VfDyL6h1iIPiq98LUkDECKVjtKlvC/EcnjhXXTl0Blj3oa/LwaFjW 2dWViUsUthid/zsaIeIlyaY= X-Google-Smtp-Source: ACHHUZ6xUaoHETrF0sYVECWCZl94G6tgLgn2yn3bb75M8xNPWyXBvNdjTmpYwLzROpQDxClSo7nrZw== X-Received: by 2002:a17:90a:7042:b0:255:cddf:a0c8 with SMTP id f60-20020a17090a704200b00255cddfa0c8mr29171624pjk.41.1687836177773; Mon, 26 Jun 2023 20:22:57 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.22.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:22:57 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 4/9] drivers: hv: Mark percpu hvcall input arg page unencrypted in SEV-SNP enlightened guest Date: Mon, 26 Jun 2023 23:22:42 -0400 Message-Id: <20230627032248.2170007-5-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan Hypervisor needs to access input arg, VMBus synic event and message pages. Mark these pages unencrypted in the SEV-SNP guest and free them only if they have been marked encrypted successfully. Signed-off-by: Tianyu Lan Reviewed-by: Michael Kelley --- drivers/hv/hv.c | 57 +++++++++++++++++++++++++++++++++++++++--- drivers/hv/hv_common.c | 13 ++++++++++ 2 files changed, 67 insertions(+), 3 deletions(-) diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index de6708dbe0df..ec6e35a0d9bf 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "hyperv_vmbus.h" =20 /* The one and only */ @@ -78,7 +79,7 @@ int hv_post_message(union hv_connection_id connection_id, =20 int hv_synic_alloc(void) { - int cpu; + int cpu, ret =3D -ENOMEM; struct hv_per_cpu_context *hv_cpu; =20 /* @@ -123,26 +124,76 @@ int hv_synic_alloc(void) goto err; } } + + if (hv_isolation_type_en_snp()) { + ret =3D set_memory_decrypted((unsigned long) + hv_cpu->synic_message_page, 1); + if (ret) { + pr_err("Failed to decrypt SYNIC msg page: %d\n", ret); + hv_cpu->synic_message_page =3D NULL; + + /* + * Free the event page here so that hv_synic_free() + * won't later try to re-encrypt it. + */ + free_page((unsigned long)hv_cpu->synic_event_page); + hv_cpu->synic_event_page =3D NULL; + goto err; + } + + ret =3D set_memory_decrypted((unsigned long) + hv_cpu->synic_event_page, 1); + if (ret) { + pr_err("Failed to decrypt SYNIC event page: %d\n", ret); + hv_cpu->synic_event_page =3D NULL; + goto err; + } + + memset(hv_cpu->synic_message_page, 0, PAGE_SIZE); + memset(hv_cpu->synic_event_page, 0, PAGE_SIZE); + } } =20 return 0; + err: /* * Any memory allocations that succeeded will be freed when * the caller cleans up by calling hv_synic_free() */ - return -ENOMEM; + return ret; } =20 =20 void hv_synic_free(void) { - int cpu; + int cpu, ret; =20 for_each_present_cpu(cpu) { struct hv_per_cpu_context *hv_cpu =3D per_cpu_ptr(hv_context.cpu_context, cpu); =20 + /* It's better to leak the page if the encryption fails. */ + if (hv_isolation_type_en_snp()) { + if (hv_cpu->synic_message_page) { + ret =3D set_memory_encrypted((unsigned long) + hv_cpu->synic_message_page, 1); + if (ret) { + pr_err("Failed to encrypt SYNIC msg page: %d\n", ret); + hv_cpu->synic_message_page =3D NULL; + } + } + + if (hv_cpu->synic_event_page) { + ret =3D set_memory_encrypted((unsigned long) + hv_cpu->synic_event_page, 1); + if (ret) { + pr_err("Failed to encrypt SYNIC event page: %d\n", ret); + hv_cpu->synic_event_page =3D NULL; + } + } + } + free_page((unsigned long)hv_cpu->synic_event_page); free_page((unsigned long)hv_cpu->synic_message_page); } diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index 4b4aa53c34c2..2d43ba2bc925 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include =20 @@ -359,6 +360,7 @@ int hv_common_cpu_init(unsigned int cpu) u64 msr_vp_index; gfp_t flags; int pgcount =3D hv_root_partition ? 2 : 1; + int ret; =20 /* hv_cpu_init() can be called with IRQs disabled from hv_resume() */ flags =3D irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL; @@ -378,6 +380,17 @@ int hv_common_cpu_init(unsigned int cpu) outputarg =3D (void **)this_cpu_ptr(hyperv_pcpu_output_arg); *outputarg =3D (char *)(*inputarg) + HV_HYP_PAGE_SIZE; } + + if (hv_isolation_type_en_snp()) { + ret =3D set_memory_decrypted((unsigned long)*inputarg, pgcount); + if (ret) { + kfree(*inputarg); + *inputarg =3D NULL; + return ret; + } + + memset(*inputarg, 0x00, pgcount * PAGE_SIZE); + } } =20 msr_vp_index =3D hv_get_register(HV_REGISTER_VP_INDEX); --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F1E1EB64DD for ; Tue, 27 Jun 2023 03:29:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229835AbjF0D3e (ORCPT ); Mon, 26 Jun 2023 23:29:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229653AbjF0D2V (ORCPT ); Mon, 26 Jun 2023 23:28:21 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B627C2117; Mon, 26 Jun 2023 20:22:59 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id 98e67ed59e1d1-25ec175b86bso2944648a91.1; Mon, 26 Jun 2023 20:22:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836179; x=1690428179; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WavXlIUDaiNvQ7Jhn4eQcl+B7LwqPDg8ntyEa3A0jyM=; b=LSRiUlnVbD1yG2LtgeNbwA5zFxbzqYOZMJQQOSULLaJJLryLZPpcKSTWUqJvZt4Qme OQc7BpsRmxlvTFgL8eFXT26zQTRFpN7CL7Eg2ZApv1ea3T1bM2OwHzOP+qQo/DMiMydl m5k1OTpGQz9kXiIzwZC6JoLBfQ+S4hVOGciYKWnEICaM8jJuWqahZu03anyCH1Vn37fU MLPDgVVy5G9paXDSIQ8wPTof6jMTMmyVPmgeKV8soGX/dB8tEkOWyMj+IiFhVedctgis jEaAe44y9o7UZhiQyYGQshMWn/x2Oe8wl5xg7bTgGHalCuVBxEdtKL7+WjGk9m1vj5FR lrqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836179; x=1690428179; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WavXlIUDaiNvQ7Jhn4eQcl+B7LwqPDg8ntyEa3A0jyM=; b=Yft2M8fpU/CZR2uz/wJU1xx6NVZ5hzyZFAkLFzJ9kZ4q2zpeje+1G9yzxolnSJ9LXw JrMh2cQs/yH2Nh54J0fun595XDsNAmT9iKBhw6Fd4qwefTkSvMZaXt4VYcjU2fTRJZOJ I2sAcpee9e5wN2a/bk1cDfZ/0u6SrIopt6CQyJx0FM5UXl93DhlOcKtUHb2Yw0esDNpy ql8h6O6rH16C46rGmMh36jaCwDQRaXr5mUW5yg5XKgsaVbNsWyq/8K6tyzI8JQ0hcNjo rwUjNzZ0TWqcOvKzNtmBMYgMo0SdK9078LOEUHS3+CUDbQt9bLPMr276r3Vgm2THr+99 zN6Q== X-Gm-Message-State: AC+VfDzUcukr1s652cTi9FhkFcJUbq8XSc+XhfHkDVfbiSOEBLZlpdTJ E1Ths7hfs0OnFbLn8+9rdMw= X-Google-Smtp-Source: ACHHUZ4bHFbhEu7fAz1E4ESsRByJd1Jqm4+f+OnoVMcgMllJUT9JvSYVvsUX0CQFnJ2sh+bC/iHADg== X-Received: by 2002:a17:90a:19ca:b0:25f:20f:2f7d with SMTP id 10-20020a17090a19ca00b0025f020f2f7dmr23774243pjj.2.1687836179114; Mon, 26 Jun 2023 20:22:59 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.22.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:22:58 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 5/9] x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp enlightened guest Date: Mon, 26 Jun 2023 23:22:43 -0400 Message-Id: <20230627032248.2170007-6-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan In sev-snp enlightened guest, Hyper-V hypercall needs to use vmmcall to trigger vmexit and notify hypervisor to handle hypercall request. Signed-off-by: Tianyu Lan --- arch/x86/include/asm/mshyperv.h | 44 ++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyper= v.h index 31c476f4e656..d859d7c5f5e8 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -61,16 +61,25 @@ static inline u64 hv_do_hypercall(u64 control, void *in= put, void *output) u64 hv_status; =20 #ifdef CONFIG_X86_64 - if (!hv_hypercall_pg) - return U64_MAX; + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__("mov %4, %%r8\n" + "vmmcall" + : "=3Da" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input_address) + : "r" (output_address) + : "cc", "memory", "r8", "r9", "r10", "r11"); + } else { + if (!hv_hypercall_pg) + return U64_MAX; =20 - __asm__ __volatile__("mov %4, %%r8\n" - CALL_NOSPEC - : "=3Da" (hv_status), ASM_CALL_CONSTRAINT, - "+c" (control), "+d" (input_address) - : "r" (output_address), - THUNK_TARGET(hv_hypercall_pg) - : "cc", "memory", "r8", "r9", "r10", "r11"); + __asm__ __volatile__("mov %4, %%r8\n" + CALL_NOSPEC + : "=3Da" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input_address) + : "r" (output_address), + THUNK_TARGET(hv_hypercall_pg) + : "cc", "memory", "r8", "r9", "r10", "r11"); + } #else u32 input_address_hi =3D upper_32_bits(input_address); u32 input_address_lo =3D lower_32_bits(input_address); @@ -104,7 +113,13 @@ static inline u64 _hv_do_fast_hypercall8(u64 control, = u64 input1) u64 hv_status; =20 #ifdef CONFIG_X86_64 - { + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__( + "vmmcall" + : "=3Da" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + :: "cc", "r8", "r9", "r10", "r11"); + } else { __asm__ __volatile__(CALL_NOSPEC : "=3Da" (hv_status), ASM_CALL_CONSTRAINT, "+c" (control), "+d" (input1) @@ -149,7 +164,14 @@ static inline u64 _hv_do_fast_hypercall16(u64 control,= u64 input1, u64 input2) u64 hv_status; =20 #ifdef CONFIG_X86_64 - { + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__("mov %4, %%r8\n" + "vmmcall" + : "=3Da" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + : "r" (input2) + : "cc", "r8", "r9", "r10", "r11"); + } else { __asm__ __volatile__("mov %4, %%r8\n" CALL_NOSPEC : "=3Da" (hv_status), ASM_CALL_CONSTRAINT, --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D4DAEB64DD for ; Tue, 27 Jun 2023 03:29:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230297AbjF0D3k (ORCPT ); Mon, 26 Jun 2023 23:29:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42256 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229623AbjF0D2V (ORCPT ); Mon, 26 Jun 2023 23:28:21 -0400 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 344622115; Mon, 26 Jun 2023 20:23:01 -0700 (PDT) Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-25f0e0bbcaaso1934878a91.3; Mon, 26 Jun 2023 20:23:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836180; x=1690428180; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1RKpz5HBE9JLrRBFLWv7AJFc4wm2/SCXversK+BtfEc=; b=akngYKBlJ50MAGPqRAGc1r0u1GU6Eqd/kAPm+PCdK8pSl+BholJ8W5Lz/XKnrwxX/f XnfqQZCHFL4E3BVGKY1SMlHs+NQCxB9766HYFgLJNBCA3n+GCUq9PX3hcptZdmN/GDK0 eTp9qy4HkSuMUg7ntO6H+3RPeDxLRQiG7+OJP1ePF1Yv3QhuQVsG52hTKdJaNcqZSxNK BKLbEut+IXrScoi0AeaKz3UqI2w/QVgciZFaj/fKtxcWnAi1BRBz1StRmLXlilAAK0uy 8I6YPeJAJwukwzJWe+IQJvM4W19WDMFMZ+Grkr6xjMFE0uoIkwKl4JiHb6muyR0mc1Xq nP7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836180; x=1690428180; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1RKpz5HBE9JLrRBFLWv7AJFc4wm2/SCXversK+BtfEc=; b=KZ5lrT1SWyVJJc4vNXmzflQch5kP/Y35+31v3Xhx0X+p3q5aacs/+sol4vKtNQFqjs 9QMk+eYyiGnGCVVbkEChKzDeP9Ictr+DBDNmlHwAV3i9z9OVm+yRj+f/qwlYeve4vHBg 43RB0gp+kgcY3ilsDTVjwBOQrCR14ynAqgYEWOo5az+kbLIHNS9lVLsbbkYIDeiWTzMU 1hVAj7wGjEH33tvvSpQea61ztlAMulC5xkKOFezVNyiZctWzDgPkJQNPq9nzinBg/LjZ /CNbXpftntwT9IBGdetvzOLZQZGjrZHiNW2NEkiq9dma/zikBDXjyAn8AZLuBGIoAMWJ 6d4Q== X-Gm-Message-State: AC+VfDz5o/OZTZJai/t7Z+Ta1Kj9xy52gdbqy7QyenfciypMcrMPPXPL eaUazSZmgh9rKP2EU3pHayw= X-Google-Smtp-Source: ACHHUZ61fTXrRa2PBCfDuHA3HaSxtIC2ROCo1URXZD186AaP5V7GRRgKnQ/505qIAD0Sk4grMj0tZA== X-Received: by 2002:a17:90a:354:b0:263:14fc:f9a6 with SMTP id 20-20020a17090a035400b0026314fcf9a6mr787598pjf.14.1687836180547; Mon, 26 Jun 2023 20:23:00 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.22.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:23:00 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 6/9] clocksource: hyper-v: Mark hyperv tsc page unencrypted in sev-snp enlightened guest Date: Mon, 26 Jun 2023 23:22:44 -0400 Message-Id: <20230627032248.2170007-7-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan Hyper-V tsc page is shared with hypervisor and mark the page unencrypted in sev-snp enlightened guest when it's used. Signed-off-by: Tianyu Lan Reviewed-by: Michael Kelley --- drivers/clocksource/hyperv_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyper= v_timer.c index bcd9042a0c9f..66e29a19770b 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -376,7 +376,7 @@ EXPORT_SYMBOL_GPL(hv_stimer_global_cleanup); static union { struct ms_hyperv_tsc_page page; u8 reserved[PAGE_SIZE]; -} tsc_pg __aligned(PAGE_SIZE); +} tsc_pg __bss_decrypted __aligned(PAGE_SIZE); =20 static struct ms_hyperv_tsc_page *tsc_page =3D &tsc_pg.page; static unsigned long tsc_pfn; --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B49EEB64DD for ; Tue, 27 Jun 2023 03:29:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230345AbjF0D3q (ORCPT ); Mon, 26 Jun 2023 23:29:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229628AbjF0D2V (ORCPT ); Mon, 26 Jun 2023 23:28:21 -0400 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F0B4211B; Mon, 26 Jun 2023 20:23:02 -0700 (PDT) Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-55767141512so1729752a12.3; Mon, 26 Jun 2023 20:23:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836182; x=1690428182; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IJTPSxR2JGN4fgMuHptSG8Hdo91/jubbscb7t5UbRBo=; b=rzDQGPnaqRnWoXI/FbVlXs2gDJI3IyYY8/Rj6Qj5+N2MJO35f66NUsLN3FZOKOxvSL WGIL4dO94S5C2vQEwP9Lsla+yoJpHQyNF7FI4yQulIMmk8XZK7HYG3Optpme14KVrNcc tfS6oMBXyV4T0FFCGQvEtsnihEy7EByahZhuCDvdAPZZ05cudg/bR6tQNmMWLeJjeokm iYflSwyu70PMdtzqFKRyKKwRSpHJaQQ41x+DJrnvt0kDrm7Pc15pRNsEbODakM1LMkiV VzHnMBb2hmaT1Gzi1v9FnexRjuNen5KcLwHnTjuv99mHPrN6biEG8TMgC0VrLPyo8C1z 6TwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836182; x=1690428182; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IJTPSxR2JGN4fgMuHptSG8Hdo91/jubbscb7t5UbRBo=; b=QcoXjYaLniFBGaqmnqiEGFJ2da7c3GCrerxgqt/H+BRq8bxTPJ8tX0zchvlSJqig+c diZvuGZqemmn4U+7Cn/YGXKcxy56FjgPbT9nOfMS5e4hA15swn+yuihLrmFZ1eg9B52r nUgvYbWdf8eDd3qipFiJnW+iD6l0yL/Pl7VqTLjujXgElXLk9s1nFMHwzwdjw6xAr36o yoSxlliyyggZyMhUPsb5uEST0LC614N7GdnYSo973mLgYAeE25vGU4d0jrPVC+TQkwPF X1DVKFEn4oaipoCwvj0fnQUiM39M343L9tsJ/hWAZ5J+oK/FUMxqNctv2sI6ELo+R9EJ Umgg== X-Gm-Message-State: AC+VfDwWmP0ZcX+2Mi/XdCRflqCzceBLhflHTDCr8FktOPnT7JW0kbl1 5YaggEr1k3YuED4CPOx9l5k= X-Google-Smtp-Source: ACHHUZ56iBuJLtVAaukBPyoox0Oef7/mADkIlMpGB/6P5aYK5LdhBfhzgwxNQaxPDd3TH6lp+RkCUg== X-Received: by 2002:a17:90a:4902:b0:259:a879:cb8f with SMTP id c2-20020a17090a490200b00259a879cb8fmr16077197pjh.7.1687836182033; Mon, 26 Jun 2023 20:23:02 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.23.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:23:01 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 7/9] x86/hyperv: Initialize cpu and memory for SEV-SNP enlightened guest Date: Mon, 26 Jun 2023 23:22:45 -0400 Message-Id: <20230627032248.2170007-8-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan Hyper-V enlightened guest doesn't have boot loader support. Boot Linux kernel directly from hypervisor with data (kernel image, initrd and parameter page) and memory for boot up that is initialized via AMD SEV PSP protocol (Please reference Section 4.5 Launching a Guest of [1]). Kernel needs to read processor and memory info from EN_SEV_ SNP_PROCESSOR/MEM_INFO_ADDR address which are populated by Hyper-V. The these data is prepared by hypervisor via SNP_ LAUNCH_UPDATE with page type SNP_PAGE_TYPE_UNMEASURED and Initialize smp cpu related ops, validate system memory and add them into e820 table. [1]: https://www.amd.com/system/files/TechDocs/56860.pdf Signed-off-by: Tianyu Lan Reviewed-by: Michael Kelley --- arch/x86/hyperv/ivm.c | 93 +++++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 17 ++++++ arch/x86/kernel/cpu/mshyperv.c | 3 ++ 3 files changed, 113 insertions(+) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 5d3ee3124e00..b1639ec07155 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -17,6 +17,11 @@ #include #include #include +#include +#include +#include +#include +#include =20 #ifdef CONFIG_AMD_MEM_ENCRYPT =20 @@ -57,6 +62,8 @@ union hv_ghcb { =20 static u16 hv_ghcb_version __ro_after_init; =20 +static u32 processor_count; + u64 hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_si= ze) { union hv_ghcb *hv_ghcb; @@ -356,6 +363,92 @@ static bool hv_is_private_mmio(u64 addr) return false; } =20 +static __init void hv_snp_get_smp_config(unsigned int early) +{ + /* + * The "early" parameter can be true only if old-style AMD + * Opteron NUMA detection is enabled, which should never be + * the case for an SEV-SNP guest. See CONFIG_AMD_NUMA. + * For safety, just do nothing if "early" is true. + */ + if (early) + return; + + /* + * There is no firmware and ACPI MADT table support in + * in the Hyper-V SEV-SNP enlightened guest. Set smp + * related config variable here. + */ + while (num_processors < processor_count) { + early_per_cpu(x86_cpu_to_apicid, num_processors) =3D num_processors; + early_per_cpu(x86_bios_cpu_apicid, num_processors) =3D num_processors; + physid_set(num_processors, phys_cpu_present_map); + set_cpu_possible(num_processors, true); + set_cpu_present(num_processors, true); + num_processors++; + } +} + +__init void hv_sev_init_mem_and_cpu(void) +{ + struct memory_map_entry *entry; + struct e820_entry *e820_entry; + u64 e820_end; + u64 ram_end; + u64 page; + + /* + * Hyper-V enlightened snp guest boots kernel + * directly without bootloader. So roms, bios + * regions and reserve resources are not available. + * Set these callback to NULL. + */ + x86_platform.legacy.rtc =3D 0; + x86_platform.legacy.reserve_bios_regions =3D 0; + x86_platform.set_wallclock =3D set_rtc_noop; + x86_platform.get_wallclock =3D get_rtc_noop; + x86_init.resources.probe_roms =3D x86_init_noop; + x86_init.resources.reserve_resources =3D x86_init_noop; + x86_init.mpparse.find_smp_config =3D x86_init_noop; + x86_init.mpparse.get_smp_config =3D hv_snp_get_smp_config; + + /* + * Hyper-V SEV-SNP enlightened guest doesn't support ioapic + * and legacy APIC page read/write. Switch to hv apic here. + */ + disable_ioapic_support(); + + /* Get processor and mem info. */ + processor_count =3D *(u32 *)__va(EN_SEV_SNP_PROCESSOR_INFO_ADDR); + entry =3D (struct memory_map_entry *)__va(EN_SEV_SNP_MEM_INFO_ADDR); + + /* + * There is no bootloader/EFI firmware in the SEV SNP guest. + * E820 table in the memory just describes memory for kernel, + * ACPI table, cmdline, boot params and ramdisk. The dynamic + * data(e.g, vcpu number and the rest memory layout) needs to + * be read from EN_SEV_SNP_PROCESSOR_INFO_ADDR. + */ + for (; entry->numpages !=3D 0; entry++) { + e820_entry =3D &e820_table->entries[ + e820_table->nr_entries - 1]; + e820_end =3D e820_entry->addr + e820_entry->size; + ram_end =3D (entry->starting_gpn + + entry->numpages) * PAGE_SIZE; + + if (e820_end < entry->starting_gpn * PAGE_SIZE) + e820_end =3D entry->starting_gpn * PAGE_SIZE; + + if (e820_end < ram_end) { + pr_info("Hyper-V: add e820 entry [mem %#018Lx-%#018Lx]\n", e820_end, ra= m_end - 1); + e820__range_add(e820_end, ram_end - e820_end, + E820_TYPE_RAM); + for (page =3D e820_end; page < ram_end; page +=3D PAGE_SIZE) + pvalidate((unsigned long)__va(page), RMP_PG_SIZE_4K, true); + } + } +} + void __init hv_vtom_init(void) { /* diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyper= v.h index d859d7c5f5e8..7a9a6cdc2ae9 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -50,6 +50,21 @@ extern bool hv_isolation_type_en_snp(void); =20 extern union hv_ghcb * __percpu *hv_ghcb_pg; =20 +/* + * Hyper-V puts processor and memory layout info + * to this address in SEV-SNP enlightened guest. + */ +#define EN_SEV_SNP_PROCESSOR_INFO_ADDR 0x802000 +#define EN_SEV_SNP_MEM_INFO_ADDR 0x802018 + +struct memory_map_entry { + u64 starting_gpn; + u64 numpages; + u16 type; + u16 flags; + u32 reserved; +}; + int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages); int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id); int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags); @@ -255,12 +270,14 @@ void hv_ghcb_msr_read(u64 msr, u64 *value); bool hv_ghcb_negotiate_protocol(void); void __noreturn hv_ghcb_terminate(unsigned int set, unsigned int reason); void hv_vtom_init(void); +void hv_sev_init_mem_and_cpu(void); #else static inline void hv_ghcb_msr_write(u64 msr, u64 value) {} static inline void hv_ghcb_msr_read(u64 msr, u64 *value) {} static inline bool hv_ghcb_negotiate_protocol(void) { return false; } static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason= ) {} static inline void hv_vtom_init(void) {} +static inline void hv_sev_init_mem_and_cpu(void) {} #endif =20 extern bool hv_isolation_type_snp(void); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 5398fb2f4d39..d3bb921ee7fe 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -529,6 +529,9 @@ static void __init ms_hyperv_init_platform(void) if (!(ms_hyperv.features & HV_ACCESS_TSC_INVARIANT)) mark_tsc_unstable("running on Hyper-V"); =20 + if (hv_isolation_type_en_snp()) + hv_sev_init_mem_and_cpu(); + hardlockup_detector_disable(); } =20 --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07EBAEB64DC for ; Tue, 27 Jun 2023 03:29:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230355AbjF0D3w (ORCPT ); Mon, 26 Jun 2023 23:29:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229759AbjF0D2W (ORCPT ); Mon, 26 Jun 2023 23:28:22 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2DD22120; Mon, 26 Jun 2023 20:23:03 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-262e5e71978so1363757a91.1; Mon, 26 Jun 2023 20:23:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836183; x=1690428183; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bj0GajAExDkGrY+2aoG6AAOFcPqRGDxZFdmu7SDntpw=; b=BSw+LTRVdT1e7ZkI8XUBBv+rzxoFPY2eJ9u1G13E4CvIdoq/K3rKVp06esoyKvEsJd 6nnDvvTRROlVn/1/AdUVrp2iR9sV2qfaeGv/SxY8ituvxBTNsi6Fz7mPYrgF5RE9eetN IoE0nB2k0DicnfUkFfT8GScnLgG6eDbf2Y4RWbSTd2G6B/9wyMLpF5qSvZLIPNkbNsD2 juhfNjwx8MuszauU5AMMdQIgb3zwHwi8r4Cqz/zSincCwLAiY6PpO57OQsXNzBlqHeny j6XtOqQybuNLQW6yjsuTAze+NjGHNaAsFMceyEW+YLYyHs268P/FHn6qC8NXmdRfOE9J J8QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836183; x=1690428183; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bj0GajAExDkGrY+2aoG6AAOFcPqRGDxZFdmu7SDntpw=; b=Qh/4YYiXfjIC1NSCGxW/7hhUC0p3pOaXE3BKUwV61n4vGl20hPJ8d6zfVJQT6UM9+w 89dNeioDXSX0pODRGIdRKrE4tZhhUh610MAilAhY/xL7sA1Dr7MgPl38+1EThvJPjYB7 h1KtBERCgw/BWYVlGa09Z9rEXRZ+z8fbTr/yb/mm4vn3+g9PeAOawKScXUt2xfHEkdgf cve1eWxJgTGFNx5RSsuhdISKJ2rvQbf572k8vAhuEFSatTTmVYl7GdsZUkamFujjXWqn 3E0KlZ/P3wbMIVWLOxEEvEKcxOb4e+q6Cvyew5C8A3o00WKJFoBPobX2eXbKTydzr+oC OygA== X-Gm-Message-State: AC+VfDzmnjqB2bjI2h4D5rBQ9zZ/HEMr3J7qMQ5yRgmIp6cTBdm00sfq eUms3Xio2JdUmVbUqcKyW3Y= X-Google-Smtp-Source: ACHHUZ5EXQgmEJGrl74Q48/kfpp6wnCCGc0IqK/zwvfcZCML9dm1cIuuwfDvmKlR+qYrDvvrBZSfGQ== X-Received: by 2002:a17:90a:6f43:b0:25e:fb6d:ce68 with SMTP id d61-20020a17090a6f4300b0025efb6dce68mr35450752pjk.6.1687836183325; Mon, 26 Jun 2023 20:23:03 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.23.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:23:02 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 8/9] x86/hyperv: Add smp support for SEV-SNP guest Date: Mon, 26 Jun 2023 23:22:46 -0400 Message-Id: <20230627032248.2170007-9-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan In the AMD SEV-SNP guest, AP needs to be started up via sev es save area and Hyper-V requires to call HVCALL_START_VP hypercall to pass the gpa of sev es save area with AP's vp index and VTL(Virtual trust level) parameters. Override wakeup_secondary_cpu_64 callback with hv_snp_boot_ap. Signed-off-by: Tianyu Lan Reviewed-by: Michael Kelley --- arch/x86/hyperv/ivm.c | 95 +++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 9 +++ arch/x86/kernel/cpu/mshyperv.c | 13 ++++- include/asm-generic/hyperv-tlfs.h | 1 + 4 files changed, 116 insertions(+), 2 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index b1639ec07155..9b307f99b540 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -22,11 +22,15 @@ #include #include #include +#include =20 #ifdef CONFIG_AMD_MEM_ENCRYPT =20 #define GHCB_USAGE_HYPERV_CALL 1 =20 +static u8 ap_start_input_arg[PAGE_SIZE] __bss_decrypted __aligned(PAGE_SIZ= E); +static u8 ap_start_stack[PAGE_SIZE] __aligned(PAGE_SIZE); + union hv_ghcb { struct ghcb ghcb; struct { @@ -449,6 +453,97 @@ __init void hv_sev_init_mem_and_cpu(void) } } =20 +#define hv_populate_vmcb_seg(seg, gdtr_base) \ +do { \ + if (seg.selector) { \ + seg.base =3D 0; \ + seg.limit =3D HV_AP_SEGMENT_LIMIT; \ + seg.attrib =3D *(u16 *)(gdtr_base + seg.selector + 5); \ + seg.attrib =3D (seg.attrib & 0xFF) | ((seg.attrib >> 4) & 0xF00); \ + } \ +} while (0) \ + +int hv_snp_boot_ap(int cpu, unsigned long start_ip) +{ + struct sev_es_save_area *vmsa =3D (struct sev_es_save_area *) + __get_free_page(GFP_KERNEL | __GFP_ZERO); + struct desc_ptr gdtr; + u64 ret, rmp_adjust, retry =3D 5; + struct hv_enable_vp_vtl *start_vp_input; + unsigned long flags; + + native_store_gdt(&gdtr); + + vmsa->gdtr.base =3D gdtr.address; + vmsa->gdtr.limit =3D gdtr.size; + + asm volatile("movl %%es, %%eax;" : "=3Da" (vmsa->es.selector)); + hv_populate_vmcb_seg(vmsa->es, vmsa->gdtr.base); + + asm volatile("movl %%cs, %%eax;" : "=3Da" (vmsa->cs.selector)); + hv_populate_vmcb_seg(vmsa->cs, vmsa->gdtr.base); + + asm volatile("movl %%ss, %%eax;" : "=3Da" (vmsa->ss.selector)); + hv_populate_vmcb_seg(vmsa->ss, vmsa->gdtr.base); + + asm volatile("movl %%ds, %%eax;" : "=3Da" (vmsa->ds.selector)); + hv_populate_vmcb_seg(vmsa->ds, vmsa->gdtr.base); + + vmsa->efer =3D native_read_msr(MSR_EFER); + + asm volatile("movq %%cr4, %%rax;" : "=3Da" (vmsa->cr4)); + asm volatile("movq %%cr3, %%rax;" : "=3Da" (vmsa->cr3)); + asm volatile("movq %%cr0, %%rax;" : "=3Da" (vmsa->cr0)); + + vmsa->xcr0 =3D 1; + vmsa->g_pat =3D HV_AP_INIT_GPAT_DEFAULT; + vmsa->rip =3D (u64)secondary_startup_64_no_verify; + vmsa->rsp =3D (u64)&ap_start_stack[PAGE_SIZE]; + + /* + * Set the SNP-specific fields for this VMSA: + * VMPL level + * SEV_FEATURES (matches the SEV STATUS MSR right shifted 2 bits) + */ + vmsa->vmpl =3D 0; + vmsa->sev_features =3D sev_status >> 2; + + /* + * Running at VMPL0 allows the kernel to change the VMSA bit for a page + * using the RMPADJUST instruction. However, for the instruction to + * succeed it must target the permissions of a lesser privileged + * (higher numbered) VMPL level, so use VMPL1 (refer to the RMPADJUST + * instruction in the AMD64 APM Volume 3). + */ + rmp_adjust =3D RMPADJUST_VMSA_PAGE_BIT | 1; + ret =3D rmpadjust((unsigned long)vmsa, RMP_PG_SIZE_4K, + rmp_adjust); + if (ret !=3D 0) { + pr_err("RMPADJUST(%llx) failed: %llx\n", (u64)vmsa, ret); + return ret; + } + + local_irq_save(flags); + start_vp_input =3D + (struct hv_enable_vp_vtl *)ap_start_input_arg; + memset(start_vp_input, 0, sizeof(*start_vp_input)); + start_vp_input->partition_id =3D -1; + start_vp_input->vp_index =3D cpu; + start_vp_input->target_vtl.target_vtl =3D ms_hyperv.vtl; + *(u64 *)&start_vp_input->vp_context =3D __pa(vmsa) | 1; + + do { + ret =3D hv_do_hypercall(HVCALL_START_VP, + start_vp_input, NULL); + } while (hv_result(ret) =3D=3D HV_STATUS_TIME_OUT && retry--); + + local_irq_restore(flags); + + if (!hv_result_success(ret)) + pr_err("HvCallStartVirtualProcessor failed: %llx\n", ret); + return ret; +} + void __init hv_vtom_init(void) { /* diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyper= v.h index 7a9a6cdc2ae9..804c67475054 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -65,6 +65,13 @@ struct memory_map_entry { u32 reserved; }; =20 +/* + * DEFAULT INIT GPAT and SEGMENT LIMIT value in struct VMSA + * to start AP in enlightened SEV guest. + */ +#define HV_AP_INIT_GPAT_DEFAULT 0x0007040600070406ULL +#define HV_AP_SEGMENT_LIMIT 0xffffffff + int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages); int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id); int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags); @@ -271,6 +278,7 @@ bool hv_ghcb_negotiate_protocol(void); void __noreturn hv_ghcb_terminate(unsigned int set, unsigned int reason); void hv_vtom_init(void); void hv_sev_init_mem_and_cpu(void); +int hv_snp_boot_ap(int cpu, unsigned long start_ip); #else static inline void hv_ghcb_msr_write(u64 msr, u64 value) {} static inline void hv_ghcb_msr_read(u64 msr, u64 *value) {} @@ -278,6 +286,7 @@ static inline bool hv_ghcb_negotiate_protocol(void) { r= eturn false; } static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason= ) {} static inline void hv_vtom_init(void) {} static inline void hv_sev_init_mem_and_cpu(void) {} +static int hv_snp_boot_ap(int cpu, unsigned long start_ip) {} #endif =20 extern bool hv_isolation_type_snp(void); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index d3bb921ee7fe..8e1d9ed6a1e0 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -295,6 +295,16 @@ static void __init hv_smp_prepare_cpus(unsigned int ma= x_cpus) =20 native_smp_prepare_cpus(max_cpus); =20 + /* + * Override wakeup_secondary_cpu_64 callback for SEV-SNP + * enlightened guest. + */ + if (hv_isolation_type_en_snp()) + apic->wakeup_secondary_cpu_64 =3D hv_snp_boot_ap; + + if (!hv_root_partition) + return; + #ifdef CONFIG_X86_64 for_each_present_cpu(i) { if (i =3D=3D 0) @@ -502,8 +512,7 @@ static void __init ms_hyperv_init_platform(void) =20 # ifdef CONFIG_SMP smp_ops.smp_prepare_boot_cpu =3D hv_smp_prepare_boot_cpu; - if (hv_root_partition) - smp_ops.smp_prepare_cpus =3D hv_smp_prepare_cpus; + smp_ops.smp_prepare_cpus =3D hv_smp_prepare_cpus; # endif =20 /* diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv= -tlfs.h index f4e4cc4f965f..fdac4a1714ec 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -223,6 +223,7 @@ enum HV_GENERIC_SET_FORMAT { #define HV_STATUS_INVALID_PORT_ID 17 #define HV_STATUS_INVALID_CONNECTION_ID 18 #define HV_STATUS_INSUFFICIENT_BUFFERS 19 +#define HV_STATUS_TIME_OUT 120 #define HV_STATUS_VTL_ALREADY_ENABLED 134 =20 /* --=20 2.25.1 From nobody Sun Feb 8 11:26:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A096EB64DC for ; Tue, 27 Jun 2023 03:30:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230363AbjF0D37 (ORCPT ); Mon, 26 Jun 2023 23:29:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229886AbjF0D2W (ORCPT ); Mon, 26 Jun 2023 23:28:22 -0400 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F8392126; Mon, 26 Jun 2023 20:23:05 -0700 (PDT) Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-54fb23ff7d3so1993391a12.0; Mon, 26 Jun 2023 20:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687836184; x=1690428184; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mOH4gSPEIhpTW98QIj0wv6cpCR35fy8xVOmhHBHDwaY=; b=ANaNPobi1LlQxW0wdCG0Glnw3AHYwUIa8Qg4DBjBLvScG0WBKr8qCsf7Mz+18u/WU3 6dnIcVY47eOtyjpkaJW2cwyZTRkxtt7WjcP/1r2C0JYR4nlZbgTajeasbRLIeXdjnpSr pgJH+0iCGnFXJZLFj9ubxwQZlVh4ApQop/80AwlYke3TSc+1HoTIcvK2Zd16Aa9T/UoJ aGzVTEb5uixiEWDzFa45ZsAj19OfxlC6Vgvv3g8/E5CrvvsVDH0Uf6cISC2eKl79G1MX o8q2alCiD3y1+06C69O4r2j5dX0F85D1sY52ObLUgxLFJWE3Iw+tDzOboKE2N/izIfNQ 1juQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687836184; x=1690428184; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mOH4gSPEIhpTW98QIj0wv6cpCR35fy8xVOmhHBHDwaY=; b=QozppQlYoYy/1RyqNBMof2wLBCn+nw5LmSRh44AhD92KPT3+IBF/tJYRB3kf0yQg2T szVu3GVp4kTnXIO2gCv+Q6NPKi6WZd3nECeUPeKj8MR+7gQU72AVo836OOytkcEdWjo1 ynR6f+5vrVfVessoJcDIlUICmr8G0LPw51XCIbCepNC2qb4mwPJs1PEJq+f9AONuhLQX HsQSmK7wIIZKyvEMozc/993M/qvMAh4ATrLkG/Uvn/Md9UwA4opkHmK+F/R6xp4ov49g QK/Fniw/pjWHQLCWk41EYnE2gnb2t3uIrLoPWD4/PiGhjzxqG62yr9zZrt+e2WKjiG3F qvlQ== X-Gm-Message-State: AC+VfDxCTL7rGs6XmZCFC7L296/xXnXxsCn9jLGANa2wjt0QtsY3cG7R /qlxkRsKJsAQMMSWAePurQA= X-Google-Smtp-Source: ACHHUZ649XFt14nTD0RKbS7T8HGDXk/ibxTGnSCTsUZF4ntA/4vA54PzrAunHYoMJoD3+SXkrRSECw== X-Received: by 2002:a17:90a:1906:b0:25e:ae0f:7311 with SMTP id 6-20020a17090a190600b0025eae0f7311mr20920818pjg.23.1687836184513; Mon, 26 Jun 2023 20:23:04 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:37:c5e9:2003:6c97:8057]) by smtp.gmail.com with ESMTPSA id mm12-20020a17090b358c00b0025ec54be16asm618756pjb.2.2023.06.26.20.23.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jun 2023 20:23:04 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de, michael.h.kelley@microsoft.com Cc: Tianyu Lan , linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, vkuznets@redhat.com Subject: [PATCH V2 9/9] x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES Date: Mon, 26 Jun 2023 23:22:47 -0400 Message-Id: <20230627032248.2170007-10-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230627032248.2170007-1-ltykernel@gmail.com> References: <20230627032248.2170007-1-ltykernel@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianyu Lan Add Hyperv-specific handling for faults caused by VMMCALL instructions. Signed-off-by: Tianyu Lan Reviewed-by: Michael Kelley --- arch/x86/kernel/cpu/mshyperv.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 8e1d9ed6a1e0..ba9a3a65f664 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -32,6 +32,7 @@ #include #include #include +#include =20 /* Is Linux running as the root partition? */ bool hv_root_partition; @@ -577,6 +578,20 @@ static bool __init ms_hyperv_msi_ext_dest_id(void) return eax & HYPERV_VS_PROPERTIES_EAX_EXTENDED_IOAPIC_RTE; } =20 +static void hv_sev_es_hcall_prepare(struct ghcb *ghcb, struct pt_regs *reg= s) +{ + /* RAX and CPL are already in the GHCB */ + ghcb_set_rcx(ghcb, regs->cx); + ghcb_set_rdx(ghcb, regs->dx); + ghcb_set_r8(ghcb, regs->r8); +} + +static bool hv_sev_es_hcall_finish(struct ghcb *ghcb, struct pt_regs *regs) +{ + /* No checking of the return state needed */ + return true; +} + const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv =3D { .name =3D "Microsoft Hyper-V", .detect =3D ms_hyperv_platform, @@ -584,4 +599,6 @@ const __initconst struct hypervisor_x86 x86_hyper_ms_hy= perv =3D { .init.x2apic_available =3D ms_hyperv_x2apic_available, .init.msi_ext_dest_id =3D ms_hyperv_msi_ext_dest_id, .init.init_platform =3D ms_hyperv_init_platform, + .runtime.sev_es_hcall_prepare =3D hv_sev_es_hcall_prepare, + .runtime.sev_es_hcall_finish =3D hv_sev_es_hcall_finish, }; --=20 2.25.1