From nobody Thu Dec 18 20:24:50 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAA9AC04A94 for ; Mon, 14 Aug 2023 05:55:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233401AbjHNFyk (ORCPT ); Mon, 14 Aug 2023 01:54:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233384AbjHNFyH (ORCPT ); Mon, 14 Aug 2023 01:54:07 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2063.outbound.protection.outlook.com [40.107.100.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9F8710D1 for ; Sun, 13 Aug 2023 22:53:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TvgzcqceTECYTvQpbpj9BG++Awc9E9bcIQZBVJgaCOJ34LQCwsA7RPQfXwV0TsjHkSg5LZrz/q/nmL9MO3hnSgL3uA9cs2BtkB/qoyITCFRxN6J62CB+suLe6d+q+rMJgASKLbQ8MCBGVC/R090Z50RDVayrWjZETHBC1sSkXIl1RA7wL/d+r77Yy7/6YY1+VpZvT4HoWc6nqiOsC5aKUSCSXHHy6NTXkfH9JZNjPT2Qd169W0u0Q5LotHUNEh9Q2XYhbE7+A5efaMN71HeHnq3DsutmJ2wfZJOZlXvwwFYXuGMeTBdCLGwWRgQTI1m60oiCp54Y97MibyN6gPvGIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5N8itChEk8GU//5mEJKtr3mUGKsCDKlVDDcch4PolT8=; b=Mu5F6xqheK32F+lIM1LphTQzJPLS8Cx2B2Ms+XZjqHZg/NWqhtvj2NiKQwLPbwvGIZlFX1gmFxSSbfror0soclFkfsdYMVFN2GtP8FvYN2XqLcpVWxh+Dsw71P4jL4WoUmKngXjuZ3ImIGKmEH4tk+YMhCBwd3qTuwfsSaayoQqW9r/MV5A09fCWWfj8h4piAniByYn97FzBbkPbDeFW9xKk+cSby/xHvDPKAb8Rp0RBtccIM5JAiLSsRwJ4LsF5ExI0JBs5kWsREkbuXGmFCdEf+hxR4LtafzlQeMKfY8j/pW5VSpClbOq8R8IMq86oyjWP7K+bfFC7PLrC1jpFDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5N8itChEk8GU//5mEJKtr3mUGKsCDKlVDDcch4PolT8=; b=JrgULRa3yOzUoQsD/P8G67exDA4KnQYbUcf7WhOJmwAuSXwnFtOflpTheMESzJjLs6wlJCz7YWpYyhe2tLCQDnPwxklaCkTiB6yki+gHMiJ9QZJntYuRy82I10g0+pbeN7bjZChEqtjuYDKH1NmYmtNVi2tfxW+EklxyxP042e0= Received: from MW4PR03CA0186.namprd03.prod.outlook.com (2603:10b6:303:b8::11) by DS7PR12MB6262.namprd12.prod.outlook.com (2603:10b6:8:96::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.24; Mon, 14 Aug 2023 05:53:56 +0000 Received: from CO1NAM11FT079.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b8:cafe::73) by MW4PR03CA0186.outlook.office365.com (2603:10b6:303:b8::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.33 via Frontend Transport; Mon, 14 Aug 2023 05:53:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT079.mail.protection.outlook.com (10.13.175.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6699.13 via Frontend Transport; Mon, 14 Aug 2023 05:53:55 +0000 Received: from gomati.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 14 Aug 2023 00:53:51 -0500 From: Nikunj A Dadhania To: , CC: , , , , , , Subject: [PATCH v4 07/14] x86/sev: Move and reorganize sev guest request api Date: Mon, 14 Aug 2023 11:22:15 +0530 Message-ID: <20230814055222.1056404-8-nikunj@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814055222.1056404-1-nikunj@amd.com> References: <20230814055222.1056404-1-nikunj@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT079:EE_|DS7PR12MB6262:EE_ X-MS-Office365-Filtering-Correlation-Id: d3e0eae0-2826-4d23-6c1d-08db9c8ad8dc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6IwMXPcIaIpMe1pBgDhMOIWxtc89Es8yoi2FDSZsdW/AWtX5diEroqnGYQyesdEZcz/CIy3gnbbrYzDj9g5FFswaf2+hlnOjit6A1GdZOS+1Twt9GIoiRQScd53N6C1vqjLSM2wyvUZIXv2txJrPOZalFEtSuOgBXJi8E4VmJDhmx+3T51ae+7I+ePSNSGR7YDxcw59aFHAti9I8/T3P8ZFOD5UDecn4Ahice6ZUwjPh1sHA2Cycb5GKH6yUoyoSsDgjpJQDUQe9M8x7a7/3G7RKVDLqzog1QIiBhwzfFn0iS6aU1906WdnXMiX4Avw/X3B6UXS5Mk6WOQLDF2DMKAja6lbLLcPYoLCAHKpVnbriUL0iu4U65wk6QvAsG+0ArsLzT4oI2KRzcw9QQl/ebY/t7kuVmwMz0FX1v1jrssN3aYkETRu0dI1xdQ4ERFZ+QY66pkCjIzBcbNd+qzp1KItSLyiLeBVUhMubXAreeNfQpSsp6HhXW8yTN60Jy94vIhrYoyM79Caz4vq2B73T/q+T/VftNv4s2ZroJ17dNLCFo0bLhNUXB8nfrvel3ZNfc5a+SJ9SAyYzv+CFx9AAe9onamfTI6A0fXuOPu1WYDvasiPTJZBB6zCUa6/1VDw6GpZymJ2nIxVh5Y7qkOQxEM+wOW9co8nZiUhycoWzVXlQS//Wul+CY6mk9OwaEfp2W3sRzsHN91NZWKoVfpk7Qf2qhe2QNlJOsLwGvVGMMBKJRB4LIZKJRWj7rh6ihgg6C2xIp3nGoIle0dE1K+fw8w== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(396003)(346002)(39860400002)(136003)(186006)(1800799006)(82310400008)(451199021)(46966006)(40470700004)(36840700001)(16526019)(26005)(1076003)(41300700001)(336012)(8936002)(70206006)(70586007)(316002)(8676002)(36860700001)(2616005)(47076005)(83380400001)(426003)(6666004)(7696005)(81166007)(40460700003)(478600001)(40480700001)(110136005)(54906003)(30864003)(36756003)(4326008)(82740400003)(356005)(5660300002)(2906002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Aug 2023 05:53:55.9082 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d3e0eae0-2826-4d23-6c1d-08db9c8ad8dc X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT079.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6262 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" For enabling Secure TSC, SEV-SNP guests need to communicate with the AMD Security Processor early during boot. Many of the required functions are implemented in the sev-guest driver and therefore not available at early boot. Move the required functions and provide an API to the driver to assign key and send guest request. Signed-off-by: Nikunj A Dadhania --- arch/x86/Kconfig | 1 + arch/x86/include/asm/sev-guest.h | 84 +++- arch/x86/include/asm/sev.h | 10 - arch/x86/kernel/sev.c | 466 ++++++++++++++++++++++- drivers/virt/coco/sev-guest/Kconfig | 1 - drivers/virt/coco/sev-guest/sev-guest.c | 486 +----------------------- 6 files changed, 555 insertions(+), 493 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 53bab123a8ee..7502903cbc6b 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1546,6 +1546,7 @@ config AMD_MEM_ENCRYPT select INSTRUCTION_DECODER select ARCH_HAS_CC_PLATFORM select X86_MEM_ENCRYPT + select CRYPTO_LIB_AESGCM help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory diff --git a/arch/x86/include/asm/sev-guest.h b/arch/x86/include/asm/sev-gu= est.h index 22ef97b55069..e6f94208173d 100644 --- a/arch/x86/include/asm/sev-guest.h +++ b/arch/x86/include/asm/sev-guest.h @@ -11,6 +11,11 @@ #define __VIRT_SEVGUEST_H__ =20 #include +#include +#include + +#define SNP_REQ_MAX_RETRY_DURATION (60*HZ) +#define SNP_REQ_RETRY_DELAY (2*HZ) =20 #define MAX_AUTHTAG_LEN 32 #define AUTHTAG_LEN 16 @@ -58,11 +63,45 @@ struct snp_guest_msg_hdr { u8 rsvd3[35]; } __packed; =20 +/* SNP Guest message request */ +struct snp_req_data { + unsigned long req_gpa; + unsigned long resp_gpa; +}; + struct snp_guest_msg { struct snp_guest_msg_hdr hdr; u8 payload[4000]; } __packed; =20 +struct sev_guest_platform_data { + /* request and response are in unencrypted memory */ + struct snp_guest_msg *request, *response; + + struct snp_secrets_page_layout *layout; + struct snp_req_data input; +}; + +struct snp_guest_dev { + struct device *dev; + struct miscdevice misc; + + /* Mutex to serialize the shared buffer access and command handling. */ + struct mutex cmd_mutex; + + void *certs_data; + struct aesgcm_ctx *ctx; + + /* + * Avoid information leakage by double-buffering shared messages + * in fields that are in regular encrypted memory + */ + struct snp_guest_msg secret_request, secret_response; + + struct sev_guest_platform_data *pdata; + unsigned int vmpck_id; +}; + struct snp_guest_req { void *req_buf, *resp_buf, *data; size_t req_sz, resp_sz, *data_npages; @@ -72,6 +111,47 @@ struct snp_guest_req { u8 msg_type; }; =20 -int snp_issue_guest_request(struct snp_guest_req *req, struct snp_req_data= *input, - struct snp_guest_request_ioctl *rio); +int snp_setup_psp_messaging(struct snp_guest_dev *snp_dev); +int snp_send_guest_request(struct snp_guest_dev *dev, struct snp_guest_req= *req, + struct snp_guest_request_ioctl *rio); +bool snp_assign_vmpck(struct snp_guest_dev *dev, unsigned int vmpck_id); +bool snp_is_vmpck_empty(unsigned int vmpck_id); + +static void free_shared_pages(void *buf, size_t sz) +{ + unsigned int npages =3D PAGE_ALIGN(sz) >> PAGE_SHIFT; + int ret; + + if (!buf) + return; + + ret =3D set_memory_encrypted((unsigned long)buf, npages); + if (ret) { + WARN_ONCE(ret, "failed to restore encryption mask (leak it)\n"); + return; + } + + __free_pages(virt_to_page(buf), get_order(sz)); +} + +static void *alloc_shared_pages(size_t sz) +{ + unsigned int npages =3D PAGE_ALIGN(sz) >> PAGE_SHIFT; + struct page *page; + int ret; + + page =3D alloc_pages(GFP_KERNEL_ACCOUNT, get_order(sz)); + if (!page) + return NULL; + + ret =3D set_memory_decrypted((unsigned long)page_address(page), npages); + if (ret) { + pr_err("%s: failed to mark page shared, ret=3D%d\n", __func__, ret); + __free_pages(page, get_order(sz)); + return NULL; + } + + return page_address(page); +} + #endif /* __VIRT_SEVGUEST_H__ */ diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index d8e972aeb22c..807f85f8014c 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -88,16 +88,6 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs); =20 #define RMPADJUST_VMSA_PAGE_BIT BIT(16) =20 -/* SNP Guest message request */ -struct snp_req_data { - unsigned long req_gpa; - unsigned long resp_gpa; -}; - -struct sev_guest_platform_data { - u64 secrets_gpa; -}; - /* * The secrets page contains 96-bytes of reserved field that can be used by * the guest OS. The guest OS uses the area to save the message sequence diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index a63a3edf13d0..72e76c58aebd 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -24,6 +24,7 @@ #include #include #include +#include =20 #include #include @@ -959,6 +960,457 @@ static void snp_cleanup_vmsa(struct sev_es_save_area = *vmsa) free_page((unsigned long)vmsa); } =20 +static struct sev_guest_platform_data *platform_data; + +static inline u8 *snp_get_vmpck(unsigned int vmpck_id) +{ + if (!platform_data) + return NULL; + + return platform_data->layout->vmpck0 + vmpck_id * VMPCK_KEY_LEN; +} + +static inline u32 *snp_get_os_area_msg_seqno(unsigned int vmpck_id) +{ + if (!platform_data) + return NULL; + + return &platform_data->layout->os_area.msg_seqno_0 + vmpck_id; +} + +bool snp_is_vmpck_empty(unsigned int vmpck_id) +{ + char zero_key[VMPCK_KEY_LEN] =3D {0}; + u8 *key =3D snp_get_vmpck(vmpck_id); + + if (key) + return !memcmp(key, zero_key, VMPCK_KEY_LEN); + + return true; +} +EXPORT_SYMBOL_GPL(snp_is_vmpck_empty); + +/* + * If an error is received from the host or AMD Secure Processor (ASP) the= re + * are two options. Either retry the exact same encrypted request or disco= ntinue + * using the VMPCK. + * + * This is because in the current encryption scheme GHCB v2 uses AES-GCM to + * encrypt the requests. The IV for this scheme is the sequence number. GCM + * cannot tolerate IV reuse. + * + * The ASP FW v1.51 only increments the sequence numbers on a successful + * guest<->ASP back and forth and only accepts messages at its exact seque= nce + * number. + * + * So if the sequence number were to be reused the encryption scheme is + * vulnerable. If the sequence number were incremented for a fresh IV the = ASP + * will reject the request. + */ +static void snp_disable_vmpck(struct snp_guest_dev *snp_dev) +{ + u8 *key =3D snp_get_vmpck(snp_dev->vmpck_id); + + pr_alert("Disabling vmpck_id %d to prevent IV reuse.\n", snp_dev->vmpck_i= d); + memzero_explicit(key, VMPCK_KEY_LEN); +} + +static inline u64 __snp_get_msg_seqno(struct snp_guest_dev *snp_dev) +{ + u32 *os_area_msg_seqno =3D snp_get_os_area_msg_seqno(snp_dev->vmpck_id); + u64 count; + + if (!os_area_msg_seqno) { + pr_err("SNP unable to get message sequence counter\n"); + return 0; + } + + lockdep_assert_held(&snp_dev->cmd_mutex); + + /* Read the current message sequence counter from secrets pages */ + count =3D *os_area_msg_seqno; + + return count + 1; +} + +/* Return a non-zero on success */ +static u64 snp_get_msg_seqno(struct snp_guest_dev *snp_dev) +{ + u64 count =3D __snp_get_msg_seqno(snp_dev); + + /* + * The message sequence counter for the SNP guest request is a 64-bit + * value but the version 2 of GHCB specification defines a 32-bit storage + * for it. If the counter exceeds the 32-bit value then return zero. + * The caller should check the return value, but if the caller happens to + * not check the value and use it, then the firmware treats zero as an + * invalid number and will fail the message request. + */ + if (count >=3D UINT_MAX) { + pr_err("SNP request message sequence counter overflow\n"); + return 0; + } + + return count; +} + +static void snp_inc_msg_seqno(struct snp_guest_dev *snp_dev) +{ + u32 *os_area_msg_seqno =3D snp_get_os_area_msg_seqno(snp_dev->vmpck_id); + + if (!os_area_msg_seqno) { + pr_err("SNP unable to get message sequence counter\n"); + return; + } + + /* + * The counter is also incremented by the PSP, so increment it by 2 + * and save in secrets page. + */ + *os_area_msg_seqno +=3D 2; +} + +static struct aesgcm_ctx *snp_init_crypto(unsigned int vmpck_id) +{ + struct aesgcm_ctx *ctx; + u8 *key; + + if (snp_is_vmpck_empty(vmpck_id)) { + pr_err("SNP: vmpck id %d is null\n", vmpck_id); + return NULL; + } + + ctx =3D kzalloc(sizeof(*ctx), GFP_KERNEL_ACCOUNT); + if (!ctx) + return NULL; + + key =3D snp_get_vmpck(vmpck_id); + if (aesgcm_expandkey(ctx, key, VMPCK_KEY_LEN, AUTHTAG_LEN)) { + pr_err("SNP: crypto init failed\n"); + kfree(ctx); + return NULL; + } + + return ctx; +} + +int snp_setup_psp_messaging(struct snp_guest_dev *snp_dev) +{ + struct sev_guest_platform_data *pdata; + int ret; + + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + pr_err("SNP not supported\n"); + return 0; + } + + if (platform_data) { + pr_debug("SNP platform data already initialized.\n"); + goto create_ctx; + } + + if (!secrets_pa) { + pr_err("SNP no secrets page\n"); + return -ENODEV; + } + + pdata =3D kzalloc(sizeof(struct sev_guest_platform_data), GFP_KERNEL); + if (!pdata) { + pr_err("SNP alloc failed\n"); + return -ENOMEM; + } + + pdata->layout =3D (__force void *)ioremap_encrypted(secrets_pa, PAGE_SIZE= ); + if (!pdata->layout) { + pr_err("Unable to locate AP jump table address: failed to map the SNP se= crets page.\n"); + goto e_free_pdata; + } + + ret =3D -ENOMEM; + /* Allocate the shared page used for the request and response message. */ + pdata->request =3D alloc_shared_pages(sizeof(struct snp_guest_msg)); + if (!pdata->request) + goto e_unmap; + + pdata->response =3D alloc_shared_pages(sizeof(struct snp_guest_msg)); + if (!pdata->response) + goto e_free_request; + + /* initial the input address for guest request */ + pdata->input.req_gpa =3D __pa(pdata->request); + pdata->input.resp_gpa =3D __pa(pdata->response); + platform_data =3D pdata; + +create_ctx: + ret =3D -EIO; + snp_dev->ctx =3D snp_init_crypto(snp_dev->vmpck_id); + if (!snp_dev->ctx) { + pr_err("SNP init crypto failed\n"); + platform_data =3D NULL; + goto e_free_response; + } + + snp_dev->pdata =3D platform_data; + return 0; + +e_free_response: + free_shared_pages(pdata->response, sizeof(struct snp_guest_msg)); +e_free_request: + free_shared_pages(pdata->request, sizeof(struct snp_guest_msg)); +e_unmap: + iounmap(pdata->layout); +e_free_pdata: + kfree(pdata); + + return ret; +} +EXPORT_SYMBOL_GPL(snp_setup_psp_messaging); + +static int __enc_payload(struct aesgcm_ctx *ctx, struct snp_guest_msg *msg, + void *plaintext, size_t len) +{ + struct snp_guest_msg_hdr *hdr =3D &msg->hdr; + u8 iv[GCM_AES_IV_SIZE] =3D {}; + + if (WARN_ON((hdr->msg_sz + ctx->authsize) > sizeof(msg->payload))) + return -EBADMSG; + + memcpy(iv, &hdr->msg_seqno, sizeof(hdr->msg_seqno)); + aesgcm_encrypt(ctx, msg->payload, plaintext, len, &hdr->algo, AAD_LEN, + iv, hdr->authtag); + return 0; +} + +static int dec_payload(struct aesgcm_ctx *ctx, struct snp_guest_msg *msg, + void *plaintext, size_t len) +{ + struct snp_guest_msg_hdr *hdr =3D &msg->hdr; + u8 iv[GCM_AES_IV_SIZE] =3D {}; + + memcpy(iv, &hdr->msg_seqno, sizeof(hdr->msg_seqno)); + if (aesgcm_decrypt(ctx, plaintext, msg->payload, len, &hdr->algo, + AAD_LEN, iv, hdr->authtag)) + return 0; + else + return -EBADMSG; +} + +static int verify_and_dec_payload(struct snp_guest_dev *snp_dev, struct sn= p_guest_req *guest_req, + struct sev_guest_platform_data *pdata) +{ + struct snp_guest_msg *resp =3D &snp_dev->secret_response; + struct snp_guest_msg *req =3D &snp_dev->secret_request; + struct snp_guest_msg_hdr *req_hdr =3D &req->hdr; + struct snp_guest_msg_hdr *resp_hdr =3D &resp->hdr; + struct aesgcm_ctx *ctx =3D snp_dev->ctx; + + pr_debug("response [seqno %lld type %d version %d sz %d]\n", + resp_hdr->msg_seqno, resp_hdr->msg_type, resp_hdr->msg_version, + resp_hdr->msg_sz); + + /* Copy response from shared memory to encrypted memory. */ + memcpy(resp, pdata->response, sizeof(*resp)); + + /* Verify that the sequence counter is incremented by 1 */ + if (unlikely(resp_hdr->msg_seqno !=3D (req_hdr->msg_seqno + 1))) + return -EBADMSG; + + /* Verify response message type and version number. */ + if (resp_hdr->msg_type !=3D (req_hdr->msg_type + 1) || + resp_hdr->msg_version !=3D req_hdr->msg_version) + return -EBADMSG; + + /* + * If the message size is greater than our buffer length then return + * an error. + */ + if (unlikely((resp_hdr->msg_sz + ctx->authsize) > guest_req->resp_sz)) + return -EBADMSG; + + return dec_payload(ctx, resp, guest_req->resp_buf, resp_hdr->msg_sz); +} + +static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, struct sn= p_guest_req *req) +{ + struct snp_guest_msg *msg =3D &snp_dev->secret_request; + struct snp_guest_msg_hdr *hdr =3D &msg->hdr; + + memset(msg, 0, sizeof(*msg)); + + hdr->algo =3D SNP_AEAD_AES_256_GCM; + hdr->hdr_version =3D MSG_HDR_VER; + hdr->hdr_sz =3D sizeof(*hdr); + hdr->msg_type =3D req->msg_type; + hdr->msg_version =3D req->msg_version; + hdr->msg_seqno =3D seqno; + hdr->msg_vmpck =3D req->vmpck_id; + hdr->msg_sz =3D req->req_sz; + + /* Verify the sequence number is non-zero */ + if (!hdr->msg_seqno) + return -ENOSR; + + pr_debug("request [seqno %lld type %d version %d sz %d]\n", + hdr->msg_seqno, hdr->msg_type, hdr->msg_version, hdr->msg_sz); + + return __enc_payload(snp_dev->ctx, msg, req->req_buf, req->req_sz); +} + +static int snp_issue_guest_request(struct snp_guest_req *req, struct snp_r= eq_data *input, + struct snp_guest_request_ioctl *rio); + +static int __handle_guest_request(struct snp_guest_dev *snp_dev, struct sn= p_guest_req *req, + struct snp_guest_request_ioctl *rio, + struct sev_guest_platform_data *pdata) +{ + unsigned long req_start =3D jiffies; + unsigned int override_npages =3D 0; + u64 override_err =3D 0; + int rc; + +retry_request: + /* + * Call firmware to process the request. In this function the encrypted + * message enters shared memory with the host. So after this call the + * sequence number must be incremented or the VMPCK must be deleted to + * prevent reuse of the IV. + */ + rc =3D snp_issue_guest_request(req, &pdata->input, rio); + switch (rc) { + case -ENOSPC: + /* + * If the extended guest request fails due to having too + * small of a certificate data buffer, retry the same + * guest request without the extended data request in + * order to increment the sequence number and thus avoid + * IV reuse. + */ + override_npages =3D *req->data_npages; + req->exit_code =3D SVM_VMGEXIT_GUEST_REQUEST; + + /* + * Override the error to inform callers the given extended + * request buffer size was too small and give the caller the + * required buffer size. + */ + override_err =3D SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVALID_LEN); + + /* + * If this call to the firmware succeeds, the sequence number can + * be incremented allowing for continued use of the VMPCK. If + * there is an error reflected in the return value, this value + * is checked further down and the result will be the deletion + * of the VMPCK and the error code being propagated back to the + * user as an ioctl() return code. + */ + goto retry_request; + + /* + * The host may return SNP_GUEST_REQ_ERR_BUSY if the request has been + * throttled. Retry in the driver to avoid returning and reusing the + * message sequence number on a different message. + */ + case -EAGAIN: + if (jiffies - req_start > SNP_REQ_MAX_RETRY_DURATION) { + rc =3D -ETIMEDOUT; + break; + } + schedule_timeout_killable(SNP_REQ_RETRY_DELAY); + goto retry_request; + } + + /* + * Increment the message sequence number. There is no harm in doing + * this now because decryption uses the value stored in the response + * structure and any failure will wipe the VMPCK, preventing further + * use anyway. + */ + snp_inc_msg_seqno(snp_dev); + + if (override_err) { + rio->exitinfo2 =3D override_err; + + /* + * If an extended guest request was issued and the supplied certificate + * buffer was not large enough, a standard guest request was issued to + * prevent IV reuse. If the standard request was successful, return -EIO + * back to the caller as would have originally been returned. + */ + if (!rc && override_err =3D=3D SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVAL= ID_LEN)) + rc =3D -EIO; + } + + if (override_npages) + *req->data_npages =3D override_npages; + + return rc; +} + +int snp_send_guest_request(struct snp_guest_dev *snp_dev, struct snp_guest= _req *req, + struct snp_guest_request_ioctl *rio) +{ + struct sev_guest_platform_data *pdata; + u64 seqno; + int rc; + + if (!snp_dev || !snp_dev->pdata || !req || !rio) + return -ENODEV; + + pdata =3D snp_dev->pdata; + + /* Get message sequence and verify that its a non-zero */ + seqno =3D snp_get_msg_seqno(snp_dev); + if (!seqno) + return -EIO; + + /* Clear shared memory's response for the host to populate. */ + memset(pdata->response, 0, sizeof(struct snp_guest_msg)); + + /* Encrypt the userspace provided payload in pdata->secret_request. */ + rc =3D enc_payload(snp_dev, seqno, req); + if (rc) + return rc; + + /* + * Write the fully encrypted request to the shared unencrypted + * request page. + */ + memcpy(pdata->request, &snp_dev->secret_request, sizeof(snp_dev->secret_r= equest)); + + rc =3D __handle_guest_request(snp_dev, req, rio, pdata); + if (rc) { + if (rc =3D=3D -EIO && + rio->exitinfo2 =3D=3D SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVALID_LE= N)) + return rc; + + pr_alert("Detected error from ASP request. rc: %d, exitinfo2: 0x%llx\n", + rc, rio->exitinfo2); + snp_disable_vmpck(snp_dev); + return rc; + } + + rc =3D verify_and_dec_payload(snp_dev, req, pdata); + if (rc) { + pr_alert("Detected unexpected decode failure from ASP. rc: %d\n", rc); + snp_disable_vmpck(snp_dev); + return rc; + } + + return 0; +} +EXPORT_SYMBOL_GPL(snp_send_guest_request); + +bool snp_assign_vmpck(struct snp_guest_dev *dev, unsigned int vmpck_id) +{ + if (WARN_ON(vmpck_id > 3)) + return false; + + dev->vmpck_id =3D vmpck_id; + + return true; +} +EXPORT_SYMBOL_GPL(snp_assign_vmpck); + static int wakeup_cpu_via_vmgexit(int apic_id, unsigned long start_ip) { struct sev_es_save_area *cur_vmsa, *vmsa; @@ -2160,8 +2612,8 @@ static int __init init_sev_config(char *str) } __setup("sev=3D", init_sev_config); =20 -int snp_issue_guest_request(struct snp_guest_req *req, struct snp_req_data= *input, - struct snp_guest_request_ioctl *rio) +static int snp_issue_guest_request(struct snp_guest_req *req, struct snp_r= eq_data *input, + struct snp_guest_request_ioctl *rio) { struct ghcb_state state; struct es_em_ctxt ctxt; @@ -2228,7 +2680,6 @@ int snp_issue_guest_request(struct snp_guest_req *req= , struct snp_req_data *inpu =20 return ret; } -EXPORT_SYMBOL_GPL(snp_issue_guest_request); =20 static struct platform_device sev_guest_device =3D { .name =3D "sev-guest", @@ -2237,18 +2688,9 @@ static struct platform_device sev_guest_device =3D { =20 static int __init snp_init_platform_device(void) { - struct sev_guest_platform_data data; - if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return -ENODEV; =20 - if (!secrets_pa) - return -ENODEV; - - data.secrets_gpa =3D secrets_pa; - if (platform_device_add_data(&sev_guest_device, &data, sizeof(data))) - return -ENODEV; - if (platform_device_register(&sev_guest_device)) return -ENODEV; =20 diff --git a/drivers/virt/coco/sev-guest/Kconfig b/drivers/virt/coco/sev-gu= est/Kconfig index bcc760bfb468..c130456ad401 100644 --- a/drivers/virt/coco/sev-guest/Kconfig +++ b/drivers/virt/coco/sev-guest/Kconfig @@ -2,7 +2,6 @@ config SEV_GUEST tristate "AMD SEV Guest driver" default m depends on AMD_MEM_ENCRYPT - select CRYPTO_LIB_AESGCM help SEV-SNP firmware provides the guest a mechanism to communicate with the PSP without risk from a malicious hypervisor who wishes to read, diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/se= v-guest/sev-guest.c index 0f519b855169..e84f7c532f4e 100644 --- a/drivers/virt/coco/sev-guest/sev-guest.c +++ b/drivers/virt/coco/sev-guest/sev-guest.c @@ -27,32 +27,6 @@ =20 #define DEVICE_NAME "sev-guest" =20 -#define SNP_REQ_MAX_RETRY_DURATION (60*HZ) -#define SNP_REQ_RETRY_DELAY (2*HZ) - -struct snp_guest_dev { - struct device *dev; - struct miscdevice misc; - - /* Mutex to serialize the shared buffer access and command handling. */ - struct mutex cmd_mutex; - - void *certs_data; - struct aesgcm_ctx *ctx; - /* request and response are in unencrypted memory */ - struct snp_guest_msg *request, *response; - - /* - * Avoid information leakage by double-buffering shared messages - * in fields that are in regular encrypted memory. - */ - struct snp_guest_msg secret_request, secret_response; - - struct snp_secrets_page_layout *layout; - struct snp_req_data input; - unsigned int vmpck_id; -}; - static u32 vmpck_id; module_param(vmpck_id, uint, 0444); MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with th= e PSP."); @@ -66,95 +40,6 @@ static inline unsigned int get_ctx_authsize(struct snp_g= uest_dev *snp_dev) return 0; } =20 -static inline u8 *snp_get_vmpck(struct snp_guest_dev *snp_dev) -{ - return snp_dev->layout->vmpck0 + snp_dev->vmpck_id * VMPCK_KEY_LEN; -} - -static inline u32 *snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev) -{ - return &snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id; -} - -static bool snp_is_vmpck_empty(struct snp_guest_dev *snp_dev) -{ - char zero_key[VMPCK_KEY_LEN] =3D {0}; - u8 *key =3D snp_get_vmpck(snp_dev); - - return !memcmp(key, zero_key, VMPCK_KEY_LEN); -} - -/* - * If an error is received from the host or AMD Secure Processor (ASP) the= re - * are two options. Either retry the exact same encrypted request or disco= ntinue - * using the VMPCK. - * - * This is because in the current encryption scheme GHCB v2 uses AES-GCM to - * encrypt the requests. The IV for this scheme is the sequence number. GCM - * cannot tolerate IV reuse. - * - * The ASP FW v1.51 only increments the sequence numbers on a successful - * guest<->ASP back and forth and only accepts messages at its exact seque= nce - * number. - * - * So if the sequence number were to be reused the encryption scheme is - * vulnerable. If the sequence number were incremented for a fresh IV the = ASP - * will reject the request. - */ -static void snp_disable_vmpck(struct snp_guest_dev *snp_dev) -{ - u8 *key =3D snp_get_vmpck(snp_dev); - - dev_alert(snp_dev->dev, "Disabling vmpck_id %d to prevent IV reuse.\n", - snp_dev->vmpck_id); - memzero_explicit(key, VMPCK_KEY_LEN); -} - -static inline u64 __snp_get_msg_seqno(struct snp_guest_dev *snp_dev) -{ - u32 *os_area_msg_seqno =3D snp_get_os_area_msg_seqno(snp_dev); - u64 count; - - lockdep_assert_held(&snp_dev->cmd_mutex); - - /* Read the current message sequence counter from secrets pages */ - count =3D *os_area_msg_seqno; - - return count + 1; -} - -/* Return a non-zero on success */ -static u64 snp_get_msg_seqno(struct snp_guest_dev *snp_dev) -{ - u64 count =3D __snp_get_msg_seqno(snp_dev); - - /* - * The message sequence counter for the SNP guest request is a 64-bit - * value but the version 2 of GHCB specification defines a 32-bit storage - * for it. If the counter exceeds the 32-bit value then return zero. - * The caller should check the return value, but if the caller happens to - * not check the value and use it, then the firmware treats zero as an - * invalid number and will fail the message request. - */ - if (count >=3D UINT_MAX) { - dev_err(snp_dev->dev, "request message sequence counter overflow\n"); - return 0; - } - - return count; -} - -static void snp_inc_msg_seqno(struct snp_guest_dev *snp_dev) -{ - u32 *os_area_msg_seqno =3D snp_get_os_area_msg_seqno(snp_dev); - - /* - * The counter is also incremented by the PSP, so increment it by 2 - * and save in secrets page. - */ - *os_area_msg_seqno +=3D 2; -} - static inline struct snp_guest_dev *to_snp_dev(struct file *file) { struct miscdevice *dev =3D file->private_data; @@ -162,255 +47,6 @@ static inline struct snp_guest_dev *to_snp_dev(struct = file *file) return container_of(dev, struct snp_guest_dev, misc); } =20 -static struct aesgcm_ctx *snp_init_crypto(struct snp_guest_dev *snp_dev) -{ - struct aesgcm_ctx *ctx; - u8 *key; - - if (snp_is_vmpck_empty(snp_dev)) { - pr_err("SNP: vmpck id %d is null\n", snp_dev->vmpck_id); - return NULL; - } - - ctx =3D kzalloc(sizeof(*ctx), GFP_KERNEL_ACCOUNT); - if (!ctx) - return NULL; - - key =3D snp_get_vmpck(snp_dev); - if (aesgcm_expandkey(ctx, key, VMPCK_KEY_LEN, AUTHTAG_LEN)) { - pr_err("SNP: crypto init failed\n"); - kfree(ctx); - return NULL; - } - - return ctx; -} - -static int __enc_payload(struct aesgcm_ctx *ctx, struct snp_guest_msg *msg, - void *plaintext, size_t len) -{ - struct snp_guest_msg_hdr *hdr =3D &msg->hdr; - u8 iv[GCM_AES_IV_SIZE] =3D {}; - - if (WARN_ON((hdr->msg_sz + ctx->authsize) > sizeof(msg->payload))) - return -EBADMSG; - - memcpy(iv, &hdr->msg_seqno, sizeof(hdr->msg_seqno)); - aesgcm_encrypt(ctx, msg->payload, plaintext, len, &hdr->algo, AAD_LEN, - iv, hdr->authtag); - return 0; -} - -static int dec_payload(struct aesgcm_ctx *ctx, struct snp_guest_msg *msg, - void *plaintext, size_t len) -{ - struct snp_guest_msg_hdr *hdr =3D &msg->hdr; - u8 iv[GCM_AES_IV_SIZE] =3D {}; - - memcpy(iv, &hdr->msg_seqno, sizeof(hdr->msg_seqno)); - if (aesgcm_decrypt(ctx, plaintext, msg->payload, len, &hdr->algo, - AAD_LEN, iv, hdr->authtag)) - return 0; - else - return -EBADMSG; -} - -static int verify_and_dec_payload(struct snp_guest_dev *snp_dev, struct sn= p_guest_req *guest_req) -{ - struct snp_guest_msg *resp =3D &snp_dev->secret_response; - struct snp_guest_msg *req =3D &snp_dev->secret_request; - struct snp_guest_msg_hdr *req_hdr =3D &req->hdr; - struct snp_guest_msg_hdr *resp_hdr =3D &resp->hdr; - struct aesgcm_ctx *ctx =3D snp_dev->ctx; - - pr_debug("response [seqno %lld type %d version %d sz %d]\n", - resp_hdr->msg_seqno, resp_hdr->msg_type, resp_hdr->msg_version, - resp_hdr->msg_sz); - - /* Copy response from shared memory to encrypted memory. */ - memcpy(resp, snp_dev->response, sizeof(*resp)); - - /* Verify that the sequence counter is incremented by 1 */ - if (unlikely(resp_hdr->msg_seqno !=3D (req_hdr->msg_seqno + 1))) - return -EBADMSG; - - /* Verify response message type and version number. */ - if (resp_hdr->msg_type !=3D (req_hdr->msg_type + 1) || - resp_hdr->msg_version !=3D req_hdr->msg_version) - return -EBADMSG; - - /* - * If the message size is greater than our buffer length then return - * an error. - */ - if (unlikely((resp_hdr->msg_sz + ctx->authsize) > guest_req->resp_sz)) - return -EBADMSG; - - /* Decrypt the payload */ - return dec_payload(ctx, resp, guest_req->resp_buf, resp_hdr->msg_sz); -} - -static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, struct sn= p_guest_req *req) -{ - struct snp_guest_msg *msg =3D &snp_dev->secret_request; - struct snp_guest_msg_hdr *hdr =3D &msg->hdr; - - memset(msg, 0, sizeof(*msg)); - - hdr->algo =3D SNP_AEAD_AES_256_GCM; - hdr->hdr_version =3D MSG_HDR_VER; - hdr->hdr_sz =3D sizeof(*hdr); - hdr->msg_type =3D req->msg_type; - hdr->msg_version =3D req->msg_version; - hdr->msg_seqno =3D seqno; - hdr->msg_vmpck =3D req->vmpck_id; - hdr->msg_sz =3D req->req_sz; - - /* Verify the sequence number is non-zero */ - if (!hdr->msg_seqno) - return -ENOSR; - - pr_debug("request [seqno %lld type %d version %d sz %d]\n", - hdr->msg_seqno, hdr->msg_type, hdr->msg_version, hdr->msg_sz); - - return __enc_payload(snp_dev->ctx, msg, req->req_buf, req->req_sz); -} - -static int __handle_guest_request(struct snp_guest_dev *snp_dev, struct sn= p_guest_req *req, - struct snp_guest_request_ioctl *rio) -{ - unsigned long req_start =3D jiffies; - unsigned int override_npages =3D 0; - u64 override_err =3D 0; - int rc; - -retry_request: - /* - * Call firmware to process the request. In this function the encrypted - * message enters shared memory with the host. So after this call the - * sequence number must be incremented or the VMPCK must be deleted to - * prevent reuse of the IV. - */ - rc =3D snp_issue_guest_request(req, &snp_dev->input, rio); - switch (rc) { - case -ENOSPC: - /* - * If the extended guest request fails due to having too - * small of a certificate data buffer, retry the same - * guest request without the extended data request in - * order to increment the sequence number and thus avoid - * IV reuse. - */ - override_npages =3D *req->data_npages; - req->exit_code =3D SVM_VMGEXIT_GUEST_REQUEST; - - /* - * Override the error to inform callers the given extended - * request buffer size was too small and give the caller the - * required buffer size. - */ - override_err =3D SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVALID_LEN); - - /* - * If this call to the firmware succeeds, the sequence number can - * be incremented allowing for continued use of the VMPCK. If - * there is an error reflected in the return value, this value - * is checked further down and the result will be the deletion - * of the VMPCK and the error code being propagated back to the - * user as an ioctl() return code. - */ - goto retry_request; - - /* - * The host may return SNP_GUEST_VMM_ERR_BUSY if the request has been - * throttled. Retry in the driver to avoid returning and reusing the - * message sequence number on a different message. - */ - case -EAGAIN: - if (jiffies - req_start > SNP_REQ_MAX_RETRY_DURATION) { - rc =3D -ETIMEDOUT; - break; - } - schedule_timeout_killable(SNP_REQ_RETRY_DELAY); - goto retry_request; - } - - /* - * Increment the message sequence number. There is no harm in doing - * this now because decryption uses the value stored in the response - * structure and any failure will wipe the VMPCK, preventing further - * use anyway. - */ - snp_inc_msg_seqno(snp_dev); - - if (override_err) { - rio->exitinfo2 =3D override_err; - - /* - * If an extended guest request was issued and the supplied certificate - * buffer was not large enough, a standard guest request was issued to - * prevent IV reuse. If the standard request was successful, return -EIO - * back to the caller as would have originally been returned. - */ - if (!rc && override_err =3D=3D SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVAL= ID_LEN)) - rc =3D -EIO; - } - - if (override_npages) - *req->data_npages =3D override_npages; - - return rc; -} - -static int snp_send_guest_request(struct snp_guest_dev *snp_dev, struct sn= p_guest_req *req, - struct snp_guest_request_ioctl *rio) -{ - u64 seqno; - int rc; - - /* Get message sequence and verify that its a non-zero */ - seqno =3D snp_get_msg_seqno(snp_dev); - if (!seqno) - return -EIO; - - /* Clear shared memory's response for the host to populate. */ - memset(snp_dev->response, 0, sizeof(struct snp_guest_msg)); - - /* Encrypt the userspace provided payload in snp_dev->secret_request. */ - rc =3D enc_payload(snp_dev, seqno, req); - if (rc) - return rc; - - /* - * Write the fully encrypted request to the shared unencrypted - * request page. - */ - memcpy(snp_dev->request, &snp_dev->secret_request, - sizeof(snp_dev->secret_request)); - - rc =3D __handle_guest_request(snp_dev, req, rio); - if (rc) { - if (rc =3D=3D -EIO && - rio->exitinfo2 =3D=3D SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVALID_LE= N)) - return rc; - - dev_alert(snp_dev->dev, - "Detected error from ASP request. rc: %d, exitinfo2: 0x%llx\n", - rc, rio->exitinfo2); - snp_disable_vmpck(snp_dev); - return rc; - } - - rc =3D verify_and_dec_payload(snp_dev, req); - if (rc) { - dev_alert(snp_dev->dev, "Detected unexpected decode failure from ASP. rc= : %d\n", rc); - snp_disable_vmpck(snp_dev); - return rc; - } - - return 0; -} - static int get_report(struct snp_guest_dev *snp_dev, struct snp_guest_requ= est_ioctl *arg) { struct snp_guest_req guest_req =3D {0}; @@ -610,7 +246,7 @@ static long snp_guest_ioctl(struct file *file, unsigned= int ioctl, unsigned long mutex_lock(&snp_dev->cmd_mutex); =20 /* Check if the VMPCK is not empty */ - if (snp_is_vmpck_empty(snp_dev)) { + if (snp_is_vmpck_empty(snp_dev->vmpck_id)) { dev_err_ratelimited(snp_dev->dev, "VMPCK is disabled\n"); mutex_unlock(&snp_dev->cmd_mutex); return -ENOTTY; @@ -638,147 +274,63 @@ static long snp_guest_ioctl(struct file *file, unsig= ned int ioctl, unsigned long return ret; } =20 -static void free_shared_pages(void *buf, size_t sz) -{ - unsigned int npages =3D PAGE_ALIGN(sz) >> PAGE_SHIFT; - int ret; - - if (!buf) - return; - - ret =3D set_memory_encrypted((unsigned long)buf, npages); - if (ret) { - WARN_ONCE(ret, "failed to restore encryption mask (leak it)\n"); - return; - } - - __free_pages(virt_to_page(buf), get_order(sz)); -} - -static void *alloc_shared_pages(struct device *dev, size_t sz) -{ - unsigned int npages =3D PAGE_ALIGN(sz) >> PAGE_SHIFT; - struct page *page; - int ret; - - page =3D alloc_pages(GFP_KERNEL_ACCOUNT, get_order(sz)); - if (!page) - return NULL; - - ret =3D set_memory_decrypted((unsigned long)page_address(page), npages); - if (ret) { - dev_err(dev, "failed to mark page shared, ret=3D%d\n", ret); - __free_pages(page, get_order(sz)); - return NULL; - } - - return page_address(page); -} - static const struct file_operations snp_guest_fops =3D { .owner =3D THIS_MODULE, .unlocked_ioctl =3D snp_guest_ioctl, }; =20 -bool snp_assign_vmpck(struct snp_guest_dev *dev, int vmpck_id) -{ - if (WARN_ON(vmpck_id > 3)) - return false; - - dev->vmpck_id =3D vmpck_id; - - return true; -} - static int __init sev_guest_probe(struct platform_device *pdev) { - struct snp_secrets_page_layout *layout; - struct sev_guest_platform_data *data; struct device *dev =3D &pdev->dev; struct snp_guest_dev *snp_dev; struct miscdevice *misc; - void __iomem *mapping; int ret; =20 if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return -ENODEV; =20 - if (!dev->platform_data) - return -ENODEV; - - data =3D (struct sev_guest_platform_data *)dev->platform_data; - mapping =3D ioremap_encrypted(data->secrets_gpa, PAGE_SIZE); - if (!mapping) - return -ENODEV; - - layout =3D (__force void *)mapping; - - ret =3D -ENOMEM; snp_dev =3D devm_kzalloc(&pdev->dev, sizeof(struct snp_guest_dev), GFP_KE= RNEL); if (!snp_dev) - goto e_unmap; + return -ENOMEM; =20 - ret =3D -EINVAL; - snp_dev->layout =3D layout; if (!snp_assign_vmpck(snp_dev, vmpck_id)) { dev_err(dev, "invalid vmpck id %d\n", vmpck_id); - goto e_unmap; + ret =3D -EINVAL; + goto e_free_snpdev; } =20 - /* Verify that VMPCK is not zero. */ - if (snp_is_vmpck_empty(snp_dev)) { - dev_err(dev, "vmpck id %d is null\n", vmpck_id); - goto e_unmap; + if (snp_setup_psp_messaging(snp_dev)) { + dev_err(dev, "Unable to setup PSP messaging vmpck id %d\n", snp_dev->vmp= ck_id); + ret =3D -ENODEV; + goto e_free_snpdev; } =20 mutex_init(&snp_dev->cmd_mutex); platform_set_drvdata(pdev, snp_dev); snp_dev->dev =3D dev; =20 - /* Allocate the shared page used for the request and response message. */ - snp_dev->request =3D alloc_shared_pages(dev, sizeof(struct snp_guest_msg)= ); - if (!snp_dev->request) - goto e_unmap; - - snp_dev->response =3D alloc_shared_pages(dev, sizeof(struct snp_guest_msg= )); - if (!snp_dev->response) - goto e_free_request; - - snp_dev->certs_data =3D alloc_shared_pages(dev, SEV_FW_BLOB_MAX_SIZE); + snp_dev->certs_data =3D alloc_shared_pages(SEV_FW_BLOB_MAX_SIZE); if (!snp_dev->certs_data) - goto e_free_response; - - ret =3D -EIO; - snp_dev->ctx =3D snp_init_crypto(snp_dev); - if (!snp_dev->ctx) - goto e_free_cert_data; + goto e_free_ctx; =20 misc =3D &snp_dev->misc; misc->minor =3D MISC_DYNAMIC_MINOR; misc->name =3D DEVICE_NAME; misc->fops =3D &snp_guest_fops; =20 - /* initial the input address for guest request */ - snp_dev->input.req_gpa =3D __pa(snp_dev->request); - snp_dev->input.resp_gpa =3D __pa(snp_dev->response); - ret =3D misc_register(misc); if (ret) - goto e_free_ctx; + goto e_free_cert_data; =20 - dev_info(dev, "Initialized SEV guest driver (using vmpck_id %d)\n", vmpck= _id); + dev_info(dev, "Initialized SEV guest driver (using vmpck_id %d)\n", snp_d= ev->vmpck_id); return 0; =20 -e_free_ctx: - kfree(snp_dev->ctx); e_free_cert_data: free_shared_pages(snp_dev->certs_data, SEV_FW_BLOB_MAX_SIZE); -e_free_response: - free_shared_pages(snp_dev->response, sizeof(struct snp_guest_msg)); -e_free_request: - free_shared_pages(snp_dev->request, sizeof(struct snp_guest_msg)); -e_unmap: - iounmap(mapping); + e_free_ctx: + kfree(snp_dev->ctx); +e_free_snpdev: + kfree(snp_dev); return ret; } =20 @@ -786,11 +338,9 @@ static int __exit sev_guest_remove(struct platform_dev= ice *pdev) { struct snp_guest_dev *snp_dev =3D platform_get_drvdata(pdev); =20 - free_shared_pages(snp_dev->certs_data, SEV_FW_BLOB_MAX_SIZE); - free_shared_pages(snp_dev->response, sizeof(struct snp_guest_msg)); - free_shared_pages(snp_dev->request, sizeof(struct snp_guest_msg)); - kfree(snp_dev->ctx); misc_deregister(&snp_dev->misc); + kfree(snp_dev->ctx); + kfree(snp_dev); =20 return 0; } --=20 2.34.1