From nobody Fri Dec 19 22:07:47 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC71A28BABC for ; Thu, 15 May 2025 22:04:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747346661; cv=none; b=MlThcT3xUTYMoExtF3Z/2T9R0wRDFBZUPA52VGsVabksODNokV5gM+wWGt0hFDF9K7BluxHKnhXcWv4RBSNVTd1Ykea4WGZUhvkgwHo90IZifVdxtXgcucjyZPixQWgma7gt+teDlrWN0gNzydFcr29GJcsapZEAGypPc5lA9qY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747346661; c=relaxed/simple; bh=6eVULcLPVdzazGE/wg9Pf+TgKc9aOyrX0eUUEgwYzJU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=S5meRohIKGujtfvslMkztNLiyv9n4xnl02xqcWfOCpbLG6xanD8FPrOfgn3l8gC+l2DhNrntn7xiQRn1Wfw3JIw9xeYSiYnuQpjguweQLyUvMyhEe2meZPrK56WPvdgDNusuQbg//NJvOaRKDCthVs5Xju/AmZ8sI9QnGSkATKQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dionnaglaze.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IfTiBaJP; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dionnaglaze.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IfTiBaJP" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-22fafbbe51fso20014025ad.0 for ; Thu, 15 May 2025 15:04:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747346657; x=1747951457; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/61l/GG8ceY1HQ/1Mbz9W8+Z6NMWlwdTU/z1QWzDW24=; b=IfTiBaJP3o9wJ19dNr8ddF9fWihDsU31Lr1QB/1+vtwdoH0PPCoJ6r2I7//b1xQqJN Y/4JQRxnSM+UWvzEgxDF3te2DMIKMWSUT7TFJA2WRCWqpXvShDwzMXyqyGXkI6tcK02l 6smFp/5AxXwILffQXqzl223sz4ryqk9hFmH8aoH4T3CHXyCIRuVl/DV+rCZX0SUv1HMj quyNefiPPzqg8TkS7xUpYunH2BuoOATtZFbnOqRwFIn6QWaeKD6tDOdcaVC0DWr0RkL+ rtKVKPcu0os/JMpweGw0u74us2Fh35e2CPcOh8c7t3e8gC1ac+vs91O2Gp3ZHc507AnT n5Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747346657; x=1747951457; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/61l/GG8ceY1HQ/1Mbz9W8+Z6NMWlwdTU/z1QWzDW24=; b=WCeMEJEcUQkG9X13f3k4yoZ6gi73gkYfh1/tClSIC3ga2dU5s2bbuDdYT/C0sxl1Bt jXE8M1mJ80KfakNjyZXm1F2zcWPHkCRmZoUK0Wrousnup63OnBQ6OmSb3fpP45q0eo+9 6zgLchBO/Ol11Y/mvhFgjoo2N9vijr+nzC/dEt3uF4cMn066ry39oOc4+fa7ecbgNefs VsE3e1B5IkwI/1s6Ef+V2ZrTTBMVvhCjxXDpeE1NjtcXp3e2u3XeUBkKs1pa6cJ/uXiq 0imhsynh1Ij66QPEKr3QBPCTrrruCdnPXvejquYesLoBoP0yVWgD2UcqVt+y2GC/0Ut2 T2GA== X-Gm-Message-State: AOJu0YzR5G9egY2v94nvBBeJjr4rNnJjzajiud/S6ylHx60/eKXyUxji O5CHjVpWQM7DDIhG4u3iHeTbU1KBB8O5z3tWGYl8V6JQiWf2hO/ulSW0JPzvVnkNxbcFX4+5f7r sM/ZWqeBDv6SlRSeV2X6G7xcJvg== X-Google-Smtp-Source: AGHT+IEIogqWVI3jhvBuBHzH/RVC4nU9qls/xo8WHotmwx8ma/XGj5KxgExpQP3h4WgyLHLR1u3cujYZAXUA4Janbg== X-Received: from plge2.prod.google.com ([2002:a17:902:cf42:b0:226:453a:f2db]) (user=dionnaglaze job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:19c4:b0:220:e655:d77 with SMTP id d9443c01a7336-231d452d0e3mr12729205ad.36.1747346657095; Thu, 15 May 2025 15:04:17 -0700 (PDT) Date: Thu, 15 May 2025 22:03:59 +0000 In-Reply-To: <20250515220400.1096945-1-dionnaglaze@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250515220400.1096945-1-dionnaglaze@google.com> X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250515220400.1096945-2-dionnaglaze@google.com> Subject: [PATCH v5 1/2] kvm: sev: Add SEV-SNP guest request throttling From: Dionna Glaze To: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, Dionna Glaze , Thomas Lendacky , Paolo Bonzini , Joerg Roedel , Peter Gonda , Borislav Petkov , Sean Christopherson Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The AMD-SP is a precious resource that doesn't have a scheduler other than a mutex lock queue. To avoid customers from causing a DoS, a mem_enc_ioctl command for rate limiting guest requests is added. Recommended values are {.interval_ms =3D 1000, .burst =3D 1} or {.interval_ms =3D 2000, .burst =3D 2} to average 1 request every second. You may need to allow 2 requests back to back to allow for the guest to query the certificate length in an extended guest request without a pause. The 1 second average is our target for quality of service since empirical tests show that 64 VMs can concurrently request an attestation report with a maximum latency of 1 second. We don't anticipate more concurrency than that for a seldom used request for a majority well-behaved set of VMs. The majority point is decided as >64 VMs given the assumed 128 VM count for "extreme load". Cc: Thomas Lendacky Cc: Paolo Bonzini Cc: Joerg Roedel Cc: Peter Gonda Cc: Borislav Petkov Cc: Sean Christopherson Signed-off-by: Dionna Glaze --- .../virt/kvm/x86/amd-memory-encryption.rst | 23 +++++++++++++ arch/x86/include/uapi/asm/kvm.h | 7 ++++ arch/x86/kvm/svm/sev.c | 33 +++++++++++++++++++ arch/x86/kvm/svm/svm.h | 3 ++ 4 files changed, 66 insertions(+) diff --git a/Documentation/virt/kvm/x86/amd-memory-encryption.rst b/Documen= tation/virt/kvm/x86/amd-memory-encryption.rst index 1ddb6a86ce7f..1b5b4fc35aac 100644 --- a/Documentation/virt/kvm/x86/amd-memory-encryption.rst +++ b/Documentation/virt/kvm/x86/amd-memory-encryption.rst @@ -572,6 +572,29 @@ Returns: 0 on success, -negative on error See SNP_LAUNCH_FINISH in the SEV-SNP specification [snp-fw-abi]_ for furth= er details on the input parameters in ``struct kvm_sev_snp_launch_finish``. =20 +21. KVM_SEV_SNP_SET_REQUEST_THROTTLE_RATE +----------------------------------------- + +The KVM_SEV_SNP_SET_REQUEST_THROTTLE_RATE command is used to set a per-VM = rate +limit on responding to requests for AMD-SP to process a guest request. +The AMD-SP is a global resource with limited capacity, so to avoid noisy +neighbor effects, the host may set a request rate for guests. + +Parameters (in): struct kvm_sev_snp_set_request_throttle_rate + +Returns: 0 on success, -negative on error + +:: + + struct kvm_sev_snp_set_request_throttle_rate { + __u32 interval_ms; + __u32 burst; + }; + +The interval will be translated into jiffies, so if it after transformation +the interval is 0, the command will return ``-EINVAL``. The ``burst`` value +must be greater than 0. + Device attribute API =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kv= m.h index 460306b35a4b..d92242d9b9af 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -708,6 +708,8 @@ enum sev_cmd_id { KVM_SEV_SNP_LAUNCH_UPDATE, KVM_SEV_SNP_LAUNCH_FINISH, =20 + KVM_SEV_SNP_SET_REQUEST_THROTTLE_RATE, + KVM_SEV_NR_MAX, }; =20 @@ -877,6 +879,11 @@ struct kvm_sev_snp_launch_finish { __u64 pad1[4]; }; =20 +struct kvm_sev_snp_set_request_throttle_rate { + __u32 interval_ms; + __u32 burst; +}; + #define KVM_X2APIC_API_USE_32BIT_IDS (1ULL << 0) #define KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK (1ULL << 1) =20 diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index a7a7dc507336..35b04a10ed73 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -12,12 +12,14 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include #include =20 @@ -2535,6 +2537,28 @@ static int snp_launch_finish(struct kvm *kvm, struct= kvm_sev_cmd *argp) return ret; } =20 +static int snp_set_request_throttle_ms(struct kvm *kvm, struct kvm_sev_cmd= *argp) +{ + struct kvm_sev_info *sev =3D to_kvm_sev_info(kvm); + struct kvm_sev_snp_set_request_throttle_rate params; + u64 jiffies; + + if (!sev_snp_guest(kvm)) + return -ENOTTY; + + if (copy_from_user(¶ms, u64_to_user_ptr(argp->data), sizeof(params))) + return -EFAULT; + + jiffies =3D ((u64)params.interval_ms * HZ) / 1000; + + if (!jiffies || !params.burst || params.burst > S32_MAX || jiffies > S32_= MAX) + return -EINVAL; + + ratelimit_state_init(&sev->snp_guest_msg_rs, jiffies, params.burst); + + return 0; +} + int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { struct kvm_sev_cmd sev_cmd; @@ -2640,6 +2664,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *a= rgp) case KVM_SEV_SNP_LAUNCH_FINISH: r =3D snp_launch_finish(kvm, &sev_cmd); break; + case KVM_SEV_SNP_SET_REQUEST_THROTTLE_RATE_MS: + r =3D snp_set_request_throttle_ms(kvm, &sev_cmd); + break; default: r =3D -EINVAL; goto out; @@ -4015,6 +4042,12 @@ static int snp_handle_guest_req(struct vcpu_svm *svm= , gpa_t req_gpa, gpa_t resp_ =20 mutex_lock(&sev->guest_req_mutex); =20 + if (!__ratelimit(&sev->snp_guest_msg_rs)) { + svm_vmgexit_no_action(svm, SNP_GUEST_ERR(SNP_GUEST_VMM_ERR_BUSY, 0)); + ret =3D 1; + goto out_unlock; + } + if (kvm_read_guest(kvm, req_gpa, sev->guest_req_buf, PAGE_SIZE)) { ret =3D -EIO; goto out_unlock; diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index f16b068c4228..2643c940d054 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -18,6 +18,7 @@ #include #include #include +#include =20 #include #include @@ -112,6 +113,8 @@ struct kvm_sev_info { void *guest_req_buf; /* Bounce buffer for SNP Guest Request input */ void *guest_resp_buf; /* Bounce buffer for SNP Guest Request output */ struct mutex guest_req_mutex; /* Must acquire before using bounce buffers= */ + + struct ratelimit_state snp_guest_msg_rs; /* Limit guest requests */ }; =20 struct kvm_svm { --=20 2.49.0.1101.gccaa498523-goog From nobody Fri Dec 19 22:07:47 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61EAC28DEEF for ; Thu, 15 May 2025 22:04:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747346660; cv=none; b=dBToX+FHIMrJ8QbWty4PFFdzTzgVFmkAKir3Xl8uV4lIXBDeGzFk5y2Nmmem0k0uf51ArlNOs8Xo53RiBdmrEXB6KaH+HJ+1AcKeVBxJHufj6HXrstk6FdqYyiju3YVCAXQnVIgI13HdQrDwUbd2ZUgzmhQtyOA/4o7+AhShHbQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747346660; c=relaxed/simple; bh=mmLKrWU9BMWCBbc2PskOjz9u++A6Zg7rZw9yObFxywE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ddK8RX3sxi4zF9k6MEP+NjJOw7bI6fx6sqavp91Y6ki3huuAFxDfUWq/HMnZwLzMJIKRtNBIVAUQO/DHdVcYkG6BkOofjnJZ2C4wpmrOhj8KiHtYCHLEue5IhiNX09IuWwxggxRwI1I7LCu9pNSF/Paj/XW8jPfh0V3KjSY1EMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dionnaglaze.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PX+fnWIN; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dionnaglaze.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PX+fnWIN" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-22e816139b8so12415355ad.3 for ; Thu, 15 May 2025 15:04:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747346658; x=1747951458; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=da2OQuvEUljiX7sOqbLfJ1kNKJczIhzC2yFs3FB784s=; b=PX+fnWIN3U99SnP2YbY6QK+x80N1QqXkdUjsmMWg18O48j1Tv+knpGZhdVSW5tQIZs EFt29EC8NR1ZDLmm4MmxLX4uBO29Ir59uQB5QN9Aq6u4cHaAFlj1YIF9UnLma/o49P/b oM0ZSY++UVbvKoN3YYpIzJWE3MRvOosvwDOa6gLnKjBAx+dj9mqwwyV/DGRv92by2Z4I uOXkmETFxF+RaGUVYR7yqCIbBM7xFuUi0USaqMddLyEUVpomwyHRQOpu93w9dqQs/r5y fOSCWAMPamB/WAOZHrvxeptATD5mx3hsPpMEOg44ntaYobsQ6zufSb0+/1Iv3/DZOUdO 2k3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747346658; x=1747951458; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=da2OQuvEUljiX7sOqbLfJ1kNKJczIhzC2yFs3FB784s=; b=D6LspGob46prMIjMhKuDI1AtIuEf8EPZrFU5kh63LghkKlHgRWbSo+g6DeGql4uTjB 1nWfC7sasJc6rl7cF6plbS5nJI5VTsXGUW2242t3BDKhAQ47+VLqMIA6cX3ysG6FVuK5 Feh77Mugy5svrYKd5Nd4SxwQgy9kM2e0MH3GjFYytvgdbna09ktVhwZPQjNDbzov/E90 69vlRwsOJuMMUzCPS37wOrHf4OxgjUos0M/9KHkbVV15nFcJLqSX1p97YFNa5bsdAd2s XU42orI/nSfKd4AdhuAJ/b665xiI6mpKpyhrbOzZZ8oxNPACkCY3cL/Oq1VYwy/0Fgql 8imw== X-Gm-Message-State: AOJu0Yx2zL1ULZ/msaa/c8ukwgt3hbjvkzymiKa4VqZ9tEakfp/Nv5Ub UhiuWnYcyRPZNdF/WbBRWm1npSEMgtpIwWtZ69h+LAj5KtV+xgAqaxs8R5U70U5P9QHvYPLGb5o kowX3GBqHKrGRSX7KIRPB8AZmBg== X-Google-Smtp-Source: AGHT+IGlf26I+aKQ0s1+qKRvgDiVtukbmuiTudxffeFLhCnrwEABrd4/+TM3/By71fTsSsPwRSUcRT9wY+f7sv5Vlw== X-Received: from plge2.prod.google.com ([2002:a17:902:cf42:b0:231:c831:9520]) (user=dionnaglaze job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e787:b0:224:26fd:82e5 with SMTP id d9443c01a7336-231d455d9ddmr12636465ad.48.1747346658529; Thu, 15 May 2025 15:04:18 -0700 (PDT) Date: Thu, 15 May 2025 22:04:00 +0000 In-Reply-To: <20250515220400.1096945-1-dionnaglaze@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250515220400.1096945-1-dionnaglaze@google.com> X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250515220400.1096945-3-dionnaglaze@google.com> Subject: [PATCH v5 2/2] kvm: sev: If ccp is busy, report busy to guest From: Dionna Glaze To: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, Dionna Glaze , Thomas Lendacky , Paolo Bonzini , Joerg Roedel , Peter Gonda , Borislav Petkov , Sean Christopherson Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The ccp driver can be overloaded even with guest request rate limits. The return value of -EBUSY means that there is no firmware error to report back to user space, so the guest VM would see this as exitinfo2 =3D 0. The false success can trick the guest to update its message sequence number when it shouldn't have. Instead, when ccp returns -EBUSY, that is reported to userspace as the throttling return value. Cc: Thomas Lendacky Cc: Paolo Bonzini Cc: Joerg Roedel Cc: Peter Gonda Cc: Borislav Petkov Cc: Sean Christopherson Signed-off-by: Dionna Glaze --- arch/x86/kvm/svm/sev.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 35b04a10ed73..884ab3f54fca 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -4063,6 +4063,11 @@ static int snp_handle_guest_req(struct vcpu_svm *svm= , gpa_t req_gpa, gpa_t resp_ * the PSP is dead and commands are timing out. */ ret =3D sev_issue_cmd(kvm, SEV_CMD_SNP_GUEST_REQUEST, &data, &fw_err); + if (ret =3D=3D -EBUSY) { + svm_vmgexit_no_action(svm, SNP_GUEST_ERR(SNP_GUEST_VMM_ERR_BUSY, fw_err)= ); + ret =3D 1; + goto out_unlock; + } if (ret && !fw_err) goto out_unlock; =20 --=20 2.49.0.1101.gccaa498523-goog