From nobody Thu Oct 2 07:46:35 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EFE92D595E for ; Fri, 19 Sep 2025 22:33:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758321188; cv=none; b=HoSQUlP3PMs2YG1RIqeg4l9owwhbWQTvLbBACMo9eXsaIkDRx/jjhRyn/oCasMhN33px2gMDng5/PQcMVCN/PBDN5mwCyn12rNZvO6AswcKtO06f+YUIRFF7TZDNGITSBWt4Ik1oFDSkOG/3YQWq6A6WcJUc+y0j+L83jN2GMQY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758321188; c=relaxed/simple; bh=MGoqbJuB9sCOizQgDYnMaXG/P8e3uUL1d4ZDtIlGQ04=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QEmuN/vz0h04h8uI7BbefL9cgqxEdtoIdbL9BWEAfFpabb3sWOAEJzZ+sDaxdfmgTkcJHnn8Gue/aslrjMj/XcZGU604diUCO9Dy9a4JwvCe9UsRuPFOiVkCApxx0oLDsDBJEpnLbyr5YGCaHXqdKAIhL5VfiQ9bUL/qRIsbZNc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=df4Pb4fN; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="df4Pb4fN" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-77dff701fb8so1506476b3a.1 for ; Fri, 19 Sep 2025 15:33:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758321186; x=1758925986; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=wqYcd9euZlCyN9BS3XgrU5fRkOaykxhfX02wnyNgUOs=; b=df4Pb4fN1+5xBwOnUfndrG3L3hSpJctLcp8n5wAXXz6bhRz1verdI56BsjXua+I3DN XS/r5MPmSQ/odHbfKz8QWEl1PsmWYjIZpCvauFWayWLWBKMnCt5NLrGldxIzQqFbswxY TuwXlXl+cWM7ZmKnQ+HI35xrn8WojK2KTAEmtJPeHWKym0ZDFjIoHnn3wn5nhY7+MEQ7 af3mLAnaIB7a0ZXbZ9yLUc9H8GOOcGhtql8z2dnFRC0gw1US5TsYlcoqB/ZVGcbfxBge bK3RYjgnmWyV8g7YrY6LNBhZhkG+FbO4u0PO2M63gsLcRZGf8rWNkWPPh8ryxC0Ge8NG blow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758321186; x=1758925986; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wqYcd9euZlCyN9BS3XgrU5fRkOaykxhfX02wnyNgUOs=; b=aWIOykviabYUBxJHCgk4kfSLO90W/msf8isDQXdQt9yiJaeMqcg2Lk3Sb9hApDgDsQ WJM4JqKkXidpM1ZShbuwPeEFrK7aKrC6wq+vjeF40zIPDg4d2A4SnmnC6ZYUjL9hS5Sb KBWEnKSOpVXEMpeIAq9smx70m3YxukLnX4pqE11RHs2x5HUJ7+s3KTXEoAst1jDJzsPS kwJ9K2PnbdGd6Q+Sh0Wi5TVLwqLCkg14gPkx6VXgpsjaXgQU+vh2bDpzrImnseUvSdNg 1mIX4R0RtP26cgSySpDuQKp1SJYVpTFhTfVsv1WER2wY9NW7d9dVkuo8r7vHlO5qk1/k RUAA== X-Forwarded-Encrypted: i=1; AJvYcCUT0/mPXFWnbr42NH1mgkMhqJT8zlyMXlOzMn0VSW4PoNgFa93lmQxwMxQIIriEZ/qZl7EO+SkKcoXqCJM=@vger.kernel.org X-Gm-Message-State: AOJu0YxxFVQ9C+IJmFfkkpJGcQa8dOS0oWTD6UmpHT7oRf6bJcB8ceDV JOW/BUSHN3LsFRpqlJDwG1hFoJFV6L/grFY5fjeEViAp3eNizBPX59O+J2E3OWY+/vwITeCBEmG mcz5F8g== X-Google-Smtp-Source: AGHT+IHa7+GTGOB/OfVBevbILqcp7CNMwVxukhCmqhE5euiMqraGlNNrlrM+jwtI07qDvfdB/ch8vTN5P0k= X-Received: from pfmm21.prod.google.com ([2002:a05:6a00:2495:b0:77f:1cc1:89eb]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1706:b0:774:615b:c8ad with SMTP id d2e1a72fcca58-77e4d127eb2mr6222337b3a.9.1758321186510; Fri, 19 Sep 2025 15:33:06 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 19 Sep 2025 15:32:09 -0700 In-Reply-To: <20250919223258.1604852-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919223258.1604852-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919223258.1604852-3-seanjc@google.com> Subject: [PATCH v16 02/51] KVM: SEV: Read save fields from GHCB exactly once From: Sean Christopherson To: Paolo Bonzini , Sean Christopherson Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Mathias Krause , John Allen , Rick Edgecombe , Chao Gao , Binbin Wu , Xiaoyao Li , Maxim Levitsky , Zhang Yi Z , Xin Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Wrap all reads of GHCB save fields with READ_ONCE() via a KVM-specific GHCB get() utility to help guard against TOCTOU bugs. Using READ_ONCE() doesn't completely prevent such bugs, e.g. doesn't prevent KVM from redoing get() after checking the initial value, but at least addresses all potential TOCTOU issues in the current KVM code base. To prevent unintentional use of the generic helpers, take only @svm for the kvm_ghcb_get_xxx() helpers and retrieve the ghcb instead of explicitly passing it in. Opportunistically reduce the indentation of the macro-defined helpers and clean up the alignment. Fixes: 4e15a0ddc3ff ("KVM: SEV: snapshot the GHCB before accessing it") Cc: Tom Lendacky Signed-off-by: Sean Christopherson Reviewed-by: Tom Lendacky --- arch/x86/kvm/svm/sev.c | 22 +++++++++++----------- arch/x86/kvm/svm/svm.h | 25 +++++++++++++++---------- 2 files changed, 26 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index f046a587ecaf..8d057dbd8a71 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3343,26 +3343,26 @@ static void sev_es_sync_from_ghcb(struct vcpu_svm *= svm) BUILD_BUG_ON(sizeof(svm->sev_es.valid_bitmap) !=3D sizeof(ghcb->save.vali= d_bitmap)); memcpy(&svm->sev_es.valid_bitmap, &ghcb->save.valid_bitmap, sizeof(ghcb->= save.valid_bitmap)); =20 - vcpu->arch.regs[VCPU_REGS_RAX] =3D kvm_ghcb_get_rax_if_valid(svm, ghcb); - vcpu->arch.regs[VCPU_REGS_RBX] =3D kvm_ghcb_get_rbx_if_valid(svm, ghcb); - vcpu->arch.regs[VCPU_REGS_RCX] =3D kvm_ghcb_get_rcx_if_valid(svm, ghcb); - vcpu->arch.regs[VCPU_REGS_RDX] =3D kvm_ghcb_get_rdx_if_valid(svm, ghcb); - vcpu->arch.regs[VCPU_REGS_RSI] =3D kvm_ghcb_get_rsi_if_valid(svm, ghcb); + vcpu->arch.regs[VCPU_REGS_RAX] =3D kvm_ghcb_get_rax_if_valid(svm); + vcpu->arch.regs[VCPU_REGS_RBX] =3D kvm_ghcb_get_rbx_if_valid(svm); + vcpu->arch.regs[VCPU_REGS_RCX] =3D kvm_ghcb_get_rcx_if_valid(svm); + vcpu->arch.regs[VCPU_REGS_RDX] =3D kvm_ghcb_get_rdx_if_valid(svm); + vcpu->arch.regs[VCPU_REGS_RSI] =3D kvm_ghcb_get_rsi_if_valid(svm); =20 - svm->vmcb->save.cpl =3D kvm_ghcb_get_cpl_if_valid(svm, ghcb); + svm->vmcb->save.cpl =3D kvm_ghcb_get_cpl_if_valid(svm); =20 if (kvm_ghcb_xcr0_is_valid(svm)) { - vcpu->arch.xcr0 =3D ghcb_get_xcr0(ghcb); + vcpu->arch.xcr0 =3D kvm_ghcb_get_xcr0(svm); vcpu->arch.cpuid_dynamic_bits_dirty =3D true; } =20 /* Copy the GHCB exit information into the VMCB fields */ - exit_code =3D ghcb_get_sw_exit_code(ghcb); + exit_code =3D kvm_ghcb_get_sw_exit_code(svm); control->exit_code =3D lower_32_bits(exit_code); control->exit_code_hi =3D upper_32_bits(exit_code); - control->exit_info_1 =3D ghcb_get_sw_exit_info_1(ghcb); - control->exit_info_2 =3D ghcb_get_sw_exit_info_2(ghcb); - svm->sev_es.sw_scratch =3D kvm_ghcb_get_sw_scratch_if_valid(svm, ghcb); + control->exit_info_1 =3D kvm_ghcb_get_sw_exit_info_1(svm); + control->exit_info_2 =3D kvm_ghcb_get_sw_exit_info_2(svm); + svm->sev_es.sw_scratch =3D kvm_ghcb_get_sw_scratch_if_valid(svm); =20 /* Clear the valid entries fields */ memset(ghcb->save.valid_bitmap, 0, sizeof(ghcb->save.valid_bitmap)); diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 5d39c0b17988..5365984e82e5 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -913,16 +913,21 @@ void __svm_sev_es_vcpu_run(struct vcpu_svm *svm, bool= spec_ctrl_intercepted, void __svm_vcpu_run(struct vcpu_svm *svm, bool spec_ctrl_intercepted); =20 #define DEFINE_KVM_GHCB_ACCESSORS(field) \ - static __always_inline bool kvm_ghcb_##field##_is_valid(const struct vcpu= _svm *svm) \ - { \ - return test_bit(GHCB_BITMAP_IDX(field), \ - (unsigned long *)&svm->sev_es.valid_bitmap); \ - } \ - \ - static __always_inline u64 kvm_ghcb_get_##field##_if_valid(struct vcpu_sv= m *svm, struct ghcb *ghcb) \ - { \ - return kvm_ghcb_##field##_is_valid(svm) ? ghcb->save.field : 0; \ - } \ +static __always_inline u64 kvm_ghcb_get_##field(struct vcpu_svm *svm) \ +{ \ + return READ_ONCE(svm->sev_es.ghcb->save.field); \ +} \ + \ +static __always_inline bool kvm_ghcb_##field##_is_valid(const struct vcpu_= svm *svm) \ +{ \ + return test_bit(GHCB_BITMAP_IDX(field), \ + (unsigned long *)&svm->sev_es.valid_bitmap); \ +} \ + \ +static __always_inline u64 kvm_ghcb_get_##field##_if_valid(struct vcpu_svm= *svm) \ +{ \ + return kvm_ghcb_##field##_is_valid(svm) ? kvm_ghcb_get_##field(svm) : 0; \ +} =20 DEFINE_KVM_GHCB_ACCESSORS(cpl) DEFINE_KVM_GHCB_ACCESSORS(rax) --=20 2.51.0.470.ga7dc726c21-goog