From nobody Wed Feb 11 15:25:16 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56DFAC6FD1D for ; Fri, 7 Apr 2023 20:20:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231289AbjDGUUo (ORCPT ); Fri, 7 Apr 2023 16:20:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231283AbjDGUU3 (ORCPT ); Fri, 7 Apr 2023 16:20:29 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF449CA1E for ; Fri, 7 Apr 2023 13:20:11 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 204-20020a250fd5000000b00b6d6655dc35so42735805ybp.6 for ; Fri, 07 Apr 2023 13:20:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680898810; x=1683490810; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=88YN7x3qppPMVMra5pRzHHPVPBEL4gMuQXheETIYTNU=; b=OMW9kzjNG7JkuumKUv1gxM5ClRnzMIgI7kcKO2DxeN5LnLC/y/rEnRoSpywnKQs6hD xlQD/OsuGnJ5npkky5Zeo0idUTx/ZeUjWrN65bVSD+9YE94mwbexeVuduMYRcJXkdWR9 c5e4/E0ZMYZkZuNx3ir9D1lXfiqPJjj8W9x34KqqTmzX6auXI6JlOD5uUwQyobkTZHkX GCYUe1C5vg1gq2woLG6c4aqn7ii1vyB75r4qVRPb3gbnB8AF+iJW/hONV03AuDt//Ykl +JKBwB+Km4olaMCfJvrbGcPiR60/pmR36bZ7vdgt09knNdseMPjEeCZ5XJB2aV2wWMRD M2Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680898810; x=1683490810; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=88YN7x3qppPMVMra5pRzHHPVPBEL4gMuQXheETIYTNU=; b=PqwT6TPWyaZCnjM5nJua/CMiBucQExiiWd3RbQaE60QBXvgbUz+HHZVcxzT0aSyZb0 1B+bMUfI4STuCQJqm8cGTdsxKGFuqx+l8AR0Y+8Gym3ZRM49C3Ejb/N2u0mf6oUurhyE 6YtOH5QSwF56wZM5SMBrD7IEh2h3UOTGTT2m2aCGPretjuWMEcLmFrlsqLOtXICfRbOn /8zhPMw0rMmG+Ao6VngTJRIQjAryn7zf5YAB/VHvwydKxMCiNIJ4qOh6O4w2b7JtNPr/ rzxWGF6O28N8rQaY8l68WK0qevvI0hBcmtlrusQIIStQpyuZ9H+SwsLEnXfMM3cjgzCp rVuw== X-Gm-Message-State: AAQBX9dzWiyNfaYe9/0FyfYAq+re0xnkNxgHb1C0HJbcZEpY4xmdFpgN LiirI6vGAnBUFN5hq/t7l4WeQYu/5A== X-Google-Smtp-Source: AKy350Z518uI8iDZqsGbGxAPlrTG82u1+tzRRv9ENlxzq3lw1oRIWVZvb/x2syQmHk/sxAmaqvastUXjlQ== X-Received: from sagi.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:241b]) (user=sagis job=sendgmr) by 2002:a25:d958:0:b0:b75:3fd4:1b31 with SMTP id q85-20020a25d958000000b00b753fd41b31mr2479838ybg.1.1680898810376; Fri, 07 Apr 2023 13:20:10 -0700 (PDT) Date: Fri, 7 Apr 2023 20:19:21 +0000 In-Reply-To: <20230407201921.2703758-1-sagis@google.com> Mime-Version: 1.0 References: <20230407201921.2703758-1-sagis@google.com> X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230407201921.2703758-6-sagis@google.com> Subject: [RFC PATCH 5/5] KVM: TDX: Add core logic for TDX intra-host migration From: Sagi Shahar To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: Sean Christopherson , Paolo Bonzini , Isaku Yamahata , Erdem Aktas , David Matlack , Kai Huang , Zhi Wang , Chao Peng , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Sagi Shahar Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adds the core logic for transferring state between source and destination TDs during intra-host migration. Signed-off-by: Sagi Shahar --- arch/x86/kvm/vmx/tdx.c | 191 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 190 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 0999a6d827c99..05b164a91437b 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -2834,9 +2834,198 @@ static __always_inline bool tdx_guest(struct kvm *k= vm) return tdx_kvm->finalized; } =20 +#define for_each_memslot_pair(memslots_1, memslots_2, memslot_iter_1, \ + memslot_iter_2) \ + for (memslot_iter_1 =3D rb_first(&memslots_1->gfn_tree), \ + memslot_iter_2 =3D rb_first(&memslots_2->gfn_tree); \ + memslot_iter_1 && memslot_iter_2; \ + memslot_iter_1 =3D rb_next(memslot_iter_1), \ + memslot_iter_2 =3D rb_next(memslot_iter_2)) + static int tdx_migrate_from(struct kvm *dst, struct kvm *src) { - return -EINVAL; + struct rb_node *src_memslot_iter, *dst_memslot_iter; + struct vcpu_tdx *dst_tdx_vcpu, *src_tdx_vcpu; + struct kvm_memslots *src_slots, *dst_slots; + struct kvm_vcpu *dst_vcpu, *src_vcpu; + struct kvm_tdx *src_tdx, *dst_tdx; + unsigned long i, j; + int ret; + + src_tdx =3D to_kvm_tdx(src); + dst_tdx =3D to_kvm_tdx(dst); + + src_slots =3D __kvm_memslots(src, 0); + dst_slots =3D __kvm_memslots(dst, 0); + + ret =3D -EINVAL; + + if (!src_tdx->finalized) { + pr_warn("Cannot migrate from a non finalized VM\n"); + goto abort; + } + + // Traverse both memslots in gfn order and compare them + for_each_memslot_pair(src_slots, dst_slots, src_memslot_iter, dst_memslot= _iter) { + struct kvm_memory_slot *src_slot, *dst_slot; + + src_slot =3D + container_of(src_memslot_iter, struct kvm_memory_slot, + gfn_node[src_slots->node_idx]); + dst_slot =3D + container_of(src_memslot_iter, struct kvm_memory_slot, + gfn_node[dst_slots->node_idx]); + + if (src_slot->base_gfn !=3D dst_slot->base_gfn || + src_slot->npages !=3D dst_slot->npages) { + pr_warn("Cannot migrate between VMs with different memory slots configu= rations\n"); + goto abort; + } + + if (src_slot->flags !=3D dst_slot->flags) { + pr_warn("Cannot migrate between VMs with different memory slots configu= rations\n"); + goto abort; + } + + if (src_slot->flags & KVM_MEM_PRIVATE) { + if (src_slot->restrictedmem.file->f_inode->i_ino !=3D + dst_slot->restrictedmem.file->f_inode->i_ino) { + pr_warn("Private memslots points to different restricted files\n"); + goto abort; + } + + if (src_slot->restrictedmem.index !=3D dst_slot->restrictedmem.index) { + pr_warn("Private memslots points to the restricted file at different o= ffsets\n"); + goto abort; + } + } + } + + if (src_memslot_iter || dst_memslot_iter) { + pr_warn("Cannot migrate between VMs with different memory slots configur= ations\n"); + goto abort; + } + + dst_tdx->hkid =3D src_tdx->hkid; + dst_tdx->tdr_pa =3D src_tdx->tdr_pa; + + dst_tdx->tdcs_pa =3D kcalloc(tdx_info.nr_tdcs_pages, sizeof(*dst_tdx->tdc= s_pa), + GFP_KERNEL_ACCOUNT | __GFP_ZERO); + if (!dst_tdx->tdcs_pa) { + ret =3D -ENOMEM; + goto late_abort; + } + memcpy(dst_tdx->tdcs_pa, src_tdx->tdcs_pa, + tdx_info.nr_tdcs_pages * sizeof(*dst_tdx->tdcs_pa)); + + dst_tdx->tsc_offset =3D src_tdx->tsc_offset; + dst_tdx->attributes =3D src_tdx->attributes; + dst_tdx->xfam =3D src_tdx->xfam; + dst_tdx->kvm.arch.gfn_shared_mask =3D src_tdx->kvm.arch.gfn_shared_mask; + + kvm_for_each_vcpu(i, src_vcpu, src) + tdx_flush_vp_on_cpu(src_vcpu); + + /* Copy per-vCPU state */ + kvm_for_each_vcpu(i, src_vcpu, src) { + src_tdx_vcpu =3D to_tdx(src_vcpu); + dst_vcpu =3D kvm_get_vcpu(dst, i); + dst_tdx_vcpu =3D to_tdx(dst_vcpu); + + vcpu_load(dst_vcpu); + + memcpy(dst_vcpu->arch.regs, src_vcpu->arch.regs, + NR_VCPU_REGS * sizeof(src_vcpu->arch.regs[0])); + dst_vcpu->arch.regs_avail =3D src_vcpu->arch.regs_avail; + dst_vcpu->arch.regs_dirty =3D src_vcpu->arch.regs_dirty; + + dst_vcpu->arch.tsc_offset =3D dst_tdx->tsc_offset; + + dst_tdx_vcpu->interrupt_disabled_hlt =3D src_tdx_vcpu->interrupt_disable= d_hlt; + dst_tdx_vcpu->buggy_hlt_workaround =3D src_tdx_vcpu->buggy_hlt_workaroun= d; + + dst_tdx_vcpu->tdvpr_pa =3D src_tdx_vcpu->tdvpr_pa; + dst_tdx_vcpu->tdvpx_pa =3D kcalloc(tdx_info.nr_tdvpx_pages, + sizeof(*dst_tdx_vcpu->tdvpx_pa), + GFP_KERNEL_ACCOUNT); + if (!dst_tdx_vcpu->tdvpx_pa) { + ret =3D -ENOMEM; + vcpu_put(dst_vcpu); + goto late_abort; + } + memcpy(dst_tdx_vcpu->tdvpx_pa, src_tdx_vcpu->tdvpx_pa, + tdx_info.nr_tdvpx_pages * sizeof(*dst_tdx_vcpu->tdvpx_pa)); + + td_vmcs_write64(dst_tdx_vcpu, POSTED_INTR_DESC_ADDR, __pa(&dst_tdx_vcpu-= >pi_desc)); + + /* Copy private EPT tables */ + if (kvm_mmu_move_private_pages_from(dst_vcpu, src_vcpu)) { + ret =3D -EINVAL; + vcpu_put(dst_vcpu); + goto late_abort; + } + + for (j =3D 0; j < tdx_info.nr_tdvpx_pages; j++) + src_tdx_vcpu->tdvpx_pa[j] =3D 0; + + src_tdx_vcpu->tdvpr_pa =3D 0; + + vcpu_put(dst_vcpu); + } + + for_each_memslot_pair(src_slots, dst_slots, src_memslot_iter, + dst_memslot_iter) { + struct kvm_memory_slot *src_slot, *dst_slot; + + src_slot =3D container_of(src_memslot_iter, + struct kvm_memory_slot, + gfn_node[src_slots->node_idx]); + dst_slot =3D container_of(src_memslot_iter, + struct kvm_memory_slot, + gfn_node[dst_slots->node_idx]); + + for (i =3D 1; i < KVM_NR_PAGE_SIZES; ++i) { + unsigned long ugfn; + int level =3D i + 1; + + /* + * If the gfn and userspace address are not aligned wrt each other, then + * large page support should already be disabled at this level. + */ + ugfn =3D dst_slot->userspace_addr >> PAGE_SHIFT; + if ((dst_slot->base_gfn ^ ugfn) & (KVM_PAGES_PER_HPAGE(level) - 1)) + continue; + + dst_slot->arch.lpage_info[i - 1] =3D + src_slot->arch.lpage_info[i - 1]; + src_slot->arch.lpage_info[i - 1] =3D NULL; + } + } + + dst->mem_attr_array.xa_head =3D src->mem_attr_array.xa_head; + src->mem_attr_array.xa_head =3D NULL; + + dst_tdx->finalized =3D true; + + /* Clear source VM to avoid freeing the hkid and pages on VM put */ + src_tdx->hkid =3D -1; + src_tdx->tdr_pa =3D 0; + for (i =3D 0; i < tdx_info.nr_tdcs_pages; i++) + src_tdx->tdcs_pa[i] =3D 0; + + return 0; + +late_abort: + /* If we aborted after the state transfer already started, the src VM + * is no longer valid. + */ + kvm_vm_dead(src); + +abort: + dst_tdx->hkid =3D -1; + dst_tdx->tdr_pa =3D 0; + + return ret; } =20 int tdx_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) --=20 2.40.0.348.gf938b09366-goog