From nobody Sat Oct 4 14:35:19 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E28BF28724D for ; Fri, 15 Aug 2025 13:37:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755265058; cv=none; b=s6Kab/V7jQw/1Q2NZ2q/RsDq/WyNhYPx2BFpmx8MmN5W49J4oFiEIG3x5reG2i8VrsBpvMyoCAyR7bs4P75Ql6slTjr4GeNQiUGxsmUCFOwcR0WeKg/78rfNWYYo21F1nmwHFT9zpsb9Gu/h7m/73VCIZgNUP7f+mD+aoqkM+ns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755265058; c=relaxed/simple; bh=EkukpV3FFtELRil9qkr6l2IlP8xkLNYzLp7/fC83roE=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=RMyuwNdKQbjJSorgEtkW+PmqrXdIRKJhM+wLXTJkD4o4LlffA8ZFHAOY1CRewmmm0KLdDrTyyiQHqH5FK2rhm1U/RJFxa5g682mO9+g/RBckXMm9I3xnNqFjCFCN9E+Kmofg8WAO3VDVcuyZ5jP9DHs63r9gH7gJHDRDdsZGDlQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=OTdG9oFV; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OTdG9oFV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1755265055; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=HAr1AlxPa2PLnBvFUB2yr2b+jA2FCI9CNGMzin6U/SU=; b=OTdG9oFVAQWJrmm92f63eWBKufJhtXYUVTm7U+XwdY7Rw5ZD0+kxE0vD/6P+eBn3FyE9uS PEXcUJEiycJDlmCcILyXV3UgzW5E5McAIfILOOgGVfOuUOWilzepVSRLWc060uGRTd4KyA mGBvrYb622iTvLUXL0zoWapzNIRxXUo= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-320-mYOouCPNMmae2ZzeMIP0fg-1; Fri, 15 Aug 2025 09:37:32 -0400 X-MC-Unique: mYOouCPNMmae2ZzeMIP0fg-1 X-Mimecast-MFC-AGG-ID: mYOouCPNMmae2ZzeMIP0fg_1755265051 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E000A1954194; Fri, 15 Aug 2025 13:37:30 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.44.32.222]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2DC2919327C0; Fri, 15 Aug 2025 13:37:26 +0000 (UTC) From: Vitaly Kuznetsov To: linux-hyperv@vger.kernel.org Cc: "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , x86@kernel.org, linux-kernel@vger.kernel.org, Nuno Das Neves , Tianyu Lan , Michael Kelley , Li Tian , Philipp Rudo Subject: [PATCH] x86/hyperv: Fix kdump on Azure CVMs Date: Fri, 15 Aug 2025 15:37:25 +0200 Message-ID: <20250815133725.1591863-1-vkuznets@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" Azure CVM instance types featuring a paravisor hang upon kdump. The investigation shows that makedumpfile causes a hang when it steps on a page which was previously share with the host (HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY). The new kernel has no knowledge of these 'special' regions (which are Vmbus connection pages, GPADL buffers, ...). There are several ways to approach the issue: - Convey the knowledge about these regions to the new kernel somehow. - Unshare these regions before accessing in the new kernel (it is unclear if there's a way to query the status for a given GPA range). - Unshare these regions before jumping to the new kernel (which this patch implements). To make the procedure as robust as possible, store PFN ranges of shared regions in a linked list instead of storing GVAs and re-using hv_vtom_set_host_visibility(). This also allows to avoid memory allocation on the kdump/kexec path. The patch skips implementing weird corner case in hv_list_enc_remove() when a PFN in the middle of a region is unshared. First, it is unlikely that such requests happen. Second, it is not a big problem if hv_list_enc_remove() doesn't actually remove some regions as this will only result in an extra hypercall doing nothing upon kexec/kdump; there's no need to be perfect. Signed-off-by: Vitaly Kuznetsov --- arch/x86/hyperv/ivm.c | 153 ++++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 2 + drivers/hv/vmbus_drv.c | 2 + 3 files changed, 157 insertions(+) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index ade6c665c97e..a6e614672836 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -462,6 +462,150 @@ void hv_ivm_msr_read(u64 msr, u64 *value) hv_ghcb_msr_read(msr, value); } =20 +/* + * Keep track of the PFN regions which were shared with the host. The acce= ss + * must be revoked upon kexec/kdump (see hv_ivm_clear_host_access()). + */ +struct hv_enc_pfn_region { + struct list_head list; + u64 pfn; + int count; +}; + +static LIST_HEAD(hv_list_enc); +static DEFINE_RAW_SPINLOCK(hv_list_enc_lock); + +static int hv_list_enc_add(const u64 *pfn_list, int count) +{ + struct hv_enc_pfn_region *ent; + unsigned long flags; + bool found =3D false; + u64 pfn; + int i; + + for (i =3D 0; i < count; i++) { + pfn =3D pfn_list[i]; + + found =3D false; + raw_spin_lock_irqsave(&hv_list_enc_lock, flags); + list_for_each_entry(ent, &hv_list_enc, list) { + if ((ent->pfn <=3D pfn) && (ent->pfn + ent->count - 1 >=3D pfn)) { + /* Nothin to do - pfn is already in the list */ + found =3D true; + } else if (ent->pfn + ent->count =3D=3D pfn) { + /* Grow existing region up */ + found =3D true; + ent->count++; + } else if (pfn + 1 =3D=3D ent->pfn) { + /* Grow existing region down */ + found =3D true; + ent->pfn--; + ent->count++; + } + }; + raw_spin_unlock_irqrestore(&hv_list_enc_lock, flags); + + if (found) + continue; + + /* No adajacent region found -- creating a new one */ + ent =3D kzalloc(sizeof(struct hv_enc_pfn_region), GFP_KERNEL); + if (!ent) + return -ENOMEM; + + ent->pfn =3D pfn; + ent->count =3D 1; + + raw_spin_lock_irqsave(&hv_list_enc_lock, flags); + list_add(&ent->list, &hv_list_enc); + raw_spin_unlock_irqrestore(&hv_list_enc_lock, flags); + } + + return 0; +} + +static void hv_list_enc_remove(const u64 *pfn_list, int count) +{ + struct hv_enc_pfn_region *ent, *t; + unsigned long flags; + u64 pfn; + int i; + + for (i =3D 0; i < count; i++) { + pfn =3D pfn_list[i]; + + raw_spin_lock_irqsave(&hv_list_enc_lock, flags); + list_for_each_entry_safe(ent, t, &hv_list_enc, list) { + if (pfn =3D=3D ent->pfn + count - 1) { + /* Removing tail pfn */ + ent->count--; + if (!ent->count) { + list_del(&ent->list); + kfree(ent); + } + } else if (pfn =3D=3D ent->pfn) { + /* Removing head pfn */ + ent->count--; + ent->pfn++; + if (!ent->count) { + list_del(&ent->list); + kfree(ent); + } + } + + /* + * Removing PFNs in the middle of a region is not implemented; the + * list is currently only used for cleanup upon kexec and there's + * no harm done if we issue an extra unneeded hypercall making some + * region encrypted when it already is. + */ + }; + raw_spin_unlock_irqrestore(&hv_list_enc_lock, flags); + } +} + +void hv_ivm_clear_host_access(void) +{ + struct hv_gpa_range_for_visibility *input; + struct hv_enc_pfn_region *ent; + unsigned long flags; + u64 hv_status; + int cur, i; + + if (!hv_is_isolation_supported()) + return; + + raw_spin_lock_irqsave(&hv_list_enc_lock, flags); + + input =3D *this_cpu_ptr(hyperv_pcpu_input_arg); + if (!input) + goto unlock; + + list_for_each_entry(ent, &hv_list_enc, list) { + for (i =3D 0, cur =3D 0; i < ent->count; i++) { + input->gpa_page_list[cur] =3D ent->pfn + i; + cur++; + + if (cur =3D=3D HV_MAX_MODIFY_GPA_REP_COUNT || i =3D=3D ent->count - 1) { + input->partition_id =3D HV_PARTITION_ID_SELF; + input->host_visibility =3D VMBUS_PAGE_NOT_VISIBLE; + input->reserved0 =3D 0; + input->reserved1 =3D 0; + hv_status =3D hv_do_rep_hypercall( + HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY, + cur, 0, input, NULL); + WARN_ON_ONCE(!hv_result_success(hv_status)); + cur =3D 0; + } + } + + }; + +unlock: + raw_spin_unlock_irqrestore(&hv_list_enc_lock, flags); +} +EXPORT_SYMBOL_GPL(hv_ivm_clear_host_access); + /* * hv_mark_gpa_visibility - Set pages visible to host via hvcall. * @@ -475,6 +619,7 @@ static int hv_mark_gpa_visibility(u16 count, const u64 = pfn[], struct hv_gpa_range_for_visibility *input; u64 hv_status; unsigned long flags; + int ret; =20 /* no-op if partition isolation is not enabled */ if (!hv_is_isolation_supported()) @@ -486,6 +631,14 @@ static int hv_mark_gpa_visibility(u16 count, const u64= pfn[], return -EINVAL; } =20 + if (visibility =3D=3D VMBUS_PAGE_NOT_VISIBLE) { + hv_list_enc_remove(pfn, count); + } else { + ret =3D hv_list_enc_add(pfn, count); + if (ret) + return ret; + } + local_irq_save(flags); input =3D *this_cpu_ptr(hyperv_pcpu_input_arg); =20 diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyper= v.h index abc4659f5809..6a988001e46f 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -263,10 +263,12 @@ static inline int hv_snp_boot_ap(u32 apic_id, unsigne= d long start_ip, void hv_vtom_init(void); void hv_ivm_msr_write(u64 msr, u64 value); void hv_ivm_msr_read(u64 msr, u64 *value); +void hv_ivm_clear_host_access(void); #else static inline void hv_vtom_init(void) {} static inline void hv_ivm_msr_write(u64 msr, u64 value) {} static inline void hv_ivm_msr_read(u64 msr, u64 *value) {} +static inline void hv_ivm_clear_host_access(void) {} #endif =20 static inline bool hv_is_synic_msr(unsigned int reg) diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 2ed5a1e89d69..2e891e108218 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -2784,6 +2784,7 @@ static void hv_kexec_handler(void) /* Make sure conn_state is set as hv_synic_cleanup checks for it */ mb(); cpuhp_remove_state(hyperv_cpuhp_online); + hv_ivm_clear_host_access(); }; =20 static void hv_crash_handler(struct pt_regs *regs) @@ -2799,6 +2800,7 @@ static void hv_crash_handler(struct pt_regs *regs) cpu =3D smp_processor_id(); hv_stimer_cleanup(cpu); hv_synic_disable_regs(cpu); + hv_ivm_clear_host_access(); }; =20 static int hv_synic_suspend(void) --=20 2.50.0