From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748056; cv=none; d=zohomail.com; s=zohoarc; b=Y4BJKRZuACoWbzsVVoGirW5/0ypfYCypdpj7hEzOgFRNRY0apOnOEYiiK6WucGUKcSPGupUoxWkbmrVL/d1Z3E7pdXJMeARCg4Y0aEFGcdlmS1bvRnhwSBtiHg4rOmbJKoihLo6rf5+NAFyPT+oZKtkj8trq+BmQg5mSPAIiSNM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748056; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=vtDia1dYtGzUpfOhr+PQWn9sLmMvXEnKI84jgzczGq4=; b=lH1ClB4CnBD+nvTCknp3uTB9mLU9kAPMRcA2Qpv5IS6nkS1+13Hth/uP446PZWl7SDmyhul8YReIPuse0mofCQaBUJa2slFSN7edwPyg4gPwdj7GFq0JTY655lsRRobzqrdPRngJjUfQvuwKHod4lPPtqIi6UjVRstNMnn6Jivc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748056031101.98334779786421; Tue, 3 Nov 2026 15:20:56 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470712.1679327 (Exim 4.92) (envelope-from ) id 1xONnh-0002kT-Sf; Tue, 03 Nov 2026 23:20:21 +0000 Received: by outflank-mailman (output) from mailman id 1470712.1679327; Tue, 03 Nov 2026 23:20:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnh-0002kL-PU; Tue, 03 Nov 2026 23:20:21 +0000 Received: by outflank-mailman (input) for mailman id 1470712; Tue, 03 Nov 2026 23:20:20 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONng-0002jw-G5 for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:20 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONng-0000000CofD-03Xm; Tue, 03 Nov 2026 23:20:20 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnf-0000000EMFa-3ZGa; Tue, 03 Nov 2026 23:20:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=vtDia1dYtGzUpfOhr+PQWn9sLmMvXEnKI84jgzczGq4=; b=f31WC/J4b87av2JxQr7luwD5FJ 3g+TvCXUdZG2ul0X5l1erbH1VuHgfOfRkhpmnsSGsYanPSEATpF7iAg4h4gWr/Bc7l9POIsEN9tu7 CNch47ImuHvOO35AmXsW+gTqR26TtubTJAH5U7UlmJ07VgwfL9BQ7n+IUXB2DPrajqoU=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 01/44] x86/mm: allocate the per-domain L3 in arch_domain_create() Date: Tue, 3 Nov 2026 17:13:23 -0500 Message-ID: <20261103-asi-hvm-v6-1-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748057023158500 Content-Type: text/plain; charset="utf-8" init_xen_l4_slots() relies on d->arch.perdomain_l3_pg being non-NULL, or it will fill in a garbage value for slot 260. Currently, this is allocated only implicitly, as a side effect of calling create_perdomain_mapping(). But depending on the specific circumstances, this may not happen until several hypercalls after the domain is created. This is not robust: callers need to keep track of when exactly certain structures happen to be created to ensure that init_xen_l4_slots() operates properly. Since fcf91ce9de66 ("x86/pv: don't populate the GDT/LDT L3 slot at domain creation"), a PV domain gets its L3 with its first vCPU (pv_create_gdt_ldt_l1tab()). Only mapcache_domain_init() makes it earlier, and release builds skip that unless memory hotplug is enabled or the host's memory reaches past the part of the directmap PV guests can see. So in release builds on most hosts, an L4 that is promoted before XEN_DOMCTL_max_vcpus has created the domain's vCPUs gets a slot 260 entry made from a NULL page (0x803e900000000063). libxl is unaffected: it creates the vCPUs before it builds or restores a domain. Allocate it explicitly in arch_domain_create() instead. Introduce allocate_perdomain_l3(), which will allocate a page with the appropriate parameters and zero it. create_perdomain_mapping() now requires the L3 to exist, and BUG()s if it does not. We return a pg rather than setting d->arch.perdomain_l3_pg directly, because later we'll use the same function to initialize the per-vCPU perdomain L3. This removes the need to allocate it semi-implicitly (calling create_perdomain_mapping() with nr =3D=3D 0) in hvm_domain_initialise(). Fixes: fcf91ce9de66 ("x86/pv: don't populate the GDT/LDT L3 slot at domain = creation") Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap Reviewed-by: Jan Beulich --- Changes in v6: - New in this version, replacing "x86/mm: refuse to promote an L4 while the domain has no per-domain L3": allocate the L3 with the domain instead (Jan). --- xen/arch/x86/domain.c | 6 ++++++ xen/arch/x86/hvm/hvm.c | 5 ----- xen/arch/x86/include/asm/mm.h | 1 + xen/arch/x86/mm.c | 32 +++++++++++++++----------------- 4 files changed, 22 insertions(+), 22 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 368e4deff3..e67c336b82 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -901,6 +901,12 @@ int arch_domain_create(struct domain *d, =20 psr_domain_init(d); =20 + if ( (d->arch.perdomain_l3_pg =3D allocate_perdomain_l3(d)) =3D=3D NUL= L ) + { + rc =3D -ENOMEM; + goto fail; + } + if ( is_hvm_domain(d) ) { if ( (rc =3D hvm_domain_initialise(d, config)) !=3D 0 ) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index ef9c7cd057..46d3a520d5 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -620,10 +620,6 @@ int hvm_domain_initialise(struct domain *d, INIT_LIST_HEAD(&d->arch.hvm.mmcfg_regions); INIT_LIST_HEAD(&d->arch.hvm.msix_tables); =20 - rc =3D create_perdomain_mapping(d, PERDOMAIN_VIRT_START, 0, NULL, NULL= ); - if ( rc ) - goto fail; - hvm_init_cacheattr_region_list(d); =20 rc =3D paging_enable(d, PG_refcounts|PG_translate|PG_external); @@ -731,7 +727,6 @@ int hvm_domain_initialise(struct domain *d, fail0: hvm_destroy_cacheattr_region_list(d); destroy_perdomain_mapping(d, PERDOMAIN_VIRT_START, 0); - fail: hvm_domain_relinquish_resources(d); XFREE(d->arch.hvm.io_handler); XFREE(d->arch.hvm.pl_time); diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index aa515efbee..e77ac5ccdf 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -606,6 +606,7 @@ int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_= HANDLE_PARAM(void) arg); #define NIL(type) ((type *)-sizeof(type)) #define IS_NIL(ptr) (!((uintptr_t)(ptr) + sizeof(*(ptr)))) =20 +struct page_info *allocate_perdomain_l3(struct domain *d); int create_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg); diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index a1b7770db0..63e6aadcda 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6235,6 +6235,16 @@ static bool perdomain_l1e_needs_freeing(l1_pgentry_t= l1e) (_PAGE_PRESENT | _PAGE_AVAIL0); } =20 +struct page_info *allocate_perdomain_l3(struct domain *d) +{ + struct page_info *pg =3D alloc_domheap_page(d, MEMF_no_owner); + + if ( pg ) + clear_domain_page(page_to_mfn(pg)); + + return pg; +} + int create_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg) @@ -6248,24 +6258,12 @@ int create_perdomain_mapping(struct domain *d, unsi= gned long va, ASSERT(va >=3D PERDOMAIN_VIRT_START && va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); =20 - if ( !d->arch.perdomain_l3_pg ) - { - pg =3D alloc_domheap_page(d, MEMF_no_owner); - if ( !pg ) - return -ENOMEM; - l3tab =3D __map_domain_page(pg); - clear_page(l3tab); - d->arch.perdomain_l3_pg =3D pg; - if ( !nr ) - { - unmap_domain_page(l3tab); - return 0; - } - } - else if ( !nr ) + BUG_ON(!d->arch.perdomain_l3_pg); + + if ( !nr ) return 0; - else - l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); + + l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); =20 ASSERT(!l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1))); =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748056; cv=none; d=zohomail.com; s=zohoarc; b=RIZoDIRTeseuUmRPAa/PWOrgx8uP8iosyx0DDUm18EQ/jXL/8MndUp9aKsFfY2cXzQJ1AZGFBZy6VDUGg6mjg+Xoe1yR2jvkpR4cUnMhFRC3kcwrnTXNwppyEzN1Gjxle8cBzKBiVHJDdg1BkF+iYUBItbAxs2SP9wJNXJziKYY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748056; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=PGokQ3fmGWi+wfOTXKwVpSYtauVAwIDFBs65aAXW+/s=; b=aSEy1wMsrYNsBVDg4Byjl12Jq2hKyr2NaxYupo8pfI4fgKcywOv0Pnq/vJ/iWxQqYEko88+raG8gZh1zW1ualIGm3Uk75HyjckT6ENjtXjfnaI9Fgroct2Xrb10oTlopasgxivY89l8Y8T3b4W1P9HuyD7SE2pLI4PkEI16bivw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748056978167.8606159271145; Tue, 3 Nov 2026 15:20:56 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470716.1679360 (Exim 4.92) (envelope-from ) id 1xONno-0003YX-9P; Tue, 03 Nov 2026 23:20:28 +0000 Received: by outflank-mailman (output) from mailman id 1470716.1679360; Tue, 03 Nov 2026 23:20:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONno-0003Xx-3B; Tue, 03 Nov 2026 23:20:28 +0000 Received: by outflank-mailman (input) for mailman id 1470716; Tue, 03 Nov 2026 23:20:26 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnm-0003QF-Q1 for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:26 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONnh-0000000CofL-0Vf8; Tue, 03 Nov 2026 23:20:21 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONng-0000000EMFa-1D4D; Tue, 03 Nov 2026 23:20:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=PGokQ3fmGWi+wfOTXKwVpSYtauVAwIDFBs65aAXW+/s=; b=IqnfxnpjNjOz+4qD0eSNG1Hmcp 3lZE3nt0vaIFMbk6bU5eTwtwgX2wUarnEe7sFegqIutB9RFfYeHImSrgc+z8s9WYlkjLz15nlwcS0 c1282hmJ+TVp7jGLoGBhJ5zvKUuYQvOoP5Kvr/66o9ug6BK9/bVobea9VLSbcLdEyU08=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 02/44] x86/mm: purge unneeded destroy_perdomain_mapping() Date: Tue, 3 Nov 2026 17:13:24 -0500 Message-ID: <20261103-asi-hvm-v6-2-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748059015158500 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that destroy_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. There are two calls made from domain-scoped contexts; both calls turn out to be unnecessary: - destroy_perdomain_mapping() is not logically the undo of create_perdomain_mapping(), as the name and its use in hvm_domain_initialise() suggest. destroy_ tears down mappings without freeing the per-domain L3; and since the call here passes nr =3D=3D 0, it tears down nothing at all. The per-domain L3 page is actually freed by free_perdomain_mappings(), which hvm_domain_initialise()'s caller, arch_domain_create(), already invokes on its failure path. - The call in pv_domain_destroy() is redundant at both of its call sites. arch_domain_destroy() unconditionally calls free_perdomain_mappings(), which tears down the same entries and additionally frees the page-table structures; and the other caller, the error path of pv_domain_initialise(), returns into arch_domain_create()'s failure path, which invokes free_perdomain_mappings() as well. One vCPU-scoped caller, pv_destroy_gdt_ldt_l1tab(), is covered the same way: a vCPU is only ever destroyed together with its domain, or on the failure path of creating it, and both routes end in free_perdomain_mappings(); the GDT/LDT slots hold no area-owned pages, so there is nothing else for the call to do. While we're getting rid of unnecessary callers, drop this one too. Note that free_compat_arg_xlat() keeps its explicit teardown: switch_compat() calls it on the failure path. While in practice most toolstacks will immediately destroy the domain, in theory a toolstack could let it live on as a 64-bit guest. Release the resources immediately rather than waiting for domain destruction. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap Reviewed-by: Jan Beulich --- Changes in v6: - Message: create_perdomain_mapping() no longer allocates the per-domain L3, since "x86/mm: allocate the per-domain L3 in arch_domain_create()". Changes in v4: - Remove an unnecessary comment (Jan). Changes in v3: - Note that pv_domain_destroy()'s other call site, the error path of pv_domain_initialise(), is covered the same way (Jan). - Also drop pv_destroy_gdt_ldt_l1tab(), the per-vCPU teardown call (Jan), and comment on why free_compat_arg_xlat() keeps its own. The Reviewed-by tags given for the previous form are dropped. Changes in v2: - Added to the series Changes since the previously posted version: - Reworked the commit message to make it more clear how it fits in with the larger series. No functional change. --- xen/arch/x86/hvm/hvm.c | 1 - xen/arch/x86/pv/domain.c | 10 ---------- 2 files changed, 11 deletions(-) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 46d3a520d5..71da15c0a4 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -726,7 +726,6 @@ int hvm_domain_initialise(struct domain *d, XFREE(d->arch.hvm.irq); fail0: hvm_destroy_cacheattr_region_list(d); - destroy_perdomain_mapping(d, PERDOMAIN_VIRT_START, 0); hvm_domain_relinquish_resources(d); XFREE(d->arch.hvm.io_handler); XFREE(d->arch.hvm.pl_time); diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 0c42ae58aa..28ea0742ab 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -319,12 +319,6 @@ static int pv_create_gdt_ldt_l1tab(struct vcpu *v) NULL); } =20 -static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v) -{ - destroy_perdomain_mapping(v->domain, GDT_VIRT_START(v), - 1U << GDT_LDT_VCPU_SHIFT); -} - void pv_vcpu_destroy(struct vcpu *v) { if ( is_pv_32bit_vcpu(v) ) @@ -333,7 +327,6 @@ void pv_vcpu_destroy(struct vcpu *v) release_compat_l4(v); } =20 - pv_destroy_gdt_ldt_l1tab(v); XFREE(v->arch.pv.trap_ctxt); } =20 @@ -381,9 +374,6 @@ void pv_domain_destroy(struct domain *d) { pv_l1tf_domain_destroy(d); =20 - destroy_perdomain_mapping(d, GDT_LDT_VIRT_START, - GDT_LDT_MBYTES << (20 - PAGE_SHIFT)); - XFREE(d->arch.pv.cpuidmasks); =20 FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab); --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748043; cv=none; d=zohomail.com; s=zohoarc; b=XmMrnMvVxPWHvMpG1hoPgZ2YbNH0SuErjEZ49rqKy6xQWkHPDUccIyQhHnPhNrLFibQigiZ4SkOt0f/qPhoekojusIv8ZTWHQ94lN6QSxc0zIcUIWgUrtL9ma56RLXxL0hYEZEHTAdZUedbE5hlWalrrCj6rBzm8CQ+qjWUyLvg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748043; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3VJN5UNE9/z4qrpuleJ1/OW1QHRZAWYrv6m65EedvqM=; b=YHNa53zMEnIbamVYgOLXPf3XjRlkyxQ/JVYKGZVYUoFRbPcPWcu8f455Lmj26Th7fzjqUXCC3BmArBrv56B0qkeueUzLKUr4VM5IcTkA9sl24Swf9SbiZRJ5eFIbrtzBD48eqNgWPf/JIM/0kwWJ8uq/lJ7h9cYRTNHs9BR6/nA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748043545511.9786789869206; Tue, 3 Nov 2026 15:20:43 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470713.1679336 (Exim 4.92) (envelope-from ) id 1xONnk-0002yh-2a; Tue, 03 Nov 2026 23:20:24 +0000 Received: by outflank-mailman (output) from mailman id 1470713.1679336; Tue, 03 Nov 2026 23:20:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnj-0002ya-Vz; Tue, 03 Nov 2026 23:20:23 +0000 Received: by outflank-mailman (input) for mailman id 1470713; Tue, 03 Nov 2026 23:20:22 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONni-0002wo-Qw for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:22 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONni-0000000CofN-1h5Q; Tue, 03 Nov 2026 23:20:22 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnh-0000000EMFa-2DVQ; Tue, 03 Nov 2026 23:20:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=3VJN5UNE9/z4qrpuleJ1/OW1QHRZAWYrv6m65EedvqM=; b=m5v61KqkHzWattc6yApMmTdIUu 6NeSqowudPi3AnVvZ/xY6Cq+vSWsWY3OVoHpwiDFPoCAz3uYlaIhjFAI/Sx70UK4rnowrxH+0GIRm Zg3t6wdFp7T8ZmxHat404orQL+sxFkYrULxgRBRRXPzBz+pN7DmgU2d7tK3FzglXW6hA=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , Teddy Astie Subject: [PATCH v6 03/44] x86/mm: prepare destroy_perdomain_mapping() for per-vCPU perdomain areas Date: Tue, 3 Nov 2026 17:13:25 -0500 Message-ID: <20261103-asi-hvm-v6-3-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748044978158500 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that destroy_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. The one remaining caller, free_compat_arg_xlat(), is already in a vCPU context, so we just need to change the parameter from a domain pointer to a vCPU pointer. No functional change. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap Reviewed-by: Roger Pau Monn=C3=A9 --- Changes in v4: - Say "No functional change." (Roger). Changes in v3: - Rebased onto staging without the GDT/LDT conversions (for the HVM-only series): free_compat_arg_xlat() is the only caller left. - Drop the linear-page-table fast path altogether (Jan): its only user would have been guest-driven LDT teardown, and without it the function does not depend on the linear mapping being available. Changes in v2: - Added to the series Changes since the previously posted version: - The fast path is gone (v3), and with it the previously posted version's sync_local_execstate() call. - Reword and retitle for clarity (was: "x86/mm: switch destroy_perdomain_mapping() parameter from domain to vCPU"). --- xen/arch/x86/include/asm/mm.h | 2 +- xen/arch/x86/mm.c | 3 ++- xen/arch/x86/x86_64/mm.c | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index e77ac5ccdf..1cdd30f7e2 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -610,7 +610,7 @@ struct page_info *allocate_perdomain_l3(struct domain *= d); int create_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg); -void destroy_perdomain_mapping(struct domain *d, unsigned long va, +void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr); void free_perdomain_mappings(struct domain *d); =20 diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 63e6aadcda..c784c4f726 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6356,10 +6356,11 @@ int create_perdomain_mapping(struct domain *d, unsi= gned long va, return rc; } =20 -void destroy_perdomain_mapping(struct domain *d, unsigned long va, +void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr) { const l3_pgentry_t *l3tab, *pl3e; + const struct domain *d =3D v->domain; =20 ASSERT(va >=3D PERDOMAIN_VIRT_START && va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index 8eadab7933..f467ed426b 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -739,7 +739,7 @@ int setup_compat_arg_xlat(struct vcpu *v) =20 void free_compat_arg_xlat(struct vcpu *v) { - destroy_perdomain_mapping(v->domain, ARG_XLAT_START(v), + destroy_perdomain_mapping(v, ARG_XLAT_START(v), PFN_UP(COMPAT_ARG_XLAT_SIZE)); } =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748057; cv=none; d=zohomail.com; s=zohoarc; b=djhMsvc0EUQDCQtNYgOpx8RN3aVMhfDRjI9WT6fBkstKdLyfds0rZvYHF94I5XKi/NfBkAXfIlpHD9tBP6xj65wsFA2ZpHQB0CmbQo7gA0Qat9U4eyhSJouL+dOR73VyunmTvXFT7CZLBb2r6neiVWPRDHlx1CE3aFHka1H/+hg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748057; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=u/xqvNkGLeAb23thkUDCLuctZV78xyIQ2mEEAVZx0bI=; b=Jq1slfs4yDixjcTXnwsLBI4sKPO6y4y2qUdNJmbEMV3DjUS+og/BLmpLZZANH7bb+kUx7/9u5iZllHbk0P4I9aQsN17dX0JsaMGRvZrwCscs7RdjbVx7uhTEBmA6u18H2eW+FiTIvjtSnF701Fd2IIQCgh9Qn5V9ILRuCCxMLr0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748057196143.57051073212654; Tue, 3 Nov 2026 15:20:57 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470714.1679341 (Exim 4.92) (envelope-from ) id 1xONnk-00031U-CT; Tue, 03 Nov 2026 23:20:24 +0000 Received: by outflank-mailman (output) from mailman id 1470714.1679341; Tue, 03 Nov 2026 23:20:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnk-00030F-79; Tue, 03 Nov 2026 23:20:24 +0000 Received: by outflank-mailman (input) for mailman id 1470714; Tue, 03 Nov 2026 23:20:23 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnj-0002y9-6j for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:23 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONni-0000000CofP-3RIZ; Tue, 03 Nov 2026 23:20:22 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONni-0000000EMFa-2uPe; Tue, 03 Nov 2026 23:20:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=u/xqvNkGLeAb23thkUDCLuctZV78xyIQ2mEEAVZx0bI=; b=ZkcVon2qXIegu7yDD+TTaB9afe mmZctwvre4xhOTA8I7W70uKmz51sT1w5cDs0fvh3u3XXb9xu0RhcvAT39ALruVviW/YXZq9cxN2TN UDy+tLvLXgizCtU1GkaYP192hqV98iKs7oTG3eUJj4sY53Vqqpoc4pQ8ocUw6bnq6aVY=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , Teddy Astie Subject: [PATCH v6 04/44] x86/domain_page: drop redundant create_perdomain_mapping() call Date: Tue, 3 Nov 2026 17:13:26 -0500 Message-ID: <20261103-asi-hvm-v6-4-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748059008158500 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that create_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. The one remaining call in a domain context turns out to be unnecessary: mapcache_domain_init() pre-plumbs L1 tables over the whole inuse/garbage bitmap range (sized for the full MAPCACHE_ENTRIES capacity), without populating any data pages. The plumbing is redundant: mapcache_vcpu_init() installs the bitmap pages the domain will actually use with populating calls, which allocate any missing page-table structure on demand -- and since d->max_vcpus is fixed before any vCPU is created, the range those calls cover never grows. The pre-plumbed tail beyond it backs virtual addresses that are never populated at all. In debug builds, and on hosts with memory hotplug or with more memory than the PV directmap covers, the call was previously what first allocated a PV domain's per-domain L3. Since "x86/mm: allocate the per-domain L3 in arch_domain_create()", the L3 comes with the domain. Drop the call. With the only fallible operation gone, mapcache_domain_init() becomes void, and arch_domain_create()'s error handling for it goes away. The per-domain-mapping work in mapcache_vcpu_init() stays where it is rather than moving into mapcache_domain_init() along with this change, ready for a later patch which introduces per-vCPU mapcaches. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap Reviewed-by: Roger Pau Monn=C3=A9 --- Changes in v6: - Message: since "x86/mm: allocate the per-domain L3 in arch_domain_create()", the L3 comes with the domain, and this is the only call left in a domain context. Changes in v3: - Rebased onto staging without the GDT/LDT conversions (for the HVM-only series); no functional difference. - Say why the per-vCPU part stays in mapcache_vcpu_init() (Jan). - Say that the call also first allocated a PV domain's per-domain L3 in some builds, which all builds now do with the first vCPU. Changes in v2: - Added to the series (split out of the following patch). Changes since the previously posted version: - Split out of "x86/mm: switch {create,destroy}_perdomain_mapping() domain parameter to vCPU", where the removal was folded into the parameter switch without its own rationale. --- xen/arch/x86/domain.c | 3 +-- xen/arch/x86/domain_page.c | 8 ++------ xen/arch/x86/include/asm/domain.h | 2 +- 3 files changed, 4 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index e67c336b82..173daf9000 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -914,8 +914,7 @@ int arch_domain_create(struct domain *d, } else if ( is_pv_domain(d) ) { - if ( (rc =3D mapcache_domain_init(d)) !=3D 0 ) - goto fail; + mapcache_domain_init(d); =20 if ( (rc =3D pv_domain_initialise(d)) !=3D 0 ) goto fail; diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 5744754f11..f02be1675b 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -223,7 +223,7 @@ void unmap_domain_page(const void *ptr) local_irq_restore(flags); } =20 -int mapcache_domain_init(struct domain *d) +void mapcache_domain_init(struct domain *d) { struct mapcache_domain *dcache =3D &d->arch.pv.mapcache; unsigned int bitmap_pages; @@ -232,7 +232,7 @@ int mapcache_domain_init(struct domain *d) =20 #ifdef NDEBUG if ( !mem_hotplug && max_page <=3D PFN_DOWN(__pa(HYPERVISOR_VIRT_END -= 1)) ) - return 0; + return; #endif =20 BUILD_BUG_ON(MAPCACHE_VIRT_END + PAGE_SIZE * (3 + @@ -244,10 +244,6 @@ int mapcache_domain_init(struct domain *d) (bitmap_pages + 1) * PAGE_SIZE / sizeof(long); =20 spin_lock_init(&dcache->lock); - - return create_perdomain_mapping(d, (unsigned long)dcache->inuse, - 2 * bitmap_pages + 1, - NIL(l1_pgentry_t *), NULL); } =20 int mapcache_vcpu_init(struct vcpu *v) diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 2d0a915410..212daae1f5 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -89,7 +89,7 @@ struct mapcache_domain { unsigned long *garbage; }; =20 -int mapcache_domain_init(struct domain *d); +void mapcache_domain_init(struct domain *d); int mapcache_vcpu_init(struct vcpu *v); =20 /* x86/64: toggle guest between kernel and user modes. */ --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748048; cv=none; d=zohomail.com; s=zohoarc; b=KAjiKqi1XHKCuNCEKXeoBIrZcYuE8ypcJKiKvAnFXYdpdtUmxbKVv5kEf8sMsEasiew5w7n079O3uJ5PFw5BXjLCa1p5MnTzpaTKOuDEWMx7l08H0zxaG5KRco3nP4W4nvkQMAtmLNTyIex8c+gENYvTByWHtJoyEXr/RIAvy7Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748048; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ULtiu4aFlCDZSTihWE2R472MpYJVuVy4Xwfo4YXyEfQ=; b=TNDriRO2az1XoQuaswOhqU6zKVi5dEVhs+p2nTfj2oOQd1nodICcdKbkP0gnZIL7ttfz4CIiRTKDA6B7UpZTgf+iDqTICJWMobgVY3AttfH+kYXgHLslMHZIfVP2mpyKrPgpSClstZrZwz26rclw9WvgYtFsLwE90+Lbqaq+FRI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748048758133.30761869108062; Tue, 3 Nov 2026 15:20:48 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470715.1679353 (Exim 4.92) (envelope-from ) id 1xONnn-0003Rc-Lg; Tue, 03 Nov 2026 23:20:27 +0000 Received: by outflank-mailman (output) from mailman id 1470715.1679353; Tue, 03 Nov 2026 23:20:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnn-0003RO-IG; Tue, 03 Nov 2026 23:20:27 +0000 Received: by outflank-mailman (input) for mailman id 1470715; Tue, 03 Nov 2026 23:20:25 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnl-0003Ok-K9 for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:25 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONnl-0000000Coff-0TD7; Tue, 03 Nov 2026 23:20:25 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnj-0000000EMFa-3OKN; Tue, 03 Nov 2026 23:20:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=ULtiu4aFlCDZSTihWE2R472MpYJVuVy4Xwfo4YXyEfQ=; b=ca0ULvjt+26YeVsKccI+kqPVYi TCTjshxtmN45+mwavA28NEIDHyW/i/Yb2DsiPUbGqza64fCLUh0d/TTdF4dnOqZ/gvq7h3KIEijvw Oouq6luYB0dIgCulc19mxuV1j+dMKRIYFLmIKzP8x8biR/fmyw3GxAcnpRmWYFOmh3J8=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 05/44] x86/mm: prepare create_perdomain_mapping() for per-vCPU perdomain areas Date: Tue, 3 Nov 2026 17:13:27 -0500 Message-ID: <20261103-asi-hvm-v6-5-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748051021158501 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that create_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. The remaining callers are already in a vCPU context. This is no accident: much of the perdomain area has always been laid out in per-vCPU slices -- each vCPU has its own GDT/LDT window and its own COMPAT_ARG_XLAT pages -- and each vCPU's slice is set up as that vCPU is created. For these callers, we just need to change the parameter from a domain pointer to a vCPU pointer. Once the perdomain area itself becomes per-vCPU, the same calls will populate the owning vCPU's own area rather than slices of a shared one. Note that we cannot yet do a parallel movement of free_perdomain_mappings(): the per-domain page-table hierarchy is still a single domain-wide structure shared by all vCPUs, so tearing it down from a per-vCPU path would pull the mappings out from under sibling vCPUs (e.g. on a partially failed, retryable XEN_DOMCTL_max_vcpus), and vCPU-create error paths can rely on domain destruction to free a partially set up hierarchy. Once a domain's area is per-vCPU, a later change gives its vCPUs a teardown of their own; the domain-wide teardown stays for domains with a shared area, PV ones included. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v6: - Rebase over "x86/mm: allocate the per-domain L3 in arch_domain_create()". - No longer add an implicit L3 creation in arch_vcpu_create(). Changes in v5: - Message: the domain-wide mapcache is one pool, not per-vCPU windows: drop it from the per-vCPU slices, and say "much of" the area. Changes in v4: - Message: drop a sentence repeating its paragraph, and say that the domain-wide teardown stays for domains with a shared area (Roger). - End the comment in arch_vcpu_create() at the colon (Roger). Changes in v3: - Rebased onto staging without the GDT/LDT conversions (for the HVM-only series): the capture parameters stay, as the PV GDT/LDT stash still uses them. Changes in v2: - Added to the series Changes since the previously posted version: - Keep free_perdomain_mappings() (and hence perdomain teardown) domain-scoped. - Keep the idle domain without a perdomain area. - Retitle (was: "x86/mm: switch {create,destroy}_perdomain_mapping() domain parameter to vCPU"); destroy_perdomain_mapping() was switched in a separate patch. - Split the removal of mapcache_domain_init()'s redundant create_perdomain_mapping() call into its own (preceding) patch. --- xen/arch/x86/domain_page.c | 6 +++--- xen/arch/x86/include/asm/mm.h | 2 +- xen/arch/x86/mm.c | 3 ++- xen/arch/x86/pv/domain.c | 2 +- xen/arch/x86/x86_64/mm.c | 2 +- 5 files changed, 8 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index f02be1675b..f3e65e2d44 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -260,15 +260,15 @@ int mapcache_vcpu_init(struct vcpu *v) if ( ents > dcache->entries ) { /* Populate page tables. */ - int rc =3D create_perdomain_mapping(d, MAPCACHE_VIRT_START, ents, + int rc =3D create_perdomain_mapping(v, MAPCACHE_VIRT_START, ents, NIL(l1_pgentry_t *), NULL); =20 /* Populate bit maps. */ if ( !rc ) - rc =3D create_perdomain_mapping(d, (unsigned long)dcache->inus= e, + rc =3D create_perdomain_mapping(v, (unsigned long)dcache->inus= e, nr, NULL, NIL(struct page_info *= )); if ( !rc ) - rc =3D create_perdomain_mapping(d, (unsigned long)dcache->garb= age, + rc =3D create_perdomain_mapping(v, (unsigned long)dcache->garb= age, nr, NULL, NIL(struct page_info *= )); =20 if ( rc ) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 1cdd30f7e2..e85a6db203 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -607,7 +607,7 @@ int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_= HANDLE_PARAM(void) arg); #define IS_NIL(ptr) (!((uintptr_t)(ptr) + sizeof(*(ptr)))) =20 struct page_info *allocate_perdomain_l3(struct domain *d); -int create_perdomain_mapping(struct domain *d, unsigned long va, +int create_perdomain_mapping(struct vcpu *v, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg); void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index c784c4f726..61bd4aa02a 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6245,10 +6245,11 @@ struct page_info *allocate_perdomain_l3(struct doma= in *d) return pg; } =20 -int create_perdomain_mapping(struct domain *d, unsigned long va, +int create_perdomain_mapping(struct vcpu *v, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg) { + struct domain *d =3D v->domain; struct page_info *pg; l3_pgentry_t *l3tab; l2_pgentry_t *l2tab; diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 28ea0742ab..f3c711b85b 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -313,7 +313,7 @@ int switch_compat(struct domain *d) =20 static int pv_create_gdt_ldt_l1tab(struct vcpu *v) { - return create_perdomain_mapping(v->domain, GDT_VIRT_START(v), + return create_perdomain_mapping(v, GDT_VIRT_START(v), 1U << GDT_LDT_VCPU_SHIFT, v->domain->arch.pv.gdt_ldt_l1tab, NULL); diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index f467ed426b..7b04537066 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -732,7 +732,7 @@ void __init zap_low_mappings(void) =20 int setup_compat_arg_xlat(struct vcpu *v) { - return create_perdomain_mapping(v->domain, ARG_XLAT_START(v), + return create_perdomain_mapping(v, ARG_XLAT_START(v), PFN_UP(COMPAT_ARG_XLAT_SIZE), NULL, NIL(struct page_info *)); } --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748046; cv=none; d=zohomail.com; s=zohoarc; b=AL4dZE6wJ3rzPuP7kf4frt6CHFrPTgwMUmyhMAR7EScT0Ts6mazSbai1f9vf7pLWjzWcKNqDHbkpxgYoqfQXSKyUAlsbYpU82ukVFSjzupUvXlDeR4x40JNuv7mcbqEhU/X/U02xN8k//XRMzPBTqs+CnxujXSxxdEPNj3ar3KQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748046; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GsQusdW8CwK9lWg0WTbN8UcrqKWkfEhBlrYNLpqfu3M=; b=ELBAF5J82SNQbyVlCz5k4MK9EZFNWdQih+IHqh1G9Hvo74tRhAVEHamXf531s+l8tVgk5zhDdFDbFJUqjQ97XHRyghhlV8Ywmb7bU8o8assWfc1mAetW6A/7yshZaMTirjxHCQCOk12qhoFvv/QCalAcX7MGdGJ7gm7j7QGKwpc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748046820208.74947886261782; Tue, 3 Nov 2026 15:20:46 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470717.1679363 (Exim 4.92) (envelope-from ) id 1xONno-0003al-IP; Tue, 03 Nov 2026 23:20:28 +0000 Received: by outflank-mailman (output) from mailman id 1470717.1679363; Tue, 03 Nov 2026 23:20:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONno-0003a3-Di; Tue, 03 Nov 2026 23:20:28 +0000 Received: by outflank-mailman (input) for mailman id 1470717; Tue, 03 Nov 2026 23:20:26 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnm-0003QW-Sc for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:26 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONnm-0000000Cofn-2hMV; Tue, 03 Nov 2026 23:20:26 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnm-0000000EMFa-0sJu; Tue, 03 Nov 2026 23:20:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=GsQusdW8CwK9lWg0WTbN8UcrqKWkfEhBlrYNLpqfu3M=; b=5bjzRnRQ0QOdV3xF/OMMo+7ue+ xi2SKeb1Z3cpSjbaLKqm8cxu+7oxpm6HWtF0YEbscdUMd9An6W2Vz33lI6eSjbWmUlqCLU/vqLkdZ DdxRzGgMWnKIldOWcoZm2dar9hr58i3DZBQpjjhtRze6SYGj9ZvzvB+5whIjTYvM5CQU=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Stefano Stabellini , Teddy Astie Subject: [PATCH v6 06/44] x86/spec-ctrl: introduce Address Space Isolation command line option Date: Tue, 3 Nov 2026 17:13:28 -0500 Message-ID: <20261103-asi-hvm-v6-6-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748049143158500 From: Roger Pau Monn=C3=A9 Introduce the `asi=3D` command line option, and the opt_vcpu_pt_hvm and opt_vcpu_pt_hwdom knobs plus the per-domain d->arch.asi.vcpu_pt setting they control. The option is introduced ahead of the functionality it enables, so that the newly added code can be keyed on it from the start; all knobs currently default to off, and any use of the option, `asi=3D0` included, adds a boot-time warning that the feature is not functional and is meant for development only. The mechanisms this option controls apply to HVM domains: an HVM vCPU already runs on its own monitor table, so giving each vCPU its own per-domain area needs no work at context switch. d->arch.asi.vcpu_pt is therefore only ever set for HVM domains, a PVH hardware domain included; PV domains, a PV hardware domain included, are unaffected. Besides the plain boolean, `pv=3D`, `hvm=3D` and `dom0=3D` select the featu= res for PV domains or HVM domains other than the hardware domain, or for the hardware domain, PV or PVH. None of the features applies to PV domains, so `pv=3D` has no effect beyond a boot-log line saying it is ignored, and `dom0=3D` acts on a PVH hardware domain only. The boot log gains "ASI features for ..." lines for Dom0 and HVM domains, so hardware-domain-only configurations remain visible. Further per-mechanism tokens arrive with their mechanisms in later patches. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - Rename opt_pvh_vcpu_pt_hwdom back to opt_vcpu_pt_hwdom: that it acts on a PVH hardware domain only is a limitation of the implementation, not of the design (Roger). - pv=3D logs that it is ignored (Roger). - parse_asi() adds the development warning, so any use of asi=3D gives it and later sub-options need not extend a condition (Roger). - print_details(): IS_ENABLED(CONFIG_HVM) rather than #ifdef (Roger). Changes in v4: - Docs: "domains" throughout, not "guests" (Roger). - Group the per-domain settings as one-bit fields of d->arch.asi (Roger). - Rename opt_vcpu_pt_hwdom to opt_pvh_vcpu_pt_hwdom: it acts on a PVH hardware domain only (Jan). - parse_asi(): break in the success branch and fall through to default:, rather than a default: label after an else (MISRA C:2012 Rule 16.2; Jan). - Message: enabling a knob adds a boot-time warning; it does not taint (Roger). - Add the pv=3D and dom0=3D sub-options here, beside hvm=3D (Roger). Changes in v3: - HVM only: drop the PV knob and its interaction with XPTI. vCPU-PT applies to HVM domains and to a PVH hardware domain; the Dom0 boot log line says None for a PV one. Changes in v2: - Added to the series Changes since the previously posted version: - Include the hardware domain in the development warning and the boot log summary. - Make the opt_vcpu_pt_* knobs plain booleans preinitialised to false, dropping the late -1 resolution. - Documentation: mention possible protection against unmitigated attacks, and state that hvm=3D does not affect the hardware domain. - Rewrite the commit message. --- docs/misc/xen-command-line.pandoc | 29 ++++++++++ xen/arch/x86/include/asm/domain.h | 6 ++ xen/arch/x86/include/asm/spec_ctrl.h | 2 + xen/arch/x86/spec_ctrl.c | 87 ++++++++++++++++++++++++++++ 4 files changed, 124 insertions(+) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index a6f08463d1..839d06da26 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -202,6 +202,35 @@ to appropriate auditing by Xen. Argo is disabled by d= efault. This option is disabled by default, to protect domains from a DoS by a buggy or malicious other domain spamming the ring. =20 +### asi (x86) +> `=3D List of [ , {pv,hvm,dom0}=3D, +> vcpu-pt=3D | vcpu-pt=3Dhvm=3D ]` + +> Default: `false` + +Offers control over whether the hypervisor will engage in Address Space +Isolation, by not having potentially sensitive information permanently map= ped +in the VMM page-tables. Using this option might avoid the need to apply +mitigations for certain speculative related attacks, at the cost of mapping +sensitive information on-demand. It might also offer some protection agai= nst +unmitigated speculation-related attacks. + +The mechanisms currently implemented apply to HVM domains, including a PVH +hardware domain. PV domains, including a PV hardware domain, are unaffect= ed. + +* `pv=3D`, `hvm=3D` and `dom0=3D` enable the features for PV domains or HV= M domains + other than the hardware domain, or for the hardware domain, PV or PVH. = The + plain boolean does all three: `asi=3D1,dom0=3D0` leaves the hardware dom= ain + out. The forms naming no domain type (an un-suffixed `vcpu-pt=3D`) + cover the hardware domain too. + +**WARNING: manual de-selection of enabled options will invalidate any +protection offered by the feature. The fine grained options provided below +are meant to be used for debugging purposes only.** + +* `vcpu-pt` gives each vCPU its own per-domain area: the per-domain slot of + each vCPU's monitor table maps a region private to that vCPU. + ### asid (x86) > `=3D ` =20 diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 212daae1f5..12f79feb42 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -474,6 +474,12 @@ struct arch_domain /* Don't unconditionally inject #GP for unhandled MSRs. */ bool msr_relaxed; =20 + /* Fine grained ASI related options. */ + struct { + /* Give each vCPU its own per-domain area (HVM only). */ + bool vcpu_pt:1; + } asi; + /* Emulated devices enabled bitmap. */ uint32_t emulation_flags; } __cacheline_aligned; diff --git a/xen/arch/x86/include/asm/spec_ctrl.h b/xen/arch/x86/include/as= m/spec_ctrl.h index 8f82533c41..b62229ab5d 100644 --- a/xen/arch/x86/include/asm/spec_ctrl.h +++ b/xen/arch/x86/include/asm/spec_ctrl.h @@ -87,6 +87,8 @@ extern uint8_t default_scf; =20 extern int8_t opt_xpti_hwdom, opt_xpti_domu; =20 +extern bool opt_vcpu_pt_hwdom, opt_vcpu_pt_hvm; + extern bool cpu_has_bug_l1tf; extern int8_t opt_pv_l1tf_hwdom, opt_pv_l1tf_domu; extern bool opt_bp_spec_reduce; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index bc8538a56f..8797523a62 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -86,6 +86,10 @@ bool __ro_after_init opt_bp_spec_reduce =3D true; =20 static bool __initdata opt_ibpb_alt; =20 +/* Give each vCPU of an HVM domain its own per-domain area (vCPU-PT). */ +bool __ro_after_init opt_vcpu_pt_hvm; +bool __ro_after_init opt_vcpu_pt_hwdom; + static int __init cf_check parse_spec_ctrl(const char *s) { const char *ss; @@ -487,6 +491,80 @@ static int __init cf_check parse_pv_l1tf(const char *s) } custom_param("pv-l1tf", parse_pv_l1tf); =20 +static int __init cf_check parse_asi(const char *s) +{ + static bool __initdata warned; + const char *ss; + int val, rc =3D 0; + + if ( !warned ) + { + warned =3D true; + warning_add( + "Address Space Isolation is not functional, this option is\n" + "intended to be used only for development purposes.\n"); + } + + /* Interpret 'asi' alone in its positive boolean form. */ + if ( *s =3D=3D '\0' ) + opt_vcpu_pt_hwdom =3D opt_vcpu_pt_hvm =3D true; + + do { + ss =3D strchr(s, ','); + if ( !ss ) + ss =3D strchr(s, '\0'); + + val =3D parse_bool(s, ss); + switch ( val ) + { + case 0: + case 1: + opt_vcpu_pt_hwdom =3D opt_vcpu_pt_hvm =3D val; + break; + + default: + if ( (val =3D parse_boolean("pv", s, ss)) >=3D 0 ) + printk(XENLOG_WARNING + "No ASI feature applies to PV domains - ignoring 'a= si=3D%.*s' setting\n", + (int)(ss - s), s); + else if ( (val =3D parse_boolean("hvm", s, ss)) >=3D 0 ) + opt_vcpu_pt_hvm =3D val; + else if ( (val =3D parse_boolean("dom0", s, ss)) >=3D 0 ) + opt_vcpu_pt_hwdom =3D val; + else if ( (val =3D parse_boolean("vcpu-pt", s, ss)) !=3D -1 ) + { + switch ( val ) + { + case 1: + case 0: + opt_vcpu_pt_hwdom =3D opt_vcpu_pt_hvm =3D val; + break; + + case -2: + s +=3D strlen("vcpu-pt=3D"); + if ( (val =3D parse_boolean("hvm", s, ss)) >=3D 0 ) + { + opt_vcpu_pt_hvm =3D val; + break; + } + fallthrough; + default: + rc =3D -EINVAL; + break; + } + } + else if ( *s ) + rc =3D -EINVAL; + break; + } + + s =3D ss + 1; + } while ( *ss ); + + return rc; +} +custom_param("asi", parse_asi); + static void __init print_details(enum ind_thunk thunk) { unsigned int _7d0 =3D 0, _7d2 =3D 0, e8b =3D 0, e21a =3D 0, e21c =3D 0= , max =3D 0, tmp; @@ -680,6 +758,12 @@ static void __init print_details(enum ind_thunk thunk) opt_pv_l1tf_hwdom ? "enabled" : "disabled", opt_pv_l1tf_domu ? "enabled" : "disabled"); #endif + + printk(" ASI features for Dom0:%s\n", + opt_dom0_pvh && opt_vcpu_pt_hwdom ? " vCPU-PT" : " None"); + if ( IS_ENABLED(CONFIG_HVM) ) + printk(" ASI features for HVM VMs:%s\n", + opt_vcpu_pt_hvm ? " vCPU-PT" : " None"); } =20 static bool __init check_smt_enabled(void) @@ -1866,6 +1950,9 @@ void spec_ctrl_init_domain(struct domain *d) if ( pv ) d->arch.pv.xpti =3D is_hardware_domain(d) ? opt_xpti_hwdom : opt_xpti_domu; + + d->arch.asi.vcpu_pt =3D !pv && (is_hardware_domain(d) ? opt_vcpu_pt_hw= dom + : opt_vcpu_pt_hvm); } =20 void __init init_speculation_mitigations(void) --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748056; cv=none; d=zohomail.com; s=zohoarc; b=cQTohbUlvSQhkBPpDlelO4zOfzNsxx5rcY6g2k7qZ2OkFQlMgFBl90wpKdCVda2b2ANkxl7IUG4gOPMpKrYhoLKUFsDyMXeC1aEE47znhQJMls6Y/lpijMHFGmeDXevP0k3vVuQrSS3sG9UpE7zRezimNJX2WgLJ/WzSOUoPuM0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748056; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SDb1ZxgXV65R0hZws53PbRDvNFXyUOAfc8HwZa5MEvU=; b=fbD1su8Wn6jgL3GMFDooJWPnSW+E1odWy2G1pF/0I2yaiTWFYCDDd2diwcFihLQDD1ekNRUcFuVYmbd8PiuEX7EaegaaHAogzjKyWPpgf5tQ9vwuqn7FgUbkW48YQjMR/6B6fMaQNPkwC/6x09YQ8uHekuLDkM2ZBWZRJ+3dVHk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748056293638.6233501595924; Tue, 3 Nov 2026 15:20:56 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470718.1679380 (Exim 4.92) (envelope-from ) id 1xONnp-000423-Uu; Tue, 03 Nov 2026 23:20:29 +0000 Received: by outflank-mailman (output) from mailman id 1470718.1679380; Tue, 03 Nov 2026 23:20:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnp-00041h-O5; Tue, 03 Nov 2026 23:20:29 +0000 Received: by outflank-mailman (input) for mailman id 1470718; Tue, 03 Nov 2026 23:20:28 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONno-0003YQ-4G for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:28 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONnn-0000000Cog2-2oTn; Tue, 03 Nov 2026 23:20:27 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnn-0000000EMFa-2ElZ; Tue, 03 Nov 2026 23:20:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=SDb1ZxgXV65R0hZws53PbRDvNFXyUOAfc8HwZa5MEvU=; b=rRG6C2ZLVRCc123dGWHhe/NrTD xjKc8krAbf/HysLL8U/tZlG9RpnYBxdTARxYTQq5GBaLrcbNdWcZBr5Z9RAJC2kws3NBf9pUlH7BC w/msqzI4mS1BQj8/ImQ0fnQODZ1Q5HD0hubX/pnfRYcZSUMhTSWIRzLof4BzDsN68vkk=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie , Tim Deegan Subject: [PATCH v6 07/44] x86/mm: have init_xen_l4_slots() take a vCPU in place of the domain Date: Tue, 3 Nov 2026 17:13:29 -0500 Message-ID: <20261103-asi-hvm-v6-7-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748057149158500 Content-Type: text/plain; charset="utf-8" A later change gives the vCPUs of some domains per-domain areas of their own, and slot 260 of a table private to one vCPU must then hold that vCPU's L3. Eventually, all guest types may be able to support either per-domain L3s or per-vCPU L3s. HVM domains always have one root page table per vCPU; this does not change. So init_xen_l4_slots() needs the appropriate L3 at vCPU creation, either the domain-wide L3 or the per-vCPU L3. PV domains with a domain-wide L3 should behave as now: the Xen slots of guest root pagetables need the domain-wide L3 installed. But PV guests with per-vCPU page tables can never run directly on guest pagetables; they will always need some separate page table, which can be initialized with the per-vCPU L3. So, create two versions of init_xen_l4_slots(). The first is used in contexts where we may be initializing a non-guest root, and must have a vCPU L3 if per-vCPU L3s are in use; this takes a vCPU as an argument. The second is used for roots that all of a domain's vCPUs share (a PV guest's own L4s, and their shadows), which only need an L3 if domain-wide L3s are in use; this takes a domain as an argument. Both wrap an internal function, which takes a domain and an L3 pointer, and leaves the slot empty if this pointer is NULL. At the moment, both always pass on the domain-wide L3, but in the future this will change if the guest is using per-vCPU pagetables. Otherwise no functional change: slot 260 still gets the domain's per-domain L3. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v6: - Add init_xen_l4_slots_pv() form, which only takes a domain as an argument; promote_l4_table(), the PV dom0 builder and the shadows of PV guests' L4s use it, so promote_l4_table() no longer needs vCPU 0. Changes in v5: - promote_l4_table() checks for vCPU 0 itself, now that "x86/mm: refuse to promote an L4 while the domain has no per-domain L3" tests for the L3 instead. Changes in v4: - New in this version, split out of "x86/hvm: introduce per-vCPU L3 page-table": init_xen_l4_slots() takes a vCPU in place of the domain (Roger, Jan). - Reflow promote_l4_table()'s call to init_xen_l4_slots() (Jan). --- xen/arch/x86/include/asm/mm.h | 4 +++- xen/arch/x86/mm.c | 35 +++++++++++++++++++++++++++++----- xen/arch/x86/mm/hap/hap.c | 2 +- xen/arch/x86/mm/shadow/hvm.c | 2 +- xen/arch/x86/mm/shadow/multi.c | 5 +++-- xen/arch/x86/pv/dom0_build.c | 4 ++-- xen/arch/x86/pv/domain.c | 2 +- 7 files changed, 41 insertions(+), 13 deletions(-) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index e85a6db203..0d2cecf1c7 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -370,7 +370,9 @@ int devalidate_page(struct page_info *page, unsigned lo= ng type, =20 void init_xen_pae_l2_slots(l2_pgentry_t *l2t, const struct domain *d); void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, - const struct domain *d, mfn_t sl4mfn, bool ro_mpt); + const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt); +void init_xen_l4_slots_pv(l4_pgentry_t *l4t, mfn_t l4mfn, + const struct domain *d, mfn_t sl4mfn, bool ro_mp= t); bool fill_ro_mpt(mfn_t mfn); void zap_ro_mpt(mfn_t mfn); =20 diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 61bd4aa02a..10106d9f03 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -1693,8 +1693,10 @@ static int promote_l3_table(struct page_info *page) * appropriate slots. Pagetables not shared with guests will gain the * extended directmap. */ -void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, - const struct domain *d, mfn_t sl4mfn, bool ro_mpt) +static void _init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, + const struct domain *d, + const struct page_info *l3_pg, + mfn_t sl4mfn, bool ro_mpt) { /* * PV vcpus need a shortened directmap. HVM and Idle vcpus get the fu= ll @@ -1723,7 +1725,7 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, =20 /* Slot 260: Per-domain mappings. */ l4t[l4_table_offset(PERDOMAIN_VIRT_START)] =3D - l4e_from_page(d->arch.perdomain_l3_pg, __PAGE_HYPERVISOR_RW); + l3_pg ? l4e_from_page(l3_pg, __PAGE_HYPERVISOR_RW) : l4e_empty(); =20 /* Slot 4: Per-domain mappings mirror. */ BUILD_BUG_ON(IS_ENABLED(CONFIG_PV32) && @@ -1770,6 +1772,29 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mf= n, } } =20 +/* + * v is the vCPU the table is private to. For shared root pagetables, use + * init_xen_l4_slots_pv(). + */ +void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, + const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt) +{ + const struct domain *d =3D v->domain; + + _init_xen_l4_slots(l4t, l4mfn, d, d->arch.perdomain_l3_pg, sl4mfn, ro_= mpt); +} + +/* + * This is a version of init_xen_l4_slots for shared root pagetables; + * it only propagates domain-wide perdomain L3s, and so only needs a + * domain pointer. + */ +void init_xen_l4_slots_pv(l4_pgentry_t *l4t, mfn_t l4mfn, + const struct domain *d, mfn_t sl4mfn, bool ro_mp= t) +{ + _init_xen_l4_slots(l4t, l4mfn, d, d->arch.perdomain_l3_pg, sl4mfn, ro_= mpt); +} + bool fill_ro_mpt(mfn_t mfn) { l4_pgentry_t *l4tab =3D map_domain_page(mfn); @@ -1878,8 +1903,8 @@ static int promote_l4_table(struct page_info *page) =20 if ( !rc ) { - init_xen_l4_slots(pl4e, l4mfn, - d, INVALID_MFN, VM_ASSIST(d, m2p_strict)); + init_xen_l4_slots_pv(pl4e, l4mfn, d, INVALID_MFN, + VM_ASSIST(d, m2p_strict)); atomic_inc(&d->arch.pv.nr_l4_pages); } unmap_domain_page(pl4e); diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c index 0ede4181a0..aba4f77df9 100644 --- a/xen/arch/x86/mm/hap/hap.c +++ b/xen/arch/x86/mm/hap/hap.c @@ -407,7 +407,7 @@ static mfn_t hap_make_monitor_table(struct vcpu *v) m4mfn =3D page_to_mfn(pg); l4e =3D map_domain_page(m4mfn); =20 - init_xen_l4_slots(l4e, m4mfn, d, INVALID_MFN, false); + init_xen_l4_slots(l4e, m4mfn, v, INVALID_MFN, false); unmap_domain_page(l4e); =20 return m4mfn; diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c index e6fb97c4b6..0b23326214 100644 --- a/xen/arch/x86/mm/shadow/hvm.c +++ b/xen/arch/x86/mm/shadow/hvm.c @@ -760,7 +760,7 @@ mfn_t sh_make_monitor_table(const struct vcpu *v, unsig= ned int shadow_levels) * shadow-linear mapping will either be inserted below when creating * lower level monitor tables, or later in sh_update_cr3(). */ - init_xen_l4_slots(l4e, m4mfn, d, INVALID_MFN, false); + init_xen_l4_slots(l4e, m4mfn, v, INVALID_MFN, false); =20 if ( shadow_levels < 4 ) { diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c index 7f72b4b7b2..1831091b95 100644 --- a/xen/arch/x86/mm/shadow/multi.c +++ b/xen/arch/x86/mm/shadow/multi.c @@ -974,8 +974,9 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_t= ype) =20 BUILD_BUG_ON(sizeof(l4_pgentry_t) !=3D sizeof(shadow_l4e_t)); =20 - init_xen_l4_slots(l4t, gmfn, d, smfn, (!is_pv_32bit_domain(d) = && - VM_ASSIST(d, m2p_strict= ))); + init_xen_l4_slots_pv(l4t, gmfn, d, smfn, + (!is_pv_32bit_domain(d) && + VM_ASSIST(d, m2p_strict))); unmap_domain_page(l4t); } break; diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index 24f2fa0b7f..dd64af5616 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -724,8 +724,8 @@ static int __init dom0_construct(const struct boot_doma= in *bd) maddr_to_page(mpt_alloc)->u.inuse.type_info =3D PGT_l4_page_table; l4start =3D l4tab =3D __va(mpt_alloc); mpt_alloc +=3D PAGE_SIZE; clear_page(l4tab); - init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)), - d, INVALID_MFN, true); + init_xen_l4_slots_pv(l4tab, _mfn(virt_to_mfn(l4start)), + d, INVALID_MFN, true); v->arch.guest_table =3D pagetable_from_paddr(__pa(l4start)); } else diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index f3c711b85b..f139d8f83c 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -127,7 +127,7 @@ static int setup_compat_l4(struct vcpu *v) mfn =3D page_to_mfn(pg); l4tab =3D map_domain_page(mfn); clear_page(l4tab); - init_xen_l4_slots(l4tab, mfn, v->domain, INVALID_MFN, false); + init_xen_l4_slots(l4tab, mfn, v, INVALID_MFN, false); unmap_domain_page(l4tab); =20 /* This page needs to look like a pagetable so that it can be shadowed= */ --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748054; cv=none; d=zohomail.com; s=zohoarc; b=AuVZwaA6TTBEMC5dhNSQmXERMRCZFvx7a6vriDUC18hvfdkFyTKjuRYjnI+kZV9+e0PJmZCtaff9MaMp1e3xSxEkOWzrUjWVfbW1cYTs006U21MJFgfP57lA7RxL0P6gS+ClW2Ak/fKWW4/1MIgcXM89r3yIewEcGizbSnLf7RE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748054; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GNHcHVUgW1gDmGbSBl5sXl7NRkq4PP8PDdfXE6DiBQQ=; b=QmcKvD+A4/ug0Sb5VImBQ5l5+IKQMAHZ9EeXI6QsWCWuRefEuGNg+Hf+qw3azxvZbmUUqCXYOnFf/QE/9JCo/nmeedqvJlLQoh/JJ+cEcgh2AvxycsDP2SNiypQ1wmeQnwgei9VOJuLHBxSfvJL3polFkd2/OJ249A3H0fL6w9s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748054130473.4271346074578; Tue, 3 Nov 2026 15:20:54 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470719.1679387 (Exim 4.92) (envelope-from ) id 1xONnq-0004Aa-Ij; Tue, 03 Nov 2026 23:20:30 +0000 Received: by outflank-mailman (output) from mailman id 1470719.1679387; Tue, 03 Nov 2026 23:20:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnq-00049E-AM; Tue, 03 Nov 2026 23:20:30 +0000 Received: by outflank-mailman (input) for mailman id 1470719; Tue, 03 Nov 2026 23:20:29 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnp-0003yV-HG for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:29 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONnp-0000000CogD-1FPr; Tue, 03 Nov 2026 23:20:29 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnn-0000000EMFa-3ykc; Tue, 03 Nov 2026 23:20:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=GNHcHVUgW1gDmGbSBl5sXl7NRkq4PP8PDdfXE6DiBQQ=; b=JB6IAy9ohjxqSmTsoSvQHCdgoJ zLWCcvVMj31ICbgua5mHcdTp/Y7qosaP80Yp9yYdWcki15/Ug2F5KJCdP9cDcg6TOGhuKByIKyTgW NRvun3gQ5IiETjM467nrCS7H32G31aBunR/NYOHOEvptjMVtWor+jwNHOtovU2TvLUq8=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 08/44] x86/hvm: introduce per-vCPU L3 page-table Date: Tue, 3 Nov 2026 17:13:30 -0500 Message-ID: <20261103-asi-hvm-v6-8-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748055025158500 From: Roger Pau Monn=C3=A9 Currently all of a domain's vCPUs share its per-domain area: one L3, referenced from every root page-table associated with the domain, so every mapping in it is visible to every vCPU of the domain. Much of the area is already laid out in per-vCPU slices (each vCPU's COMPAT_ARG_XLAT pages, and for PV each vCPU's GDT/LDT window), but the visibility is domain-wide. Meanwhile, much of the per-vCPU state Xen maintains -- the VMCB, the VMX MSR load/save areas, FPU/XSAVE state -- lives in always-mapped memory. For HVM domains, allow each vCPU a per-domain area of its own instead ("vCPU-PT"). HVM monitor tables are already per-vCPU, so this needs nothing at context switch: every monitor table carries its vCPU's own L3 from creation, and the per-vCPU mappings are isolated from other running vCPUs at once. We will later build on this, adding a per-vCPU mapcache and a per-vCPU state window, into which we can move the vCPU state mentioned above. Add pervcpu_l3_pg to the arch_vcpu struct, to correspond to the perdomain_l3_pg in the domain struct. (We retain both so that vCPU-PT can be chosen domain by domain; PV domains always use the domain-wide L3.) Modify arch_domain_create() to only allocate a per-domain L3 if vcpu_pt is not enabled for the domain; modify arch_vcpu_create() to allocate a per-vCPU L3 if it is. In {create,destroy}_perdomain_mapping(), if d->arch.asi.vcpu_pt, use pervcpu_l3_pg as the per-domain L3; otherwise, use perdomain_l3_pg. Introduce a helper, perdomain_l3(), to consistently choose the correct one. Introduce free_pervcpu_mappings() to free this tree, called from arch_vcpu_destroy() on normal teardown. Since the vcpu structure holds the only reference to pervcpu_l3_pg, arch_vcpu_create() must also call it on its error paths: nothing else records the allocation once the vcpu struct is torn down. The domain-wide free_perdomain_mappings() keeps covering non-vCPU-PT domains. init_xen_l4_slots() is given the vCPU that a table is private to. Have it install that vCPU's perdomain_l3() in slot 260. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-fable-5-1, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v6: - Port to new explicit L3 allocation regime. Changes in v5: - init_xen_l4_slots()'s BUG_ON() follows slot 260 to perdomain_l3(). - Message: all of a domain's vCPUs share its per-domain area; the mapcache is not among the per-vCPU slices; the mode is "vCPU-PT". The comment above init_xen_l4_slots() names perdomain_l3(v) and the domain-wide L3. Changes in v4: - Rebase over earlier changes - d->arch.vcpu_pt is now d->arch.asi.vcpu_pt. - init_xen_l4_slots() taking a vCPU in place of the domain, and promote_l4_table() failing while the domain has none, are now patches of their own (Roger, Jan). - Drop the fail_early label: every cleanup step after fail: is safe before its setup has run (Roger). - free_pervcpu_mappings(): clear the pointer, then free, as XFREE() does (Roger). - Likewise in free_perdomain_mappings(). Changes in v3: - HVM only: PV domains keep the domain-wide area, so the PV parts go (the single-vCPU limit, the shadow refusals, the XPTI slot install). init_xen_l4_slots() takes the vCPU as an extra, optional argument rather than in place of the domain, so PV guest L4s, which no vCPU owns, are promoted exactly as before. - Retitle (was: "x86/mm: introduce per-vCPU L3 page-table"). - free_compat_arg_xlat()'s comment names free_pervcpu_mappings() as the teardown for a vCPU-PT domain. Changes in v2: - Added to the series Changes since the previously posted version: - Expand the commit message with the motivation and the design rationale. - Keep domain-wide freeing intact and introduce a vCPU-scoped free_pervcpu_mappings() instead of re-scoping free_perdomain_mappings(); fix the arch_vcpu_create() error-path leaks of a partially built per-vCPU hierarchy. - Add a perdomain_l3() helper for the root selection, rather than open-coding the vcpu_pt choice (and testing both root pointers) at each site. --- xen/arch/x86/domain.c | 17 +++++++-- xen/arch/x86/include/asm/domain.h | 4 ++ xen/arch/x86/include/asm/mm.h | 2 + xen/arch/x86/mm.c | 61 +++++++++++++++++++++++-------- 4 files changed, 66 insertions(+), 18 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 173daf9000..46ca945b48 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -511,16 +511,23 @@ int arch_vcpu_create(struct vcpu *v) =20 v->arch.flags =3D TF_kernel_mode; =20 + if ( d->arch.asi.vcpu_pt && + (v->arch.pervcpu_l3_pg =3D allocate_perdomain_l3(d)) =3D=3D NULL ) + { + rc =3D -ENOMEM; + goto fail; + } + rc =3D mapcache_vcpu_init(v); if ( rc ) - return rc; + goto fail; =20 if ( !is_idle_domain(d) ) { paging_vcpu_init(v); =20 if ( (rc =3D vcpu_init_fpu(v)) !=3D 0 ) - return rc; + goto fail; =20 vmce_init_vcpu(v); =20 @@ -568,6 +575,7 @@ int arch_vcpu_create(struct vcpu *v) vcpu_destroy_fpu(v); xfree(v->arch.msrs); v->arch.msrs =3D NULL; + free_pervcpu_mappings(v); =20 return rc; } @@ -588,6 +596,8 @@ void arch_vcpu_destroy(struct vcpu *v) pv_vcpu_destroy(v); else ASSERT_UNREACHABLE(); + + free_pervcpu_mappings(v); } =20 int arch_sanitise_domain_config(const struct xen_domctl_createdomain *conf= ig) @@ -901,7 +911,8 @@ int arch_domain_create(struct domain *d, =20 psr_domain_init(d); =20 - if ( (d->arch.perdomain_l3_pg =3D allocate_perdomain_l3(d)) =3D=3D NUL= L ) + if ( !d->arch.asi.vcpu_pt && + (d->arch.perdomain_l3_pg =3D allocate_perdomain_l3(d)) =3D=3D NUL= L ) { rc =3D -ENOMEM; goto fail; diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 12f79feb42..4140a92f15 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -332,6 +332,7 @@ struct monitor_write_data { =20 struct arch_domain { + /* Domain-wide L3 for the per-domain slot; NULL in a vCPU-PT domain. */ struct page_info *perdomain_l3_pg; =20 /* I/O-port admin-specified access capabilities. */ @@ -688,6 +689,9 @@ struct arch_vcpu =20 struct vcpu_msrs *msrs; =20 + /* Per-vCPU L3 for the per-domain slot; NULL in a non-vCPU-PT domain. = */ + struct page_info *pervcpu_l3_pg; + struct { bool next_interrupt_enabled; } monitor; diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 0d2cecf1c7..7b9a0f561d 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -369,6 +369,7 @@ int devalidate_page(struct page_info *page, unsigned lo= ng type, int preemptible); =20 void init_xen_pae_l2_slots(l2_pgentry_t *l2t, const struct domain *d); +struct page_info *perdomain_l3(const struct vcpu *v); void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt); void init_xen_l4_slots_pv(l4_pgentry_t *l4t, mfn_t l4mfn, @@ -615,6 +616,7 @@ int create_perdomain_mapping(struct vcpu *v, unsigned l= ong va, void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr); void free_perdomain_mappings(struct domain *d); +void free_pervcpu_mappings(struct vcpu *v); =20 void __iomem *ioremap_wc(paddr_t pa, size_t len); =20 diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 10106d9f03..f52c72a605 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -1682,6 +1682,18 @@ static int promote_l3_table(struct page_info *page) } #endif /* CONFIG_PV */ =20 +/* + * The root of the per-domain area in use by @v: the vCPU's own L3 for a + * vCPU-PT domain, the domain-wide one otherwise. + */ +struct page_info *perdomain_l3(const struct vcpu *v) +{ + const struct domain *d =3D v->domain; + + return d->arch.asi.vcpu_pt ? v->arch.pervcpu_l3_pg + : d->arch.perdomain_l3_pg; +} + /* * Fill an L4 with Xen entries. * @@ -1780,8 +1792,11 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mf= n, const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt) { const struct domain *d =3D v->domain; + struct page_info *l3_pg =3D perdomain_l3(v); =20 - _init_xen_l4_slots(l4t, l4mfn, d, d->arch.perdomain_l3_pg, sl4mfn, ro_= mpt); + BUG_ON(!l3_pg); + + _init_xen_l4_slots(l4t, l4mfn, d, l3_pg, sl4mfn, ro_mpt); } =20 /* @@ -6275,7 +6290,7 @@ int create_perdomain_mapping(struct vcpu *v, unsigned= long va, struct page_info **ppg) { struct domain *d =3D v->domain; - struct page_info *pg; + struct page_info *pg, *l3_pg =3D perdomain_l3(v); l3_pgentry_t *l3tab; l2_pgentry_t *l2tab; l1_pgentry_t *l1tab; @@ -6284,12 +6299,12 @@ int create_perdomain_mapping(struct vcpu *v, unsign= ed long va, ASSERT(va >=3D PERDOMAIN_VIRT_START && va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); =20 - BUG_ON(!d->arch.perdomain_l3_pg); + BUG_ON(!l3_pg); =20 if ( !nr ) return 0; =20 - l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); + l3tab =3D __map_domain_page(l3_pg); =20 ASSERT(!l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1))); =20 @@ -6386,16 +6401,16 @@ void destroy_perdomain_mapping(const struct vcpu *v= , unsigned long va, unsigned int nr) { const l3_pgentry_t *l3tab, *pl3e; - const struct domain *d =3D v->domain; + struct page_info *l3_pg =3D perdomain_l3(v); =20 ASSERT(va >=3D PERDOMAIN_VIRT_START && va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); ASSERT(!nr || !l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1))); =20 - if ( !d->arch.perdomain_l3_pg ) + if ( !l3_pg ) return; =20 - l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); + l3tab =3D __map_domain_page(l3_pg); pl3e =3D l3tab + l3_table_offset(va); =20 if ( l3e_get_flags(*pl3e) & _PAGE_PRESENT ) @@ -6434,16 +6449,11 @@ void destroy_perdomain_mapping(const struct vcpu *v= , unsigned long va, unmap_domain_page(l3tab); } =20 -void free_perdomain_mappings(struct domain *d) +static void free_perdomain_l3(struct page_info *l3pg) { - l3_pgentry_t *l3tab; + l3_pgentry_t *l3tab =3D __map_domain_page(l3pg); unsigned int i; =20 - if ( !d->arch.perdomain_l3_pg ) - return; - - l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); - for ( i =3D 0; i < PERDOMAIN_SLOTS; ++i) if ( l3e_get_flags(l3tab[i]) & _PAGE_PRESENT ) { @@ -6479,8 +6489,29 @@ void free_perdomain_mappings(struct domain *d) } =20 unmap_domain_page(l3tab); - free_domheap_page(d->arch.perdomain_l3_pg); + free_domheap_page(l3pg); +} + +void free_perdomain_mappings(struct domain *d) +{ + struct page_info *l3pg =3D d->arch.perdomain_l3_pg; + + if ( !l3pg ) + return; + d->arch.perdomain_l3_pg =3D NULL; + free_perdomain_l3(l3pg); +} + +void free_pervcpu_mappings(struct vcpu *v) +{ + struct page_info *l3pg =3D v->arch.pervcpu_l3_pg; + + if ( !l3pg ) + return; + + v->arch.pervcpu_l3_pg =3D NULL; + free_perdomain_l3(l3pg); } =20 static void write_sss_token(unsigned long *ptr) --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748061; cv=none; d=zohomail.com; s=zohoarc; b=ZgVCn9Uur7OpNI+1zuAfPayCCNdhzzV5Axsn93fr+R0t9Nm2SH9cXjYa5NLTw9Mw0PO+KHLOYXhUk6bQBSmcU5DbDp9J5+OalP4fGnjpTrvHv2+OUfqAagANwvoUyrRsXyo7EJzgLbMdlc0D3Wxix7T2KB2aj8oFqrWUozxl/Bk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748061; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=S/tsd9lGldh/FCRRWK8hPBso1PhupuXumoKhOFQc+oM=; b=ab9uiMqgh9HETWEppKVoLJQEJPr/gY3DlDAtZMMPIQFvwYooor5+hPIYnOnIDwsu+TaOPpU3cTWl9X10DekCdVvzxW8kGhwX00P0CHP66H/1ClhkZ44TvSG7C1rdyCDKFdjJsV+WudJuc6Vgyl7HQOL1vFtni2qKcdTi6he8OYE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 179374806158472.05895089727585; Tue, 3 Nov 2026 15:21:01 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470721.1679397 (Exim 4.92) (envelope-from ) id 1xONns-0004XF-1Y; Tue, 03 Nov 2026 23:20:32 +0000 Received: by outflank-mailman (output) from mailman id 1470721.1679397; Tue, 03 Nov 2026 23:20:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnr-0004Vy-Q1; Tue, 03 Nov 2026 23:20:31 +0000 Received: by outflank-mailman (input) for mailman id 1470721; Tue, 03 Nov 2026 23:20:30 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONnq-0004BH-GN for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:20:30 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONnq-0000000CogL-1AtX; Tue, 03 Nov 2026 23:20:30 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnq-0000000EMFa-0gX1; Tue, 03 Nov 2026 23:20:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=S/tsd9lGldh/FCRRWK8hPBso1PhupuXumoKhOFQc+oM=; b=UlvwapHs2AsUIdTh+zfX8imUtG OiqPjv+mk1oqHQbYf0Yvk/W5Tnz20CBCPnYp4PErPN8RuEE2Ng8VtEEOlf6PuBWxQoR9mBojsr1wK lk9GUe6rhZrjHk0AmygTqYGhiv+irBDy+9wldu12mgdKG2Mq369Ng4LxWw+DsnpLK96U=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 09/44] x86/domain_page: drop redundant is_pv_vcpu() checks Date: Tue, 3 Nov 2026 17:13:31 -0500 Message-ID: <20261103-asi-hvm-v6-9-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748063015158500 Content-Type: text/plain; charset="utf-8" The is_pv_vcpu() checks after mapcache_current_vcpu() never fail: they have been redundant since 622c9a5ba95d ("x86/mm: accurately track which vCPU page-tables are loaded") made mapcache_current_vcpu() do the check itself. mapcache_current_vcpu() is now guaranteed to return a PV vCPU with a caching structure (which is what is needed for the subsequent dcache->inuse check): either a scheduled PV vCPU, or this CPU's idle vCPU (possibly after finishing a lazy context switch). Drop the checks; their speculation barriers (evaluate_nospec()) go too. mapcache_current_vcpu() returns NULL if this CPU's pgtable_vcpu is not yet set (early boot) or is not a PV vCPU. Otherwise it reads pgtable_vcpu again and returns that. In between, pgtable_vcpu changes only when a lazy context switch from a PV vCPU is completed, by mapcache_current_vcpu() itself (sync_local_execstate()) or first by an interrupt such as a FLUSH_VCPU_STATE IPI; completing it loads the idle vCPU's page-tables and sets pgtable_vcpu to the idle vCPU. No other writer of pgtable_vcpu runs in an interrupt that returns to the code it interrupted. During an EFI runtime call pgtable_vcpu is the idle vCPU: efi_rs_enter() completes any lazy switch, then loads EFI's page-tables, which carry the directmap. So the vCPU returned is either the PV vCPU checked or this CPU's idle vCPU. The idle vCPU passes is_pv_vcpu(), as the idle domain is created without XEN_DOMCTL_CDF_hvm, but its mapcache is never set up: domain_create() returns for the idle domain before calling arch_domain_create(), which is what calls mapcache_domain_init(). With its dcache->inuse NULL, map_domain_page() uses the directmap, and unmap_domain_page()'s ASSERT() on dcache->inuse still catches a mapcache address unmapped with no mapcache in use. In builds without CONFIG_PV, is_pv_vcpu() is always false and mapcache_current_vcpu() always returns NULL. No functional change. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v4: - New in this version, split out of "x86/domain_page: prepare the mapcache for per-vCPU accounting", with a justification of its own. NB I've included Claude's complete analysis in the commit message. I've reworded the commit message so that the middle two paragraphs can be dropped, or reworded more briefly on commit. --- xen/arch/x86/domain_page.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index f3e65e2d44..ed68483695 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -74,7 +74,7 @@ void *map_domain_page(mfn_t mfn) #endif =20 v =3D mapcache_current_vcpu(); - if ( !v || !is_pv_vcpu(v) ) + if ( !v ) return mfn_to_virt(mfn_x(mfn)); =20 dcache =3D &v->domain->arch.pv.mapcache; @@ -179,7 +179,7 @@ void unmap_domain_page(const void *ptr) ASSERT(va >=3D MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); =20 v =3D mapcache_current_vcpu(); - ASSERT(v && is_pv_vcpu(v)); + ASSERT(v); =20 dcache =3D &v->domain->arch.pv.mapcache; ASSERT(dcache->inuse); --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748700; cv=none; d=zohomail.com; s=zohoarc; b=Bol/BUsxO//ftw7ql+HestMEoF7uTlzcxlxfbncTrFGfRG0KLV4PJvTq6WMkdrvsQq3ni5rHHr2y6JkmTBEvVl7Bs5KufxAWQx6v4RSSzE5T94Hx30mF+M0hSZZDHiUXUkwhLmssTQRUnSOQE651KMhMLEckckw9QJsdymuI800= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748700; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Qz96shbhY184atVnv3ju4Bmbb5aXR1JV+WrIp8a3N3k=; b=JArvvE4KT2x73kEGF/I5/Lacgyqmj1+O2eOtwIwiX55+372o4maRrOAFyQN9eeiMPI0kwdWHMcUpnN3cyJsjwVQF7M1Mbvh9JINXMAGzn3yTn839SYQRgZHElCBIVnpOsLiHDFZwAXPNKXYe4fgQs1Y2VuSvCRiylUz1cxK8O98= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748700103864.8702801540829; Tue, 3 Nov 2026 15:31:40 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470803.1679514 (Exim 4.92) (envelope-from ) id 1xONxo-0002yX-73; Tue, 03 Nov 2026 23:30:48 +0000 Received: by outflank-mailman (output) from mailman id 1470803.1679514; Tue, 03 Nov 2026 23:30:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxm-0002sD-Tt; Tue, 03 Nov 2026 23:30:46 +0000 Received: by outflank-mailman (input) for mailman id 1470803; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000dG-9U for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CouW-0EqG; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnq-0000000EMFa-2enA; Tue, 03 Nov 2026 23:20:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=Qz96shbhY184atVnv3ju4Bmbb5aXR1JV+WrIp8a3N3k=; b=vMMvAm+YRur4tvjh5Ue+qJ12+7 6h/cOgzRHvxd0W4kjN+PkgCjEibWGwcr+qxfsDNPNrtB5yDwIQdteqDwtIcjPrpmRYtU/3ZHEQNVQ i0yXJCTLpPxsGo3eTNfjJVqubxuBKOdkF1bxdT83Vo1hHZqc4BaegmOX5l1yi+eXNI54=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 10/44] x86/domain_page: prepare the mapcache for per-vCPU accounting Date: Tue, 3 Nov 2026 17:13:32 -0500 Message-ID: <20261103-asi-hvm-v6-10-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748701722158500 From: Roger Pau Monn=C3=A9 Currently, only PV domains have a mapcache, which is a domain-wide one. In preparation for enabling per-vCPU mapcaches for HVM domains, split the accounting out into a struct mapcache, embedded in struct mapcache_domain. Introduce has_mapcache(), for whether a vCPU has a mapcache, and vcpu_mapcache(), which returns the accounting for a vCPU's mapcache and, in *dcache, the domain-wide mapcache whose lock and TLB epoch the caller has to take into account. For now has_mapcache() is is_pv_vcpu(), and vcpu_mapcache() always returns the domain's accounting. Move map_domain_page()'s handling of the domain-wide lock and TLB epoch into three helpers: mapcache_enter() takes the lock and catches up with a wrap another vCPU caused, mapcache_new_epoch() starts a new epoch after a wrap, and mapcache_exit() drops the lock. A per-vCPU mapcache can then skip them without map_domain_page() being re-indented. struct mapcache_domain keeps its size (40 bytes, x86_64 debug build), and so do the structures containing it. No functional change. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v4: - Reword has_mapcache()'s comment (Roger). - Move the comment on the domain-wide lock and TLB epoch into the function body (Roger). - Rename mapcache_lock() / mapcache_unlock() to mapcache_enter() / mapcache_exit() (Roger). - mapcache_new_epoch(): assert that the domain-wide lock is held (Roger). - mapcache_vcpu_init(): drop the comment on the early return for a disabled domain-wide mapcache (Jan). - mapcache_vcpu_init(): initialise ents and nr at their declarations, as before (Jan). - Dropping the redundant is_pv_vcpu() checks in map_domain_page() and unmap_domain_page() is now a patch of its own (Jan). Changes in v3: - New in this version, split out of "x86/hvm: introduce per-vCPU mapcache for vCPU-PT domains" for review. --- xen/arch/x86/domain_page.c | 137 ++++++++++++++++++++---------- xen/arch/x86/include/asm/domain.h | 19 +++-- 2 files changed, 102 insertions(+), 54 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index ed68483695..e4ea7a254f 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -18,17 +18,19 @@ #include #include =20 +/* Whether the vCPU supports mapcache usage, might not be in use. */ +static bool has_mapcache(const struct vcpu *v) +{ + return is_pv_vcpu(v); +} + static inline struct vcpu *mapcache_current_vcpu(void) { const struct vcpu *v =3D this_cpu(pgtable_vcpu); const struct vcpu *curr =3D current; =20 - /* - * During early boot pgtable_vcpu is not set, callers must handle NULL. - * Non-PV domains don't have a mapcache, the directmap covers all phys= ical - * address space. - */ - if ( !v || !is_pv_vcpu(v) ) + /* During early boot pgtable_vcpu is not set, callers must handle NULL= . */ + if ( !v || !has_mapcache(v) ) return NULL; =20 /* @@ -54,6 +56,52 @@ static inline struct vcpu *mapcache_current_vcpu(void) return ACCESS_ONCE(this_cpu(pgtable_vcpu)); } =20 +/* + * The accounting for v's mapcache: the domain's, with *dcache set so the + * caller takes the domain-wide lock and TLB-epoch machinery into account. + */ +static struct mapcache *vcpu_mapcache(struct vcpu *v, + struct mapcache_domain **dcache) +{ + struct domain *d =3D v->domain; + + *dcache =3D &d->arch.pv.mapcache; + return &(*dcache)->cache; +} + +static void mapcache_enter(struct mapcache_domain *dcache, + struct mapcache_vcpu *vcache) +{ + /* The domain's vCPUs share the mapcache's slots. */ + spin_lock(&dcache->lock); + + /* Has some other CPU caused a wrap? We must flush if so. */ + if ( unlikely(dcache->epoch !=3D vcache->shadow_epoch) ) + { + vcache->shadow_epoch =3D dcache->epoch; + if ( NEED_FLUSH(this_cpu(tlbflush_time), dcache->tlbflush_timestam= p) ) + { + perfc_incr(domain_page_tlb_flush); + flush_tlb_local(); + } + } +} + +/* A wrap has just flushed this CPU's TLB: start a new epoch. */ +static void mapcache_new_epoch(struct mapcache_domain *dcache, + struct mapcache_vcpu *vcache) +{ + ASSERT(spin_is_locked(&dcache->lock)); + + vcache->shadow_epoch =3D ++dcache->epoch; + dcache->tlbflush_timestamp =3D tlbflush_current_time(); +} + +static void mapcache_exit(struct mapcache_domain *dcache) +{ + spin_unlock(&dcache->lock); +} + #define mapcache_l2_entry(e) ((e) >> PAGETABLE_ORDER) #define MAPCACHE_L2_ENTRIES (mapcache_l2_entry(MAPCACHE_ENTRIES - 1) + 1) #define MAPCACHE_L1ENT(idx) \ @@ -66,6 +114,7 @@ void *map_domain_page(mfn_t mfn) struct vcpu *v; struct mapcache_domain *dcache; struct mapcache_vcpu *vcache; + struct mapcache *cache; struct vcpu_maphash_entry *hashent; =20 #ifdef NDEBUG @@ -77,9 +126,9 @@ void *map_domain_page(mfn_t mfn) if ( !v ) return mfn_to_virt(mfn_x(mfn)); =20 - dcache =3D &v->domain->arch.pv.mapcache; + cache =3D vcpu_mapcache(v, &dcache); vcache =3D &v->arch.pv.mapcache; - if ( !dcache->inuse ) + if ( !cache->inuse ) return mfn_to_virt(mfn_x(mfn)); =20 perfc_incr(map_domain_page_count); @@ -90,41 +139,30 @@ void *map_domain_page(mfn_t mfn) if ( hashent->mfn =3D=3D mfn_x(mfn) ) { idx =3D hashent->idx; - ASSERT(idx < dcache->entries); + ASSERT(idx < cache->entries); hashent->refcnt++; ASSERT(hashent->refcnt); ASSERT(mfn_eq(l1e_get_mfn(MAPCACHE_L1ENT(idx)), mfn)); goto out; } =20 - spin_lock(&dcache->lock); + mapcache_enter(dcache, vcache); =20 - /* Has some other CPU caused a wrap? We must flush if so. */ - if ( unlikely(dcache->epoch !=3D vcache->shadow_epoch) ) - { - vcache->shadow_epoch =3D dcache->epoch; - if ( NEED_FLUSH(this_cpu(tlbflush_time), dcache->tlbflush_timestam= p) ) - { - perfc_incr(domain_page_tlb_flush); - flush_tlb_local(); - } - } - - idx =3D find_next_zero_bit(dcache->inuse, dcache->entries, dcache->cur= sor); - if ( unlikely(idx >=3D dcache->entries) ) + idx =3D find_next_zero_bit(cache->inuse, cache->entries, cache->cursor= ); + if ( unlikely(idx >=3D cache->entries) ) { unsigned long accum =3D 0, prev =3D 0; =20 /* /First/, clean the garbage map and update the inuse list. */ - for ( i =3D 0; i < BITS_TO_LONGS(dcache->entries); i++ ) + for ( i =3D 0; i < BITS_TO_LONGS(cache->entries); i++ ) { accum |=3D prev; - dcache->inuse[i] &=3D ~xchg(&dcache->garbage[i], 0); - prev =3D ~dcache->inuse[i]; + cache->inuse[i] &=3D ~xchg(&cache->garbage[i], 0); + prev =3D ~cache->inuse[i]; } =20 - if ( accum | (prev & BITMAP_LAST_WORD_MASK(dcache->entries)) ) - idx =3D find_first_zero_bit(dcache->inuse, dcache->entries); + if ( accum | (prev & BITMAP_LAST_WORD_MASK(cache->entries)) ) + idx =3D find_first_zero_bit(cache->inuse, cache->entries); else { /* Replace a hash entry instead. */ @@ -144,19 +182,18 @@ void *map_domain_page(mfn_t mfn) i =3D 0; } while ( i !=3D MAPHASH_HASHFN(mfn_x(mfn)) ); } - BUG_ON(idx >=3D dcache->entries); + BUG_ON(idx >=3D cache->entries); =20 /* /Second/, flush TLBs. */ perfc_incr(domain_page_tlb_flush); flush_tlb_local(); - vcache->shadow_epoch =3D ++dcache->epoch; - dcache->tlbflush_timestamp =3D tlbflush_current_time(); + mapcache_new_epoch(dcache, vcache); } =20 - set_bit(idx, dcache->inuse); - dcache->cursor =3D idx + 1; + set_bit(idx, cache->inuse); + cache->cursor =3D idx + 1; =20 - spin_unlock(&dcache->lock); + mapcache_exit(dcache); =20 l1e_write(&MAPCACHE_L1ENT(idx), l1e_from_mfn(mfn, __PAGE_HYPERVISOR_RW= )); =20 @@ -170,6 +207,7 @@ void unmap_domain_page(const void *ptr) unsigned int idx; struct vcpu *v; struct mapcache_domain *dcache; + struct mapcache *cache; unsigned long va =3D (unsigned long)ptr, mfn, flags; struct vcpu_maphash_entry *hashent; =20 @@ -181,8 +219,8 @@ void unmap_domain_page(const void *ptr) v =3D mapcache_current_vcpu(); ASSERT(v); =20 - dcache =3D &v->domain->arch.pv.mapcache; - ASSERT(dcache->inuse); + cache =3D vcpu_mapcache(v, &dcache); + ASSERT(cache->inuse); =20 idx =3D PFN_DOWN(va - MAPCACHE_VIRT_START); mfn =3D l1e_get_pfn(MAPCACHE_L1ENT(idx)); @@ -205,7 +243,7 @@ void unmap_domain_page(const void *ptr) hashent->mfn); l1e_write(&MAPCACHE_L1ENT(hashent->idx), l1e_empty()); /* /Second/, mark as garbage. */ - set_bit(hashent->idx, dcache->garbage); + set_bit(hashent->idx, cache->garbage); } =20 /* Add newly-freed mapping to the maphash. */ @@ -217,7 +255,7 @@ void unmap_domain_page(const void *ptr) /* /First/, zap the PTE. */ l1e_write(&MAPCACHE_L1ENT(idx), l1e_empty()); /* /Second/, mark as garbage. */ - set_bit(idx, dcache->garbage); + set_bit(idx, cache->garbage); } =20 local_irq_restore(flags); @@ -239,9 +277,9 @@ void mapcache_domain_init(struct domain *d) 2 * PFN_UP(BITS_TO_LONGS(MAPCACHE_ENTRIES) * sizeof(long)= )) > MAPCACHE_VIRT_START + (PERDOMAIN_SLOT_MBYTES << 20)); bitmap_pages =3D PFN_UP(BITS_TO_LONGS(MAPCACHE_ENTRIES) * sizeof(long)= ); - dcache->inuse =3D (void *)MAPCACHE_VIRT_END + PAGE_SIZE; - dcache->garbage =3D dcache->inuse + - (bitmap_pages + 1) * PAGE_SIZE / sizeof(long); + dcache->cache.inuse =3D (void *)MAPCACHE_VIRT_END + PAGE_SIZE; + dcache->cache.garbage =3D dcache->cache.inuse + + (bitmap_pages + 1) * PAGE_SIZE / sizeof(long); =20 spin_lock_init(&dcache->lock); } @@ -249,15 +287,20 @@ void mapcache_domain_init(struct domain *d) int mapcache_vcpu_init(struct vcpu *v) { struct domain *d =3D v->domain; - struct mapcache_domain *dcache =3D &d->arch.pv.mapcache; + struct mapcache_domain *dcache; + struct mapcache *cache; unsigned long i; unsigned int ents =3D d->max_vcpus * MAPCACHE_VCPU_ENTRIES; unsigned int nr =3D PFN_UP(BITS_TO_LONGS(ents) * sizeof(long)); =20 - if ( !is_pv_vcpu(v) || !dcache->inuse ) + if ( !has_mapcache(v) ) + return 0; + + cache =3D vcpu_mapcache(v, &dcache); + if ( !cache->inuse ) return 0; =20 - if ( ents > dcache->entries ) + if ( ents > cache->entries ) { /* Populate page tables. */ int rc =3D create_perdomain_mapping(v, MAPCACHE_VIRT_START, ents, @@ -265,16 +308,16 @@ int mapcache_vcpu_init(struct vcpu *v) =20 /* Populate bit maps. */ if ( !rc ) - rc =3D create_perdomain_mapping(v, (unsigned long)dcache->inus= e, + rc =3D create_perdomain_mapping(v, (unsigned long)cache->inuse, nr, NULL, NIL(struct page_info *= )); if ( !rc ) - rc =3D create_perdomain_mapping(v, (unsigned long)dcache->garb= age, + rc =3D create_perdomain_mapping(v, (unsigned long)cache->garba= ge, nr, NULL, NIL(struct page_info *= )); =20 if ( rc ) return rc; =20 - dcache->entries =3D ents; + cache->entries =3D ents; } =20 /* Mark all maphash entries as not in use. */ diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 4140a92f15..542c6b5eca 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -57,6 +57,16 @@ struct trap_bounce { unsigned long eip; }; =20 +struct mapcache { + /* The number of array entries, and a cursor into the array. */ + unsigned int entries; + unsigned int cursor; + + /* Which mappings are in use, and which are garbage to reap next epoch= ? */ + unsigned long *inuse; + unsigned long *garbage; +}; + #define MAPHASH_ENTRIES 8 #define MAPHASH_HASHFN(pfn) ((pfn) & (MAPHASH_ENTRIES-1)) #define MAPHASHENT_NOTINUSE ((u32)~0U) @@ -73,10 +83,6 @@ struct mapcache_vcpu { }; =20 struct mapcache_domain { - /* The number of array entries, and a cursor into the array. */ - unsigned int entries; - unsigned int cursor; - /* Protects map_domain_page(). */ spinlock_t lock; =20 @@ -84,9 +90,8 @@ struct mapcache_domain { unsigned int epoch; u32 tlbflush_timestamp; =20 - /* Which mappings are in use, and which are garbage to reap next epoch= ? */ - unsigned long *inuse; - unsigned long *garbage; + /* Accounting of the domain-wide mapcache. */ + struct mapcache cache; }; =20 void mapcache_domain_init(struct domain *d); --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748684; cv=none; d=zohomail.com; s=zohoarc; b=RgSoFZy6QgRvPGRqjDh+th2xEgccc7sYDL67co+hyPyY+Zd3Po87FOymT6tiKnmwf/7nKyW3bWyzWfUORDcsZ6QprB94k3x4dS3+O+p1lQKhAfYVqtLWIhQNnByC9+6iLPxH312xxuPjmVqEQreKhB/sRYPJvIX+gGc1r0i3oTs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748684; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6RX48R0QNYubdYR5xyvgMIQXSDrHO/YymvMThV7Bj0U=; b=Vqj/ilTsAKLzq9JGHQzJ71Ug/F3ZhLHJVUVjaZ+WEgxxDpNjQXInoV3Vs6yC7iEZURw7paG5ng/78Xk98Ovo7VThczucwhG7sZX5RKYBOXgEqrYZzr0rwjKNStgEP8k3pq2i7HCo9nUpv+eaMOpWsOZDYDOxwaPLKp5xiz5tSP8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 179374868419237.75758227896972; Tue, 3 Nov 2026 15:31:24 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470806.1679547 (Exim 4.92) (envelope-from ) id 1xONxv-0004Eo-D6; Tue, 03 Nov 2026 23:30:55 +0000 Received: by outflank-mailman (output) from mailman id 1470806.1679547; Tue, 03 Nov 2026 23:30:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxt-000480-V9; Tue, 03 Nov 2026 23:30:53 +0000 Received: by outflank-mailman (input) for mailman id 1470806; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000di-EP for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CovY-1XYK; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnr-0000000EMFa-1M9j; Tue, 03 Nov 2026 23:20:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=6RX48R0QNYubdYR5xyvgMIQXSDrHO/YymvMThV7Bj0U=; b=Gc+TtgwSAdtYCMBpSUQ553r+Rb 3cx8waJiOAJJuN+3+TQ55waQlxg1Wo5ffa0U9BRUzY1UjcaY5wbfdzPu7tSJl5PpXwQT5zuvK/ha4 DvPVOMqOUrkAbyeZJF5sUoU6yhl8w1/rZFbWp1UIEq3+I4oGMAlMt+vXqJ1xP4Nee660=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , Teddy Astie Subject: [PATCH v6 11/44] x86/domain_page: move struct mapcache_vcpu to struct arch_vcpu Date: Tue, 3 Nov 2026 17:13:33 -0500 Message-ID: <20261103-asi-hvm-v6-11-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748685606158500 From: Roger Pau Monn=C3=A9 struct mapcache_vcpu, the per-vCPU part of the mapcache (the maphash and the vCPU's shadow of the domain's TLB epoch), lives in struct pv_vcpu, as only PV vCPUs have a mapcache. In preparation for implementing mapcaches for HVM domains, move it to struct arch_vcpu: v->arch.pv.mapcache becomes v->arch.mapcache. struct pv_vcpu shrinks from 512 to 376 bytes, and struct vcpu grows from 2880 to 3008 bytes (x86_64, debug build), within its page: from here on HVM vCPUs carry the 136-byte struct mapcache_vcpu too, ahead of its use by those of vCPU-PT domains. No functional change. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap Reviewed-by: Roger Pau Monn=C3=A9 --- Changes in v3: - New in this version, split out of "x86/hvm: introduce per-vCPU mapcache for vCPU-PT domains" for review. --- xen/arch/x86/domain_page.c | 6 +++--- xen/arch/x86/include/asm/domain.h | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index e4ea7a254f..205d7b832a 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -127,7 +127,7 @@ void *map_domain_page(mfn_t mfn) return mfn_to_virt(mfn_x(mfn)); =20 cache =3D vcpu_mapcache(v, &dcache); - vcache =3D &v->arch.pv.mapcache; + vcache =3D &v->arch.mapcache; if ( !cache->inuse ) return mfn_to_virt(mfn_x(mfn)); =20 @@ -224,7 +224,7 @@ void unmap_domain_page(const void *ptr) =20 idx =3D PFN_DOWN(va - MAPCACHE_VIRT_START); mfn =3D l1e_get_pfn(MAPCACHE_L1ENT(idx)); - hashent =3D &v->arch.pv.mapcache.hash[MAPHASH_HASHFN(mfn)]; + hashent =3D &v->arch.mapcache.hash[MAPHASH_HASHFN(mfn)]; =20 local_irq_save(flags); =20 @@ -324,7 +324,7 @@ int mapcache_vcpu_init(struct vcpu *v) BUILD_BUG_ON(MAPHASHENT_NOTINUSE < MAPCACHE_ENTRIES); for ( i =3D 0; i < MAPHASH_ENTRIES; i++ ) { - struct vcpu_maphash_entry *hashent =3D &v->arch.pv.mapcache.hash[i= ]; + struct vcpu_maphash_entry *hashent =3D &v->arch.mapcache.hash[i]; =20 hashent->mfn =3D ~0UL; /* never valid to map */ hashent->idx =3D MAPHASHENT_NOTINUSE; diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 542c6b5eca..45d308407e 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -545,9 +545,6 @@ struct arch_domain =20 struct pv_vcpu { - /* map_domain_page() mapping cache. */ - struct mapcache_vcpu mapcache; - unsigned int vgc_flags; =20 struct trap_info *trap_ctxt; @@ -643,6 +640,9 @@ struct arch_vcpu #define async_exception_state(t) async_exception_state[(t)-1] uint8_t async_exception_mask; =20 + /* map_domain_page() mapping cache. */ + struct mapcache_vcpu mapcache; + /* Virtual Machine Extensions */ union { struct pv_vcpu pv; --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748662; cv=none; d=zohomail.com; s=zohoarc; b=K3739c9SzppdQHZOxppy8KlPO3yfdLpVzhVOXNDM868SjegrWK1snx5CfZx/h6M00FeZTarxNje3mFd7Lf7Tb30uj17RjEDLS6LbKfJ9Hx736Gj5hI74roPEpyzvRxcL3hDp/aqcnQzK8dxMgOWPi0y0p7LZpyGx9RlF1XolW40= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748662; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=KFvkL5WqfDQDRrp7P3qFC/Nmk3HS+ngDzI09Wi+qTuQ=; b=anosiG8Z1Iej1FwPa+OfWpZxNwNe0EFzSqbbMl6vFbKX841U1pBzaJSxhEY4d6FztacTphVe/CLACFoGuTNl7qplZ9eVpTPpZmhThCFBv5UgiDEGvOp8TAVXjvolqmPuI1BTiTQwmUR/fovwLU85DJetm1WpbTaCO1TW+N5bI4A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 179374866220975.09607243200594; Tue, 3 Nov 2026 15:31:02 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470802.1679507 (Exim 4.92) (envelope-from ) id 1xONxm-0002nA-Oh; Tue, 03 Nov 2026 23:30:46 +0000 Received: by outflank-mailman (output) from mailman id 1470802.1679507; Tue, 03 Nov 2026 23:30:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxl-0002ip-Ts; Tue, 03 Nov 2026 23:30:45 +0000 Received: by outflank-mailman (input) for mailman id 1470802; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000dM-AT for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Cov2-143P; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnr-0000000EMFa-3BRw; Tue, 03 Nov 2026 23:20:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=KFvkL5WqfDQDRrp7P3qFC/Nmk3HS+ngDzI09Wi+qTuQ=; b=B/0h5gJI7p7Eo47YEgnum4NZOn wacxYk7wL7yFZOaJwWu7ibel1X8VU3imtiTvTeeWuu9UgltEJwqZ3VT6JtB1uv6ZEpr/zhUUvPaoM RlS2xBxF7T3c/MnNI9fbVXnVj4yStfnvq0GDHntJ69gowwN0jcrSEOX1yFjBoaKVMV+s=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , Teddy Astie Subject: [PATCH v6 12/44] x86/domain_page: don't unmap a mapcache entry that is not present Date: Tue, 3 Nov 2026 17:13:34 -0500 Message-ID: <20261103-asi-hvm-v6-12-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748663729158500 Mappings must be torn down by the vCPU that created them, and only once. Have unmap_domain_page() check that the entry it is handed is present and bail out otherwise (ASSERT_UNREACHABLE()), rather than read an MFN out of an empty PTE and act on it, as a second unmap of an entry the first one zapped would. (A second unmap of an entry the maphash kept is caught, as before, only by the refcount ASSERT() in debug builds.) Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap Reviewed-by: Roger Pau Monn=C3=A9 --- Changes in v4: - Reword the comment on the new check (Roger). Changes in v3: - New in this version, split out of "x86/hvm: introduce per-vCPU mapcache for vCPU-PT domains" for review. --- xen/arch/x86/domain_page.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 205d7b832a..47ccefa4e9 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -223,6 +223,15 @@ void unmap_domain_page(const void *ptr) ASSERT(cache->inuse); =20 idx =3D PFN_DOWN(va - MAPCACHE_VIRT_START); + /* + * Sanity check: an entry that isn't present means a double or stray + * unmap. Play it safe: don't act on the MFN in an empty PTE. + */ + if ( !(l1e_get_flags(MAPCACHE_L1ENT(idx)) & _PAGE_PRESENT) ) + { + ASSERT_UNREACHABLE(); + return; + } mfn =3D l1e_get_pfn(MAPCACHE_L1ENT(idx)); hashent =3D &v->arch.mapcache.hash[MAPHASH_HASHFN(mfn)]; =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748700; cv=none; d=zohomail.com; s=zohoarc; b=NcqXcj5p+VjbiVw2vNk+BjpGyDPS3+pmfhgDx3ARCT/cAIFz2Bn+DFOdMressNBfYbjkQdkZXGtRhef0q2h064t5O7icFBRNQ6AP6Q1Ql+n13sg+Bj+3cTr4Nn7GduHBBYy+ApVC4Hm2bsJ0s2+9K0nFmOCHhlAQ74Uw2Pka8Uw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748700; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=q61QcfEPbpij49BHXIR9hDLEn3Fryo2EnMJYGWj3XbE=; b=ew28fMn0YAwnP1g1efCw3NL/Hex23JisaqlUWm2EBhjDji7HMZnwat/vPpjyz2nkmU8nQ2K3ULRd1c0z9GzrrWbF2zfvD4A8jwFrT+w9UNd2Kg9nJcKq1Ic13CpRJWkedgpVCqxSTNS1q1soPe4Wwa4GFr33hjmR1dGLwM1gLsY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 179374870017536.25796762122218; Tue, 3 Nov 2026 15:31:40 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470820.1679663 (Exim 4.92) (envelope-from ) id 1xONyN-0001pX-SX; Tue, 03 Nov 2026 23:31:23 +0000 Received: by outflank-mailman (output) from mailman id 1470820.1679663; Tue, 03 Nov 2026 23:31:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyL-0001gN-UJ; Tue, 03 Nov 2026 23:31:21 +0000 Received: by outflank-mailman (input) for mailman id 1470820; Tue, 03 Nov 2026 23:30:41 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxh-0001dv-Az for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:41 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CovE-1Agc; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONns-0000000EMFa-0rTu; Tue, 03 Nov 2026 23:20:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=q61QcfEPbpij49BHXIR9hDLEn3Fryo2EnMJYGWj3XbE=; b=bqANd3lCanoJt+icnl190R5Cnl jOpFZvf6OXXpZhi8rL7sialzz1CAMtRNA4hhjW/dyor+EF8MNzteCFYkQnz2cq+dFpAII85leMYTw /yJHKWtM6kA0u0BJElFEHPRQhghvm9CbN3diYI91z50KU8gphUKYAnc5vZLCyEZNwLRM=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 13/44] x86/hvm: introduce per-vCPU mapcache for vCPU-PT domains Date: Tue, 3 Nov 2026 17:13:35 -0500 Message-ID: <20261103-asi-hvm-v6-13-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748701727158500 From: Roger Pau Monn=C3=A9 Add a struct mapcache to struct mapcache_vcpu for the per-vCPU accounting, with MAPCACHE_PERVCPU_ENTRIES (64) slots. Its inuse and garbage bitmaps are then an unsigned long each: declare them inline in struct mapcache_vcpu, rather than mapping them in the per-domain area as the domain-wide bitmaps are. A per-vCPU mapcache is only ever used by its own vCPU, and so by one pCPU at a time. We can therefore skip both the lock and the TLB-epoch tracking the domain-wide mapcache needs; a wrap still flushes the local TLB. struct mapcache_vcpu grows from 136 to 176 bytes, and struct vcpu from 3008 to 3072 bytes (x86_64, debug build), within its page. Unlike a domain-wide mapcache, whose one pool all the domain's vCPUs draw on, a per-vCPU one cannot, by design, use more than its own slots: MAPCACHE_PERVCPU_ENTRIES (64) is the most single-page mappings such a vCPU can hold at once, interrupt handlers' included, and one more hits the BUG_ON() in map_domain_page(). The PoD zero-page sweep held up to 16 on top of its callers' until 41ddcfe808ec ("x86/PoD: map one page at a time in p2m_pod_zero_check()"). A domain-wide mapcache is sized at MAPCACHE_VCPU_ENTRIES (16) slots per vCPU, in one shared pool; a per-vCPU one gets 64, for a wider margin. The extra slots take no memory: the area is a whole L1 page whatever the number of slots, and each bitmap is still one unsigned long. Wraps, each of which flushes the local TLB, also come less often. mapcache_current_vcpu()'s lazy-context-switch handling (sync_local_execstate() plus a pgtable_vcpu re-read) now applies to any vCPU with a mapcache: when the idle vCPU runs lazily on an HVM vCPU's monitor table, the full switch happens before the idle context uses a mapcache. In release builds map_domain_page() still tries the directmap first, for any vCPU; debug builds exercise the new mapcache. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v6: - Message: "x86/PoD: map one page at a time in p2m_pod_zero_check()" is in staging now, so cite it by its hash. Changes in v5: - Message: a domain-wide mapcache is one pool that all the domain's vCPUs draw on, not per-vCPU slots another vCPU could borrow. Changes in v4: - d->arch.vcpu_pt is now d->arch.asi.vcpu_pt. - mapcache_lock() / mapcache_unlock() are now mapcache_enter() / mapcache_exit(). has_mapcache()'s comment no longer changes, and the early return in mapcache_enter() has a one-line comment instead of a sentence added to the block (Roger). - mapcache_vcpu_init(): ents and nr become assignments here, ents now depending on the kind of mapcache (Jan). - A per-vCPU mapcache gets 64 slots (MAPCACHE_PERVCPU_ENTRIES) in place of MAPCACHE_VCPU_ENTRIES (16), which still sizes the domain-wide mapcache; its bitmaps, now full, are described as an unsigned long each (Roger). - Move the comment on why a per-vCPU mapcache needs only a local flush from map_domain_page()'s TLB flush to mapcache_new_epoch()'s early return, as one line (Roger). Changes in v3: - HVM only, and per-vCPU from the start: the HVM enablement of "x86: Initialize the mapcache for PV, HVM, and idle domains" (from the directmap removal series) is folded in, without the idle domain and without a domain-wide HVM mapcache. - Retitle (was: "x86/mm: introduce a per-vCPU mapcache when using ASI"). - Say which second unmaps the not-present check in unmap_domain_page() catches: those of an entry the first unmap zapped. - State the budget of single-page mappings a vCPU can hold at once. - The mapcache accounting refactor, the move of struct mapcache_vcpu to struct arch_vcpu and the not-present check in unmap_domain_page() are split out, for review, into the three patches before this one. --- xen/arch/x86/domain_page.c | 61 +++++++++++++++++++++++-------- xen/arch/x86/include/asm/config.h | 2 + xen/arch/x86/include/asm/domain.h | 6 +++ 3 files changed, 53 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 47ccefa4e9..770fee80a4 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -21,7 +21,7 @@ /* Whether the vCPU supports mapcache usage, might not be in use. */ static bool has_mapcache(const struct vcpu *v) { - return is_pv_vcpu(v); + return is_pv_vcpu(v) || v->domain->arch.asi.vcpu_pt; } =20 static inline struct vcpu *mapcache_current_vcpu(void) @@ -34,10 +34,10 @@ static inline struct vcpu *mapcache_current_vcpu(void) return NULL; =20 /* - * If we are in a lazy context-switch state from a PV vCPU do a full s= witch - * to the idle vCPU now, otherwise an incoming FLUSH_VCPU_STATE IPI wo= uld - * change the page tables under our feet an invalidate any in-use mapc= ache - * entries. + * If we are in a lazy context-switch state from a vCPU with a mapcach= e do + * a full switch to the idle vCPU now, otherwise an incoming + * FLUSH_VCPU_STATE IPI would change the page tables under our feet and + * invalidate any in-use mapcache entries. */ if ( unlikely(this_cpu(curr_vcpu) !=3D curr) ) { @@ -57,14 +57,21 @@ static inline struct vcpu *mapcache_current_vcpu(void) } =20 /* - * The accounting for v's mapcache: the domain's, with *dcache set so the - * caller takes the domain-wide lock and TLB-epoch machinery into account. + * The accounting for v's mapcache: its own for a vCPU-PT domain, the doma= in's + * otherwise, in which case *dcache is set so the caller takes the domain-= wide + * lock and TLB-epoch machinery into account. */ static struct mapcache *vcpu_mapcache(struct vcpu *v, struct mapcache_domain **dcache) { struct domain *d =3D v->domain; =20 + if ( d->arch.asi.vcpu_pt ) + { + *dcache =3D NULL; + return &v->arch.mapcache.cache; + } + *dcache =3D &d->arch.pv.mapcache; return &(*dcache)->cache; } @@ -72,6 +79,10 @@ static struct mapcache *vcpu_mapcache(struct vcpu *v, static void mapcache_enter(struct mapcache_domain *dcache, struct mapcache_vcpu *vcache) { + /* Only its own vCPU uses a per-vCPU mapcache: no lock or epoch needed= . */ + if ( !dcache ) + return; + /* The domain's vCPUs share the mapcache's slots. */ spin_lock(&dcache->lock); =20 @@ -91,6 +102,10 @@ static void mapcache_enter(struct mapcache_domain *dcac= he, static void mapcache_new_epoch(struct mapcache_domain *dcache, struct mapcache_vcpu *vcache) { + /* Per-vCPU: no epoch, as page-table loads flush its non-global entrie= s. */ + if ( !dcache ) + return; + ASSERT(spin_is_locked(&dcache->lock)); =20 vcache->shadow_epoch =3D ++dcache->epoch; @@ -99,7 +114,8 @@ static void mapcache_new_epoch(struct mapcache_domain *d= cache, =20 static void mapcache_exit(struct mapcache_domain *dcache) { - spin_unlock(&dcache->lock); + if ( dcache ) + spin_unlock(&dcache->lock); } =20 #define mapcache_l2_entry(e) ((e) >> PAGETABLE_ORDER) @@ -299,15 +315,25 @@ int mapcache_vcpu_init(struct vcpu *v) struct mapcache_domain *dcache; struct mapcache *cache; unsigned long i; - unsigned int ents =3D d->max_vcpus * MAPCACHE_VCPU_ENTRIES; - unsigned int nr =3D PFN_UP(BITS_TO_LONGS(ents) * sizeof(long)); + unsigned int ents, nr; =20 if ( !has_mapcache(v) ) return 0; =20 cache =3D vcpu_mapcache(v, &dcache); - if ( !cache->inuse ) + if ( !dcache ) + { + /* Per-vCPU bitmaps are an unsigned long each, in the vCPU itself.= */ + cache->inuse =3D v->arch.mapcache.inuse; + cache->garbage =3D v->arch.mapcache.garbage; + ents =3D MAPCACHE_PERVCPU_ENTRIES; + } + else if ( !cache->inuse ) return 0; + else + ents =3D d->max_vcpus * MAPCACHE_VCPU_ENTRIES; + + nr =3D PFN_UP(BITS_TO_LONGS(ents) * sizeof(long)); =20 if ( ents > cache->entries ) { @@ -315,13 +341,16 @@ int mapcache_vcpu_init(struct vcpu *v) int rc =3D create_perdomain_mapping(v, MAPCACHE_VIRT_START, ents, NIL(l1_pgentry_t *), NULL); =20 - /* Populate bit maps. */ - if ( !rc ) + /* Populate the domain-wide bit maps (kept in the per-domain area)= . */ + if ( !rc && dcache ) + { rc =3D create_perdomain_mapping(v, (unsigned long)cache->inuse, nr, NULL, NIL(struct page_info *= )); - if ( !rc ) - rc =3D create_perdomain_mapping(v, (unsigned long)cache->garba= ge, - nr, NULL, NIL(struct page_info *= )); + if ( !rc ) + rc =3D create_perdomain_mapping(v, (unsigned long)cache->g= arbage, + nr, NULL, + NIL(struct page_info *)); + } =20 if ( rc ) return rc; diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/c= onfig.h index aebd3e8f45..ee84c57432 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -279,6 +279,8 @@ extern unsigned long xen_phys_start; =20 /* map_domain_page() map cache. The second per-domain-mapping sub-area. */ #define MAPCACHE_VCPU_ENTRIES (CONFIG_PAGING_LEVELS * CONFIG_PAGING_LEV= ELS) +/* Single-page slots of a per-vCPU mapcache (vCPU-PT domains). */ +#define MAPCACHE_PERVCPU_ENTRIES 64 #define MAPCACHE_ENTRIES (MAX_VIRT_CPUS * MAPCACHE_VCPU_ENTRIES) #define MAPCACHE_VIRT_START PERDOMAIN_VIRT_SLOT(1) #define MAPCACHE_VIRT_END (MAPCACHE_VIRT_START + \ diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 45d308407e..7076e38848 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -80,6 +80,12 @@ struct mapcache_vcpu { uint32_t idx; uint32_t refcnt; } hash[MAPHASH_ENTRIES]; + + /* Accounting of a per-vCPU mapcache (vCPU-PT domains). */ + struct mapcache cache; + /* Backing for cache.{inuse,garbage}. */ + DECLARE_BITMAP(inuse, MAPCACHE_PERVCPU_ENTRIES); + DECLARE_BITMAP(garbage, MAPCACHE_PERVCPU_ENTRIES); }; =20 struct mapcache_domain { --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748659; cv=none; d=zohomail.com; s=zohoarc; b=KPAzJLa7Eyqf69nfRZBFNQzVstOyoE0TadpBFLyGqPtNYyBbaj1vANxeeUMqvVWqa/8tO+jxVeUhnyoQd8V9LY6PNEY08/zUu1ByJBk2J8Bg8O1/e0CSZ6yq2R0+FsJjWf6uHNnvmj/c+wiAQQ17AzSfAiINgzfWlhKWlyEhqeo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748659; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=HsBQFYbAaJl6u0wptC/IWMz5BiLtWpaI/vOuJRWYogs=; b=Rik1NARO6N18A4Qv6UbgoVc0LmbrWF/GV4uknx9ogDnu+pYWW/nrfPv9NzW02JhEFi6Bo4BtT5+Fjmx0SL0EcsEpKv+3bCMv9eF46nY9YBTQxUhsEvNlSzDCkEyjZsMTZUzW/tjoyDjKu8XJsEIwcVVNFeHko4qFmocg5+7GZqQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748659093837.6027254669; Tue, 3 Nov 2026 15:30:59 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470800.1679491 (Exim 4.92) (envelope-from ) id 1xONxk-0002KR-Ly; Tue, 03 Nov 2026 23:30:44 +0000 Received: by outflank-mailman (output) from mailman id 1470800.1679491; Tue, 03 Nov 2026 23:30:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxk-0002GV-8s; Tue, 03 Nov 2026 23:30:44 +0000 Received: by outflank-mailman (input) for mailman id 1470800; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000d9-7L for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Coud-0N9q; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONns-0000000EMFa-2cO9; Tue, 03 Nov 2026 23:20:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=HsBQFYbAaJl6u0wptC/IWMz5BiLtWpaI/vOuJRWYogs=; b=CmNnCx1ZNqgpbIWYzRjf6vR++Y Zv1khxuautkhQOXO+CfVKSYZioovyhqp19GnAwzWeWWp6i2EQEvnaXxrRdEKCOYUAyfTalyiN2Dv+ 9Qj6+wMrcgIUg+lIyr0j+oZfkoJQ+kfzCWkpm//nQ4kqXMV9ctC6vqVUoEUGj9JdsT0w=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 14/44] x86/mm: let Xen's page-table walker operate on a given root Date: Tue, 3 Nov 2026 17:13:36 -0500 Message-ID: <20261103-asi-hvm-v6-14-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748659618158500 Content-Type: text/plain; charset="utf-8" map_pages_to_xen() and modify_xen_mappings() maintain Xen's own mappings, walking from idle_pg_table, whose Xen slots every other root page-table copies. A later patch introduces a second, sparse directmap hierarchy: page tables for the directmap virtual address range that some guest contexts use in place of idle_pg_table's. It needs the same maintenance at the same virtual addresses. Give virt_to_xen_l{3,2,1}e() the L4 to walk from, and move the bodies of map_pages_to_xen() and modify_xen_mappings() into map_pages_in() and modify_mappings_in(), which take the root as well. The public functions pass idle_pg_table. The EFI runtime-services page-tables mirror idle_pg_table's L4 entries only, so a newly allocated L3 is propagated to them only when the root is idle_pg_table. map_pages_in() and modify_mappings_in() free page-table pages through free_pagetable_in(), which takes the root too and for now only calls free_xen_pagetable(). TLB flushing is unaffected: flushes are by virtual address and go to every online CPU, which covers both hierarchies. No functional change. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - Free page-table pages through free_pagetable_in(), for now a plain wrapper, split out of "x86/mm: build and maintain a sparse view of the directmap". Changes in v4: - Message: name the second hierarchy as sparse; flushes reach every CPU (Roger). Changes in v3: - New in this version. --- xen/arch/x86/mm.c | 78 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 56 insertions(+), 22 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index f52c72a605..39c6fa3d56 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5247,13 +5247,14 @@ static DEFINE_SPINLOCK(map_pgdir_lock); =20 /* * For virt_to_xen_lXe() functions, they take a linear address and return a - * pointer to Xen's LX entry. Caller needs to unmap the pointer. + * pointer to Xen's LX entry in the page-table hierarchy rooted at @root + * (idle_pg_table for Xen's own mappings). Caller needs to unmap the poin= ter. */ -static l3_pgentry_t *virt_to_xen_l3e(unsigned long v) +static l3_pgentry_t *virt_to_xen_l3e(l4_pgentry_t *root, unsigned long v) { l4_pgentry_t *pl4e; =20 - pl4e =3D &idle_pg_table[l4_table_offset(v)]; + pl4e =3D &root[l4_table_offset(v)]; if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) ) { bool locking =3D system_state > SYS_STATE_boot; @@ -5269,7 +5270,8 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v) l4_pgentry_t l4e =3D l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR); =20 l4e_write(pl4e, l4e); - efi_update_l4_pgtable(l4_table_offset(v), l4e); + if ( root =3D=3D idle_pg_table ) + efi_update_l4_pgtable(l4_table_offset(v), l4e); l3mfn =3D INVALID_MFN; } if ( locking ) @@ -5280,11 +5282,11 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long = v) return map_l3t_from_l4e(*pl4e) + l3_table_offset(v); } =20 -static l2_pgentry_t *virt_to_xen_l2e(unsigned long v) +static l2_pgentry_t *virt_to_xen_l2e(l4_pgentry_t *root, unsigned long v) { l3_pgentry_t *pl3e, l3e; =20 - pl3e =3D virt_to_xen_l3e(v); + pl3e =3D virt_to_xen_l3e(root, v); if ( !pl3e ) return NULL; =20 @@ -5318,11 +5320,11 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long = v) return map_l2t_from_l3e(l3e) + l2_table_offset(v); } =20 -static l1_pgentry_t *virt_to_xen_l1e(unsigned long v) +static l1_pgentry_t *virt_to_xen_l1e(l4_pgentry_t *root, unsigned long v) { l2_pgentry_t *pl2e, l2e; =20 - pl2e =3D virt_to_xen_l2e(v); + pl2e =3D virt_to_xen_l2e(root, v); if ( !pl2e ) return NULL; =20 @@ -5398,7 +5400,7 @@ mfn_t xen_map_to_mfn(unsigned long va) bool locking =3D system_state > SYS_STATE_boot; unsigned int l2_offset =3D l2_table_offset(va); unsigned int l1_offset =3D l1_table_offset(va); - const l3_pgentry_t *pl3e =3D virt_to_xen_l3e(va); + const l3_pgentry_t *pl3e =3D virt_to_xen_l3e(idle_pg_table, va); const l2_pgentry_t *pl2e =3D NULL; const l1_pgentry_t *pl1e =3D NULL; struct page_info *l3page; @@ -5438,7 +5440,21 @@ mfn_t xen_map_to_mfn(unsigned long va) return ret; } =20 -int map_pages_to_xen( +/* + * Free a page-table page that the hierarchy rooted at @root has stopped + * using. + */ +static void free_pagetable_in(const l4_pgentry_t *root, mfn_t mfn) +{ + free_xen_pagetable(mfn); +} + +/* + * map_pages_to_xen() in the hierarchy rooted at @root: idle_pg_table's, or + * another hierarchy sharing its virtual addresses. + */ +static int map_pages_in( + l4_pgentry_t *root, unsigned long virt, mfn_t mfn, unsigned long nr_mfns, @@ -5489,7 +5505,7 @@ int map_pages_to_xen( UNMAP_DOMAIN_PAGE(pl3e); UNMAP_DOMAIN_PAGE(pl2e); =20 - pl3e =3D virt_to_xen_l3e(virt); + pl3e =3D virt_to_xen_l3e(root, virt); if ( !pl3e ) goto out; =20 @@ -5542,10 +5558,10 @@ int map_pages_to_xen( ol2e =3D l2t[i]; if ( (l2e_get_flags(ol2e) & _PAGE_PRESENT) && !(l2e_get_flags(ol2e) & _PAGE_PSE) ) - free_xen_pagetable(l2e_get_mfn(ol2e)); + free_pagetable_in(root, l2e_get_mfn(ol2e)); } unmap_domain_page(l2t); - free_xen_pagetable(l3e_get_mfn(ol3e)); + free_pagetable_in(root, l3e_get_mfn(ol3e)); } } =20 @@ -5613,7 +5629,7 @@ int map_pages_to_xen( free_xen_pagetable(l2mfn); } =20 - pl2e =3D virt_to_xen_l2e(virt); + pl2e =3D virt_to_xen_l2e(root, virt); if ( !pl2e ) goto out; =20 @@ -5643,7 +5659,7 @@ int map_pages_to_xen( flush_flags(l1e_get_flags(l1t[i])); flush_area(virt, flush_flags); unmap_domain_page(l1t); - free_xen_pagetable(l2e_get_mfn(ol2e)); + free_pagetable_in(root, l2e_get_mfn(ol2e)); } } =20 @@ -5658,7 +5674,7 @@ int map_pages_to_xen( /* Normal page mapping. */ if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { - pl1e =3D virt_to_xen_l1e(virt); + pl1e =3D virt_to_xen_l1e(root, virt); if ( pl1e =3D=3D NULL ) goto out; } @@ -5775,7 +5791,7 @@ int map_pages_to_xen( flush_area(virt - PAGE_SIZE, FLUSH_TLB_GLOBAL | FLUSH_ORDER(PAGETABLE_ORDER)); - free_xen_pagetable(l2e_get_mfn(ol2e)); + free_pagetable_in(root, l2e_get_mfn(ol2e)); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -5822,7 +5838,7 @@ int map_pages_to_xen( flush_area(virt - PAGE_SIZE, FLUSH_TLB_GLOBAL | FLUSH_ORDER(2*PAGETABLE_ORDER)); - free_xen_pagetable(l3e_get_mfn(ol3e)); + free_pagetable_in(root, l3e_get_mfn(ol3e)); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -5843,6 +5859,15 @@ int map_pages_to_xen( return rc; } =20 +int map_pages_to_xen( + unsigned long virt, + mfn_t mfn, + unsigned long nr_mfns, + pte_attr_t flags) +{ + return map_pages_in(idle_pg_table, virt, mfn, nr_mfns, flags); +} + int __init populate_pt_range(unsigned long virt, unsigned long nr_mfns) { return map_pages_to_xen(virt, INVALID_MFN, nr_mfns, MAP_SMALL_PAGES); @@ -5859,8 +5884,12 @@ int __init populate_pt_range(unsigned long virt, uns= igned long nr_mfns) * flags are ignored. * * It is an error to call with present flags over an unpopulated range. + * + * modify_mappings_in() does so in the hierarchy rooted at @root, as + * map_pages_in() does. */ -int modify_xen_mappings(unsigned long s, unsigned long e, pte_attr_t nf) +static int modify_mappings_in(l4_pgentry_t *root, unsigned long s, + unsigned long e, pte_attr_t nf) { bool locking =3D system_state > SYS_STATE_boot; l3_pgentry_t *pl3e =3D NULL; @@ -5887,7 +5916,7 @@ int modify_xen_mappings(unsigned long s, unsigned lon= g e, pte_attr_t nf) UNMAP_DOMAIN_PAGE(pl2e); UNMAP_DOMAIN_PAGE(pl3e); =20 - pl3e =3D virt_to_xen_l3e(v); + pl3e =3D virt_to_xen_l3e(root, v); if ( !pl3e ) goto out; =20 @@ -6070,7 +6099,7 @@ int modify_xen_mappings(unsigned long s, unsigned lon= g e, pte_attr_t nf) if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(NULL, FLUSH_TLB_GLOBAL); /* flush before free */ - free_xen_pagetable(l1mfn); + free_pagetable_in(root, l1mfn); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -6114,7 +6143,7 @@ int modify_xen_mappings(unsigned long s, unsigned lon= g e, pte_attr_t nf) if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(NULL, FLUSH_TLB_GLOBAL); /* flush before free */ - free_xen_pagetable(l2mfn); + free_pagetable_in(root, l2mfn); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -6138,6 +6167,11 @@ int modify_xen_mappings(unsigned long s, unsigned lo= ng e, pte_attr_t nf) =20 #undef flush_area =20 +int modify_xen_mappings(unsigned long s, unsigned long e, pte_attr_t nf) +{ + return modify_mappings_in(idle_pg_table, s, e, nf); +} + int destroy_xen_mappings(unsigned long s, unsigned long e) { return modify_xen_mappings(s, e, _PAGE_NONE); --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748700; cv=none; d=zohomail.com; s=zohoarc; b=oHuRPTgJKG6lM3ADL8Gv4l4mMXB4MVrFjvDuVPPqmbH4Eo9l2iImZ4SzHOWwn9aDC7Auws0xOnXC4OaaQnSvr07MTWWyQkuhePApV+2KZvqWtzEmht5rv61i6BJbRCStoHmKRUquQ3TeYpYdJ5tnnK04SUYiXENivhjFMFYHDAY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748700; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Lb7F1xsyX+9AldokNzH9xX+s7w05jag/1Hr7PjUMcMQ=; b=LqtAqYpFNEaEAXm+5tGSMBEdX+SwTslTimbbk/VZtQAqVJubo+yL6SNnp535ypH5SKQHwe8qfHadq1liIHHvnZnUX+ZJvflYnsb/7bacaOgMH0ZNK959uoxa+x3uHd98Vx/y95fZU7D3xDLG8sAx3N/432XqV0PK7XG8EQq2liE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748699924270.87120172213963; Tue, 3 Nov 2026 15:31:39 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470815.1679617 (Exim 4.92) (envelope-from ) id 1xONyC-0007q6-45; Tue, 03 Nov 2026 23:31:12 +0000 Received: by outflank-mailman (output) from mailman id 1470815.1679617; Tue, 03 Nov 2026 23:31:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyA-0007gb-J5; Tue, 03 Nov 2026 23:31:10 +0000 Received: by outflank-mailman (input) for mailman id 1470815; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000eY-Oo for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cou6-48d4; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnt-0000000EMFa-0lsh; Tue, 03 Nov 2026 23:20:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=Lb7F1xsyX+9AldokNzH9xX+s7w05jag/1Hr7PjUMcMQ=; b=DhpHWtVtRhvNWPOJI1DNHhivft JKoOPLcH5d8/imTCWzWnOp3VmVXKUhGwesWGBXYypnOHy/NO0qVF0Uh+jelHhnTq9WA+cMzorot9h 8QzjWGsj859X4dEoxAmVerVS4I/GILqQh9dMgP/+WKl84HHIetsS/6+NotlxnnhCsHTw=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Stefano Stabellini Subject: [PATCH v6 15/44] xen/gunzip: process pending softirqs once per deflate block Date: Tue, 3 Nov 2026 17:13:37 -0500 Message-ID: <20261103-asi-hvm-v6-15-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748701665158500 Content-Type: text/plain; charset="utf-8" Nothing in the gzip decompressor lets the CPU run softirqs. The NMI watchdog treats a CPU as stuck once its timer softirq has not run for the watchdog's timeout, 5s by default, so a decompression that runs that long panics the host. Decompression is normally quick, but it need not be. Every compressed deflate block builds its Huffman tables with xmalloc() and frees them with xfree(), and those calls reach the xenheap whenever a request is a page or more, or a free empties an xmalloc pool page. A subsequent change can make every xenheap free end in a TLB flush on all online CPUs. With it in effect, decompressing the PV dom0 kernel and reading its ELF headers on a 480-CPU host took 5.2s instead of 0.5s; with cpufreq=3Dxen:performance max_cstate=3D1,0 dom0_nodes=3D0,relaxed added to the command line as well, the watchdog fired before decompression had finished, on each of eight boots. Process pending softirqs after each deflate block, as libelf does for each ELF note since c5089fae0f6f ("x86: avoid tripping watchdog when constructing dom0"), and as the PV and PVH dom0 builders do in their long loops. Every caller of perform_gunzip() runs during domain construction at boot, on the boot CPU with interrupts enabled and no lock held: x86's PV and PVH dom0 builders, which call process_pending_softirqs() themselves, and kernel_decompress(), which Arm uses for dom0 and for dom0less domains. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/common/gzip/gunzip.c | 1 + xen/common/gzip/inflate.c | 1 + 2 files changed, 2 insertions(+) diff --git a/xen/common/gzip/gunzip.c b/xen/common/gzip/gunzip.c index 89f45d4050..9551d470aa 100644 --- a/xen/common/gzip/gunzip.c +++ b/xen/common/gzip/gunzip.c @@ -3,6 +3,7 @@ #include #include #include +#include =20 #define WSIZE 0x80000000U =20 diff --git a/xen/common/gzip/inflate.c b/xen/common/gzip/inflate.c index cb146555c8..737aff505e 100644 --- a/xen/common/gzip/inflate.c +++ b/xen/common/gzip/inflate.c @@ -1018,6 +1018,7 @@ static int __init inflate(struct gunzip_state *s) r =3D inflate_block(s, &e); if (r) return r; + process_pending_softirqs(); } while (!e); =20 /* Undo too much lookahead. The next read will be byte aligned so we --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748657; cv=none; d=zohomail.com; s=zohoarc; b=JVI1sFk4l+fahOxC2dVXo8PCKgQb1lC2PGNZVGki4QygDv/uUeZhZNoaSLHwdVDt4h6+hkyFBbT5eWMeAG7fiV4C8+dq7uzhi4atkOMwDpnT2CkcSQXWEozXerdPcY+FkWisYsZeOdRhViWbxEqudcfsgSnI/7trIt3TwuH9ySA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748657; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=XSMwTIeZ5h90J4h/y2ZToyj+1NftG+yZwUVpCV46iA0=; b=QVptcRuHvAwI8n/0YvfnAjL0g7kgyni1VnciFJJwxiK0UMOtEUj56dmIKpLR9uaUNQT683wCyCHTz1vOLIIWiSaoYjakeFVHmc3k0pkA+EUoCw+S4pMVtiKrFvY56zdbAJlAIOm/OY5Nnkr+X02nLFfoMHHE4wy/fgVYEtVkmjo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748657123883.267312971178; Tue, 3 Nov 2026 15:30:57 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470790.1679419 (Exim 4.92) (envelope-from ) id 1xONxe-0000k2-EK; Tue, 03 Nov 2026 23:30:38 +0000 Received: by outflank-mailman (output) from mailman id 1470790.1679419; Tue, 03 Nov 2026 23:30:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxe-0000iG-79; Tue, 03 Nov 2026 23:30:38 +0000 Received: by outflank-mailman (input) for mailman id 1470790; Tue, 03 Nov 2026 23:30:35 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxb-0000cB-Sa for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:35 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cot8-2LqQ; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnv-0000000EMFa-1eLx; Tue, 03 Nov 2026 23:20:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=XSMwTIeZ5h90J4h/y2ZToyj+1NftG+yZwUVpCV46iA0=; b=18K9dXGFatSNazLPoV4skeQSFT 9m9zBHvy/T7zHnZlWGYHWrkTe0ijP/75HmQF7E8VJPDCVhRoE51XkXQMha6iBx0ziBX//YzDTUZoO hTBTN45UpuF5b872RHoYxA9J8NAfVKxdqZRIG1bo2WPGaG7m9IibQF7TFAcAjCp8Hdsc=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v6 16/44] x86/mm: add hooks for a sparse view of the directmap Date: Tue, 3 Nov 2026 17:13:38 -0500 Message-ID: <20261103-asi-hvm-v6-16-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748657728158500 Content-Type: text/plain; charset="utf-8" The directmap maps all of RAM, and every root page-table carries it, so while any guest context runs, all guest memory is reachable from Xen's address space. For ASI, guest contexts are to run on a sparse view of the directmap which maps only what Xen must reach through it at all times, leaving guest memory to transient mappings (map_domain_page()). A later patch builds such a sparse view alongside the full directmap, once the boot allocator has handed its memory to the heap. It is then kept in step: - xenheap allocations are added, and taken out again before the pages are freed. If that fails the pages stay mapped, and are leaked with a warning rather than reused; - memory handed to the heap later (boot modules, .init) is taken out, and is not handed over if that fails; - a PV dom0's initrd, assigned to dom0 in place, is taken out; - page-tables from the boot allocator, which the sparse view maps with every other boot-time allocation, are taken out when idle_pg_table's hierarchy frees them, and are not freed if that fails, so that the sparse view never keeps a page the heap may hand out; - changes to the full directmap through map_pages_to_xen() and modify_xen_mappings() are mirrored. After boot, map_pages_to_xen() maps into the directmap only memory Xen keeps (stack guard and shadow stack pages, firmware tables); memory hot-add maps the new range into the full directmap only, so that the sparse view never needs page-table pages for it. Add a hook at each of these points, and one where the view is to be built; x86's are stubs in a new asm/sparse-directmap.h. map_pages_in() and modify_mappings_in() are no longer static: memory hot-add calls map_pages_in() on idle_pg_table, and the sparse view will use both. Common code gains four hooks for this, arch_heap_pages_init(), arch_xenheap_needs_scrub() and arch_xenheap_pages_{alloc,free}(), no-ops until an architecture implements them. A failed arch_xenheap_pages_alloc() may have mapped part of the pages: alloc_xenheap_pages() then takes them out through arch_xenheap_pages_free(), as a free would, before freeing them, and leaks them if that fails too. alloc_xenheap_pages() accepts pages still awaiting scrubbing (MEMF_no_scrub). That is harmless while the directmap maps all memory anyway, but the sparse view would make such a page's contents, a dying domain's data for instance, reachable from every context on it until overwritten. With a sparse view configured, arch_xenheap_needs_scrub() makes xenheap allocations scrub those pages first, as domain heap allocations do; clean pages need nothing extra. arch_mfns_in_directmap() says whether an MFN range is reachable through the directmap; it now answers for every context, so false once a sparse view is configured. Its one caller, init_node_heap(), then keeps a NUMA node's heap metadata in the xenheap instead of carving it out of the node's memory through the directmap: carved out of a range handed to the heap, it would be left out of the sparse view, which every heap allocation needs it in. The hooks for now are just stubs. No functional change. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version, split out of "x86/mm: build and maintain a sparse view of the directmap". --- xen/arch/x86/include/asm/mm.h | 12 +++++- xen/arch/x86/include/asm/sparse-directmap.h | 30 +++++++++++++++ xen/arch/x86/mm.c | 30 ++++++++++++--- xen/arch/x86/pv/dom0_build.c | 4 ++ xen/arch/x86/setup.c | 4 ++ xen/arch/x86/x86_64/mm.c | 10 ++++- xen/common/page_alloc.c | 41 ++++++++++++++++++++- xen/include/xen/mm.h | 21 +++++++++++ 8 files changed, 142 insertions(+), 10 deletions(-) create mode 100644 xen/arch/x86/include/asm/sparse-directmap.h diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 7b9a0f561d..61dc317dd4 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -7,6 +7,7 @@ #include #include #include +#include #include =20 /* @@ -599,6 +600,11 @@ mfn_t alloc_xen_pagetable(void); void free_xen_pagetable(mfn_t mfn); void *alloc_mapped_pagetable(mfn_t *pmfn); =20 +int map_pages_in(l4_pgentry_t *root, unsigned long virt, mfn_t mfn, + unsigned long nr_mfns, pte_attr_t flags); +int modify_mappings_in(l4_pgentry_t *root, unsigned long s, unsigned long = e, + pte_attr_t nf); + int __sync_local_execstate(void); =20 /* Arch-specific portion of memory_op hypercall. */ @@ -660,12 +666,16 @@ void write_32bit_pse_identmap(uint32_t *l2); =20 /* * x86 maps part of physical memory via the directmap region. - * Return whether the range of MFN falls in the directmap region. + * Return whether the range of MFN falls in the directmap region in every + * context: never, with a sparse directmap view configured. */ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long= nr) { unsigned long eva =3D min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END); =20 + if ( sparse_dmap_configured() ) + return false; + return (mfn + nr) <=3D (virt_to_mfn(eva - 1) + 1); } =20 diff --git a/xen/arch/x86/include/asm/sparse-directmap.h b/xen/arch/x86/inc= lude/asm/sparse-directmap.h new file mode 100644 index 0000000000..53747a98ad --- /dev/null +++ b/xen/arch/x86/include/asm/sparse-directmap.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef X86_SPARSE_DIRECTMAP_H +#define X86_SPARSE_DIRECTMAP_H + +#include +#include +#include + +static inline bool sparse_dmap_configured(void) { return false; } +static inline void sparse_dmap_init(void) {} +static inline int sparse_dmap_mirror_map(unsigned long virt, mfn_t mfn, + unsigned long nr, pte_attr_t flag= s) +{ + return 0; +} +static inline int sparse_dmap_mirror_modify(unsigned long s, unsigned long= e, + pte_attr_t nf) +{ + return 0; +} +static inline int sparse_dmap_remove(mfn_t mfn, unsigned long nr) +{ + return 0; +} +static inline int sparse_dmap_drop_pagetable(mfn_t mfn) +{ + return 0; +} + +#endif /* X86_SPARSE_DIRECTMAP_H */ diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 39c6fa3d56..904ed7a258 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -127,6 +127,7 @@ #include #include #include +#include #include #include #include @@ -5442,10 +5443,17 @@ mfn_t xen_map_to_mfn(unsigned long va) =20 /* * Free a page-table page that the hierarchy rooted at @root has stopped - * using. + * using. One of idle_pg_table's may be in the sparse directmap view: it = is + * taken out first, under the sparse view's L3 lock inside an L3 lock of + * idle_pg_table's hierarchy (the only order in which the two nest), and + * kept if that fails. The sparse view's own page-tables are not in it, a= nd + * are freed directly: the caller may hold the sparse view's L3 lock. */ static void free_pagetable_in(const l4_pgentry_t *root, mfn_t mfn) { + if ( root =3D=3D idle_pg_table && sparse_dmap_drop_pagetable(mfn) ) + return; + free_xen_pagetable(mfn); } =20 @@ -5453,7 +5461,7 @@ static void free_pagetable_in(const l4_pgentry_t *roo= t, mfn_t mfn) * map_pages_to_xen() in the hierarchy rooted at @root: idle_pg_table's, or * another hierarchy sharing its virtual addresses. */ -static int map_pages_in( +int map_pages_in( l4_pgentry_t *root, unsigned long virt, mfn_t mfn, @@ -5865,7 +5873,12 @@ int map_pages_to_xen( unsigned long nr_mfns, pte_attr_t flags) { - return map_pages_in(idle_pg_table, virt, mfn, nr_mfns, flags); + int rc =3D map_pages_in(idle_pg_table, virt, mfn, nr_mfns, flags); + + if ( !rc ) + rc =3D sparse_dmap_mirror_map(virt, mfn, nr_mfns, flags); + + return rc; } =20 int __init populate_pt_range(unsigned long virt, unsigned long nr_mfns) @@ -5888,8 +5901,8 @@ int __init populate_pt_range(unsigned long virt, unsi= gned long nr_mfns) * modify_mappings_in() does so in the hierarchy rooted at @root, as * map_pages_in() does. */ -static int modify_mappings_in(l4_pgentry_t *root, unsigned long s, - unsigned long e, pte_attr_t nf) +int modify_mappings_in(l4_pgentry_t *root, unsigned long s, + unsigned long e, pte_attr_t nf) { bool locking =3D system_state > SYS_STATE_boot; l3_pgentry_t *pl3e =3D NULL; @@ -6169,7 +6182,12 @@ static int modify_mappings_in(l4_pgentry_t *root, un= signed long s, =20 int modify_xen_mappings(unsigned long s, unsigned long e, pte_attr_t nf) { - return modify_mappings_in(idle_pg_table, s, e, nf); + int rc =3D modify_mappings_in(idle_pg_table, s, e, nf); + + if ( !rc ) + rc =3D sparse_dmap_mirror_modify(s, e, nf); + + return rc; } =20 int destroy_xen_mappings(unsigned long s, unsigned long e) diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index dd64af5616..829eafd67d 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -22,6 +22,7 @@ #include #include #include +#include =20 /* Allow ring-3 access in long mode as guest cannot use ring 1 ... */ #define BASE_PROT (_PAGE_PRESENT|_PAGE_RW|_PAGE_ACCESSED|_PAGE_USER) @@ -650,6 +651,9 @@ static int __init dom0_construct(const struct boot_doma= in *bd) { if ( assign_pages(mfn_to_page(_mfn(initrd_mfn)), count, d, 0) ) BUG(); + /* The initrd is dom0's now: out of the sparse directmap. */ + if ( sparse_dmap_remove(_mfn(initrd_mfn), PFN_UP(initrd_len)) ) + panic("Cannot unmap dom0's initrd\n"); /* * We have mapped the initrd directly into dom0, and assigned = the * pages. Tell the boot_module handling that we've freed it, s= o the diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index df7cfcc563..e23ded0c86 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -54,6 +54,7 @@ #include #include #include +#include #include #include =20 @@ -1946,6 +1947,9 @@ void asmlinkage __init noreturn __start_xen(void) } } =20 + /* The heap now has all memory, and page-tables come from it. */ + sparse_dmap_init(); + bsp_stack =3D cpu_alloc_stack(0); /* Needs to know IDT vs FRED */ if ( !bsp_stack ) panic("No memory for BSP stack\n"); diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index 7b04537066..c5d9dd64a3 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -1231,8 +1231,14 @@ int memory_add(unsigned long spfn, unsigned long epf= n, unsigned int pxm) i =3D virt_to_mfn(HYPERVISOR_VIRT_END - 1) + 1; if ( spfn < i ) { - ret =3D map_pages_to_xen((unsigned long)mfn_to_virt(spfn), _mfn(sp= fn), - min(epfn, i) - spfn, PAGE_HYPERVISOR); + /* + * Into the full directmap only, not a sparse directmap view: the + * frame table and M2P carved from the start of the range are + * reached through their own mappings, and the rest goes to the + * heap, which the sparse view leaves out. + */ + ret =3D map_pages_in(idle_pg_table, (unsigned long)mfn_to_virt(spf= n), + _mfn(spfn), min(epfn, i) - spfn, PAGE_HYPERVISO= R); if ( ret ) goto destroy_directmap; } diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 5243dba29f..2150dcbd5e 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -1971,6 +1971,9 @@ static void init_heap_pages( unsigned long i; bool need_scrub =3D scrub_debug; =20 + if ( !arch_heap_pages_init(page_to_mfn(pg), nr_pages) ) + return; + /* * Keep MFN 0 away from the buddy allocator to avoid crossing zone * boundary when merging two buddies. @@ -2540,10 +2543,33 @@ void *alloc_xenheap_pages(unsigned int order, unsig= ned int memflags) if ( !(memflags >> _MEMF_bits) ) memflags |=3D MEMF_bits(xenheap_bits); =20 - pg =3D alloc_domheap_pages(NULL, order, memflags | MEMF_no_scrub); + /* + * Pages still awaiting scrubbing may do as xenheap pages: the directm= ap + * maps them anyway. Not so if an arch's sparse directmap view makes = them + * reachable where the rest of the heap's memory is not. + */ + if ( !arch_xenheap_needs_scrub() ) + memflags |=3D MEMF_no_scrub; + + pg =3D alloc_domheap_pages(NULL, order, memflags); if ( unlikely(pg =3D=3D NULL) ) return NULL; =20 + if ( arch_xenheap_pages_alloc(page_to_mfn(pg), order) ) + { + /* + * The hook may have mapped part of the pages before failing: take + * them out again as a free would, and leak them if that fails too. + */ + if ( arch_xenheap_pages_free(page_to_mfn(pg), order) ) + printk(XENLOG_WARNING + "Leaking xenheap pages %"PRI_mfn"+%u: still mapped\n", + mfn_x(page_to_mfn(pg)), 1U << order); + else + free_domheap_pages(pg, order); + return NULL; + } + for ( i =3D 0; i < (1u << order); i++ ) pg[i].count_info |=3D PGC_xen_heap; =20 @@ -2562,6 +2588,19 @@ void free_xenheap_pages(void *v, unsigned int order) =20 pg =3D virt_to_page(v); =20 + /* + * If the pages cannot be taken out of an arch's sparse directmap view, + * they may remain reachable through it: leak them rather than hand th= em + * out. + */ + if ( arch_xenheap_pages_free(page_to_mfn(pg), order) ) + { + printk(XENLOG_WARNING + "Leaking xenheap pages %"PRI_mfn"+%u: still mapped\n", + mfn_x(page_to_mfn(pg)), 1U << order); + return; + } + for ( i =3D 0; i < (1u << order); i++ ) pg[i].count_info &=3D ~PGC_xen_heap; =20 diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index 153c57f5dd..e8f9993286 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -94,6 +94,27 @@ bool scrub_free_pages(void); #define alloc_xenheap_page() (alloc_xenheap_pages(0,0)) #define free_xenheap_page(v) (free_xenheap_pages(v,0)) =20 +/* + * Hooks for an architecture keeping a second, sparse view of its directma= p, + * which maps the xenheap but not the rest of the heap's memory. + */ +static inline bool arch_heap_pages_init(mfn_t mfn, unsigned long nr) +{ + return true; +} +static inline bool arch_xenheap_needs_scrub(void) +{ + return false; +} +static inline int arch_xenheap_pages_alloc(mfn_t mfn, unsigned int order) +{ + return 0; +} +static inline int arch_xenheap_pages_free(mfn_t mfn, unsigned int order) +{ + return 0; +} + /* Free an allocation, and zero the pointer to it. */ #define FREE_XENHEAP_PAGES(p, o) do { \ void *_ptr_ =3D (p); \ --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748685; cv=none; d=zohomail.com; s=zohoarc; b=UBXRdBxlWPPvq/qrUu1dVocli349TOXNsy2lAPMYA4hBVjARL0QjHOUeChuTA1xOenH2GGSXIzyHOcuaUuupH5/9AA6uCrCsbygo5F789lHHrlpEIEtSoDid1zCCuwLSJ0D4CuYvNoBPF0UMvLvMLQgicXqxFoyCHCiF41USj2k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748685; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Y6ajORWqf8gpkH5v5C/jSbFAFNJo4H5wi/PdnzFQF1g=; b=TK9dEMm5y+V4W8svrlsge1QA0c8pWePAXkUfVDoIr4p3pZL8wgaPFMJ2djsDre+BdoL3fd4XMEptx0wrSFxAKI6CbCXsz9vvy2MOixK+VmM2w96pISqJYVK15tg9ZZ4mFx7zRNZ1iDkiK7FkKfS4xMSl+Nj8+IxNpVA9Bh5C/To= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748685176575.6116395700784; Tue, 3 Nov 2026 15:31:25 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470811.1679576 (Exim 4.92) (envelope-from ) id 1xONy1-0005IG-D9; Tue, 03 Nov 2026 23:31:01 +0000 Received: by outflank-mailman (output) from mailman id 1470811.1679576; Tue, 03 Nov 2026 23:31:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxz-00059M-4k; Tue, 03 Nov 2026 23:30:59 +0000 Received: by outflank-mailman (input) for mailman id 1470811; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000dy-HU for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CovM-1Iuh; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnw-0000000EMFa-012A; Tue, 03 Nov 2026 23:20:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=Y6ajORWqf8gpkH5v5C/jSbFAFNJo4H5wi/PdnzFQF1g=; b=hnVGcJnwAdVsodcqFoBr8j881f tyczsZaPBND8awqFECvJBujntyhSWR7/o81iKMLrTmD7CM+HCQ5l+Iz2tSeO56xcouyrEqfFkA4ml +wwX4U5Q6MHkCUsk1RD6tof9V2WDzyVTPDMdElcixiVchJG+8PJ0YibBwa9Lv09utzKk=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v6 17/44] x86/mm: build and maintain a sparse view of the directmap Date: Tue, 3 Nov 2026 17:13:39 -0500 Message-ID: <20261103-asi-hvm-v6-17-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748685697158500 Content-Type: text/plain; charset="utf-8" The directmap maps all of RAM, and every root page-table carries it, so while any guest context runs, all guest memory is reachable from Xen's address space. For ASI, guest contexts are to run on a view of the directmap which maps only what Xen must reach through it at all times, leaving guest memory to transient mappings (map_domain_page()). Rather than remove the directmap for everyone, keep the full directmap as it is, for the idle domain and for guest contexts not using ASI, and build a second, sparse view: page-tables for the same virtual addresses (the part of the directmap PV guests can see, where everything Xen reaches through the directmap lives), mapping a subset of what the full directmap maps, each entry a copy of the full directmap's. Boot and the idle domain keep the full directmap, so nothing in them needs converting. The sparse view maps the xenheap, and memory never handed to the heap allocator (Xen's image, boot-time allocations, firmware tables). It is built once the boot allocator has handed its memory to the heap, from the full directmap minus the ranges the heap was given, plus the xenheap allocations made meanwhile. The page-tables then come from the heap, and no other CPU is up. The sparse view has an L3 for each of its L4 slots from the start, so its L4 entries never change and root page-tables can take copies of them. It is then kept in step through the hooks added by "x86/mm: add hooks for a sparse view of the directmap", which this patch implements for CONFIG_SPARSE_DIRECTMAP. modify_xen_mappings_lite() needs no change: it only changes the mappings of Xen's image, in L4 slot 261, which every root page-table shares with idle_pg_table, outside the range the sparse view replaces. xen_mapping_lookup(), built only with CONFIG_SPARSE_DIRECTMAP, reads an entry of either hierarchy under the L3 table lock. The allocator may still have to scrub a page before returning it; this will be mapped with map_domain_page(). This is fine for HVM domains on the sparse view, as the mapcache is per-vCPU, so no other vCPU can read unscrubbed data. PV domains' mapcache is still domain-wide, so for PV domains on the sparse view, this opens up a window where other vCPUs could potentially speculatively access the unscrubbed data. This would be fixed by having per-vCPU mapcaches for PV domains, or synchronously scrubbing pages on free. Neither are in the scope of the current work. The sparse view is built only when some domain type is to use it, which nothing can request yet: this patch changes nothing on its own. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - Split: free_pagetable_in() moves to "x86/mm: let Xen's page-table walker operate on a given root", and the hooks with their stubs and callers to the new "x86/mm: add hooks for a sparse view of the directmap". Changes in v4: - Message: "domain type", not "guest type". - Wrap a line that reached 80 columns. - Memory hot-add maps the new range into the full directmap only. - Define opt_sparse_dmap_hwdom here, beside the PV and HVM knobs (Roger). - add_to_sparse_view(): assert that the lookup finds the MFN expected (Roger). - cmp_mfn_range(): explicit return values, as in cmp_node() (Roger). - Comments and variable name changes in sparse_dmap_init()'s loop over the heap's ranges (Roger). - Message: drop "(NUMA node heap metadata)" (Roger). - Message: drop the reference to the directmap removal series (Roger). - Message: modify_xen_mappings_lite() needs no change (Roger). - Message: note the limit of scrubbing at allocation for PV domains (Roger). - Message: only boot and the idle domain keep the full directmap. - Set the NX bit in this patch, rather than adding it in a later patch. Changes in v3: - New in this version. --- xen/arch/x86/Kconfig | 1 + xen/arch/x86/Makefile | 1 + xen/arch/x86/include/asm/mm.h | 4 + xen/arch/x86/include/asm/sparse-directmap.h | 37 ++ xen/arch/x86/mm.c | 74 ++++ xen/arch/x86/sparse-directmap.c | 431 ++++++++++++++++++++ xen/common/Kconfig | 18 + xen/include/xen/mm.h | 14 + 8 files changed, 580 insertions(+) create mode 100644 xen/arch/x86/sparse-directmap.c diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index e5535ac484..cd72102ccc 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -31,6 +31,7 @@ config X86 select HAS_SCHED_GRANULARITY select HAS_SHARED_INFO imply HAS_SOFT_RESET + select HAS_SPARSE_DIRECTMAP select HAS_UBSAN select HAS_VMAP select HAS_VPCI if HVM diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile index 2be90ea098..86adf042e9 100644 --- a/xen/arch/x86/Makefile +++ b/xen/arch/x86/Makefile @@ -63,6 +63,7 @@ obj-y +=3D shutdown.o obj-y +=3D smp.o obj-y +=3D smpboot.o obj-y +=3D spec_ctrl.o +obj-$(CONFIG_SPARSE_DIRECTMAP) +=3D sparse-directmap.o obj-y +=3D srat.o obj-y +=3D string.o obj-$(CONFIG_SYSCTL) +=3D sysctl.o diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 61dc317dd4..113ade0bbd 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -604,6 +604,10 @@ int map_pages_in(l4_pgentry_t *root, unsigned long vir= t, mfn_t mfn, unsigned long nr_mfns, pte_attr_t flags); int modify_mappings_in(l4_pgentry_t *root, unsigned long s, unsigned long = e, pte_attr_t nf); +#ifdef CONFIG_SPARSE_DIRECTMAP +mfn_t xen_mapping_lookup(l4_pgentry_t *root, unsigned long va, + pte_attr_t *flags, unsigned long *nr); +#endif =20 int __sync_local_execstate(void); =20 diff --git a/xen/arch/x86/include/asm/sparse-directmap.h b/xen/arch/x86/inc= lude/asm/sparse-directmap.h index 53747a98ad..ea14d44334 100644 --- a/xen/arch/x86/include/asm/sparse-directmap.h +++ b/xen/arch/x86/include/asm/sparse-directmap.h @@ -6,6 +6,41 @@ #include #include =20 +#ifdef CONFIG_SPARSE_DIRECTMAP + +/* + * The part of the directmap the sparse view covers: the part PV guests can + * see, which is where everything Xen reaches through the directmap lives. + * SPARSE_DMAP_END_MFN is the first MFN past it. + */ +#define SPARSE_DMAP_START DIRECTMAP_VIRT_START +#define SPARSE_DMAP_END HYPERVISOR_VIRT_END +#define SPARSE_DMAP_END_MFN (virt_to_mfn(SPARSE_DMAP_END - 1) + 1) + +/* + * Which domains use the sparse view: PV domains and HVM domains other than + * the hardware domain, and the hardware domain, PV or PVH. + */ +extern bool opt_sparse_dmap_pv, opt_sparse_dmap_hvm, opt_sparse_dmap_hwdom; + +/* Root of the sparse view, once built (NULL before, or if not configured)= . */ +extern l4_pgentry_t *sparse_dmap_root; + +static inline bool sparse_dmap_configured(void) +{ + return opt_sparse_dmap_pv || opt_sparse_dmap_hvm || opt_sparse_dmap_hw= dom; +} + +void sparse_dmap_init(void); +int sparse_dmap_mirror_map(unsigned long virt, mfn_t mfn, unsigned long nr, + pte_attr_t flags); +int sparse_dmap_mirror_modify(unsigned long s, unsigned long e, + pte_attr_t nf); +int sparse_dmap_remove(mfn_t mfn, unsigned long nr); +int sparse_dmap_drop_pagetable(mfn_t mfn); + +#else /* !CONFIG_SPARSE_DIRECTMAP */ + static inline bool sparse_dmap_configured(void) { return false; } static inline void sparse_dmap_init(void) {} static inline int sparse_dmap_mirror_map(unsigned long virt, mfn_t mfn, @@ -27,4 +62,6 @@ static inline int sparse_dmap_drop_pagetable(mfn_t mfn) return 0; } =20 +#endif /* CONFIG_SPARSE_DIRECTMAP */ + #endif /* X86_SPARSE_DIRECTMAP_H */ diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 904ed7a258..0d475432ff 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5441,6 +5441,80 @@ mfn_t xen_map_to_mfn(unsigned long va) return ret; } =20 +#ifdef CONFIG_SPARSE_DIRECTMAP +/* + * Look @va up in the hierarchy rooted at @root, without allocating anythi= ng. + * Returns the MFN mapped at @va, or INVALID_MFN, with the mapping's flags= in + * L1 form in *flags, and in *nr the number of pages from @va to the end of + * the entry that maps @va (or that finds it unmapped). + */ +mfn_t xen_mapping_lookup(l4_pgentry_t *root, unsigned long va, + pte_attr_t *flags, unsigned long *nr) +{ + bool locking =3D system_state > SYS_STATE_boot; + l4_pgentry_t l4e =3D root[l4_table_offset(va)]; + const l3_pgentry_t *pl3e =3D NULL; + const l2_pgentry_t *pl2e =3D NULL; + const l1_pgentry_t *pl1e =3D NULL; + struct page_info *l3page; + unsigned int shift =3D L4_PAGETABLE_SHIFT; + mfn_t ret =3D INVALID_MFN; + + L3T_INIT(l3page); + *flags =3D 0; + + if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) ) + goto out; + + pl3e =3D map_l3t_from_l4e(l4e) + l3_table_offset(va); + l3page =3D l4e_get_page(l4e); + L3T_LOCK(l3page); + + shift =3D L3_PAGETABLE_SHIFT; + if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ) + goto out; + if ( l3e_get_flags(*pl3e) & _PAGE_PSE ) + { + /* The superpage PAT bit sits among the address bits. */ + *flags =3D lNf_to_l1f(l3e_get_flags(*pl3e)) | + (l3e_get_intpte(*pl3e) & _PAGE_PSE_PAT ? _PAGE_PAT : 0); + ret =3D _mfn((l3e_get_pfn(*pl3e) & + ~((1UL << (2 * PAGETABLE_ORDER)) - 1)) + + (PFN_DOWN(va) & ((1UL << (2 * PAGETABLE_ORDER)) - 1))); + goto out; + } + + pl2e =3D map_l2t_from_l3e(*pl3e) + l2_table_offset(va); + shift =3D L2_PAGETABLE_SHIFT; + if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) + goto out; + if ( l2e_get_flags(*pl2e) & _PAGE_PSE ) + { + *flags =3D lNf_to_l1f(l2e_get_flags(*pl2e)) | + (l2e_get_intpte(*pl2e) & _PAGE_PSE_PAT ? _PAGE_PAT : 0); + ret =3D _mfn((l2e_get_pfn(*pl2e) & ~((1UL << PAGETABLE_ORDER) - 1)= ) + + l1_table_offset(va)); + goto out; + } + + pl1e =3D map_l1t_from_l2e(*pl2e) + l1_table_offset(va); + shift =3D PAGE_SHIFT; + if ( l1e_get_flags(*pl1e) & _PAGE_PRESENT ) + { + *flags =3D l1e_get_flags(*pl1e); + ret =3D l1e_get_mfn(*pl1e); + } + + out: + L3T_UNLOCK(l3page); + unmap_domain_page(pl1e); + unmap_domain_page(pl2e); + unmap_domain_page(pl3e); + *nr =3D ((((va >> shift) + 1) << shift) - va) >> PAGE_SHIFT; + return ret; +} +#endif /* CONFIG_SPARSE_DIRECTMAP */ + /* * Free a page-table page that the hierarchy rooted at @root has stopped * using. One of idle_pg_table's may be in the sparse directmap view: it = is diff --git a/xen/arch/x86/sparse-directmap.c b/xen/arch/x86/sparse-directma= p.c new file mode 100644 index 0000000000..d35a8d6979 --- /dev/null +++ b/xen/arch/x86/sparse-directmap.c @@ -0,0 +1,431 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * A sparse view of the directmap. + * + * Xen maps all of RAM at DIRECTMAP_VIRT_START, and every root page-table + * carries those mappings. The sparse view is a second set of page-tables + * for the same virtual addresses (the part of the directmap PV guests can + * see), mapping only what Xen needs to reach through the directmap at all + * times: + * + * - Memory never handed to the heap: Xen's image, boot-time allocations, + * firmware tables. Not a PV dom0's initrd, used in place, nor the frame + * table and M2P pages carved from hot-added memory. + * - Memory handed to the heap, then allocated to the xenheap. + * + * The rest of the heap's memory, where guest memory comes from, is not + * mapped. Every mapping in the sparse view is either identical to the + * full directmap's at the same address, or empty. + * + * Guest contexts selected on the command line run on root page-tables + * carrying the sparse view's L3s in the directmap slots. + */ + +#include +#include +#include +#include +#include +#include + +#include +#include + +bool __ro_after_init opt_sparse_dmap_pv; +bool __ro_after_init opt_sparse_dmap_hvm; +bool __ro_after_init opt_sparse_dmap_hwdom; + +/* Root of the sparse view: only ever walked and copied from, never loaded= . */ +static l4_pgentry_t __aligned(PAGE_SIZE) sparse_l4[L4_PAGETABLE_ENTRIES]; +l4_pgentry_t *__ro_after_init sparse_dmap_root; + +/* + * Before the sparse view is built: the MFN ranges handed to the heap, whi= ch + * it leaves out, and the xenheap allocations made since, which it puts ba= ck. + */ +struct mfn_range { + unsigned long s, e; +}; +static struct mfn_range early_heap[256]; +static unsigned int nr_early_heap; +static struct mfn_range early_xenheap[2 * MAX_NUMNODES]; +static unsigned int nr_early_xenheap; +static bool early_overflow; + +/* + * Clip [*mfn, *mfn + *nr) to what the sparse view covers; false if nothin= g is + * left. + */ +static bool clip_to_sparse_view(unsigned long *mfn, unsigned long *nr) +{ + unsigned long end =3D SPARSE_DMAP_END_MFN; + + if ( *mfn >=3D end ) + return false; + if ( *nr > end - *mfn ) + *nr =3D end - *mfn; + + return *nr; +} + +static void early_range_add(struct mfn_range *r, unsigned int *nr, + unsigned int max, unsigned long s, unsigned lo= ng e) +{ + if ( *nr && r[*nr - 1].e =3D=3D s ) + r[*nr - 1].e =3D e; + else if ( *nr < max ) + r[(*nr)++] =3D (struct mfn_range){ s, e }; + else + early_overflow =3D true; +} + +static void early_range_del(struct mfn_range *r, unsigned int *nr, + unsigned long s, unsigned long e) +{ + unsigned int i; + + for ( i =3D 0; i < *nr; i++ ) + if ( r[i].s =3D=3D s && r[i].e =3D=3D e ) + { + r[i] =3D r[--*nr]; + return; + } +} + +/* + * Copy the full directmap's mappings of MFNs [mfn, mfn + nr) into the spa= rse + * view. + */ +static int add_to_sparse_view(l4_pgentry_t *root, unsigned long mfn, + unsigned long nr) +{ + unsigned long start =3D (unsigned long)mfn_to_virt(mfn); + unsigned long va =3D start, end =3D start + (nr << PAGE_SHIFT); + + while ( va < end ) + { + pte_attr_t flags; + unsigned long n; + mfn_t va_mfn =3D xen_mapping_lookup(idle_pg_table, va, &flags, &n); + int rc; + + ASSERT(mfn_eq(va_mfn, INVALID_MFN) || + mfn_x(va_mfn) =3D=3D mfn + ((va - start) >> PAGE_SHIFT)); + n =3D min(n, (end - va) >> PAGE_SHIFT); + if ( !mfn_eq(va_mfn, INVALID_MFN) && + (rc =3D map_pages_in(root, va, va_mfn, n, flags)) ) + return rc; + va +=3D n << PAGE_SHIFT; + } + + return 0; +} + +static int remove_from_sparse_view(unsigned long mfn, unsigned long nr) +{ + unsigned long va =3D (unsigned long)mfn_to_virt(mfn); + + return modify_mappings_in(sparse_dmap_root, va, va + (nr << PAGE_SHIFT= ), + _PAGE_NONE); +} + +/* Memory is about to be handed to the heap allocator. */ +bool arch_heap_pages_init(mfn_t mfn, unsigned long nr) +{ + unsigned long s =3D mfn_x(mfn); + + if ( !sparse_dmap_configured() || !clip_to_sparse_view(&s, &nr) ) + return true; + + if ( !sparse_dmap_root ) + { + early_range_add(early_heap, &nr_early_heap, ARRAY_SIZE(early_heap), + s, s + nr); + return true; + } + + if ( remove_from_sparse_view(s, nr) ) + { + printk(XENLOG_ERR + "sparse directmap: cannot unmap MFNs %#lx-%#lx, not freeing= \n", + s, s + nr - 1); + return false; + } + + return true; +} + +/* + * Xenheap pages enter the sparse view, which guest contexts run on: those + * still holding a previous owner's data must be scrubbed first. + */ +bool arch_xenheap_needs_scrub(void) +{ + return sparse_dmap_configured(); +} + +/* Pages have been allocated from the heap as xenheap pages. */ +int arch_xenheap_pages_alloc(mfn_t mfn, unsigned int order) +{ + unsigned long s =3D mfn_x(mfn), nr =3D 1UL << order; + + if ( !sparse_dmap_configured() ) + return 0; + + /* Xenheap pages must be reachable through the sparse view. */ + BUG_ON(s + nr > SPARSE_DMAP_END_MFN); + + if ( !sparse_dmap_root ) + { + early_range_add(early_xenheap, &nr_early_xenheap, + ARRAY_SIZE(early_xenheap), s, s + nr); + return 0; + } + + return add_to_sparse_view(sparse_dmap_root, s, nr); +} + +/* + * Xenheap pages are about to be freed. A failure means the pages may sti= ll + * be reachable through the sparse view, and must not be reused. + */ +int arch_xenheap_pages_free(mfn_t mfn, unsigned int order) +{ + unsigned long s =3D mfn_x(mfn), nr =3D 1UL << order; + + if ( !sparse_dmap_configured() ) + return 0; + + BUG_ON(s + nr > SPARSE_DMAP_END_MFN); + + if ( !sparse_dmap_root ) + { + early_range_del(early_xenheap, &nr_early_xenheap, s, s + nr); + return 0; + } + + return remove_from_sparse_view(s, nr); +} + +/* Memory leaves Xen's hands without passing through the heap. */ +int sparse_dmap_remove(mfn_t mfn, unsigned long nr) +{ + return arch_heap_pages_init(mfn, nr) ? 0 : -ENOMEM; +} + +/* + * A page-table page of idle_pg_table's hierarchy is about to be freed. O= ne + * the boot allocator provided is in the sparse view, as every boot-time + * allocation is: take it out before the heap may hand it on. One allocat= ed + * since came from the heap and is not in the sparse view, which a lookup + * finds without changing, or flushing, anything. Non-zero: the page may + * still be mapped, and must not be freed. + */ +int sparse_dmap_drop_pagetable(mfn_t mfn) +{ + unsigned long s =3D mfn_x(mfn), nr =3D 1, n; + pte_attr_t flags; + int rc; + + if ( !sparse_dmap_root || !clip_to_sparse_view(&s, &nr) || + mfn_eq(xen_mapping_lookup(sparse_dmap_root, + (unsigned long)mfn_to_virt(s), &flags, = &n), + INVALID_MFN) ) + return 0; + + rc =3D remove_from_sparse_view(s, 1); + if ( rc ) + printk(XENLOG_WARNING + "Leaking page-table page %"PRI_mfn": still in the sparse di= rectmap\n", + mfn_x(mfn)); + + return rc; +} + +int sparse_dmap_mirror_map(unsigned long virt, mfn_t mfn, unsigned long nr, + pte_attr_t flags) +{ + if ( !sparse_dmap_root || virt < SPARSE_DMAP_START || + virt >=3D SPARSE_DMAP_END ) + return 0; + + nr =3D min(nr, (SPARSE_DMAP_END - virt) >> PAGE_SHIFT); + + return map_pages_in(sparse_dmap_root, virt, mfn, nr, flags); +} + +int sparse_dmap_mirror_modify(unsigned long s, unsigned long e, + pte_attr_t nf) +{ + unsigned long va; + + if ( !sparse_dmap_root || e <=3D SPARSE_DMAP_START || + s >=3D SPARSE_DMAP_END ) + return 0; + + s =3D max(s, SPARSE_DMAP_START); + e =3D min(e, SPARSE_DMAP_END); + + if ( !(nf & _PAGE_PRESENT) ) + return modify_mappings_in(sparse_dmap_root, s, e, nf); + + /* + * A permission change applies where the sparse view maps anything: it + * never creates mappings, and the sparse view need not map all of [s,= e). + */ + for ( va =3D s; va < e; ) + { + pte_attr_t flags; + unsigned long n; + mfn_t mfn =3D xen_mapping_lookup(sparse_dmap_root, va, &flags, &n); + int rc; + + n =3D min(n, (e - va) >> PAGE_SHIFT); + if ( !mfn_eq(mfn, INVALID_MFN) && + (rc =3D modify_mappings_in(sparse_dmap_root, va, + va + (n << PAGE_SHIFT), nf)) ) + return rc; + va +=3D n << PAGE_SHIFT; + } + + return 0; +} + +static int __init cf_check cmp_mfn_range(const void *a, const void *b) +{ + const struct mfn_range *l =3D a, *r =3D b; + + if ( l->s > r->s ) + return 1; + if ( l->s < r->s ) + return -1; + + return 0; +} + +static void __init cf_check swap_mfn_range(void *a, void *b) +{ + SWAP(*(struct mfn_range *)a, *(struct mfn_range *)b); +} + +/* + * Build the sparse view: the full directmap, minus the ranges handed to t= he + * heap so far, plus the xenheap allocations made from them. Called once = the + * heap has taken over from the boot allocator, so that the page-tables co= me + * from the heap, and before any other CPU is up. + */ +void __init sparse_dmap_init(void) +{ + unsigned long va, mapped =3D 0; + unsigned int i, heap_idx; + + if ( !sparse_dmap_configured() ) + return; + + if ( early_overflow ) + { + printk(XENLOG_WARNING + "Sparse directmap: too many early heap ranges, disabled\n"); + opt_sparse_dmap_pv =3D opt_sparse_dmap_hvm =3D opt_sparse_dmap_hwd= om =3D + false; + return; + } + + /* An L3 for every slot, so that the L4 entries never change. */ + for ( va =3D SPARSE_DMAP_START; va < SPARSE_DMAP_END; + va +=3D 1UL << L4_PAGETABLE_SHIFT ) + { + mfn_t l3mfn; + void *l3t =3D alloc_mapped_pagetable(&l3mfn); + + if ( !l3t ) + panic("Sparse directmap: out of memory\n"); + unmap_domain_page(l3t); + sparse_l4[l4_table_offset(va)] =3D + l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR); + } + + sort(early_heap, nr_early_heap, sizeof(*early_heap), cmp_mfn_range, + swap_mfn_range); + + /* + * The full directmap, leaf by leaf, minus the ranges added to the + * heap. Leaves come in address order, which is MFN order in the + * directmap, and the heap ranges are sorted and disjoint, so + * heap_idx only moves forward. + */ + for ( va =3D SPARSE_DMAP_START, heap_idx =3D 0; va < SPARSE_DMAP_END; ) + { + pte_attr_t flags; + unsigned long n, s, e; + + /* Get information about the leaf PTE at va. */ + mfn_t mfn =3D xen_mapping_lookup(idle_pg_table, va, &flags, &n); + + /* Clip the va range we're considering to SPARSE_DMAP_END. */ + n =3D min(n, (SPARSE_DMAP_END - va) >> PAGE_SHIFT); + if ( mfn_eq(mfn, INVALID_MFN) ) + { + va +=3D n << PAGE_SHIFT; + continue; + } + + /* + * Walk through the range [mfn, mfn + n) looking for ranges not + * handed to the heap. + */ + for ( s =3D mfn_x(mfn), e =3D s + n; s < e; ) + { + unsigned long upto =3D e; + + /* Step past heap ranges that end at or before s. */ + while ( heap_idx < nr_early_heap && early_heap[heap_idx].e <= =3D s ) + heap_idx++; + + /* s lies in a heap range: skip to its end. */ + if ( heap_idx < nr_early_heap && early_heap[heap_idx].s <=3D s= ) + { + s =3D min(e, early_heap[heap_idx].e); + continue; + } + + /* Map up to the next heap range, or to the end of the leaf. */ + if ( heap_idx < nr_early_heap ) + upto =3D min(e, early_heap[heap_idx].s); + + if ( map_pages_in(sparse_l4, va + ((s - mfn_x(mfn)) << PAGE_SH= IFT), + _mfn(s), upto - s, flags) ) + panic("Sparse directmap: out of memory\n"); + mapped +=3D upto - s; + s =3D upto; + } + + va +=3D n << PAGE_SHIFT; + } + + /* + * Now add back in ranges from the heap which were allocated to + * the xenheap. + */ + for ( i =3D 0; i < nr_early_xenheap; i++ ) + { + unsigned long nr =3D early_xenheap[i].e - early_xenheap[i].s; + + if ( add_to_sparse_view(sparse_l4, early_xenheap[i].s, nr) ) + panic("Sparse directmap: out of memory\n"); + mapped +=3D nr; + } + + /* + * Nothing in the directmap is executed: its slots are NX, as the full + * directmap's are. + */ + if ( cpu_has_nx ) + for ( i =3D l4_table_offset(SPARSE_DMAP_START); + i < l4_table_offset(SPARSE_DMAP_END); i++ ) + l4e_add_flags(sparse_l4[i], _PAGE_NX_BIT); + + sparse_dmap_root =3D sparse_l4; + + printk(XENLOG_INFO "Sparse directmap: %lu pages mapped\n", mapped); +} diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 5b289e444f..8f2f605e45 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -167,6 +167,9 @@ config HAS_STATIC_MEMORY config HAS_SOFT_RESET bool =20 +config HAS_SPARSE_DIRECTMAP + bool + config HAS_STACK_PROTECTOR bool =20 @@ -299,6 +302,21 @@ config SPECULATIVE_HARDEN_LOCK This option is disabled by default at run time, and needs to be enabled on the command line. =20 + +config SPARSE_DIRECTMAP + bool "Sparse directmap view" + depends on HAS_SPARSE_DIRECTMAP + help + Build support for a sparse view of the directmap, alongside the + full one: it maps only the memory Xen needs to reach at all times + (the Xen heap, Xen's own image, firmware tables), while guest + memory is reached through transient mappings. Guest contexts + selected with the `asi=3D` command line option run on the sparse + directmap view, narrowing the memory reachable from Xen's address + space while they run, and hence the memory disclosure surface of + speculative side channels. + + If unsure, say N. endmenu =20 menu "Other hardening" diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index e8f9993286..2bcc45cf58 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -98,6 +98,19 @@ bool scrub_free_pages(void); * Hooks for an architecture keeping a second, sparse view of its directma= p, * which maps the xenheap but not the rest of the heap's memory. */ +#ifdef CONFIG_SPARSE_DIRECTMAP +/* Memory is about to be handed to the heap. False: don't, keep it out. */ +bool arch_heap_pages_init(mfn_t mfn, unsigned long nr); +/* Xenheap pages are to be allocated. True: scrub those needing it first.= */ +bool arch_xenheap_needs_scrub(void); +/* + * Xenheap pages were allocated. Non-zero: failed, possibly part-way; the + * pages go through arch_xenheap_pages_free() before being freed again. + */ +int arch_xenheap_pages_alloc(mfn_t mfn, unsigned int order); +/* Xenheap pages are to be freed. Non-zero: failed, don't reuse them. */ +int arch_xenheap_pages_free(mfn_t mfn, unsigned int order); +#else static inline bool arch_heap_pages_init(mfn_t mfn, unsigned long nr) { return true; @@ -114,6 +127,7 @@ static inline int arch_xenheap_pages_free(mfn_t mfn, un= signed int order) { return 0; } +#endif =20 /* Free an allocation, and zero the pointer to it. */ #define FREE_XENHEAP_PAGES(p, o) do { \ --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748663; cv=none; d=zohomail.com; s=zohoarc; b=FQjuF9+xRygqy4EduW1Y5mX7jSVycrrsMnNKDO/Uur4ZLstK1JfrOfe8JlMaQSCcFXfhhOXWniaYIodN3WI8nhJ1VE83VEsaoQY5wSevCbH70PPpD3I/IVP+aUm5plS0z069eo9RiyeY2NWWhGA4bmC2QMMedHoNeSTfMfiizc0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748663; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RVROnRhnJ2CS9YJZ5lc9Ti4SNS+seTPBJxsek/UlS70=; b=F/t8AxBlQAOhQTSgTeU3HBfJTKNnHbD/JqA28S9PiAECgmyH6J505Crz6tcIFUzzmr6RFZHmCKr+PCgK95nV/qCcm2+J70LkqoMgXkfs3Q9rXudR9HIHsn5ym88sHXMLGpBsemK0L7uh4j4MV+pAJ0dnb6DzBJKVBYSr1TmWguc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748663338258.7924170097035; Tue, 3 Nov 2026 15:31:03 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470788.1679407 (Exim 4.92) (envelope-from ) id 1xONxd-0000ew-TU; Tue, 03 Nov 2026 23:30:37 +0000 Received: by outflank-mailman (output) from mailman id 1470788.1679407; Tue, 03 Nov 2026 23:30:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxd-0000ep-Qy; Tue, 03 Nov 2026 23:30:37 +0000 Received: by outflank-mailman (input) for mailman id 1470788; Tue, 03 Nov 2026 23:30:35 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxb-0000c7-Rs for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:35 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotA-2VvB; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnx-0000000EMFa-2Ybs; Tue, 03 Nov 2026 23:20:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=RVROnRhnJ2CS9YJZ5lc9Ti4SNS+seTPBJxsek/UlS70=; b=VmLahDTeSZ7gRrgAACDuCbK0mK EG88uRdkFAszPuE9CCY26HvPqblM6gPUNGZvxlcgCDyHiQgXfEkyTXRb+trlIiV+ZyB4gUiug/y7N pZ2gCnlty+ZYdzv/oYCsYVW+4RwHxGFEph4l2kBJS4qM9AsQKPi0X4wF4QV2yFO8WxsU=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 18/44] x86/domain_page: key the directmap fast paths on the loaded view Date: Tue, 3 Nov 2026 17:13:40 -0500 Message-ID: <20261103-asi-hvm-v6-18-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748665552158500 Content-Type: text/plain; charset="utf-8" map_domain_page() returns directmap addresses whenever it can: for any MFN the directmap covers in release builds, and for vCPUs without a mapcache. Once guest contexts may run on the sparse view of the directmap, modify this to take the fast path returning the directmap address only when the full directmap is loaded. The release-build shortcut ahead of the vCPU lookup looks at the vCPU whose page-tables are loaded, without completing a lazy context switch: if that vCPU is not on the sparse view, its page-tables map the page through the directmap, and completing the switch would only move the CPU to the idle vCPU's page-tables, which map it too. A vCPU on the sparse view always has a mapcache: HVM ones through per-vCPU page-tables, PV ones through the domain-wide mapcache, which mapcache_domain_init() now sets up for a domain using the sparse view in release builds too, rather than only when the directmap does not cover all memory. map_domain_page_global() mappings must be valid in every context: use vmap() for all of them once a sparse view is configured, as arch_mfns_in_directmap() then says. Add d->arch.asi.sparse_dmap for this; nothing sets it yet. No functional change. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v4: - d->arch.sparse_dmap is now a one-bit field of d->arch.asi, beside vcpu_pt. - Wrap a line that reached 80 columns. - Drop sparse_dmap_active: d->arch.asi.sparse_dmap alone says whether a domain's root page-tables carry the sparse view (Roger). Builds without CONFIG_SPARSE_DIRECTMAP compile the test out, as before. - Drop the second directmap check after the vCPU lookup: the !cache->inuse case covers it. Changes in v3: - New in this version. --- xen/arch/x86/domain_page.c | 34 ++++++++++++++++++++++++++++--- xen/arch/x86/include/asm/domain.h | 3 +++ 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 770fee80a4..6b662706fb 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -56,6 +56,13 @@ static inline struct vcpu *mapcache_current_vcpu(void) return ACCESS_ONCE(this_cpu(pgtable_vcpu)); } =20 +/* Whether @v's page-tables carry the sparse view of the directmap. */ +static bool on_sparse_dmap(const struct vcpu *v) +{ + return IS_ENABLED(CONFIG_SPARSE_DIRECTMAP) && + v->domain->arch.asi.sparse_dmap; +} + /* * The accounting for v's mapcache: its own for a vCPU-PT domain, the doma= in's * otherwise, in which case *dcache is set so the caller takes the domain-= wide @@ -135,9 +142,19 @@ void *map_domain_page(mfn_t mfn) =20 #ifdef NDEBUG if ( mfn_x(mfn) <=3D PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) - return mfn_to_virt(mfn_x(mfn)); + { + const struct vcpu *pt_vcpu =3D this_cpu(pgtable_vcpu); + + /* + * The page-tables loaded map @mfn through the directmap, unless t= hey + * carry the sparse view. + */ + if ( !pt_vcpu || !on_sparse_dmap(pt_vcpu) ) + return mfn_to_virt(mfn_x(mfn)); + } #endif =20 + /* No vCPU (early boot), or no mapcache: the full directmap is loaded.= */ v =3D mapcache_current_vcpu(); if ( !v ) return mfn_to_virt(mfn_x(mfn)); @@ -145,7 +162,10 @@ void *map_domain_page(mfn_t mfn) cache =3D vcpu_mapcache(v, &dcache); vcache =3D &v->arch.mapcache; if ( !cache->inuse ) + { + ASSERT(!on_sparse_dmap(v)); return mfn_to_virt(mfn_x(mfn)); + } =20 perfc_incr(map_domain_page_count); =20 @@ -294,7 +314,11 @@ void mapcache_domain_init(struct domain *d) ASSERT(is_pv_domain(d)); =20 #ifdef NDEBUG - if ( !mem_hotplug && max_page <=3D PFN_DOWN(__pa(HYPERVISOR_VIRT_END -= 1)) ) + /* + * Not needed when the directmap loaded for the domain covers everythi= ng. + */ + if ( !d->arch.asi.sparse_dmap && !mem_hotplug && + max_page <=3D PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) return; #endif =20 @@ -378,8 +402,12 @@ void *map_domain_page_global(mfn_t mfn) system_state < SYS_STATE_active) || local_irq_is_enabled())); =20 + /* + * Global mappings must hold in every context: a sparse view of the + * directmap rules the directmap out. + */ #ifdef NDEBUG - if ( mfn_x(mfn) <=3D PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) return mfn_to_virt(mfn_x(mfn)); #endif =20 diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 7076e38848..f84ac15759 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -490,6 +490,9 @@ struct arch_domain struct { /* Give each vCPU its own per-domain area (HVM only). */ bool vcpu_pt:1; + + /* The domain's root page-tables carry the sparse directmap view. = */ + bool sparse_dmap:1; } asi; =20 /* Emulated devices enabled bitmap. */ --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748684; cv=none; d=zohomail.com; s=zohoarc; b=m1O2q0+vjvZybZXDeSvnwiVMFYDrchfjNDryJcdhLiY87AcRC3I2KZNLHpuNkHEs3DX585AhA6FEaEuRhD2akdBW1/REoE8nHmuPulqncSENzVaf9sR7ekxEQsjIK2ixH566nws6A9X+4yddMoEBLTlvj8fmTdGnZjv3hGpuqlw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748684; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=r1VmGidvH1AZLdhk+0ig3hYGT6yhxdDnEfjoDf/SEiU=; b=e3OKLgn3UuGV/Ha1x83sv54MVk7kA+BHkUbcXK/7sjqsnQkn/I87VeCsTfHQFx+QLMFHzEYujn6WN1CjNUTWppQANK3ikCz8pxjhRi6IFOL+4Awcg/KgvuIC0fHic7OPCeP2BstK5UldOgEiPB3X0YULHrCdNdisgVYcI/jX4Ac= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748684897318.5098806070632; Tue, 3 Nov 2026 15:31:24 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470810.1679593 (Exim 4.92) (envelope-from ) id 1xONy5-0006Cy-Be; Tue, 03 Nov 2026 23:31:05 +0000 Received: by outflank-mailman (output) from mailman id 1470810.1679593; Tue, 03 Nov 2026 23:31:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONy3-00062i-DD; Tue, 03 Nov 2026 23:31:03 +0000 Received: by outflank-mailman (input) for mailman id 1470810; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000dv-Gq for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Covq-1ubp; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONny-0000000EMFa-2t77; Tue, 03 Nov 2026 23:20:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=r1VmGidvH1AZLdhk+0ig3hYGT6yhxdDnEfjoDf/SEiU=; b=d9FXRJIUIaGrZetJr5AfUpk4yz Zmu4v3VghhJWbE2i/nNLQayqltm599DBQ5PA5cnze4SlLx/8/YeSDY/uKnCfZqKy2T0ueQDCI6ApC aCBRP6UXgplQQHzZBwG1akKDVWZ4yeNx8RkWeBwV+oNkGAVX+053X8w1zCf2sjwmigCA=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 19/44] x86/domain_page: clear a mapcache slot at its last unmap Date: Tue, 3 Nov 2026 17:13:41 -0500 Message-ID: <20261103-asi-hvm-v6-19-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748685680158500 Content-Type: text/plain; charset="utf-8" unmap_domain_page() keeps a recently unmapped slot in the vCPU's maphash (MAPHASH_ENTRIES, 8 per vCPU) for a quick remap of the same MFN, and leaves its PTE in place until the entry is displaced by a later unmap or reclaimed by a full cache. Until then the page stays mapped, although nothing refers to the mapping and the mapcache holds no reference to the page. Where the mapcache clears a slot's PTE, it defers the flush: the slot's translation can stay in the TLB of the CPU which used it until that CPU next flushes. While the full directmap is loaded, that reaches nothing it does not map anyway. On the sparse view of the directmap, Xen reaches domain-heap memory only through mappings made for the purpose. A mapping or a translation left behind keeps the page reachable after its last unmap: while Xen serves another of the guest's contexts on the same vCPU, and, once the page is freed and handed to another domain, from a context which has nothing to do with it. So clear a slot's PTE when its last reference goes, maphash slots included: the hash entry keeps the slot, not the mapping, and a later hit maps the page again in the same slot. The slot has mapped nothing else in between, so that needs no flush. Reusing the slot for another page still goes through the garbage map, or through the replacement of a hash entry when the cache is full, both of which flush first. In a per-vCPU mapcache, as vCPU-PT domains have, also invalidate the slot's translation on the local CPU, right after clearing the PTE. The slot has a second address, in the mirror of the per-domain area that every domain but a 64-bit PV one gets for the compat argument translation area, and its translation is invalidated too: two INVLPGs. That leaves no translation on any CPU, as the vCPU's page-tables are loaded on one CPU at a time, and a CPU loading other page-tables (switch_cr3_cr4()) flushes non-global translations, the mapcache's included. A PV domain's domain-wide mapcache has its PTEs cleared but no INVLPG, as CPUs running the domain's other vCPUs may hold the slot's translation too, which a local INVLPG would not reach. For it, as before, builds with PV support flush a page freed by its owner from TLBs before the page is handed out again. The cost is one PTE write per unmap into the maphash and per remap from it, and in a per-vCPU mapcache two INVLPGs per slot's last unmap. A second unmap of an entry in the maphash now finds its PTE not present and is caught by the check unmap_domain_page() has for entries the first unmap zapped, rather than by the reference count's ASSERT() in debug builds only. The change does not depend on asi=3D or on a sparse view of the directmap being configured: PV vCPUs use the domain-wide mapcache on the default command line too. For them it adds the PTE writes above, and leaves up to MAPHASH_ENTRIES fewer pages per vCPU mapped in the domain-wide mapcache. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v4: - Merge v3's 15/18 ("x86/mm: flush owned pages before reuse with the sparse directmap view") and 16/18 ("x86/domain_page: keep the maphash's slots, not their mappings"). Rather than flush pages freed by their owner before reuse wherever a sparse view is configured, invalidate a per-vCPU mapcache slot's translations on the local CPU right after its PTE is cleared (Roger). - Message: don't restate when PV vCPUs use the mapcache (Roger). Changes in v3: - New in this version. For reviewers: with a slot cleared and invalidated at its last unmap, a per-vCPU mapcache's maphash saves only the search for a free slot, and a second slot for a page mapped twice at once. One page in two slots is fine, and already happens when it is not in the maphash. So per-vCPU mapcaches could drop the maphash, and with it their garbage map and wrap-time flush; that is left for a follow-up. --- xen/arch/x86/domain_page.c | 53 +++++++++++++++++++++++++------ xen/arch/x86/include/asm/domain.h | 5 ++- 2 files changed, 48 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 6b662706fb..4198ee7c18 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -130,6 +130,26 @@ static void mapcache_exit(struct mapcache_domain *dcac= he) #define MAPCACHE_L1ENT(idx) \ __linear_l1_table[l1_linear_offset(MAPCACHE_VIRT_START + pfn_to_paddr(= idx))] =20 +/* + * Clear a slot's PTE. For a per-vCPU mapcache (HVM domains only), then a= lso + * drop the slot's translations from this CPU's TLB, at its address and at= its + * alias in the per-domain mirror, which every HVM domain has. No other C= PU + * can hold them: the vCPU's page-tables are loaded on one CPU at a time, = and + * loading other page-tables flushes non-global translations. + */ +static void mapcache_zap(const struct mapcache_domain *dcache, + unsigned int idx) +{ + unsigned long va =3D MAPCACHE_VIRT_START + pfn_to_paddr(idx); + + l1e_write(&MAPCACHE_L1ENT(idx), l1e_empty()); + if ( dcache ) + return; + + flush_tlb_one_local(va); + flush_tlb_one_local(va - PERDOMAIN_VIRT_START + PERDOMAIN_ALT_VIRT_STA= RT); +} + void *map_domain_page(mfn_t mfn) { unsigned long flags; @@ -176,6 +196,14 @@ void *map_domain_page(mfn_t mfn) { idx =3D hashent->idx; ASSERT(idx < cache->entries); + /* + * An entry the maphash kept holds on to its slot, not to its mapp= ing + * (see unmap_domain_page()): map the page again. No flush is nee= ded, + * as the slot has mapped nothing else since. + */ + if ( !hashent->refcnt ) + l1e_write(&MAPCACHE_L1ENT(idx), + l1e_from_mfn(mfn, __PAGE_HYPERVISOR_RW)); hashent->refcnt++; ASSERT(hashent->refcnt); ASSERT(mfn_eq(l1e_get_mfn(MAPCACHE_L1ENT(idx)), mfn)); @@ -208,8 +236,8 @@ void *map_domain_page(mfn_t mfn) if ( hashent->idx !=3D MAPHASHENT_NOTINUSE && !hashent->re= fcnt ) { idx =3D hashent->idx; - ASSERT(l1e_get_pfn(MAPCACHE_L1ENT(idx)) =3D=3D hashent= ->mfn); - l1e_write(&MAPCACHE_L1ENT(idx), l1e_empty()); + ASSERT(!(l1e_get_flags(MAPCACHE_L1ENT(idx)) & + _PAGE_PRESENT)); hashent->idx =3D MAPHASHENT_NOTINUSE; hashent->mfn =3D ~0UL; break; @@ -273,32 +301,39 @@ void unmap_domain_page(const void *ptr) =20 local_irq_save(flags); =20 + /* + * The maphash keeps a freed slot, not its mapping: the page may change + * hands meanwhile. A domain-wide mapcache's translation left in a TLB + * is covered by the flush the allocator makes before reusing an owned + * page. + */ if ( hashent->idx =3D=3D idx ) { ASSERT(hashent->mfn =3D=3D mfn); ASSERT(hashent->refcnt); hashent->refcnt--; + if ( !hashent->refcnt ) + mapcache_zap(dcache, idx); } else if ( !hashent->refcnt ) { + /* The entry replaced has no mapping left: mark its slot as garbag= e. */ if ( hashent->idx !=3D MAPHASHENT_NOTINUSE ) { - /* /First/, zap the PTE. */ - ASSERT(l1e_get_pfn(MAPCACHE_L1ENT(hashent->idx)) =3D=3D - hashent->mfn); - l1e_write(&MAPCACHE_L1ENT(hashent->idx), l1e_empty()); - /* /Second/, mark as garbage. */ + ASSERT(!(l1e_get_flags(MAPCACHE_L1ENT(hashent->idx)) & + _PAGE_PRESENT)); set_bit(hashent->idx, cache->garbage); } =20 - /* Add newly-freed mapping to the maphash. */ + /* Add the newly-freed slot to the maphash, without its mapping. */ + mapcache_zap(dcache, idx); hashent->mfn =3D mfn; hashent->idx =3D idx; } else { /* /First/, zap the PTE. */ - l1e_write(&MAPCACHE_L1ENT(idx), l1e_empty()); + mapcache_zap(dcache, idx); /* /Second/, mark as garbage. */ set_bit(idx, cache->garbage); } diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index f84ac15759..0e405157b4 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -74,7 +74,10 @@ struct mapcache_vcpu { /* Shadow of mapcache_domain.epoch. */ unsigned int shadow_epoch; =20 - /* Lock-free per-VCPU hash of recently-used mappings. */ + /* + * Lock-free per-VCPU hash of recently-used mappings. An entry without + * references keeps its slot but not its mapping (see unmap_domain_pag= e()). + */ struct vcpu_maphash_entry { unsigned long mfn; uint32_t idx; --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748685; cv=none; d=zohomail.com; s=zohoarc; b=Snm1Ak/MxziUaXJN237uikVKo2h5zUDEg+H0QDwiZBr5ljNMUnT6UFPne+eprqvVGdIEcnctZKAyMdhk3Mg2P/kyZ5J/wtTM7bBkLZdgCfPNHj5Lt/DZk56ItNPPM+0AMKvFoLXkw3tfbKbGRt1hiYKeQO2vYx+HeJKz60FiFm4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748685; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=q/qAmagFc51kjK9l6VNA/qd139u3x0RJhFQgQhsvBr0=; b=P1uggSkIglsLmvkakObeWwIyItdG1r9YSZcY6dVabLrrYSHpc+DafxLcr7RqPpPQi8jI8ZdExgOhLY5ZbyYxmorciR51+pTHpxKvgmeUkaMU7u11dsr3AzdW3m5E3vEiY9xigVoWi2FpV2iG2cWDuKZcLHcJHQCXczbuEmaQlX0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748685493978.5327713299438; Tue, 3 Nov 2026 15:31:25 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470809.1679584 (Exim 4.92) (envelope-from ) id 1xONy3-0005mO-Jc; Tue, 03 Nov 2026 23:31:03 +0000 Received: by outflank-mailman (output) from mailman id 1470809.1679584; Tue, 03 Nov 2026 23:31:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONy1-0005cJ-FR; Tue, 03 Nov 2026 23:31:01 +0000 Received: by outflank-mailman (input) for mailman id 1470809; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000e2-Hj for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CovU-1Q6V; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONnz-0000000EMFa-0qxf; Tue, 03 Nov 2026 23:20:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=q/qAmagFc51kjK9l6VNA/qd139u3x0RJhFQgQhsvBr0=; b=H0D515jYxLVoIp5Z8X3NjblJ3Z WyIh54JUx1ENVuehDiLvIXhgK8EiqTzF4wO7hk4xiklLTsu7BvH9nWOGH8aJRtRL44QNRPKwTWccp cRL63PE7PWz+akQrObCLp5nzMdjAtVY0nw/c3ang53KLEGiPjwFq5Kf1vD2dF+dtzwLY=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 20/44] x86/mm: run guest contexts on the sparse directmap view Date: Tue, 3 Nov 2026 17:13:42 -0500 Message-ID: <20261103-asi-hvm-v6-20-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748687686158500 Content-Type: text/plain; charset="utf-8" In _init_xen_l4_slots(), if d->arch.asi.sparse_dmap is set, point the directmap slots at the sparse view, copying from idle_pg_table only the Xen slots below them. A root page-table not shared with a PV guest gets no extended directmap, which the sparse view does not cover. d->arch.asi.sparse_dmap is decided at domain creation: PV domains all have a mapcache, HVM domains need per-vCPU page-tables for one. No option asks for the sparse view yet. The sparse view is built before any domain is created, so the root page-tables of every domain using it carry it from their creation, dom0's included. Dom0's builders reach dom0's memory only through guest virtual addresses and map_domain_page(). In PV shim mode, replace_va_mapping() reads the guest's L4 through the directmap; map it with map_domain_page() instead, as the lower levels already are. Shadow L4s, dom0=3Dshadow's included, get the sparse view from init_xen_l4_slots_pv(). The two directmaps differ where the sparse view leaves memory out, so no translation of the full directmap may remain usable on page-tables carrying the sparse view. Directmap mappings stay global nevertheless: every CR3 write which can move a CPU onto the sparse view goes through switch_cr3_cr4(), which flushes the old page-tables' translations, global entries included, before it returns. The CR3 writes which keep entries -- a PV vCPU's switch between its kernel and user page-tables, XPTI's entry and exit -- stay within one vCPU's page-tables, which carry the same directmap, or go to XPTI's per-CPU root, which maps only Xen's own per-CPU pages there. Removals from the sparse view are flushed on every CPU, global entries included. PV guests that run with global pages, among them a PV dom0 not using the sparse view, therefore keep their directmap TLB entries across switches between guest kernel and user mode. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v6: - Port onto new dual-init_xen_l4_slots system. Changes in v4: - d->arch.vcpu_pt and d->arch.sparse_dmap are now d->arch.asi.vcpu_pt and d->arch.asi.sparse_dmap. - Drop sparse_dmap_active and sparse_dmap_activate(): root page-tables carry the sparse view from their creation, dom0's included (Roger). - The sparse view's slots are now marked NX in "x86/mm: build and maintain a sparse view of the directmap". - PV shim: replace_va_mapping() maps the guest's L4 with map_domain_page(). - Give sparse_dmap_wanted() its hardware-domain case (Roger). - init_xen_l4_slots() takes the directmap slots from the sparse view, rather than copying the full directmap's and overwriting them; sparse_dmap_install() goes (Roger). - Message: drop the sentence on NX (Roger). Changes in v3: - New in this version. --- xen/arch/x86/include/asm/sparse-directmap.h | 9 +++++++ xen/arch/x86/mm.c | 28 +++++++++++++++++++-- xen/arch/x86/pv/shim.c | 4 ++- xen/arch/x86/spec_ctrl.c | 7 ++++++ 4 files changed, 45 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/include/asm/sparse-directmap.h b/xen/arch/x86/inc= lude/asm/sparse-directmap.h index ea14d44334..2f8b94f4c1 100644 --- a/xen/arch/x86/include/asm/sparse-directmap.h +++ b/xen/arch/x86/include/asm/sparse-directmap.h @@ -31,6 +31,14 @@ static inline bool sparse_dmap_configured(void) return opt_sparse_dmap_pv || opt_sparse_dmap_hvm || opt_sparse_dmap_hw= dom; } =20 +/* Whether a new domain of the given type is to use the sparse view. */ +static inline bool sparse_dmap_wanted(bool pv, bool hwdom) +{ + return sparse_dmap_root && + (hwdom ? opt_sparse_dmap_hwdom + : pv ? opt_sparse_dmap_pv : opt_sparse_dmap_hvm); +} + void sparse_dmap_init(void); int sparse_dmap_mirror_map(unsigned long virt, mfn_t mfn, unsigned long nr, pte_attr_t flags); @@ -42,6 +50,7 @@ int sparse_dmap_drop_pagetable(mfn_t mfn); #else /* !CONFIG_SPARSE_DIRECTMAP */ =20 static inline bool sparse_dmap_configured(void) { return false; } +static inline bool sparse_dmap_wanted(bool pv, bool hwdom) { return false;= } static inline void sparse_dmap_init(void) {} static inline int sparse_dmap_mirror_map(unsigned long virt, mfn_t mfn, unsigned long nr, pte_attr_t flag= s) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 0d475432ff..36a241a41c 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -1704,7 +1704,8 @@ struct page_info *perdomain_l3(const struct vcpu *v) * l4t, l4mfn, and d are mandatory, but l4mfn doesn't need to be the mfn u= nder * *l4t. All other parameters are optional and will either fill or zero t= he * appropriate slots. Pagetables not shared with guests will gain the - * extended directmap. + * extended directmap, unless the domain runs on the sparse view of the + * directmap, which replaces the directmap slots. */ static void _init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, const struct domain *d, @@ -1713,7 +1714,7 @@ static void _init_xen_l4_slots(l4_pgentry_t *l4t, mfn= _t l4mfn, { /* * PV vcpus need a shortened directmap. HVM and Idle vcpus get the fu= ll - * directmap. + * directmap, unless the domain runs on the sparse view (below). */ bool short_directmap =3D !paging_mode_external(d); =20 @@ -1748,6 +1749,29 @@ static void _init_xen_l4_slots(l4_pgentry_t *l4t, mf= n_t l4mfn, l4t[l4_table_offset(PERDOMAIN_VIRT_START)]; =20 /* Slot 261-: text/data/bss, RW M2P, vmap, frametable, directmap. */ +#ifdef CONFIG_SPARSE_DIRECTMAP + if ( d->arch.asi.sparse_dmap ) + { + unsigned int i =3D l4_table_offset(SPARSE_DMAP_START); + + memcpy(&l4t[l4_table_offset(XEN_VIRT_START)], + &idle_pg_table[l4_table_offset(XEN_VIRT_START)], + (i - l4_table_offset(XEN_VIRT_START)) * sizeof(*l4t)); + + /* + * Every slot the sparse view covers gets its L3, whatever the full + * directmap has there, so that memory added later is covered too.= A + * PV root has guest slots above; any other root loses the rest of= the + * directmap, which the sparse view does not cover. + */ + for ( ; i < l4_table_offset(SPARSE_DMAP_END); i++ ) + l4t[i] =3D sparse_dmap_root[i]; + if ( !short_directmap ) + for ( ; i < l4_table_offset(DIRECTMAP_VIRT_END); i++ ) + l4t[i] =3D l4e_empty(); + } + else +#endif #ifndef NDEBUG if ( short_directmap && unlikely(root_pgt_pv_xen_slots < ROOT_PAGETABLE_PV_XEN_SLOTS) ) diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c index 41000b0523..2a47dbc56c 100644 --- a/xen/arch/x86/pv/shim.c +++ b/xen/arch/x86/pv/shim.c @@ -148,12 +148,14 @@ const struct platform_bad_page *__init pv_shim_reserv= ed_pages(unsigned int *size static void __init replace_va_mapping(struct domain *d, l4_pgentry_t *l4st= art, unsigned long va, mfn_t mfn) { - l4_pgentry_t l4e =3D l4start[l4_table_offset(va)]; + const l4_pgentry_t *l4t =3D map_domain_page(virt_to_mfn(l4start)); + l4_pgentry_t l4e =3D l4t[l4_table_offset(va)]; l3_pgentry_t l3e =3D l3e_from_l4e(l4e, l3_table_offset(va)); l2_pgentry_t l2e =3D l2e_from_l3e(l3e, l2_table_offset(va)); l1_pgentry_t *pl1e =3D map_l1t_from_l2e(l2e) + l1_table_offset(va); struct page_info *page =3D mfn_to_page(l1e_get_mfn(*pl1e)); =20 + unmap_domain_page(l4t); put_page_and_type(page); =20 *pl1e =3D l1e_from_mfn(mfn, (!is_pv_32bit_domain(d) ? L1_PROT diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index 8797523a62..bcdcdf0476 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -1953,6 +1953,13 @@ void spec_ctrl_init_domain(struct domain *d) =20 d->arch.asi.vcpu_pt =3D !pv && (is_hardware_domain(d) ? opt_vcpu_pt_hw= dom : opt_vcpu_pt_hvm); + + /* + * The sparse directmap view needs a mapcache: PV domains all have one, + * HVM domains through per-vCPU page-tables. + */ + d->arch.asi.sparse_dmap =3D (pv || d->arch.asi.vcpu_pt) && + sparse_dmap_wanted(pv, is_hardware_domain(d)= ); } =20 void __init init_speculation_mitigations(void) --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748670; cv=none; d=zohomail.com; s=zohoarc; b=Tr52vMWAQO9Thmgnz+v5KsQvlX3peMahglMKF2VWU76GlgFQNAHv/EuUC67VZhcnNh0hZKaBhHgTrUqxs1eLa3sSKEB3yFGCZ9ztU9qsUxYx74tjfqaDeH8og1E621FGQjV7HGd0/3Tv6M0+ju6yDpwDsGXr35H69miKNiv7lBY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748670; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Wl4nV3ZTDHnsWL9MvyITa+VNEqVOjsoYMCNqs+DR/ME=; b=TURscuDP5h6ORDzY15cZuWQLPi7PNStiQVgST02wH0Cajeh/dejwhY9seIqe41wt3+sjBmV0wUgq/aXQ8FwaMoZSrvmndIZ0ldcNtrNv1OSemd2/pTI8QbKCwo/iKiHF+4pJU5oRl7zHdu0PtkefP4GW4SZXsHdCAF1SLxoDxZY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748670216293.2565029518346; Tue, 3 Nov 2026 15:31:10 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470793.1679442 (Exim 4.92) (envelope-from ) id 1xONxf-0001AL-Tq; Tue, 03 Nov 2026 23:30:39 +0000 Received: by outflank-mailman (output) from mailman id 1470793.1679442; Tue, 03 Nov 2026 23:30:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxf-00019S-HT; Tue, 03 Nov 2026 23:30:39 +0000 Received: by outflank-mailman (input) for mailman id 1470793; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxb-0000cT-VZ for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:35 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cot4-20Zm; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo0-0000000EMFa-0SDF; Tue, 03 Nov 2026 23:20:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=Wl4nV3ZTDHnsWL9MvyITa+VNEqVOjsoYMCNqs+DR/ME=; b=DCH67ffzT10GOdQvjyNrsUDtsk KqP+JueGJBLXpmb3KMbMH1q3a8vxAlgitOTu9jhWg+30vnk5tp9epw1w+/6JUIvF+ARAZhMGsavgz /xZl/mSzoPv3jXEyJ+s/7DCme3WxS2zyP9I+nVev9cjulsmEJwtM65fhOflBqrYDAVBs=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Stefano Stabellini , Teddy Astie Subject: [PATCH v6 21/44] x86/spec-ctrl: wire the sparse directmap view into the asi= option Date: Tue, 3 Nov 2026 17:13:43 -0500 Message-ID: <20261103-asi-hvm-v6-21-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748671649158500 Content-Type: text/plain; charset="utf-8" Let asi=3D choose which domains run on the sparse view of the directmap, when the hypervisor is built with CONFIG_SPARSE_DIRECTMAP: * `asi` and `asi=3D` enable or disable the sparse view, along with vCPU-PT, for every domain including the hardware domain, PV or PVH. HVM domains, a PVH hardware domain included, use the sparse view together with vCPU-PT, which gives them the mapcache it needs; PV domains all have one. A PV domain on the sparse view keeps its domain-wide mapcache and per-domain mappings: the sparse view separates it from other domains, not its vCPUs from each other. * `asi=3Dhvm=3D`, `asi=3Dpv=3D` and `asi=3Ddom0=3D` do t= he same for HVM or PV domains other than the hardware domain, or for the hardware domain, whatever its type: `asi=3D1,dom0=3D0` leaves the hardware domain out of ASI altogether. * A new `asi=3Ddirectmap=3D` selects the sparse view for all domains, as `asi=3Dvcpu-pt=3D` selects vCPU-PT; `vcpu-pt=3D` alone leaves t= he sparse view off. On a hypervisor built without CONFIG_SPARSE_DIRECTMAP the forms enabling all features leave the sparse view off, and `directmap=3D` is rejected as invalid, as compiled-out `spec-ctrl=3D` sub-options are. A domain type the build lacks is ignored, so that the sparse view is not built for nothing. * For now, a PV domain uses one of XPTI and the sparse view, not both. Enabled explicitly, for dom0, domUs or both, XPTI takes precedence: the PV domains it covers keep the full directmap, and the boot log says so. Left to its default, XPTI yields: it is disabled for the PV domains using the sparse view, with a boot-log warning where it would otherwise be enabled, as the sparse view does not replace it yet (it still maps the xenheap and Xen's stacks). Each domain's case is settled at its creation, and again if it becomes 32-bit, which XPTI does not cover: switch_compat() then puts a domain that XPTI kept off the sparse view on it, with the domain-wide mapcache, which release builds may not have set up for it. The boot log gains a line with the directmap mode, "sparse" once a sparse view is configured and "full" otherwise, and the "ASI features for ..." lines, one for PV domUs now among them, say which domains use the sparse view. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - Follow "x86/spec-ctrl: introduce Address Space Isolation command line option": opt_vcpu_pt_hwdom's name, the development warning in parse_asi(), IS_ENABLED() in the boot log; pv=3D now replaces the line saying it is ignored. Changes in v4: - Docs, comments and message: "domains" throughout, not "guests" (Roger). - Follow the rename of opt_vcpu_pt_hwdom to opt_pvh_vcpu_pt_hwdom and the move of the per-domain settings into d->arch.asi. - pv=3D and dom0=3D move to the patch introducing asi=3D, beside hvm=3D; opt_sparse_dmap_hwdom and sparse_dmap_wanted()'s hardware-domain case to the patches that add the other knobs and sparse_dmap_wanted() (Roger). - directmap=3D on a build without CONFIG_SPARSE_DIRECTMAP is rejected as invalid, as compiled-out spec-ctrl=3D sub-options are, rather than ignored with a warning (Roger). - A PV domain that XPTI kept off the sparse view goes on it when it becomes 32-bit, which ends XPTI; it used to end up with neither. Changes in v3: - New in this version. --- docs/misc/xen-command-line.pandoc | 30 +++++- xen/arch/x86/include/asm/sparse-directmap.h | 7 ++ xen/arch/x86/pv/domain.c | 16 +++ xen/arch/x86/spec_ctrl.c | 102 +++++++++++++++++--- 4 files changed, 138 insertions(+), 17 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index 839d06da26..092c858f34 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -203,7 +203,7 @@ to appropriate auditing by Xen. Argo is disabled by de= fault. buggy or malicious other domain spamming the ring. =20 ### asi (x86) -> `=3D List of [ , {pv,hvm,dom0}=3D, +> `=3D List of [ , {pv,hvm,dom0}=3D, directmap=3D, > vcpu-pt=3D | vcpu-pt=3Dhvm=3D ]` =20 > Default: `false` @@ -215,14 +215,16 @@ mitigations for certain speculative related attacks, = at the cost of mapping sensitive information on-demand. It might also offer some protection agai= nst unmitigated speculation-related attacks. =20 -The mechanisms currently implemented apply to HVM domains, including a PVH -hardware domain. PV domains, including a PV hardware domain, are unaffect= ed. +The mechanisms currently implemented are `vcpu-pt`, for HVM domains includ= ing +a PVH hardware domain, and `directmap`, for all domains including the hard= ware +domain, PV or PVH. The plain boolean enables or disables both, wherever t= hey +apply. =20 * `pv=3D`, `hvm=3D` and `dom0=3D` enable the features for PV domains or HV= M domains other than the hardware domain, or for the hardware domain, PV or PVH. = The plain boolean does all three: `asi=3D1,dom0=3D0` leaves the hardware dom= ain - out. The forms naming no domain type (an un-suffixed `vcpu-pt=3D`) - cover the hardware domain too. + out. The forms naming no domain type (an un-suffixed `vcpu-pt=3D`= , and + `directmap=3D`) cover the hardware domain too. =20 **WARNING: manual de-selection of enabled options will invalidate any protection offered by the feature. The fine grained options provided below @@ -231,6 +233,18 @@ are meant to be used for debugging purposes only.** * `vcpu-pt` gives each vCPU its own per-domain area: the per-domain slot of each vCPU's monitor table maps a region private to that vCPU. =20 +* `directmap` (only available when Xen is built with + `CONFIG_SPARSE_DIRECTMAP`) runs guest contexts on a sparse view of the + directmap, which maps the memory Xen needs to reach at all times (the Xen + heap, Xen's own image, firmware tables), while guest memory is mapped + transiently. This selection is system-wide; `pv=3D`, `hvm=3D` and `dom0= =3D` + make it for one domain type, along with the other features. HVM domains= use + the sparse view only together with `vcpu-pt`, and PV domains only without + XPTI, which yields to the sparse view unless enabled explicitly (see + `xpti`). The whole-feature forms (`asi`, `asi=3D`, and `pv=3D`, `= hvm=3D` + and `dom0=3D`) set this too, on builds that support it; on other builds = they + leave it off. + ### asid (x86) > `=3D ` =20 @@ -3132,6 +3146,12 @@ Meltdown for all domains. With `dom0` and `domu` it is possible to control page table isolation for dom0 or guest domains only. =20 +With `asi`, XPTI and the sparse directmap view (`asi=3Ddirectmap`) exclude= each +other for a PV domain. Enabled explicitly, as a whole or with `dom0` or +`domu`, XPTI keeps the PV domains it covers off the sparse view, and the b= oot +log says so. Left to its default, XPTI is disabled for the PV domains usi= ng +the sparse view, with a boot-log warning where it would otherwise be enabl= ed. + ### xsave (x86) > `=3D ` =20 diff --git a/xen/arch/x86/include/asm/sparse-directmap.h b/xen/arch/x86/inc= lude/asm/sparse-directmap.h index 2f8b94f4c1..db4df82215 100644 --- a/xen/arch/x86/include/asm/sparse-directmap.h +++ b/xen/arch/x86/include/asm/sparse-directmap.h @@ -31,6 +31,10 @@ static inline bool sparse_dmap_configured(void) return opt_sparse_dmap_pv || opt_sparse_dmap_hvm || opt_sparse_dmap_hw= dom; } =20 +static inline bool sparse_dmap_pv(void) { return opt_sparse_dmap_pv; } +static inline bool sparse_dmap_hvm(void) { return opt_sparse_dmap_hvm; } +static inline bool sparse_dmap_hwdom(void) { return opt_sparse_dmap_hwdom;= } + /* Whether a new domain of the given type is to use the sparse view. */ static inline bool sparse_dmap_wanted(bool pv, bool hwdom) { @@ -51,6 +55,9 @@ int sparse_dmap_drop_pagetable(mfn_t mfn); =20 static inline bool sparse_dmap_configured(void) { return false; } static inline bool sparse_dmap_wanted(bool pv, bool hwdom) { return false;= } +static inline bool sparse_dmap_pv(void) { return false; } +static inline bool sparse_dmap_hvm(void) { return false; } +static inline bool sparse_dmap_hwdom(void) { return false; } static inline void sparse_dmap_init(void) {} static inline int sparse_dmap_mirror_map(unsigned long virt, mfn_t mfn, unsigned long nr, pte_attr_t flag= s) diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index f139d8f83c..579d6133f5 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -248,6 +248,20 @@ int switch_compat(struct domain *d) d->arch.has_32bit_shinfo =3D 1; d->arch.pv.is_32bit =3D true; =20 + /* + * XPTI, cleared below, is for 64-bit domains. A domain it kept off t= he + * sparse directmap view goes on it now if wanted, and the domain-wide + * mapcache the view needs is set up for the domain and its vCPUs. + */ + if ( d->arch.pv.xpti && sparse_dmap_wanted(true, is_hardware_domain(d)= ) ) + { + d->arch.asi.sparse_dmap =3D true; + mapcache_domain_init(d); + for_each_vcpu ( d, v ) + if ( (rc =3D mapcache_vcpu_init(v)) ) + goto undo_and_fail; + } + /* * For 32bit PV guests the shared_info machine address must fit in a 3= 2-bit * field within the guest's start_info structure. We might need to fr= ee @@ -301,6 +315,8 @@ int switch_compat(struct domain *d) =20 undo_and_fail: d->arch.pv.is_32bit =3D d->arch.has_32bit_shinfo =3D false; + if ( d->arch.pv.xpti ) + d->arch.asi.sparse_dmap =3D false; for_each_vcpu( d, v ) { free_compat_arg_xlat(v); diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index bcdcdf0476..c86f5ff886 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -387,6 +387,19 @@ int8_t __ro_after_init opt_xpti_domu =3D -1; =20 static __init void xpti_init_default(void) { + /* + * XPTI and the sparse directmap view exclude each other for a PV doma= in + * (see spec_ctrl_init_domain()). Enabled explicitly, XPTI takes + * precedence: the PV domains it covers stay off the sparse view. Lef= t to + * its default, it yields: it is off where the sparse view is asked fo= r. + */ + if ( !opt_dom0_pvh && opt_xpti_hwdom =3D=3D 1 && sparse_dmap_hwdom() ) + printk(XENLOG_ERR + "XPTI enabled, disabling Dom0 sparse-directmap\n"); + if ( opt_xpti_domu =3D=3D 1 && sparse_dmap_pv() ) + printk(XENLOG_ERR + "XPTI enabled, disabling PV DomU sparse-directmap\n"); + if ( (boot_cpu_data.vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON)) || cpu_has_rdcl_no ) { @@ -397,10 +410,21 @@ static __init void xpti_init_default(void) } else { + bool yield_hwdom =3D opt_xpti_hwdom < 0 && !opt_dom0_pvh && + sparse_dmap_hwdom(); + bool yield_domu =3D opt_xpti_domu < 0 && sparse_dmap_pv(); + + /* The default would be on here: say where it yields. */ + if ( yield_hwdom || yield_domu ) + printk(XENLOG_WARNING + "XPTI defaults off for PV %s on the sparse directmap\n", + yield_hwdom ? (yield_domu ? "Dom0 and DomU" : "Dom0") + : "DomU"); + if ( opt_xpti_hwdom < 0 ) - opt_xpti_hwdom =3D 1; + opt_xpti_hwdom =3D !sparse_dmap_hwdom(); if ( opt_xpti_domu < 0 ) - opt_xpti_domu =3D 1; + opt_xpti_domu =3D !sparse_dmap_pv(); } } =20 @@ -491,6 +515,23 @@ static int __init cf_check parse_pv_l1tf(const char *s) } custom_param("pv-l1tf", parse_pv_l1tf); =20 +/* + * Set the sparse directmap view's knob for each domain type given. A type + * the build lacks stays off, so that no view is built for it, and without + * CONFIG_SPARSE_DIRECTMAP all stay off. + */ +static void __init set_sparse_dmap(bool pv, bool hvm, bool hwdom, bool val) +{ +#ifdef CONFIG_SPARSE_DIRECTMAP + if ( pv && IS_ENABLED(CONFIG_PV) ) + opt_sparse_dmap_pv =3D val; + if ( hvm && IS_ENABLED(CONFIG_HVM) ) + opt_sparse_dmap_hvm =3D val; + if ( hwdom ) + opt_sparse_dmap_hwdom =3D val; +#endif +} + static int __init cf_check parse_asi(const char *s) { static bool __initdata warned; @@ -507,7 +548,10 @@ static int __init cf_check parse_asi(const char *s) =20 /* Interpret 'asi' alone in its positive boolean form. */ if ( *s =3D=3D '\0' ) + { opt_vcpu_pt_hwdom =3D opt_vcpu_pt_hvm =3D true; + set_sparse_dmap(true, true, true, true); + } =20 do { ss =3D strchr(s, ','); @@ -520,17 +564,33 @@ static int __init cf_check parse_asi(const char *s) case 0: case 1: opt_vcpu_pt_hwdom =3D opt_vcpu_pt_hvm =3D val; + set_sparse_dmap(true, true, true, val); break; =20 default: if ( (val =3D parse_boolean("pv", s, ss)) >=3D 0 ) - printk(XENLOG_WARNING - "No ASI feature applies to PV domains - ignoring 'a= si=3D%.*s' setting\n", - (int)(ss - s), s); + set_sparse_dmap(true, false, false, val); else if ( (val =3D parse_boolean("hvm", s, ss)) >=3D 0 ) + { opt_vcpu_pt_hvm =3D val; + set_sparse_dmap(false, true, false, val); + } else if ( (val =3D parse_boolean("dom0", s, ss)) >=3D 0 ) + { opt_vcpu_pt_hwdom =3D val; + set_sparse_dmap(false, false, true, val); + } + else if ( (val =3D parse_boolean("directmap", s, ss)) >=3D 0 ) + { + /* System-wide: every domain type, the hardware domain too= . */ + if ( IS_ENABLED(CONFIG_SPARSE_DIRECTMAP) ) + set_sparse_dmap(true, true, true, val); + else + { + no_config_param("SPARSE_DIRECTMAP", "asi", s, ss); + rc =3D -EINVAL; + } + } else if ( (val =3D parse_boolean("vcpu-pt", s, ss)) !=3D -1 ) { switch ( val ) @@ -759,11 +819,28 @@ static void __init print_details(enum ind_thunk thunk) opt_pv_l1tf_domu ? "enabled" : "disabled"); #endif =20 - printk(" ASI features for Dom0:%s\n", - opt_dom0_pvh && opt_vcpu_pt_hwdom ? " vCPU-PT" : " None"); - if ( IS_ENABLED(CONFIG_HVM) ) - printk(" ASI features for HVM VMs:%s\n", - opt_vcpu_pt_hvm ? " vCPU-PT" : " None"); + { + bool hwdom_pt =3D opt_dom0_pvh && opt_vcpu_pt_hwdom; + bool hwdom_sd =3D sparse_dmap_hwdom() && + (opt_dom0_pvh ? hwdom_pt : !opt_xpti_hwdom); + + printk(" ASI directmap mode: %s\n", + sparse_dmap_configured() ? "sparse" : "full"); + printk(" ASI features for Dom0:%s%s%s\n", + hwdom_pt || hwdom_sd ? "" : " None", + hwdom_pt ? " vCPU-PT" : "", + hwdom_sd ? " sparse-directmap" : ""); + if ( IS_ENABLED(CONFIG_HVM) ) + printk(" ASI features for HVM VMs:%s%s%s\n", + opt_vcpu_pt_hvm ? "" : " None", + opt_vcpu_pt_hvm ? " vCPU-PT" : "", + opt_vcpu_pt_hvm && sparse_dmap_hvm() + ? " sparse-directmap" : ""); + if ( IS_ENABLED(CONFIG_PV) ) + printk(" ASI features for PV VMs:%s\n", + sparse_dmap_pv() && !opt_xpti_domu + ? " sparse-directmap" : " None"); + } } =20 static bool __init check_smt_enabled(void) @@ -1956,9 +2033,10 @@ void spec_ctrl_init_domain(struct domain *d) =20 /* * The sparse directmap view needs a mapcache: PV domains all have one, - * HVM domains through per-vCPU page-tables. + * HVM domains through per-vCPU page-tables. A PV domain using XPTI + * stays on the full directmap. */ - d->arch.asi.sparse_dmap =3D (pv || d->arch.asi.vcpu_pt) && + d->arch.asi.sparse_dmap =3D (pv ? !d->arch.pv.xpti : d->arch.asi.vcpu_= pt) && sparse_dmap_wanted(pv, is_hardware_domain(d)= ); } =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748706; cv=none; d=zohomail.com; s=zohoarc; b=D65SYDZov2/loGuN4jPxnVfP1agy2dBPUs4Pdd4DCch50OI9+xN/ipqIGyTJL0jxIRtXBWYoLS7sIhFocfHhfKPaWN7u1lZjXpLWHjoFsRaSMB1hn76dQqp81szzGMRPLOLnyaQdUYMbB3X3MiX22HlUYwzCE6/TUHivx1Chqag= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748706; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lmWRg/5OLo0R/NPywMyA45Uq6cr7VvMb3O4KA5pNTBc=; b=n8z22wu60dJ0WoVY1dydBfn41On8oibGapWd+ULtdL7lTf0Ral0eGb7ZgbCLDFi2yinBe7nDsfn9aZuDiOZMAKXLQf1TwsCQEjA9pdX+6TurOjMnfjivqvCBr7Lm50mrqGLjjvweKmVRTXSPm1R7P/gnXuWwDy0bXVu6xRNEWmA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748706550646.0080581209385; Tue, 3 Nov 2026 15:31:46 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470821.1679672 (Exim 4.92) (envelope-from ) id 1xONyQ-0002Ie-Af; Tue, 03 Nov 2026 23:31:26 +0000 Received: by outflank-mailman (output) from mailman id 1470821.1679672; Tue, 03 Nov 2026 23:31:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyO-00029Z-Eq; Tue, 03 Nov 2026 23:31:24 +0000 Received: by outflank-mailman (input) for mailman id 1470821; Tue, 03 Nov 2026 23:30:42 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxh-0001if-Jj for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:41 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Covy-22qa; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo0-0000000EMFa-3A1t; Tue, 03 Nov 2026 23:20:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=lmWRg/5OLo0R/NPywMyA45Uq6cr7VvMb3O4KA5pNTBc=; b=hxD5jd6dH1zu9XMJu3d5HMrRd9 q++HMQfq0fOpxLNOdCkQR1KaU829Zgm8HmlGa8txftQJkMKSJmy+6ILhfG+xw50/yMIjfTgc0cQ5I yJrBhVxbpMwgnqlPK28xDknAHSm6erqNOCi96IvU7Rjm2lQ2kgeoe05YgGUZSujpU2n4=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Stefano Stabellini Subject: [PATCH v6 22/44] xen/xmalloc: return an emptied region after dropping the pool lock Date: Tue, 3 Nov 2026 17:13:44 -0500 Message-ID: <20261103-asi-hvm-v6-22-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748707681158500 Content-Type: text/plain; charset="utf-8" When a free leaves a whole region of an xmalloc pool empty, xmem_pool_free() hands the region back through pool->put_mem() with pool->lock still held. For the xenheap pool that is free_xenheap_pages(), which takes the heap lock inside the pool lock and runs arch_xenheap_pages_free(). On x86 with a sparse directmap view configured, the latter takes the page out of the sparse view, at the cost of one to three rounds of TLB-flush IPIs to every CPU, while every other CPU's sub-page xmalloc() and xfree() waits on the pool lock. By then the region is on no free list and no longer counted in the pool, so nothing else can reach it. Drop the lock first and return the region afterwards, as xmem_pool_alloc() already does when it calls pool->get_mem(), and say at both callbacks that they are called without the pool lock, so that they keep not relying on it. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- xen/common/xmalloc_tlsf.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/xen/common/xmalloc_tlsf.c b/xen/common/xmalloc_tlsf.c index b43219b4af..caf33f1a04 100644 --- a/xen/common/xmalloc_tlsf.c +++ b/xen/common/xmalloc_tlsf.c @@ -106,7 +106,10 @@ struct xmem_pool { unsigned long used_size; unsigned long num_regions; =20 - /* User provided functions for expanding/shrinking pool */ + /* + * User provided functions for expanding/shrinking pool. They are + * called without the pool lock held, and must not rely on it. + */ xmem_pool_get_memory *get_mem; xmem_pool_put_memory *put_mem; =20 @@ -488,17 +491,18 @@ void xmem_pool_free(void *ptr, struct xmem_pool *pool) =20 if ( (b->prev_hdr =3D=3D NULL) && ((tmp_b->size & BLOCK_SIZE_MASK) =3D= =3D 0) ) { - pool->put_mem(b); pool->num_regions--; pool->used_size -=3D BHDR_OVERHEAD; /* sentinel block header */ - goto out; + spin_unlock(&pool->lock); + pool->put_mem(b); + return; } =20 INSERT_BLOCK(b, pool, fl, sl); =20 tmp_b->size |=3D PREV_FREE; tmp_b->prev_hdr =3D b; - out: + spin_unlock(&pool->lock); } =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748706; cv=none; d=zohomail.com; s=zohoarc; b=HMvys8we6HHUpCRf4o5n8mMkiW1pVSdCV4R7oKZAHCnKLUrVca+d2EDAD1UWKZyTG02aaTDj+pOPgOJZ6QmGlpVZFGSE7HuifiiOpqgXBYWvufRGC47CykOS6OOvTRR1vBXXjj0KKq5+iZTKswRnR6NE2+a6spHL2Xe3gFdRttI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748706; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Tc/jXlrPlGsmLRuq65Xv7tixNRP0XkPfYKeLggB3t5w=; b=dNBjYWfIiaMwm251vAwUOMo2alPja6ibZtKgCOrWbodSJxtYokg+WDISNzga52INRstL3wwNDr6EEnEKIHHe2rfwjNEOvefr7MQoUI99CVSvb7Ksa03ZrdJXrEo8gf4VZDa8qm5QVOmqOL5zE8lYjVqFQ00vTvtQTv3nJQmnX/A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748706459861.57635345483; Tue, 3 Nov 2026 15:31:46 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470817.1679639 (Exim 4.92) (envelope-from ) id 1xONyH-0000b0-ND; Tue, 03 Nov 2026 23:31:17 +0000 Received: by outflank-mailman (output) from mailman id 1470817.1679639; Tue, 03 Nov 2026 23:31:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyF-0000RM-Qi; Tue, 03 Nov 2026 23:31:15 +0000 Received: by outflank-mailman (input) for mailman id 1470817; Tue, 03 Nov 2026 23:30:41 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxg-0001VG-SO for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:40 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotC-2eSg; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo1-0000000EMFa-11KK; Tue, 03 Nov 2026 23:20:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=Tc/jXlrPlGsmLRuq65Xv7tixNRP0XkPfYKeLggB3t5w=; b=f3GROwuGazB+8axEW19HJyW0zn Z3mo3hXlBvI573/q1/wsQTyK9bi7a7dO+ofJ0acKPOq+pgntfxT0IcUJLKWf01lQc0QdTlpoo4s1D taJC27B0OTNhOUQioYT18177bRgKQlW6sm74at3zZg5a93ygdRhcYr4gmyS38eiuNCFE=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v6 23/44] x86/mm: split a sparse directmap superpage with a page being freed Date: Tue, 3 Nov 2026 17:13:45 -0500 Message-ID: <20261103-asi-hvm-v6-23-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748707768158500 Content-Type: text/plain; charset="utf-8" A xenheap free takes its pages out of the sparse directmap view. Where a superpage covers them, the superpage has to be split, and modify_mappings_in() allocates the page-table page for that. The sparse view has such superpages wherever a xenheap block of 2MiB or more was mapped whole, and wherever map_pages_in() merged the entries of a 2MiB region into one, as it does in the full directmap, once all 512 pages were mapped. The allocation can fail once the hardware domain exists, and the free then keeps its pages out of the heap for good, with a warning. Use one of the pages being freed instead. The sparse view's free hook makes the first of them what alloc_xen_pagetable() would return, and passes it to modify_mappings_in(), which uses it, should a split need a page, before allocating one. The new L1 maps the rest of the superpage, and the page's own entry is cleared along with the others being freed. The hook tells free_xenheap_pages() whether it kept the page, and free_xenheap_pages() then frees the rest of the block as blocks of orders 0 to order - 1. The page goes back to the heap once the L1 has emptied, as any page-table page of the sparse view does. One page is enough if the sparse view has no 1GiB pages: a block smaller than 2MiB lies within one 2MiB page and needs one L1, and a larger block covers whole 2MiB pages and needs none, where a block within a 1GiB page could need an L2 as well. So map_pages_in() makes 1GiB pages, by mapping or by merging, only in idle_pg_table's hierarchy; the sparse view keeps its 2MiB pages, mapped and merged. A xenheap free then allocates nothing in the sparse view. The costs: a 2MiB region split this way can merge again only once it has emptied, as its L1 is one of its own pages; where the sparse view maps a whole 1GiB region, which takes memory never handed to the heap and xenheap memory filling it, contexts on it use 2MiB pages instead of one 1GiB page; and arch_xenheap_pages_free() gains a parameter. The other removals from the sparse view (memory handed to the heap, idle_pg_table's boot-time page-tables, and the undo of a failed xenheap allocation) still allocate when they split a superpage. The sparse view is correct without this, as a free that cannot split its superpage fails safe. It is kept apart from "x86/mm: build and maintain a sparse view of the directmap" so that it can be weighed on its own. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New patch This is a fairly complicated patch to handle a very niche corner case: - All the 4k pages in a superpage happen to have been allocated into the xenheap (and the page table automatically merged into a superpage) - The system is completely out of singleton pages to allocate as pagetables - A free_xenheap_page() is called on a subset of those pages Without this patch, the freed page is leaked; with this patch, one of the freed pages is used as a pagetable page instead. Happy to drop this patch, or move it further to the end of the series, if it helps the main bit of functionality get in. --- xen/arch/x86/include/asm/mm.h | 2 +- xen/arch/x86/mm.c | 34 ++++++++++++++++++++++------ xen/arch/x86/sparse-directmap.c | 40 ++++++++++++++++++++++++++------- xen/common/page_alloc.c | 14 ++++++++---- xen/include/xen/mm.h | 11 ++++++--- 5 files changed, 78 insertions(+), 23 deletions(-) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 113ade0bbd..50ace61ce7 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -603,7 +603,7 @@ void *alloc_mapped_pagetable(mfn_t *pmfn); int map_pages_in(l4_pgentry_t *root, unsigned long virt, mfn_t mfn, unsigned long nr_mfns, pte_attr_t flags); int modify_mappings_in(l4_pgentry_t *root, unsigned long s, unsigned long = e, - pte_attr_t nf); + pte_attr_t nf, mfn_t *spare); #ifdef CONFIG_SPARSE_DIRECTMAP mfn_t xen_mapping_lookup(l4_pgentry_t *root, unsigned long va, pte_attr_t *flags, unsigned long *nr); diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 36a241a41c..e1a80e9a5c 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5558,6 +5558,10 @@ static void free_pagetable_in(const l4_pgentry_t *ro= ot, mfn_t mfn) /* * map_pages_to_xen() in the hierarchy rooted at @root: idle_pg_table's, or * another hierarchy sharing its virtual addresses. + * + * Only idle_pg_table's hierarchy gets 1GiB pages: a xenheap free splits a + * superpage of the sparse directmap view with one of the pages it frees, + * which is enough for a 2MiB page only. */ int map_pages_in( l4_pgentry_t *root, @@ -5567,6 +5571,7 @@ int map_pages_in( pte_attr_t flags) { bool locking =3D system_state > SYS_STATE_boot; + bool page1gb =3D cpu_has_page1gb && root =3D=3D idle_pg_table; l3_pgentry_t *pl3e =3D NULL, ol3e; l2_pgentry_t *pl2e =3D NULL, ol2e; l1_pgentry_t *pl1e, ol1e; @@ -5619,7 +5624,7 @@ int map_pages_in( L3T_LOCK(current_l3page); ol3e =3D *pl3e; =20 - if ( cpu_has_page1gb && + if ( page1gb && IS_L3E_ALIGNED(virt, flags & _PAGE_PRESENT ? mfn : _mfn(0)) && nr_mfns >=3D (1UL << (L3_PAGETABLE_SHIFT - PAGE_SHIFT)) && !(flags & (_PAGE_PAT | MAP_SMALL_PAGES)) ) @@ -5905,7 +5910,7 @@ int map_pages_in( } =20 check_l3: - if ( cpu_has_page1gb && + if ( page1gb && (flags =3D=3D PAGE_HYPERVISOR) && ((nr_mfns =3D=3D 0) || IS_L3E_ALIGNED(virt, mfn)) ) { @@ -5984,6 +5989,18 @@ int __init populate_pt_range(unsigned long virt, uns= igned long nr_mfns) return map_pages_to_xen(virt, INVALID_MFN, nr_mfns, MAP_SMALL_PAGES); } =20 +static mfn_t take_or_alloc_pagetable(mfn_t *spare) +{ + mfn_t mfn =3D spare ? *spare : INVALID_MFN; + + if ( mfn_eq(mfn, INVALID_MFN) ) + return alloc_xen_pagetable(); + + *spare =3D INVALID_MFN; + + return mfn; +} + /* * Alter the permissions of a range of Xen virtual address space. * @@ -5997,10 +6014,13 @@ int __init populate_pt_range(unsigned long virt, un= signed long nr_mfns) * It is an error to call with present flags over an unpopulated range. * * modify_mappings_in() does so in the hierarchy rooted at @root, as - * map_pages_in() does. + * map_pages_in() does. @spare, if not NULL, may hold a page the caller + * gives up for a page-table page, which shattering a superpage then uses + * before allocating one. *@spare reads INVALID_MFN once it has been used, + * and the page is then no longer the caller's. */ int modify_mappings_in(l4_pgentry_t *root, unsigned long s, - unsigned long e, pte_attr_t nf) + unsigned long e, pte_attr_t nf, mfn_t *spare) { bool locking =3D system_state > SYS_STATE_boot; l3_pgentry_t *pl3e =3D NULL; @@ -6064,7 +6084,7 @@ int modify_mappings_in(l4_pgentry_t *root, unsigned l= ong s, } =20 /* PAGE1GB: shatter the superpage and fall through. */ - l2mfn =3D alloc_xen_pagetable(); + l2mfn =3D take_or_alloc_pagetable(spare); if ( mfn_eq(l2mfn, INVALID_MFN) ) goto out; =20 @@ -6122,7 +6142,7 @@ int modify_mappings_in(l4_pgentry_t *root, unsigned l= ong s, { l1_pgentry_t *l1t; /* PSE: shatter the superpage and try again. */ - mfn_t l1mfn =3D alloc_xen_pagetable(); + mfn_t l1mfn =3D take_or_alloc_pagetable(spare); =20 if ( mfn_eq(l1mfn, INVALID_MFN) ) goto out; @@ -6280,7 +6300,7 @@ int modify_mappings_in(l4_pgentry_t *root, unsigned l= ong s, =20 int modify_xen_mappings(unsigned long s, unsigned long e, pte_attr_t nf) { - int rc =3D modify_mappings_in(idle_pg_table, s, e, nf); + int rc =3D modify_mappings_in(idle_pg_table, s, e, nf, NULL); =20 if ( !rc ) rc =3D sparse_dmap_mirror_modify(s, e, nf); diff --git a/xen/arch/x86/sparse-directmap.c b/xen/arch/x86/sparse-directma= p.c index d35a8d6979..adec474f72 100644 --- a/xen/arch/x86/sparse-directmap.c +++ b/xen/arch/x86/sparse-directmap.c @@ -121,12 +121,13 @@ static int add_to_sparse_view(l4_pgentry_t *root, uns= igned long mfn, return 0; } =20 -static int remove_from_sparse_view(unsigned long mfn, unsigned long nr) +static int remove_from_sparse_view(unsigned long mfn, unsigned long nr, + mfn_t *spare) { unsigned long va =3D (unsigned long)mfn_to_virt(mfn); =20 return modify_mappings_in(sparse_dmap_root, va, va + (nr << PAGE_SHIFT= ), - _PAGE_NONE); + _PAGE_NONE, spare); } =20 /* Memory is about to be handed to the heap allocator. */ @@ -144,7 +145,7 @@ bool arch_heap_pages_init(mfn_t mfn, unsigned long nr) return true; } =20 - if ( remove_from_sparse_view(s, nr) ) + if ( remove_from_sparse_view(s, nr, NULL) ) { printk(XENLOG_ERR "sparse directmap: cannot unmap MFNs %#lx-%#lx, not freeing= \n", @@ -188,10 +189,18 @@ int arch_xenheap_pages_alloc(mfn_t mfn, unsigned int = order) /* * Xenheap pages are about to be freed. A failure means the pages may sti= ll * be reachable through the sparse view, and must not be reused. + * + * If the caller allows, the first page serves as the page-table page that + * splitting a superpage of the sparse view needs: the new L1 maps the rest + * of the superpage, and the page leaves the sparse view with the others. + * One page is enough: the sparse view has no 1GiB pages, a block below 2M= iB + * lies within one 2MiB page, and a larger one covers whole ones. */ -int arch_xenheap_pages_free(mfn_t mfn, unsigned int order) +int arch_xenheap_pages_free(mfn_t mfn, unsigned int order, bool *first_kep= t) { unsigned long s =3D mfn_x(mfn), nr =3D 1UL << order; + mfn_t spare =3D INVALID_MFN; + int rc; =20 if ( !sparse_dmap_configured() ) return 0; @@ -204,7 +213,22 @@ int arch_xenheap_pages_free(mfn_t mfn, unsigned int or= der) return 0; } =20 - return remove_from_sparse_view(s, nr); + if ( first_kept ) + { + struct page_info *pg =3D mfn_to_page(mfn); + + /* Made what alloc_xen_pagetable() would return, in case it is use= d. */ + pg->count_info &=3D ~PGC_xen_heap; + page_set_owner(pg, NULL); + spare =3D mfn; + } + + rc =3D remove_from_sparse_view(s, nr, &spare); + + if ( first_kept && mfn_eq(spare, INVALID_MFN) ) + *first_kept =3D true; + + return rc; } =20 /* Memory leaves Xen's hands without passing through the heap. */ @@ -233,7 +257,7 @@ int sparse_dmap_drop_pagetable(mfn_t mfn) INVALID_MFN) ) return 0; =20 - rc =3D remove_from_sparse_view(s, 1); + rc =3D remove_from_sparse_view(s, 1, NULL); if ( rc ) printk(XENLOG_WARNING "Leaking page-table page %"PRI_mfn": still in the sparse di= rectmap\n", @@ -267,7 +291,7 @@ int sparse_dmap_mirror_modify(unsigned long s, unsigned= long e, e =3D min(e, SPARSE_DMAP_END); =20 if ( !(nf & _PAGE_PRESENT) ) - return modify_mappings_in(sparse_dmap_root, s, e, nf); + return modify_mappings_in(sparse_dmap_root, s, e, nf, NULL); =20 /* * A permission change applies where the sparse view maps anything: it @@ -283,7 +307,7 @@ int sparse_dmap_mirror_modify(unsigned long s, unsigned= long e, n =3D min(n, (e - va) >> PAGE_SHIFT); if ( !mfn_eq(mfn, INVALID_MFN) && (rc =3D modify_mappings_in(sparse_dmap_root, va, - va + (n << PAGE_SHIFT), nf)) ) + va + (n << PAGE_SHIFT), nf, NULL)) ) return rc; va +=3D n << PAGE_SHIFT; } diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 2150dcbd5e..349de3ddf3 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -2561,7 +2561,7 @@ void *alloc_xenheap_pages(unsigned int order, unsigne= d int memflags) * The hook may have mapped part of the pages before failing: take * them out again as a free would, and leak them if that fails too. */ - if ( arch_xenheap_pages_free(page_to_mfn(pg), order) ) + if ( arch_xenheap_pages_free(page_to_mfn(pg), order, NULL) ) printk(XENLOG_WARNING "Leaking xenheap pages %"PRI_mfn"+%u: still mapped\n", mfn_x(page_to_mfn(pg)), 1U << order); @@ -2580,6 +2580,7 @@ void free_xenheap_pages(void *v, unsigned int order) { struct page_info *pg; unsigned int i; + bool first_kept =3D false; =20 ASSERT_ALLOC_CONTEXT(); =20 @@ -2593,7 +2594,7 @@ void free_xenheap_pages(void *v, unsigned int order) * they may remain reachable through it: leak them rather than hand th= em * out. */ - if ( arch_xenheap_pages_free(page_to_mfn(pg), order) ) + if ( arch_xenheap_pages_free(page_to_mfn(pg), order, &first_kept) ) { printk(XENLOG_WARNING "Leaking xenheap pages %"PRI_mfn"+%u: still mapped\n", @@ -2601,10 +2602,15 @@ void free_xenheap_pages(void *v, unsigned int order) return; } =20 - for ( i =3D 0; i < (1u << order); i++ ) + for ( i =3D first_kept ? 1 : 0; i < (1u << order); i++ ) pg[i].count_info &=3D ~PGC_xen_heap; =20 - free_heap_pages(pg, order, true); + if ( !first_kept ) + free_heap_pages(pg, order, true); + else + /* The rest of the block, as the blocks of orders 0 to order - 1. = */ + for ( i =3D 0; i < order; i++ ) + free_heap_pages(pg + (1u << i), i, true); } =20 #endif /* CONFIG_SEPARATE_XENHEAP */ diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index 2bcc45cf58..debd511caf 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -108,8 +108,12 @@ bool arch_xenheap_needs_scrub(void); * pages go through arch_xenheap_pages_free() before being freed again. */ int arch_xenheap_pages_alloc(mfn_t mfn, unsigned int order); -/* Xenheap pages are to be freed. Non-zero: failed, don't reuse them. */ -int arch_xenheap_pages_free(mfn_t mfn, unsigned int order); +/* + * Xenheap pages are to be freed. Non-zero: failed, don't reuse them. Wi= th + * @first_kept not NULL, the arch may keep the first page for itself, and + * then sets *@first_kept to true: that page is not the caller's to free. + */ +int arch_xenheap_pages_free(mfn_t mfn, unsigned int order, bool *first_kep= t); #else static inline bool arch_heap_pages_init(mfn_t mfn, unsigned long nr) { @@ -123,7 +127,8 @@ static inline int arch_xenheap_pages_alloc(mfn_t mfn, u= nsigned int order) { return 0; } -static inline int arch_xenheap_pages_free(mfn_t mfn, unsigned int order) +static inline int arch_xenheap_pages_free(mfn_t mfn, unsigned int order, + bool *first_kept) { return 0; } --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748680; cv=none; d=zohomail.com; s=zohoarc; b=TfFE2SRMFSlCbiN/XhuvXPRtDXHOCLQEe1HPAcLCHty7QpyZzg6so5geWPj2ZeaG+JwWF4nFSso5mKVCs6xZoDemilLtXchZ0Qjs1LDcQHD+pB5Y5iKDXNA0XXKv//r/GRPQ0eyI/1LkPxLu6KV5+a0Izj5eD51um0CCLjjwfx4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748680; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=doZGPedjl6t4BduuLLhxAllV1eJixwYX8iV31mVCa6U=; b=mqb2qLvIRC9aZHVGbbkPKBnSGhbReIBD0MZhu4arddHNRNFRHQMXyxyhv1MRthiN0Ee1PpDU/hk5i884ixh/K9bfBTqX95GnTfBzqFEEQ0yN497tBRnlQ11zI7Wx9Affs0ce5/9cvW16tLsABu2CuRyof81Rmjtn+hWwpU6PMiI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 179374868013647.44585251590365; Tue, 3 Nov 2026 15:31:20 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470805.1679531 (Exim 4.92) (envelope-from ) id 1xONxr-0003Y2-T5; Tue, 03 Nov 2026 23:30:51 +0000 Received: by outflank-mailman (output) from mailman id 1470805.1679531; Tue, 03 Nov 2026 23:30:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxq-0003T5-8F; Tue, 03 Nov 2026 23:30:50 +0000 Received: by outflank-mailman (input) for mailman id 1470805; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000da-Cv for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Coun-0Y5z; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo2-0000000EMFa-2rqY; Tue, 03 Nov 2026 23:20:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=doZGPedjl6t4BduuLLhxAllV1eJixwYX8iV31mVCa6U=; b=1FC68PBmjd3pbBZ/jYcxL7sPEx SjTf2yfTDgVkTShKZDahj+YjBxkZ2wRREY7ORP9osVC6yRusmepkoAOzJY5OZ1zxyEPguC1K5ol57 xKXpgqEqO/7Rl0r0su+afVmxjLxawrmlVi/BMJrKwcZ1gRmnRAdY3a5dtZd97k+cfnp4=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 24/44] x86/mm: introduce populate_perdomain_mapping() Date: Tue, 3 Nov 2026 17:13:46 -0500 Message-ID: <20261103-asi-hvm-v6-24-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748681636158500 From: Roger Pau Monn=C3=A9 The per-domain area has central machinery for building its page-tables and for managing the backing pages it owns itself: create_perdomain_mapping() / destroy_perdomain_mapping(), used by the mapcache and the compat argument-translation area. What the interface lacks is a way to install a caller's own pages at a chosen address, which a per-vCPU state window, introduced by a later patch, needs: its users allocate their pages themselves and map them there. Introduce populate_perdomain_mapping(v, va, mfn, nr, flags) for this. It writes the given MFNs, with the given page-table flags, into v's view of the per-domain area, walking the per-domain page-table structures with transient mappings, so v need not be current. Callers don't need to know where the page-tables live, how the area is structured, or whether v has a per-domain area of its own or shares its domain's. We require the range to already have been populated down to the L1 tables by create_perdomain_mapping(). TLB flushing is left to the caller. A present entry not owned by the area (!_PAGE_AVAIL0) is replaced. A present entry owned by the area (_PAGE_AVAIL0, installed by create_perdomain_mapping() itself) is never expected: such a page is referenced only by the mapping, and no caller replaces area-owned backing. Should one ever do so, the page is leaked with a message rather than freed. The function is not for contexts with interrupts off: its users map their pages when allocating them. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claud= e Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - Comment above populate_perdomain_mapping(): the range lies within one sub-area (one PERDOMAIN_VIRT_SLOT()), not one "per-domain slot". - Message: "whether v has a per-domain area of its own or shares its domain's". Changes in v4: - Re-introduced into the series. - HVM only: the users are the state window's, all of which map their pages at allocation time, so drop the linear-mapping fast path and the IRQs-off walk that context-switch callers needed, along with the map_domain_page_irqoff() variants. --- xen/arch/x86/include/asm/mm.h | 3 ++ xen/arch/x86/mm.c | 62 +++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 50ace61ce7..d04a935aff 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -623,6 +623,9 @@ struct page_info *allocate_perdomain_l3(struct domain *= d); int create_perdomain_mapping(struct vcpu *v, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg); +void populate_perdomain_mapping(const struct vcpu *v, unsigned long va, + const mfn_t *mfn, unsigned int nr, + unsigned int flags); void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr); void free_perdomain_mappings(struct domain *d); diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index e1a80e9a5c..0edf01f363 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6567,6 +6567,68 @@ int create_perdomain_mapping(struct vcpu *v, unsigne= d long va, return rc; } =20 +/* + * Map @nr pages, @mfn[0..nr-1], at consecutive pages from @va in v's view= of + * the per-domain area, with page-table @flags. The range must lie within= a + * single sub-area (one PERDOMAIN_VIRT_SLOT()), and must already have been + * plumbed down to the L1 tables by create_perdomain_mapping(). Present + * entries are replaced, and no TLB flushing is done. + * + * The walk maps the per-domain page-table pages transiently, so v need not + * be current. It is not for contexts with interrupts disabled. + */ +void populate_perdomain_mapping(const struct vcpu *v, unsigned long va, + const mfn_t *mfn, unsigned int nr, + unsigned int flags) +{ + struct page_info *l3_pg =3D perdomain_l3(v); + const l3_pgentry_t *l3tab; + const l2_pgentry_t *l2tab; + l1_pgentry_t *l1tab =3D NULL; + unsigned int i; + + ASSERT(va >=3D PERDOMAIN_VIRT_START && + va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); + ASSERT(!nr || !l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1))); + ASSERT(local_irq_is_enabled()); + + if ( !nr ) + return; + + BUG_ON(!l3_pg); + l3tab =3D __map_domain_page(l3_pg); + BUG_ON(!(l3e_get_flags(l3tab[l3_table_offset(va)]) & _PAGE_PRESENT)); + l2tab =3D map_l2t_from_l3e(l3tab[l3_table_offset(va)]); + unmap_domain_page(l3tab); + + for ( i =3D 0; i < nr; i++, va +=3D PAGE_SIZE ) + { + l1_pgentry_t *pl1e; + + if ( !l1tab || !l1_table_offset(va) ) + { + const l2_pgentry_t *pl2e =3D &l2tab[l2_table_offset(va)]; + + unmap_domain_page(l1tab); + BUG_ON(!(l2e_get_flags(*pl2e) & _PAGE_PRESENT)); + l1tab =3D map_l1t_from_l2e(*pl2e); + } + + pl1e =3D &l1tab[l1_table_offset(va)]; + if ( perdomain_l1e_needs_freeing(*pl1e) ) + { + printk(XENLOG_ERR + "%pv: leaking area-owned page %"PRI_mfn" at %#lx\n", + v, mfn_x(l1e_get_mfn(*pl1e)), va); + ASSERT_UNREACHABLE(); + } + l1e_write(pl1e, l1e_from_mfn(mfn[i], flags)); + } + + unmap_domain_page(l1tab); + unmap_domain_page(l2tab); +} + void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr) { --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748680; cv=none; d=zohomail.com; s=zohoarc; b=EEm9/zCv4jtqVtwDv+nfZIpiBGAFM0EdrvF2SvaD5Dzp59XtzUJwDYW3ibE7Guok8f0BsfLxg/u5X9rSCsnfRVHx4Ps4zzZMacgBeOc53RTvH+szOGFUfCAzyMJKhfKGJ0U5v4j2VBVCnwtZFxm9d8YY69+jgAD2CTPvC6qExyQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748680; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=bXTFHOrcLhUOodXtLyqMdEEiIJaFfjlDFEIJV1YrfvQ=; b=YIyrtgNr0crro4b4zOsSoJrMvrEmNrb0ofQmGeO3eT/AxaAXCt7VhK8jlmbZWtORdv+In1qr0k45i/QjLo8n2lHGFLxMxUTJU0ZuuQ6+ZOKyQUMO/jWMQPDd79RMJ/FE/GJxycDAl6N9TgTFCocUGoF2LxcUPUie7FjCy5Xd+yw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 17937486801861007.4843477855233; Tue, 3 Nov 2026 15:31:20 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470801.1679497 (Exim 4.92) (envelope-from ) id 1xONxl-0002XH-JA; Tue, 03 Nov 2026 23:30:45 +0000 Received: by outflank-mailman (output) from mailman id 1470801.1679497; Tue, 03 Nov 2026 23:30:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxl-0002RV-04; Tue, 03 Nov 2026 23:30:45 +0000 Received: by outflank-mailman (input) for mailman id 1470801; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000dC-8n for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Couu-0oEt; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo3-0000000EMFa-0hqE; Tue, 03 Nov 2026 23:20:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=bXTFHOrcLhUOodXtLyqMdEEiIJaFfjlDFEIJV1YrfvQ=; b=PejvEQ5O3VI5tzJZp2wI//NpQ2 m+Ot+wXdCQWCAa1tBiuo2FktJtMbdK/nYweFNsEpGmGhDnVkMe8BLymF+CFsNp6iaPscRKz4AKkC4 trz+cw4sGR/stbOAXVZoBYqpbi+aWRZ86/U/XFt5lfVPobCuP/UZx/mVgsVDWluuFDOs=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 25/44] x86/mm: introduce a per-vCPU state window in the perdomain area Date: Tue, 3 Nov 2026 17:13:47 -0500 Message-ID: <20261103-asi-hvm-v6-25-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748681691158500 Content-Type: text/plain; charset="utf-8" Under Address Space Isolation, guest-owned register state that Xen keeps in always-mapped memory moves to the domain heap, mapped only in the contexts that use it. The long-lived per-vCPU structures involved (the SVM VMCB, the VMX guest MSR load/save list, the guest XSAVE area, and later the aggregated small per-vCPU allocations) all want the same shape: one or more domain-heap pages, mapped once at fixed per-vCPU virtual addresses for the life of the vCPU. Add a fourth per-domain-mapping sub-area providing that destination: a small per-vCPU stride of page slots, with a fixed index assigned to each relocated structure and the page tables created at vCPU creation time. Consumers allocate their own pages and install them with populate_perdomain_mapping(); the backing pages remain caller-owned, so their lifetime stays with the consumer, while the page tables are torn down with the rest of the per-domain area. Before freeing their pages, consumers take them out of the window with unmap_vcpu_state(), on failure paths as at teardown: vCPU creation can fail once a consumer has mapped its pages, and the domain lives on. It flushes the domain's dirty CPUs, which include every CPU whose page-tables carry the window, in the vCPU's own per-domain area or in the one its domain's vCPUs share; so far that mask is empty on every call, as the vCPU has not run or the domain is paused. A consumer reaching a page from another context, through a transient map_domain_page() mapping, drops the mapping with unmap_vcpu_state_page(), which also flushes this CPU's translations of it, there and at its alias in the per-domain mirror: a domain-wide mapcache defers that flush, and the translations would otherwise outlive the mapping. For a domain running without per-vCPU page tables the window has the same domain-wide visibility as the other per-domain sub-areas: the virtual addresses are keyed on vcpu_id, so the same VA arithmetic serves both modes. With vcpu-pt enabled, a vCPU's window is simply absent from its siblings' address spaces. This is infrastructure for moving the SVM VMCB, the VMX MSR load/save lists, the saved register frames, the HVM emulator's caches and the guest XSAVE areas off always-mapped memory. Nothing occupies the area yet; the only behavioural change is the page-table allocation itself and the failure path that comes with it: vCPU creation can now fail with -ENOMEM. Both apply to every guest vCPU, with or without asi=3D, PV ones included. A domain with per-vCPU page tables gets an L2 and an L1 per vCPU; any other domain, every PV domain among them, one L2 plus an L1 per 64 vCPUs: 2 pages for up to 64 vCPUs. A PV domain's window stays empty until the register frames and XSAVE areas move there. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v6: - Port over change to L3 allocation. Changes in v5: - Say that the PV register frames move into the window too, not only the XSAVE areas. - Message and comments: "per-domain area", and which vCPUs share it. Changes in v4: - New in this version. --- xen/arch/x86/domain.c | 12 ++++++++ xen/arch/x86/include/asm/config.h | 21 +++++++++++++- xen/arch/x86/include/asm/mm.h | 2 ++ xen/arch/x86/mm.c | 47 +++++++++++++++++++++++++++++++ 4 files changed, 81 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 46ca945b48..633619b400 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -524,6 +524,18 @@ int arch_vcpu_create(struct vcpu *v) =20 if ( !is_idle_domain(d) ) { + BUILD_BUG_ON((MAX_VIRT_CPUS << VCPU_STATE_VA_SHIFT) > + (PERDOMAIN_SLOT_MBYTES << 20)); + BUILD_BUG_ON(VCPU_STATE_IDX_VMCB >=3D VCPU_STATE_NR_PAGES); + BUILD_BUG_ON(VCPU_STATE_IDX_N2VMCB >=3D VCPU_STATE_NR_PAGES); + BUILD_BUG_ON(VCPU_STATE_IDX_VMX_MSR >=3D VCPU_STATE_NR_PAGES); + /* Page tables only; torn down with the rest of the per-domain are= a. */ + rc =3D create_perdomain_mapping(v, VCPU_STATE_START(v), + VCPU_STATE_NR_PAGES, + NIL(l1_pgentry_t *), NULL); + if ( rc !=3D 0 ) + goto fail; + paging_vcpu_init(v); =20 if ( (rc =3D vcpu_init_fpu(v)) !=3D 0 ) diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/c= onfig.h index ee84c57432..8a265e25c8 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -158,7 +158,7 @@ /* Slot 260: per-domain mappings (including map cache). */ #define PERDOMAIN_VIRT_START (PML4_ADDR(260)) #define PERDOMAIN_SLOT_MBYTES (PML4_ENTRY_BYTES >> (20 + PAGETABLE_ORDER= )) -#define PERDOMAIN_SLOTS 3 +#define PERDOMAIN_SLOTS 4 #define PERDOMAIN_VIRT_SLOT(s) (PERDOMAIN_VIRT_START + (s) * \ (PERDOMAIN_SLOT_MBYTES << 20)) /* Slot 4: mirror of per-domain mappings (for compat xlat area accesses). = */ @@ -293,6 +293,25 @@ extern unsigned long xen_phys_start; #define ARG_XLAT_START(v) \ (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT)) =20 +/* + * Per-vCPU state window. The fourth per-domain-mapping sub-area: a small + * per-vCPU stride of page slots holding long-lived per-vCPU structures ke= pt + * out of always-mapped memory, each mapped once for the life of the vCPU = at + * a fixed index within the stride. + */ +#define VCPU_STATE_SLOT 3 +#define VCPU_STATE_VIRT_START PERDOMAIN_VIRT_SLOT(VCPU_STATE_SLOT) +#define VCPU_STATE_ORDER 3 +#define VCPU_STATE_NR_PAGES (1U << VCPU_STATE_ORDER) +#define VCPU_STATE_VA_SHIFT (VCPU_STATE_ORDER + PAGE_SHIFT) +#define VCPU_STATE_START(v) \ + (VCPU_STATE_VIRT_START + ((v)->vcpu_id << VCPU_STATE_VA_SHIFT)) +#define VCPU_STATE_ADDR(v, idx) \ + (VCPU_STATE_START(v) + (idx) * PAGE_SIZE) +#define VCPU_STATE_IDX_VMCB 0 +#define VCPU_STATE_IDX_N2VMCB 1 +#define VCPU_STATE_IDX_VMX_MSR 2 + #define ELFSIZE 64 =20 #define ARCH_CRASH_SAVE_VMCOREINFO diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index d04a935aff..b253f4a807 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -628,6 +628,8 @@ void populate_perdomain_mapping(const struct vcpu *v, u= nsigned long va, unsigned int flags); void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr); +void unmap_vcpu_state(const struct vcpu *v, unsigned int idx, unsigned int= nr); +void unmap_vcpu_state_page(const void *va); void free_perdomain_mappings(struct domain *d); void free_pervcpu_mappings(struct vcpu *v); =20 diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 0edf01f363..768270ec36 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6681,6 +6681,53 @@ void destroy_perdomain_mapping(const struct vcpu *v,= unsigned long va, unmap_domain_page(l3tab); } =20 +/* + * Take @nr slots of @v's state window, from @idx, out of the per-domain + * area, before the caller frees the pages behind them: the pages are the + * caller's own, and must not stay mapped once freed. The window's + * translations are usable only on CPUs whose page-tables carry it, all in + * the domain's dirty mask: with per-vCPU page-tables, the CPU running v or + * holding it lazily; otherwise, those of any of the domain's vCPUs. + */ +void unmap_vcpu_state(const struct vcpu *v, unsigned int idx, unsigned int= nr) +{ + const struct domain *d =3D v->domain; + + /* + * Beyond the stride is the next vCPU's window, where the domain's vCP= Us + * share a per-domain area. + */ + BUG_ON(idx >=3D VCPU_STATE_NR_PAGES || nr > VCPU_STATE_NR_PAGES - idx); + + destroy_perdomain_mapping(v, VCPU_STATE_ADDR(v, idx), nr); + + flush_tlb_mask(d->dirty_cpumask); +} + +/* + * Drop a transient mapping of a page behind a vCPU's state window, and th= is + * CPU's translations of it: at its address, and at its alias in the + * per-domain mirror, which every domain but a 64-bit PV one has (flushing + * that address there is harmless). A per-vCPU mapcache flushes both + * already, with the slot's last mapping; a domain-wide one defers the + * flush, and CPUs running other vCPUs of its domain may have cached the + * slot too. A directmap address, where the caller has no mapcache, needs + * nothing. + */ +void unmap_vcpu_state_page(const void *va) +{ + unsigned long addr =3D (unsigned long)va; + + unmap_domain_page(va); + + if ( addr < DIRECTMAP_VIRT_START ) + { + flush_tlb_one_local(addr); + flush_tlb_one_local(addr - PERDOMAIN_VIRT_START + + PERDOMAIN_ALT_VIRT_START); + } +} + static void free_perdomain_l3(struct page_info *l3pg) { l3_pgentry_t *l3tab =3D __map_domain_page(l3pg); --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748670; cv=none; d=zohomail.com; s=zohoarc; b=PdibGaGUyBMFQm1EpLMgKmA+7il9/t2y9JIg3e2QfuzOZEy+dz42GbP7KSj5LPdnYw7KI38aT1QU6P4AYbzmxTH2cIoiCaejTzsAz97oupaxNTAKP4t0kbMoBHvN4CnF8sc7n5AyO/XSQEIGAFbq/xNKHjd6cyOsUGJoqOIc+QI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748670; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LIuKm5SuhINw4v3U+2tzQzALxjgkASc2Xl749bMONok=; b=V52vKe3y6ai+j4L1bEFpYePucsr9XKKC45izUKGV/Lx6H5Iw6fi87dB9BM68J8LE415eUs4sYpGYPegddNJsIsZpieVzaBlXvxiPPiZkY8GmIMhtUALXNPin88+9Ny4JBajw94HW7Tok3RHNH/ZQm4FJ4loNrc/52C1cFYtjfk4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748670136275.6942723033983; Tue, 3 Nov 2026 15:31:10 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470798.1679469 (Exim 4.92) (envelope-from ) id 1xONxi-0001o1-Kz; Tue, 03 Nov 2026 23:30:42 +0000 Received: by outflank-mailman (output) from mailman id 1470798.1679469; Tue, 03 Nov 2026 23:30:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxh-0001kt-TS; Tue, 03 Nov 2026 23:30:41 +0000 Received: by outflank-mailman (input) for mailman id 1470798; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000cv-3D for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cots-3qoi; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo3-0000000EMFa-3OLA; Tue, 03 Nov 2026 23:20:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=LIuKm5SuhINw4v3U+2tzQzALxjgkASc2Xl749bMONok=; b=ePeXpb1Mh7jUMF3PfGRZTYMZZM bkbxwnP++45FDS7zVDwtzdsw9mDdhLoY0jiznArR1b3LzG6KCMV4Cf901E0dJy9pbnbk3w6fa+RSG H55VCMcyJfK0v1FNpM6AWsA/lZMJjbw5xcSks0BPiWk8puBsYV+jqbvm0rcUBW7RkNIQ=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Ross Lagerwall , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jason Andryuk , Teddy Astie Subject: [PATCH v6 26/44] x86/svm: introduce SVM_MAP_VMCB() and SVM_UNMAP_VMCB() Date: Tue, 3 Nov 2026 17:13:48 -0500 Message-ID: <20261103-asi-hvm-v6-26-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748671642158500 Content-Type: text/plain; charset="utf-8" Some accessors of a vCPU's VMCB can run with that vCPU paused, from another vCPU's context: context save/load, the segment and CPL getters, PAT/TSC/intercept updates reachable from toolstack operations, vm_event, the 'v' debug key, and construct_vmcb() itself, which runs before the vCPU ever entered guest context. svm_nested_features_on_efer_update() is among them: it is reachable remotely via hvm_load_cpu_ctxt() -> svm_update_guest_efer(). Route them through the new SVM_MAP_VMCB() and SVM_UNMAP_VMCB(); construct_vmcb() takes the VMCB from its caller. svm_set_tsc_offset() needs both VMCBs rather than just the active one, so it maps them individually via SVM_MAP_VMCB_ONE(). A vCPU stays in nested guest mode across a deschedule or a pause, so this path can run remotely with the L2 VMCB active; guest mode does not imply v =3D=3D current. Hot vmexit-path accesses all have v =3D=3D current and keep dereferencing svm->vmcb directly. SVM_MAP_VMCB(), SVM_MAP_VMCB_ONE() and SVM_UNMAP_VMCB() are macros over svm_vmcb_map(), svm_vmcb_map_one() and svm_vmcb_unmap(), so that SVM_UNMAP_VMCB() can clear the caller's pointer, as VCPU_UNMAP_XSAVE_AREA() does. No functional change: the helpers return the VMCB's existing address. They prepare for moving VMCBs off always-mapped memory, where a VMCB's mapping is valid only in its own vCPU's context. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- CC: Ross Lagerwall Changes in v5: - Map and unmap through SVM_MAP_VMCB(), SVM_MAP_VMCB_ONE() and SVM_UNMAP_VMCB(), which clears the pointer (George). - Retitle (was: "x86/svm: introduce svm_vmcb_map() and svm_vmcb_unmap()"). Changes in v4: - New in this version. --- xen/arch/x86/hvm/svm/nestedsvm.c | 4 +- xen/arch/x86/hvm/svm/svm.c | 133 +++++++++++++++++++++++-------- xen/arch/x86/hvm/svm/vmcb.c | 29 ++++++- xen/arch/x86/hvm/svm/vmcb.h | 18 +++++ 4 files changed, 145 insertions(+), 39 deletions(-) diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nested= svm.c index 128008321d..f5ce3e485c 100644 --- a/xen/arch/x86/hvm/svm/nestedsvm.c +++ b/xen/arch/x86/hvm/svm/nestedsvm.c @@ -1528,7 +1528,7 @@ void svm_vmexit_do_clgi(struct cpu_user_regs *regs, s= truct vcpu *v) */ void svm_nested_features_on_efer_update(struct vcpu *v) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); struct nestedsvm *svm =3D &vcpu_nestedsvm(v); u32 general2_intercepts; vintr_t vintr; @@ -1585,6 +1585,8 @@ void svm_nested_features_on_efer_update(struct vcpu *= v) vmcb_set_general2_intercepts(vmcb, general2_intercepts); } } + + SVM_UNMAP_VMCB(v, vmcb); } =20 void __init start_nested_svm(struct hvm_function_table *hvm_function_table) diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c index 1628c5e0ae..b539406f0c 100644 --- a/xen/arch/x86/hvm/svm/svm.c +++ b/xen/arch/x86/hvm/svm/svm.c @@ -107,7 +107,7 @@ static void cf_check svm_cpu_down(void) static void cf_check svm_update_guest_cr( struct vcpu *v, unsigned int cr, unsigned int flags) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); uint64_t value; =20 switch ( cr ) @@ -173,11 +173,13 @@ static void cf_check svm_update_guest_cr( default: BUG(); } + + SVM_UNMAP_VMCB(v, vmcb); } =20 static void cf_check svm_update_guest_efer(struct vcpu *v) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); unsigned long guest_efer =3D v->arch.hvm.guest_efer, xen_efer =3D read_efer(); =20 @@ -204,6 +206,7 @@ static void cf_check svm_update_guest_efer(struct vcpu = *v) guest_efer |=3D EFER_SVME; =20 vmcb_set_efer(vmcb, guest_efer); + SVM_UNMAP_VMCB(v, vmcb); =20 ASSERT(nestedhvm_enabled(v->domain) || !nsvm_efer_svm_enabled(v)); @@ -352,7 +355,7 @@ static void svm_restore_dr(struct vcpu *v) =20 static int svm_vmcb_save(struct vcpu *v, struct hvm_hw_cpu *c) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); =20 if ( vmcb->event_inj.v && hvm_event_needs_reinjection(vmcb->event_inj.type, @@ -362,13 +365,15 @@ static int svm_vmcb_save(struct vcpu *v, struct hvm_h= w_cpu *c) c->error_code =3D vmcb->event_inj.ec; } =20 + SVM_UNMAP_VMCB(v, vmcb); + return 1; } =20 static int svm_vmcb_restore(struct vcpu *v, struct hvm_hw_cpu *c) { struct page_info *page =3D NULL; - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb; struct p2m_domain *p2m =3D p2m_get_hostp2m(v->domain); =20 if ( c->pending_valid ) @@ -417,6 +422,8 @@ static int svm_vmcb_restore(struct vcpu *v, struct hvm_= hw_cpu *c) svm_update_guest_cr(v, 0, 0); svm_update_guest_cr(v, 4, 0); =20 + vmcb =3D SVM_MAP_VMCB(v); + if ( !paging_mode_shadow(v->domain) ) { vmcb_set_np(vmcb, true); @@ -436,6 +443,8 @@ static int svm_vmcb_restore(struct vcpu *v, struct hvm_= hw_cpu *c) vmcb->event_inj.raw =3D 0; =20 vmcb->cleanbits.raw =3D 0; + SVM_UNMAP_VMCB(v, vmcb); + paging_update_paging_modes(v); =20 return 0; @@ -443,7 +452,7 @@ static int svm_vmcb_restore(struct vcpu *v, struct hvm_= hw_cpu *c) =20 static void svm_save_cpu_state(struct vcpu *v, struct hvm_hw_cpu *data) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + const struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); =20 data->sysenter_cs =3D vmcb->sysenter_cs; data->sysenter_esp =3D vmcb->sysenter_esp; @@ -453,11 +462,13 @@ static void svm_save_cpu_state(struct vcpu *v, struct= hvm_hw_cpu *data) data->msr_star =3D vmcb->star; data->msr_cstar =3D vmcb->cstar; data->msr_syscall_mask =3D vmcb->sfmask; + + SVM_UNMAP_VMCB(v, vmcb); } =20 static void svm_load_cpu_state(struct vcpu *v, struct hvm_hw_cpu *data) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); =20 vmcb->lstar =3D data->msr_lstar; vmcb->star =3D data->msr_star; @@ -467,6 +478,9 @@ static void svm_load_cpu_state(struct vcpu *v, struct h= vm_hw_cpu *data) vmcb->sysenter_cs =3D data->sysenter_cs; vmcb->sysenter_esp =3D data->sysenter_esp; vmcb->sysenter_eip =3D data->sysenter_eip; + + SVM_UNMAP_VMCB(v, vmcb); + v->arch.hvm.guest_efer =3D data->msr_efer; svm_update_guest_efer(v); } @@ -492,7 +506,7 @@ static int cf_check svm_load_vmcb_ctxt(struct vcpu *v, = struct hvm_hw_cpu *ctxt) =20 static unsigned cf_check int svm_get_interrupt_shadow(struct vcpu *v) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); unsigned int intr_shadow =3D 0; =20 if ( vmcb->int_stat.intr_shadow ) @@ -503,13 +517,15 @@ static unsigned cf_check int svm_get_interrupt_shadow= (struct vcpu *v) : (vmcb_get_general1_intercepts(vmcb) & GENERAL1_INTERCEPT_IRET) ) intr_shadow |=3D HVM_INTR_SHADOW_NMI; =20 + SVM_UNMAP_VMCB(v, vmcb); + return intr_shadow; } =20 static void cf_check svm_set_interrupt_shadow( struct vcpu *v, unsigned int intr_shadow) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); bool block_nmi =3D intr_shadow & HVM_INTR_SHADOW_NMI; =20 vmcb->int_stat.intr_shadow =3D @@ -527,10 +543,13 @@ static void cf_check svm_set_interrupt_shadow( =20 vmcb_set_general1_intercepts(vmcb, gen1); } + + SVM_UNMAP_VMCB(v, vmcb); } =20 static int cf_check svm_guest_x86_mode(struct vcpu *v) { + /* Current only (see guest_cpu_user_regs() below): no mapping needed. = */ struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; =20 if ( unlikely(!(v->arch.hvm.guest_cr[0] & X86_CR0_PE)) ) @@ -544,8 +563,7 @@ static int cf_check svm_guest_x86_mode(struct vcpu *v) =20 static void cf_check svm_cpuid_policy_changed(struct vcpu *v) { - struct svm_vcpu *svm =3D &v->arch.hvm.svm; - struct vmcb_struct *vmcb =3D svm->vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); const struct cpu_policy *cp =3D v->domain->arch.cpu_policy; u32 bitmap =3D vmcb_get_exception_intercepts(vmcb); =20 @@ -555,6 +573,7 @@ static void cf_check svm_cpuid_policy_changed(struct vc= pu *v) bitmap &=3D ~(1U << X86_EXC_UD); =20 vmcb_set_exception_intercepts(vmcb, bitmap); + SVM_UNMAP_VMCB(v, vmcb); =20 /* Give access to MSR_SPEC_CTRL if the guest has been told about it. */ svm_intercept_msr(v, MSR_SPEC_CTRL, @@ -598,13 +617,18 @@ void svm_sync_vmcb(struct vcpu *v, enum vmcb_sync_sta= te new_state) =20 static unsigned int cf_check svm_get_cpl(struct vcpu *v) { - return vmcb_get_cpl(v->arch.hvm.svm.vmcb); + const struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); + unsigned int cpl =3D vmcb_get_cpl(vmcb); + + SVM_UNMAP_VMCB(v, vmcb); + + return cpl; } =20 static void cf_check svm_get_segment_register( struct vcpu *v, enum x86_segment seg, struct segment_register *reg) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); =20 ASSERT((v =3D=3D current) || !vcpu_runnable(v)); =20 @@ -645,12 +669,14 @@ static void cf_check svm_get_segment_register( *reg =3D (struct segment_register){}; break; } + + SVM_UNMAP_VMCB(v, vmcb); } =20 static void cf_check svm_set_segment_register( struct vcpu *v, enum x86_segment seg, struct segment_register *reg) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); =20 ASSERT((v =3D=3D current) || !vcpu_runnable(v)); =20 @@ -679,6 +705,7 @@ static void cf_check svm_set_segment_register( default: ASSERT_UNREACHABLE(); domain_crash(v->domain); + SVM_UNMAP_VMCB(v, vmcb); return; } =20 @@ -716,33 +743,41 @@ static void cf_check svm_set_segment_register( ASSERT_UNREACHABLE(); break; } + + SVM_UNMAP_VMCB(v, vmcb); } =20 static int cf_check svm_set_guest_pat(struct vcpu *v, u64 gpat) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb; =20 if ( paging_mode_shadow(v->domain) ) return 0; =20 + vmcb =3D SVM_MAP_VMCB(v); vmcb_set_g_pat(vmcb, gpat); + SVM_UNMAP_VMCB(v, vmcb); + return 1; } =20 static int cf_check svm_get_guest_pat(struct vcpu *v, u64 *gpat) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + const struct vmcb_struct *vmcb; =20 if ( paging_mode_shadow(v->domain) ) return 0; =20 + vmcb =3D SVM_MAP_VMCB(v); *gpat =3D vmcb_get_g_pat(vmcb); + SVM_UNMAP_VMCB(v, vmcb); + return 1; } =20 static void cf_check svm_set_tsc_offset(struct vcpu *v, uint64_t offset) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); struct vmcb_struct *n1vmcb, *n2vmcb; uint64_t n2_tsc_offset =3D 0; struct domain *d =3D v->domain; @@ -750,25 +785,36 @@ static void cf_check svm_set_tsc_offset(struct vcpu *= v, uint64_t offset) if ( !nestedhvm_enabled(d) ) { vmcb_set_tsc_offset(vmcb, offset); + SVM_UNMAP_VMCB(v, vmcb); return; } =20 - n1vmcb =3D vcpu_nestedhvm(v).nv_n1vmcx; - n2vmcb =3D vcpu_nestedhvm(v).nv_n2vmcx; - + /* + * A vCPU stays in nested guest mode across a deschedule or a pause, so + * this can run remotely with the L2 VMCB active: map both explicitly. + */ if ( nestedhvm_vcpu_in_guestmode(v) ) { + const struct nestedvcpu *nv =3D &vcpu_nestedhvm(v); + + n1vmcb =3D SVM_MAP_VMCB_ONE(v, nv->nv_n1vmcx, nv->nv_n1vmcx_pa); + n2vmcb =3D SVM_MAP_VMCB_ONE(v, nv->nv_n2vmcx, nv->nv_n2vmcx_pa); + n2_tsc_offset =3D vmcb_get_tsc_offset(n2vmcb) - vmcb_get_tsc_offset(n1vmcb); vmcb_set_tsc_offset(n1vmcb, offset); + + SVM_UNMAP_VMCB(v, n2vmcb); + SVM_UNMAP_VMCB(v, n1vmcb); } =20 vmcb_set_tsc_offset(vmcb, offset + n2_tsc_offset); + SVM_UNMAP_VMCB(v, vmcb); } =20 static void cf_check svm_set_rdtsc_exiting(struct vcpu *v, bool enable) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); u32 general1_intercepts =3D vmcb_get_general1_intercepts(vmcb); u32 general2_intercepts =3D vmcb_get_general2_intercepts(vmcb); =20 @@ -783,13 +829,14 @@ static void cf_check svm_set_rdtsc_exiting(struct vcp= u *v, bool enable) =20 vmcb_set_general1_intercepts(vmcb, general1_intercepts); vmcb_set_general2_intercepts(vmcb, general2_intercepts); + SVM_UNMAP_VMCB(v, vmcb); } =20 #ifdef CONFIG_VM_EVENT static void cf_check svm_set_descriptor_access_exiting( struct vcpu *v, bool enable) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); u32 general1_intercepts =3D vmcb_get_general1_intercepts(vmcb); u32 mask =3D GENERAL1_INTERCEPT_IDTR_READ | GENERAL1_INTERCEPT_GDTR_RE= AD | GENERAL1_INTERCEPT_LDTR_READ | GENERAL1_INTERCEPT_TR_READ @@ -802,6 +849,7 @@ static void cf_check svm_set_descriptor_access_exiting( general1_intercepts &=3D ~mask; =20 vmcb_set_general1_intercepts(vmcb, general1_intercepts); + SVM_UNMAP_VMCB(v, vmcb); } #endif /* CONFIG_VM_EVENT */ =20 @@ -1360,7 +1408,12 @@ static void cf_check svm_inject_event(const struct x= 86_event *event) =20 static bool cf_check svm_event_pending(const struct vcpu *v) { - return v->arch.hvm.svm.vmcb->event_inj.v; + const struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); + bool pending =3D vmcb->event_inj.v; + + SVM_UNMAP_VMCB(v, vmcb); + + return pending; } =20 static void cf_check svm_cpu_dead(unsigned int cpu) @@ -2318,45 +2371,55 @@ static void cf_check svm_invlpg(struct vcpu *v, uns= igned long linear) static bool cf_check svm_get_pending_event( struct vcpu *v, struct x86_event *info) { - const struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + const struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); + bool pending =3D vmcb->event_inj.v; =20 - if ( !vmcb->event_inj.v ) - return false; + if ( pending ) + { + info->vector =3D vmcb->event_inj.vector; + info->type =3D vmcb->event_inj.type; + info->error_code =3D vmcb->event_inj.ec; + } =20 - info->vector =3D vmcb->event_inj.vector; - info->type =3D vmcb->event_inj.type; - info->error_code =3D vmcb->event_inj.ec; + SVM_UNMAP_VMCB(v, vmcb); =20 - return true; + return pending; } =20 static uint64_t cf_check svm_get_reg(struct vcpu *v, unsigned int reg) { struct vcpu *curr =3D current; - const struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + const struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); struct domain *d =3D v->domain; + uint64_t val =3D 0; =20 switch ( reg ) { case MSR_SPEC_CTRL: - return vmcb->spec_ctrl; + val =3D vmcb->spec_ctrl; + break; =20 case MSR_SHADOW_GS_BASE: if ( v =3D=3D curr ) svm_sync_vmcb(v, vmcb_in_sync); - return vmcb->kerngsbase; + val =3D vmcb->kerngsbase; + break; =20 default: printk(XENLOG_G_ERR "%s(%pv, 0x%08x) Bad register\n", __func__, v, reg); domain_crash(d); - return 0; + break; } + + SVM_UNMAP_VMCB(v, vmcb); + + return val; } =20 static void cf_check svm_set_reg(struct vcpu *v, unsigned int reg, uint64_= t val) { - struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; + struct vmcb_struct *vmcb =3D SVM_MAP_VMCB(v); struct domain *d =3D v->domain; =20 switch ( reg ) @@ -2371,6 +2434,8 @@ static void cf_check svm_set_reg(struct vcpu *v, unsi= gned int reg, uint64_t val) domain_crash(d); break; } + + SVM_UNMAP_VMCB(v, vmcb); } =20 static struct hvm_function_table __initdata_cf_clobber svm_function_table = =3D { diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c index a6f09672a7..5e193be9cd 100644 --- a/xen/arch/x86/hvm/svm/vmcb.c +++ b/xen/arch/x86/hvm/svm/vmcb.c @@ -43,11 +43,25 @@ void free_vmcb(struct vmcb_struct *vmcb) free_xenheap_page(vmcb); } =20 +struct vmcb_struct *svm_vmcb_map_one(const struct vcpu *v, + struct vmcb_struct *va, paddr_t pa) +{ + return va; +} + +struct vmcb_struct *svm_vmcb_map(const struct vcpu *v) +{ + return svm_vmcb_map_one(v, v->arch.hvm.svm.vmcb, v->arch.hvm.svm.vmcb_= pa); +} + +void svm_vmcb_unmap(const struct vcpu *v, const struct vmcb_struct *vmcb) +{ +} + /* This function can directly access fields which are covered by clean bit= s. */ -static int construct_vmcb(struct vcpu *v) +static int construct_vmcb(struct vcpu *v, struct vmcb_struct *vmcb) { struct svm_vcpu *svm =3D &v->arch.hvm.svm; - struct vmcb_struct *vmcb =3D svm->vmcb; =20 vmcb->_general1_intercepts =3D GENERAL1_INTERCEPT_INTR | GENERAL1_INTERCEPT_NMI | @@ -201,6 +215,7 @@ int svm_create_vmcb(struct vcpu *v) { struct nestedvcpu *nv =3D &vcpu_nestedhvm(v); struct svm_vcpu *svm =3D &v->arch.hvm.svm; + struct vmcb_struct *vmcb; int rc; =20 if ( (nv->nv_n1vmcx =3D=3D NULL) && @@ -211,7 +226,10 @@ int svm_create_vmcb(struct vcpu *v) } =20 svm->vmcb =3D nv->nv_n1vmcx; - rc =3D construct_vmcb(v); + + vmcb =3D SVM_MAP_VMCB(v); + rc =3D construct_vmcb(v, vmcb); + SVM_UNMAP_VMCB(v, vmcb); if ( rc !=3D 0 ) { free_vmcb(nv->nv_n1vmcx); @@ -407,6 +425,7 @@ static void cf_check vmcb_dump(unsigned char ch) { struct domain *d; struct vcpu *v; + struct vmcb_struct *vmcb; =20 printk("*********** VMCB Areas **************\n"); =20 @@ -425,7 +444,9 @@ static void cf_check vmcb_dump(unsigned char ch) continue; } printk("\tVCPU %d\n", v->vcpu_id); - svm_vmcb_dump("key_handler", v->arch.hvm.svm.vmcb); + vmcb =3D SVM_MAP_VMCB(v); + svm_vmcb_dump("key_handler", vmcb); + SVM_UNMAP_VMCB(v, vmcb); =20 process_pending_softirqs(); } diff --git a/xen/arch/x86/hvm/svm/vmcb.h b/xen/arch/x86/hvm/svm/vmcb.h index 28dae325ce..d49c910d46 100644 --- a/xen/arch/x86/hvm/svm/vmcb.h +++ b/xen/arch/x86/hvm/svm/vmcb.h @@ -559,6 +559,24 @@ struct vmcb_struct { struct vmcb_struct *alloc_vmcb(void); void free_vmcb(struct vmcb_struct *vmcb); =20 +/* + * Access a VMCB of a vCPU that may not be current. SVM_MAP_VMCB() takes + * the active VMCB, SVM_MAP_VMCB_ONE() a specific one. + */ +struct vmcb_struct *svm_vmcb_map_one(const struct vcpu *v, + struct vmcb_struct *va, paddr_t pa); +struct vmcb_struct *svm_vmcb_map(const struct vcpu *v); +void svm_vmcb_unmap(const struct vcpu *v, const struct vmcb_struct *vmcb); + +#define SVM_MAP_VMCB(v) svm_vmcb_map(v) +#define SVM_MAP_VMCB_ONE(v, va, pa) svm_vmcb_map_one(v, va, pa) + +/* Drops the mapping of a VMCB and nullifies its pointer. */ +#define SVM_UNMAP_VMCB(v, vmcb) do { \ + svm_vmcb_unmap(v, vmcb); \ + (vmcb) =3D NULL; \ +} while ( false ) + int svm_create_vmcb(struct vcpu *v); void svm_destroy_vmcb(struct vcpu *v); =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748703; cv=none; d=zohomail.com; s=zohoarc; b=Fiqgy/CP0ygIQq38nSXU+pPk1ywWSS+kiVn8u5YbLvDYCSStuLdE2O7y9UryBRjP4K0ePoV5pgfXV1k1NQO6xTINRGLvU9Br5oYyy2gCiph0ng2SvEhrabWbSKB6A2YLwn5wvPWelRkIcrnri7tlguYStUU9RGmKgbqYs38LEGg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748703; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xyRRTvzAZ1uxBpB7gEOaI29dfhB9uSJD02k1XOtgRUc=; b=PWGNlssSSiAw7ozNu6FvspmJ/4fn0miAzznVPUEKHsixWH4t9XL5u/em79vJXnx/0Ep/M6a1ZzMiVI/pJ3J2rR8mTlMKS6WU5BvMmJ4tiGd8BPevbsPReAKJqeqXCWHeDz4dhJVi/OUa65o6WO4gDz5z88780gVvHyi6lgkRXSM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748703674386.551056048075; Tue, 3 Nov 2026 15:31:43 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470822.1679684 (Exim 4.92) (envelope-from ) id 1xONyT-0002qF-0k; Tue, 03 Nov 2026 23:31:29 +0000 Received: by outflank-mailman (output) from mailman id 1470822.1679684; Tue, 03 Nov 2026 23:31:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyQ-0002gQ-UM; Tue, 03 Nov 2026 23:31:26 +0000 Received: by outflank-mailman (input) for mailman id 1470822; Tue, 03 Nov 2026 23:30:42 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxh-0001k2-OQ for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:41 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotI-34NX; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo4-0000000EMFa-1Ffa; Tue, 03 Nov 2026 23:20:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=xyRRTvzAZ1uxBpB7gEOaI29dfhB9uSJD02k1XOtgRUc=; b=ml4/9THZCGPDm8R3VmKn92z6ID ydbYgOHSK2gWLbJAFp9pugzI1yQNry3IVCqcoLL0nR5/rWy2zluQkmsPqhUe1txxQIMqoEKOhW6we j+MChVpg7FY8VFkkIO2WdU2IuKTR2nxkUB75ynmRjLnlM8URa4ERTz1Je3AUx9FYuSUE=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Ross Lagerwall , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jason Andryuk , Teddy Astie Subject: [PATCH v6 27/44] x86/svm: restore guest debug registers in svm_do_resume() Date: Tue, 3 Nov 2026 17:13:49 -0500 Message-ID: <20261103-asi-hvm-v6-27-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748705771158500 Content-Type: text/plain; charset="utf-8" svm_ctxt_switch_to() restores the guest's debug registers when its DR7 enables any of them, which also updates the VMCB. Do it in svm_do_resume() instead, which follows it whenever the vCPU is scheduled in. svm_do_resume() also runs when a vCPU keeps running on its pCPU (continue_running()), without svm_ctxt_switch_to(). Where the guest's debug registers are already loaded, the restore does nothing: flag_dr_dirty records that they are, and only svm_save_dr() clears it, from svm_ctxt_switch_from(). This prepares for moving VMCBs off always-mapped memory: svm_ctxt_switch_to() runs before write_ptbase(), where the incoming vCPU's VMCB will not be mapped. Not yet run with a guest using debug registers. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- CC: Ross Lagerwall Changes in v4: - New in this version. --- xen/arch/x86/hvm/svm/svm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c index b539406f0c..df56245311 100644 --- a/xen/arch/x86/hvm/svm/svm.c +++ b/xen/arch/x86/hvm/svm/svm.c @@ -953,8 +953,6 @@ static void cf_check svm_ctxt_switch_to(struct vcpu *v) */ disable_each_ist(per_cpu(idt, cpu)); =20 - svm_restore_dr(v); - svm_tsc_ratio_load(v); =20 if ( cpu_has_msr_tsc_aux ) @@ -978,6 +976,8 @@ static void noreturn cf_check svm_do_resume(void) bool vcpu_guestmode =3D false; struct vlapic *vlapic =3D vcpu_vlapic(v); =20 + svm_restore_dr(v); + if ( nestedhvm_enabled(v->domain) && nestedhvm_vcpu_in_guestmode(v) ) vcpu_guestmode =3D 1; =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748668; cv=none; d=zohomail.com; s=zohoarc; b=EYXPk+bfYixYOzLW3zqpg6m1PEQ5sgofPvSF7qmtZojfFh7i//zwQuMiqVFErjjE1nYgkfoddwWZAnjHLRGrLseakGEkiVc5aasBdwYLym/rQayCnwZOfHAExZ9ByqGy+WTihab6/xYUvXF68FVExmXotGg7aXQQw2zQl8hQEDc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748668; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=x3Vx8q2pS91S3VIbmLZaIAr7uWnHJgNTJ/Nzz6y5x7k=; b=EL6mnCSkkYDdFNrPf4jJoASi8SjuxS2ZEZynYsV8lY+/dG1Q4m57PmLh5XNiXOV8sS8gAbtAeqgyFu3zRVqHwWCyixAgDAB4H0bepypL2lfyT/KLsZEGC37pTXwqVPW80K57SphyvzbWMADTeMOT9oFylOCLyHy4N4kvlslppOs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748668525929.8750280722437; Tue, 3 Nov 2026 15:31:08 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470804.1679521 (Exim 4.92) (envelope-from ) id 1xONxp-0003Gr-Oi; Tue, 03 Nov 2026 23:30:49 +0000 Received: by outflank-mailman (output) from mailman id 1470804.1679521; Tue, 03 Nov 2026 23:30:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxo-0003CF-Na; Tue, 03 Nov 2026 23:30:48 +0000 Received: by outflank-mailman (input) for mailman id 1470804; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000dS-BI for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Cov0-0w8D; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo4-0000000EMFa-2viA; Tue, 03 Nov 2026 23:20:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=x3Vx8q2pS91S3VIbmLZaIAr7uWnHJgNTJ/Nzz6y5x7k=; b=wbvQCI/AKOeHsMMV8YDR6d+O+i wcQfJ59sX90HfnIR+e2NYtcVK8N+/sZrDPi1gnCubI/xaC9loStf9App35e6PbyaDj39wmLkuQq44 Bphh6KMQg9/Gks66bMmehJyBAOqtvdwzddlb8E+6kIEB6K4EewkvCSOVPYMw+rDkYiI8=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Ross Lagerwall , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jason Andryuk , Teddy Astie Subject: [PATCH v6 28/44] x86/svm: drop svm_vmcb_dump()'s paddr Date: Tue, 3 Nov 2026 17:13:50 -0500 Message-ID: <20261103-asi-hvm-v6-28-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748669500158500 Content-Type: text/plain; charset="utf-8" svm_vmcb_dump() prints virt_to_maddr() of the VMCB, which is right only while VMCBs are reached through the directmap. Print the virtual address alone. This prepares for moving VMCBs off always-mapped memory. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- CC: Ross Lagerwall Changes in v5: - Keep VCPU_svm_vmcb: svm_asm_do_resume() now uses it, since df8d101a3854 ("x86/svm: Work around VMRUN's corruption of VMCB.INTR_SHADOW") in staging. - Retitle (was: "x86/svm: drop svm_vmcb_dump()'s paddr and the unused VCPU_svm_vmcb"). Changes in v4: - New in this version. --- xen/arch/x86/hvm/svm/vmcb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c index 5e193be9cd..21005dc57f 100644 --- a/xen/arch/x86/hvm/svm/vmcb.c +++ b/xen/arch/x86/hvm/svm/vmcb.c @@ -280,8 +280,8 @@ void svm_vmcb_dump(const char *from, const struct vmcb_= struct *vmcb) svm_sync_vmcb(curr, vmcb_in_sync); =20 printk("Dumping guest's current state at %s...\n", from); - printk("Size of VMCB =3D %zu, paddr =3D %"PRIpaddr", vaddr =3D %p\n", - sizeof(struct vmcb_struct), virt_to_maddr(vmcb), vmcb); + printk("Size of VMCB =3D %zu, vaddr =3D %p\n", + sizeof(struct vmcb_struct), vmcb); =20 printk("cr_intercepts =3D %#x dr_intercepts =3D %#x " "exception_intercepts =3D %#x\n", --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748696; cv=none; d=zohomail.com; s=zohoarc; b=hCSAv+M46LfjVQbnf2+6K5XzRUYRUqbn+Ctgns0fVkMIQMDMi+aRd+zKvIhvQTYdN2hVz+XtnTCHKfeK/EEafW0Xzgcq/3rdENtIQmHpLggJwAhzz4y/Etc30ynE7Ags5SjDsClMcKv274Yh8XPnTw8YcFCh46Ek8EfmJ1yqKQc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748696; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=bDsg//HU5JkYv43W8arroEfT2q3XVYeShKGP6Zhzsc4=; b=MSfgGW65eepo13ACBkSFtEGmnWMsyycvRrmqsy1P1l8WkHHuV99pXsIUOMOIRK0VxfmqFXQIhTS3RSIxnNTvutqJEsSFPOxtVibVTfSC6kRL2pHsFkWZSeE8xLd3d1fjOLWZXAq4QjQgpcLjTt2Kct/lQhWQ65e6hHiRW8E17CQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748696212499.41821723698445; Tue, 3 Nov 2026 15:31:36 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470818.1679649 (Exim 4.92) (envelope-from ) id 1xONyJ-0000w2-NH; Tue, 03 Nov 2026 23:31:19 +0000 Received: by outflank-mailman (output) from mailman id 1470818.1679649; Tue, 03 Nov 2026 23:31:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyH-0000kj-N0; Tue, 03 Nov 2026 23:31:17 +0000 Received: by outflank-mailman (input) for mailman id 1470818; Tue, 03 Nov 2026 23:30:41 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxg-0001X0-Vb for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:40 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotG-2wkD; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo5-0000000EMFa-0SLy; Tue, 03 Nov 2026 23:20:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=bDsg//HU5JkYv43W8arroEfT2q3XVYeShKGP6Zhzsc4=; b=IaeKtIM09IITDGFhVxw5daTmBw 05cXLdXdqSllw9DVEZWNS6b+iSWTXDRyyPO/R83l8PgzLaEGFw7A48bf5K1oFeZzFm8pYJQT1Zkb8 2LabEpETrRkC43DaDlhv5v6yIMPih+/C5liGZDQOQ9YvMjxs6WJrrqtPn8rzzRlhc1vA=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Ross Lagerwall , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jason Andryuk , Teddy Astie Subject: [PATCH v6 29/44] x86/svm: allocate VMCBs from the domain heap behind the vCPU state window Date: Tue, 3 Nov 2026 17:13:51 -0500 Message-ID: <20261103-asi-hvm-v6-29-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748697725158500 Content-Type: text/plain; charset="utf-8" The VMCB holds the full guest register save area, but lives on the xenheap, where it stays reachable from every context. Move it to the domain heap and map it at a fixed slot in the owning vCPU's state window, so that under Address Space Isolation it is only mapped while that vCPU's page tables are loaded. Hardware is unaffected: VMRUN, VMLOAD and VMSAVE address the VMCB physically, and vmcb_pa now derives from the page rather than from virt_to_maddr() on a directmap VA. A vCPU can have two VMCBs: the primary (nv_n1vmcx) and the nested L2 one (nv_n2vmcx). Both move, taking the first two slots in the window stride. The nested code's swapping of svm->vmcb / svm->vmcb_pa between the two works unchanged, now swapping between two window VAs. free_vmcb() takes a VMCB out of the window with unmap_vcpu_state() before freeing the page: without per-vCPU page tables the window is in the per-domain area that all its domain's vCPUs share, which outlives a vCPU whose creation fails. The window VA is only valid in the owning vCPU's context. SVM_MAP_VMCB() and SVM_MAP_VMCB_ONE() return it for v =3D=3D current, and a transient map_domain_page() mapping otherwise, which SVM_UNMAP_VMCB() drops with unmap_vcpu_state_page(), along with this CPU's translations of it. svm_ctxt_switch_to() no longer touches the VMCB, as it runs before write_ptbase(); svm_ctxt_switch_from() keeps dereferencing the window VA, as the outgoing page tables are still loaded at that point. svm_asm_do_resume() reads the VMCB's interrupt shadow through the window VA just before VMRUN, with the vCPU's own page tables loaded. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- CC: Ross Lagerwall Changes in v5: - Describe the mapping in terms of SVM_MAP_VMCB(), SVM_MAP_VMCB_ONE() and SVM_UNMAP_VMCB(), which clears the pointer (George). - v4's 26/30, "x86/svm: clear the VMCB clean bits in svm_ctxt_switch_from()", is dropped. Ross's "x86/svm: Don't clear clean bits on every context switch", now in staging as 69633efbca3c, removes the clearing from svm_ctxt_switch_to() instead. - svm_asm_do_resume() now reads the VMCB through svm.vmcb (staging's "x86/svm: Work around VMRUN's corruption of VMCB.INTR_SHADOW"); it runs in the vCPU's own context, where the window VA is valid. Changes in v4: - New in this version. --- xen/arch/x86/hvm/svm/nestedsvm.c | 8 +++--- xen/arch/x86/hvm/svm/svm.c | 10 ++++++++ xen/arch/x86/hvm/svm/vmcb.c | 43 ++++++++++++++++++++++---------- xen/arch/x86/hvm/svm/vmcb.h | 15 ++++++++--- 4 files changed, 56 insertions(+), 20 deletions(-) diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nested= svm.c index f5ce3e485c..9a308cf628 100644 --- a/xen/arch/x86/hvm/svm/nestedsvm.c +++ b/xen/arch/x86/hvm/svm/nestedsvm.c @@ -71,10 +71,9 @@ int cf_check nsvm_vcpu_initialise(struct vcpu *v) goto err; memset(msrpm, 0x0, MSRPM_SIZE); =20 - nv->nv_n2vmcx =3D alloc_vmcb(); + nv->nv_n2vmcx =3D alloc_vmcb(v, VCPU_STATE_IDX_N2VMCB, &nv->nv_n2vmcx_= pa); if ( nv->nv_n2vmcx =3D=3D NULL ) goto err; - nv->nv_n2vmcx_pa =3D virt_to_maddr(nv->nv_n2vmcx); =20 return 0; =20 @@ -95,7 +94,10 @@ void cf_check nsvm_vcpu_destroy(struct vcpu *v) * of l1 vmcb page. */ if ( nv->nv_n1vmcx ) + { v->arch.hvm.svm.vmcb =3D nv->nv_n1vmcx; + v->arch.hvm.svm.vmcb_pa =3D nv->nv_n1vmcx_pa; + } =20 if ( svm->ns_cached_msrpm ) { @@ -113,7 +115,7 @@ void cf_check nsvm_vcpu_destroy(struct vcpu *v) =20 if ( nv->nv_n2vmcx ) { - free_vmcb(nv->nv_n2vmcx); + free_vmcb(v, VCPU_STATE_IDX_N2VMCB, nv->nv_n2vmcx_pa); nv->nv_n2vmcx =3D NULL; nv->nv_n2vmcx_pa =3D INVALID_PADDR; } diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c index df56245311..8b593b1b39 100644 --- a/xen/arch/x86/hvm/svm/svm.c +++ b/xen/arch/x86/hvm/svm/svm.c @@ -265,6 +265,10 @@ static void cf_check svm_enable_msr_interception(struc= t domain *d, uint32_t msr) =20 static void svm_save_dr(struct vcpu *v) { + /* + * @v is the outgoing vCPU, whose page tables are still loaded: the + * VMCB's window VA serves, although current has moved on. + */ struct vmcb_struct *vmcb =3D v->arch.hvm.svm.vmcb; unsigned int flag_dr_dirty =3D v->arch.hvm.flag_dr_dirty; =20 @@ -940,6 +944,12 @@ static void cf_check svm_ctxt_switch_to(struct vcpu *v) { int cpu =3D smp_processor_id(); =20 + /* + * NB we cannot access the VMCB here: this runs before + * write_ptbase(), so the incoming vCPU's state window is not + * reachable yet. + */ + /* * This is required, because VMRUN does consistency check and some of = the * DOM0 selectors are pointing to invalid GDT locations, and cause AMD diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c index 21005dc57f..6588ec87e6 100644 --- a/xen/arch/x86/hvm/svm/vmcb.c +++ b/xen/arch/x86/hvm/svm/vmcb.c @@ -6,6 +6,7 @@ * */ =20 +#include #include #include #include @@ -23,30 +24,40 @@ #include "svm.h" #include "vmcb.h" =20 -struct vmcb_struct *alloc_vmcb(void) +struct vmcb_struct *alloc_vmcb(struct vcpu *v, unsigned int idx, paddr_t *= pa) { - struct vmcb_struct *vmcb; + struct page_info *pg =3D alloc_domheap_page(v->domain, MEMF_no_owner); + mfn_t mfn; + + ASSERT(idx < VCPU_STATE_NR_PAGES); =20 - vmcb =3D alloc_xenheap_page(); - if ( vmcb =3D=3D NULL ) + if ( pg =3D=3D NULL ) { printk(XENLOG_WARNING "Warning: failed to allocate vmcb.\n"); return NULL; } =20 - clear_page(vmcb); - return vmcb; + mfn =3D page_to_mfn(pg); + clear_domain_page(mfn); + populate_perdomain_mapping(v, VCPU_STATE_ADDR(v, idx), &mfn, 1, + __PAGE_HYPERVISOR_RW); + *pa =3D mfn_to_maddr(mfn); + + return (struct vmcb_struct *)VCPU_STATE_ADDR(v, idx); } =20 -void free_vmcb(struct vmcb_struct *vmcb) +void free_vmcb(struct vcpu *v, unsigned int idx, paddr_t pa) { - free_xenheap_page(vmcb); + ASSERT(idx < VCPU_STATE_NR_PAGES); + + unmap_vcpu_state(v, idx, 1); + free_domheap_page(maddr_to_page(pa)); } =20 struct vmcb_struct *svm_vmcb_map_one(const struct vcpu *v, struct vmcb_struct *va, paddr_t pa) { - return va; + return v =3D=3D current ? va : map_domain_page(maddr_to_mfn(pa)); } =20 struct vmcb_struct *svm_vmcb_map(const struct vcpu *v) @@ -56,6 +67,8 @@ struct vmcb_struct *svm_vmcb_map(const struct vcpu *v) =20 void svm_vmcb_unmap(const struct vcpu *v, const struct vmcb_struct *vmcb) { + if ( v !=3D current ) + unmap_vcpu_state_page(vmcb); } =20 /* This function can directly access fields which are covered by clean bit= s. */ @@ -219,26 +232,29 @@ int svm_create_vmcb(struct vcpu *v) int rc; =20 if ( (nv->nv_n1vmcx =3D=3D NULL) && - (nv->nv_n1vmcx =3D alloc_vmcb()) =3D=3D NULL ) + (nv->nv_n1vmcx =3D alloc_vmcb(v, VCPU_STATE_IDX_VMCB, + &nv->nv_n1vmcx_pa)) =3D=3D NULL ) { printk("Failed to create a new VMCB\n"); return -ENOMEM; } =20 svm->vmcb =3D nv->nv_n1vmcx; + svm->vmcb_pa =3D nv->nv_n1vmcx_pa; =20 vmcb =3D SVM_MAP_VMCB(v); rc =3D construct_vmcb(v, vmcb); SVM_UNMAP_VMCB(v, vmcb); if ( rc !=3D 0 ) { - free_vmcb(nv->nv_n1vmcx); + free_vmcb(v, VCPU_STATE_IDX_VMCB, nv->nv_n1vmcx_pa); nv->nv_n1vmcx =3D NULL; + nv->nv_n1vmcx_pa =3D INVALID_PADDR; svm->vmcb =3D NULL; + svm->vmcb_pa =3D INVALID_PADDR; return rc; } =20 - svm->vmcb_pa =3D nv->nv_n1vmcx_pa =3D virt_to_maddr(svm->vmcb); return 0; } =20 @@ -248,7 +264,7 @@ void svm_destroy_vmcb(struct vcpu *v) struct svm_vcpu *svm =3D &v->arch.hvm.svm; =20 if ( nv->nv_n1vmcx !=3D NULL ) - free_vmcb(nv->nv_n1vmcx); + free_vmcb(v, VCPU_STATE_IDX_VMCB, nv->nv_n1vmcx_pa); =20 if ( svm->msrpm !=3D NULL ) { @@ -260,6 +276,7 @@ void svm_destroy_vmcb(struct vcpu *v) nv->nv_n1vmcx =3D NULL; nv->nv_n1vmcx_pa =3D INVALID_PADDR; svm->vmcb =3D NULL; + svm->vmcb_pa =3D INVALID_PADDR; } =20 static void svm_dump_sel(const char *name, const struct segment_register *= s) diff --git a/xen/arch/x86/hvm/svm/vmcb.h b/xen/arch/x86/hvm/svm/vmcb.h index d49c910d46..9e1e9ec1cd 100644 --- a/xen/arch/x86/hvm/svm/vmcb.h +++ b/xen/arch/x86/hvm/svm/vmcb.h @@ -556,12 +556,19 @@ struct vmcb_struct { uint64_t res18[291]; }; =20 -struct vmcb_struct *alloc_vmcb(void); -void free_vmcb(struct vmcb_struct *vmcb); +/* + * VMCBs are domain-heap pages: hardware addresses them physically, and + * Xen maps each one at a fixed slot in the owning vCPU's state window, + * so the returned virtual address is valid only in that vCPU's context. + * @idx is the VCPU_STATE_IDX_* slot to place the mapping in. + */ +struct vmcb_struct *alloc_vmcb(struct vcpu *v, unsigned int idx, paddr_t *= pa); +void free_vmcb(struct vcpu *v, unsigned int idx, paddr_t pa); =20 /* - * Access a VMCB of a vCPU that may not be current. SVM_MAP_VMCB() takes - * the active VMCB, SVM_MAP_VMCB_ONE() a specific one. + * Access a VMCB of a vCPU that may not be current: the window VA for + * current, a transient mapping otherwise. SVM_MAP_VMCB() takes the active + * VMCB, SVM_MAP_VMCB_ONE() a specific one. */ struct vmcb_struct *svm_vmcb_map_one(const struct vcpu *v, struct vmcb_struct *va, paddr_t pa); --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748666; cv=none; d=zohomail.com; s=zohoarc; b=MD6CfL2Xpr9/5BDMNthpUs99prXXkaDbpWW2x9dAqE5/xAWzDRnImHDlnbsa7hUIOXvzzpYwRaeG1KzJm3xwuD6WY/vygu82oJpe0EEcVMZMC4GaktcjYGAj6brh+XgATNwxMiQAkWMN7/ejes7QTdsNR4wGrrShRLl7ayGTbMg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748666; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=0lX6CHwDNVccQVi6T81X3E5uhExKP3Yedzo3WsWaf6A=; b=VRyfWvOJsqqAxXnVWa09fAJUh9CoBPkySNfyHT1dcrESkzu7xnWuwoDOyA8DMwPpXQefE4AZLI7rVR0dZwxKZPoFmmZ/c7apG9wdljuRXRnOgQStVRKhrJK30s5oCjkrRchFgsueZK+RBDsuhDtWpDWlNjwn5fr+OMItcWwiUgs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748666281276.35335480698814; Tue, 3 Nov 2026 15:31:06 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470799.1679483 (Exim 4.92) (envelope-from ) id 1xONxj-000264-SC; Tue, 03 Nov 2026 23:30:43 +0000 Received: by outflank-mailman (output) from mailman id 1470799.1679483; Tue, 03 Nov 2026 23:30:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxj-00024B-BK; Tue, 03 Nov 2026 23:30:43 +0000 Received: by outflank-mailman (input) for mailman id 1470799; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000d1-4Q for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotU-3ZsP; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo5-0000000EMFa-24Wc; Tue, 03 Nov 2026 23:20:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=0lX6CHwDNVccQVi6T81X3E5uhExKP3Yedzo3WsWaf6A=; b=uCm6qzsi7xypOqhnHV2zL8csxu yR57HCBK+VvVmaCED4p8Pp9r5Limx6YykPo5W7LPM87i24TLgBU0NSeGTPjCTCyKUy+kLpx/5ZNUL nHLqhOTjnYxiq0x01gxY08viKc3peNXTplMBzL27zNcOAdwqoVCMMx9VDs8CMLYdaPOU=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 30/44] x86/vmx: move the guest MSR load/save list off always-mapped memory Date: Tue, 3 Nov 2026 17:13:52 -0500 Message-ID: <20261103-asi-hvm-v6-30-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748667627158500 Content-Type: text/plain; charset="utf-8" The guest MSR load/save list holds guest MSR values (SPEC_CTRL, LBR, PT, vPMU global control, and EFER on hardware without dedicated controls), but lives on the xenheap, where it stays reachable from every context. Move the page to the domain heap and map it at its fixed slot in the owning vCPU's state window, so that under Address Space Isolation it is only mapped while that vCPU's page tables are loaded. Hardware is unaffected: the VMCS MSR-area pointers hold the physical address, now derived from the page rather than from virt_to_maddr() on a directmap VA. The host MSR load list is left on the xenheap: it holds Xen's own exit-time values, not guest data, so relocating it would serve no isolation purpose. The window VA is only valid in the owning vCPU's context. Almost all accesses have v =3D=3D current and keep going through vmx->msr_area unchanged (the vmentry-path LBR fixups, the failed-vmentry dump, vPMU, and the MSR intercepts). The remote accessors -- vmx_add_msr() / vmx_del_msr() from vCPU construction, policy updates and context load, and single-entry reads/writes from the HVM context save/load and get/set-reg paths, all of which already require the target vCPU to be paused -- dispatch through a transient mapping of the page, dropped with unmap_vcpu_state_page(), which also drops this CPU's translations of it: * vmx_add_msr() and vmx_del_msr() map internally around their search/insert/remove. * vmx_read_guest_msr(), vmx_read_guest_loadonly_msr() and vmx_write_guest_msr() move out of line and do the same, via vmx_find_msr_in(), the existing lookup parameterised on the mapped area. vmx_find_msr() itself is unchanged for its remaining callers, which all run in current context and dereference the returned pointer through the window. vmx_destroy_vmcs() takes the page out of the window before freeing it, with unmap_vcpu_state(): without per-vCPU page tables the window is in the per-domain area that all its domain's vCPUs share, which outlives a vCPU whose creation fails. The new page pointer goes with the existing page_info pointers after pi_desc rather than next to msr_area: struct pi_desc is 64-byte aligned, so inserting eight bytes ahead of it would have added 56 bytes of padding, growing struct vmx_vcpu from 448 to 512 bytes and, via the union in struct arch_vcpu, every struct vcpu by 64 bytes (x86_64 defconfig). Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - Map and unmap through VMX_MAP_MSR_AREA() and VMX_UNMAP_MSR_AREA(), which clears the pointer (George). Changes in v4: - New in this version. --- xen/arch/x86/hvm/vmx/vmcs.c | 142 ++++++++++++++++++++++-- xen/arch/x86/include/asm/hvm/vmx/vmcs.h | 63 +++-------- 2 files changed, 147 insertions(+), 58 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index 8ec7ae7403..592722f1ca 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -1416,8 +1416,11 @@ static struct vmx_msr_entry *locate_msr_entry( return start; } =20 -struct vmx_msr_entry *vmx_find_msr(const struct vcpu *v, uint32_t msr, - enum vmx_msr_list_type type) +/* As vmx_find_msr(), but reaching the guest MSR area through @area. */ +static struct vmx_msr_entry *vmx_find_msr_in(const struct vcpu *v, + struct vmx_msr_entry *area, + uint32_t msr, + enum vmx_msr_list_type type) { const struct vmx_vcpu *vmx =3D &v->arch.hvm.vmx; struct vmx_msr_entry *start =3D NULL, *ent, *end; @@ -1435,11 +1438,11 @@ struct vmx_msr_entry *vmx_find_msr(const struct vcp= u *v, uint32_t msr, break; =20 case VMX_MSR_GUEST: - start =3D vmx->msr_area; + start =3D area; break; =20 case VMX_MSR_GUEST_LOADONLY: - start =3D vmx->msr_area; + start =3D area; substart =3D subend; subend =3D total; break; @@ -1458,6 +1461,77 @@ struct vmx_msr_entry *vmx_find_msr(const struct vcpu= *v, uint32_t msr, return ((ent < end) && (ent->index =3D=3D msr)) ? ent : NULL; } =20 +struct vmx_msr_entry *vmx_find_msr(const struct vcpu *v, uint32_t msr, + enum vmx_msr_list_type type) +{ + ASSERT(v =3D=3D current || type =3D=3D VMX_MSR_HOST); + + return vmx_find_msr_in(v, v->arch.hvm.vmx.msr_area, msr, type); +} + +static struct vmx_msr_entry *vmx_msr_area_map(const struct vcpu *v) +{ + const struct vmx_vcpu *vmx =3D &v->arch.hvm.vmx; + + if ( v =3D=3D current || !vmx->msr_area ) + return vmx->msr_area; + + return map_domain_page(page_to_mfn(vmx->msr_area_pg)); +} + +static void vmx_msr_area_unmap(const struct vcpu *v, + const struct vmx_msr_entry *area) +{ + if ( v !=3D current && area ) + unmap_vcpu_state_page(area); +} + +#define VMX_MAP_MSR_AREA(v) vmx_msr_area_map(v) + +/* Drops the mapping of a guest MSR area and nullifies its pointer. */ +#define VMX_UNMAP_MSR_AREA(v, area) do { \ + vmx_msr_area_unmap(v, area); \ + (area) =3D NULL; \ +} while ( false ) + +int vmx_read_guest_msr(const struct vcpu *v, uint32_t msr, uint64_t *val) +{ + struct vmx_msr_entry *area =3D VMX_MAP_MSR_AREA(v); + const struct vmx_msr_entry *ent =3D + area ? vmx_find_msr_in(v, area, msr, VMX_MSR_GUEST) : NULL; + + *val =3D ent ? ent->data : 0; + VMX_UNMAP_MSR_AREA(v, area); + + return ent ? 0 : -ESRCH; +} + +int vmx_read_guest_loadonly_msr(const struct vcpu *v, uint32_t msr, + uint64_t *val) +{ + struct vmx_msr_entry *area =3D VMX_MAP_MSR_AREA(v); + const struct vmx_msr_entry *ent =3D + area ? vmx_find_msr_in(v, area, msr, VMX_MSR_GUEST_LOADONLY) : NUL= L; + + *val =3D ent ? ent->data : 0; + VMX_UNMAP_MSR_AREA(v, area); + + return ent ? 0 : -ESRCH; +} + +int vmx_write_guest_msr(struct vcpu *v, uint32_t msr, uint64_t val) +{ + struct vmx_msr_entry *area =3D VMX_MAP_MSR_AREA(v); + struct vmx_msr_entry *ent =3D + area ? vmx_find_msr_in(v, area, msr, VMX_MSR_GUEST) : NULL; + + if ( ent ) + ent->data =3D val; + VMX_UNMAP_MSR_AREA(v, area); + + return ent ? 0 : -ESRCH; +} + int vmx_add_msr(struct vcpu *v, uint32_t msr, uint64_t val, enum vmx_msr_list_type type) { @@ -1503,13 +1577,38 @@ int vmx_add_msr(struct vcpu *v, uint32_t msr, uint6= 4_t val, { paddr_t addr; =20 - if ( (*ptr =3D alloc_xenheap_page()) =3D=3D NULL ) + if ( type =3D=3D VMX_MSR_HOST ) { - rc =3D -ENOMEM; - goto out; + if ( (*ptr =3D alloc_xenheap_page()) =3D=3D NULL ) + { + rc =3D -ENOMEM; + goto out; + } + + addr =3D virt_to_maddr(*ptr); } + else + { + struct page_info *pg =3D alloc_domheap_page(v->domain, + MEMF_no_owner); + mfn_t mfn; =20 - addr =3D virt_to_maddr(*ptr); + if ( pg =3D=3D NULL ) + { + rc =3D -ENOMEM; + goto out; + } + + mfn =3D page_to_mfn(pg); + clear_domain_page(mfn); + populate_perdomain_mapping(v, + VCPU_STATE_ADDR(v, + VCPU_STATE_IDX_VMX_= MSR), + &mfn, 1, __PAGE_HYPERVISOR_RW); + vmx->msr_area_pg =3D pg; + *ptr =3D (void *)VCPU_STATE_ADDR(v, VCPU_STATE_IDX_VMX_MSR); + addr =3D mfn_to_maddr(mfn); + } =20 switch ( type ) { @@ -1525,7 +1624,7 @@ int vmx_add_msr(struct vcpu *v, uint32_t msr, uint64_= t val, } } =20 - start =3D *ptr; + start =3D type =3D=3D VMX_MSR_HOST ? *ptr : VMX_MAP_MSR_AREA(v); end =3D start + total; ent =3D locate_msr_entry(start + substart, start + subend, msr); =20 @@ -1565,6 +1664,8 @@ int vmx_add_msr(struct vcpu *v, uint32_t msr, uint64_= t val, rc =3D 0; =20 out: + if ( type !=3D VMX_MSR_HOST ) + VMX_UNMAP_MSR_AREA(v, start); vmx_vmcs_exit(v); =20 return rc; @@ -1576,6 +1677,7 @@ int vmx_del_msr(struct vcpu *v, uint32_t msr, enum vm= x_msr_list_type type) struct vmx_msr_entry *start =3D NULL, *ent, *end; unsigned int substart =3D 0, subend =3D vmx->msr_save_count; unsigned int total =3D vmx->msr_load_count; + int rc =3D 0; =20 ASSERT(v =3D=3D current || !vcpu_runnable(v)); =20 @@ -1605,11 +1707,17 @@ int vmx_del_msr(struct vcpu *v, uint32_t msr, enum = vmx_msr_list_type type) if ( !start ) return -ESRCH; =20 + if ( type !=3D VMX_MSR_HOST ) + start =3D VMX_MAP_MSR_AREA(v); + end =3D start + total; ent =3D locate_msr_entry(start + substart, start + subend, msr); =20 if ( (ent =3D=3D end) || (ent->index !=3D msr) ) - return -ESRCH; + { + rc =3D -ESRCH; + goto out; + } =20 memmove(ent, ent + 1, sizeof(*ent) * (end - ent - 1)); =20 @@ -1632,7 +1740,11 @@ int vmx_del_msr(struct vcpu *v, uint32_t msr, enum v= mx_msr_list_type type) =20 vmx_vmcs_exit(v); =20 - return 0; + out: + if ( type !=3D VMX_MSR_HOST ) + VMX_UNMAP_MSR_AREA(v, start); + + return rc; } =20 void vmx_set_eoi_exit_bitmap(struct vcpu *v, u8 vector) @@ -1829,7 +1941,13 @@ void vmx_destroy_vmcs(struct vcpu *v) vmx_free_vmcs(vmx->vmcs_pa); =20 free_xenheap_page(v->arch.hvm.vmx.host_msr_area); - free_xenheap_page(v->arch.hvm.vmx.msr_area); + if ( v->arch.hvm.vmx.msr_area_pg ) + { + unmap_vcpu_state(v, VCPU_STATE_IDX_VMX_MSR, 1); + free_domheap_page(v->arch.hvm.vmx.msr_area_pg); + v->arch.hvm.vmx.msr_area_pg =3D NULL; + v->arch.hvm.vmx.msr_area =3D NULL; + } free_xenheap_page(v->arch.hvm.vmx.msr_bitmap); } =20 diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h b/xen/arch/x86/include= /asm/hvm/vmx/vmcs.h index d0716e97d2..e5730780e2 100644 --- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h +++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h @@ -140,6 +140,10 @@ struct vmx_vcpu { * context. However, the data can be modified by toolstack/migration * actions. Remote access is only permitted for paused vcpus, and is * protected under the domctl lock. + * + * The guest list is a domain-heap page (msr_area_pg, below) mapped at= a + * fixed slot in the owning vCPU's state window: msr_area holds that + * window VA, which is only valid in the owning vCPU's context. */ struct vmx_msr_entry *msr_area; struct vmx_msr_entry *host_msr_area; @@ -185,6 +189,9 @@ struct vmx_vcpu { =20 struct page_info *pml_pg; =20 + /* Backing page of msr_area; see the comment there. */ + struct page_info *msr_area_pg; + /* Bitmask of trapped CR4 bits. */ unsigned long cr4_host_mask; =20 @@ -669,55 +676,19 @@ static inline int vmx_add_host_load_msr(struct vcpu *= v, uint32_t msr, return vmx_add_msr(v, msr, val, VMX_MSR_HOST); } =20 +/* + * Look an MSR up in the guest or host list of @v. The returned pointer + * reaches the guest lists through the vCPU state window, so for those it + * is only valid (and the call only permitted) in current context; the + * single-entry accessors below handle paused remote vCPUs. + */ struct vmx_msr_entry *vmx_find_msr(const struct vcpu *v, uint32_t msr, enum vmx_msr_list_type type); =20 -static inline int vmx_read_guest_msr(const struct vcpu *v, uint32_t msr, - uint64_t *val) -{ - const struct vmx_msr_entry *ent =3D vmx_find_msr(v, msr, VMX_MSR_GUEST= ); - - if ( !ent ) - { - *val =3D 0; - return -ESRCH; - } - - *val =3D ent->data; - - return 0; -} - -static inline int vmx_read_guest_loadonly_msr( - const struct vcpu *v, uint32_t msr, uint64_t *val) -{ - const struct vmx_msr_entry *ent =3D - vmx_find_msr(v, msr, VMX_MSR_GUEST_LOADONLY); - - if ( !ent ) - { - *val =3D 0; - return -ESRCH; - } - - *val =3D ent->data; - - return 0; -} - -static inline int vmx_write_guest_msr(struct vcpu *v, uint32_t msr, - uint64_t val) -{ - struct vmx_msr_entry *ent =3D vmx_find_msr(v, msr, VMX_MSR_GUEST); - - if ( !ent ) - return -ESRCH; - - ent->data =3D val; - - return 0; -} - +int vmx_read_guest_msr(const struct vcpu *v, uint32_t msr, uint64_t *val); +int vmx_read_guest_loadonly_msr(const struct vcpu *v, uint32_t msr, + uint64_t *val); +int vmx_write_guest_msr(struct vcpu *v, uint32_t msr, uint64_t val); =20 /* MSR intercept bitmap infrastructure. */ enum vmx_msr_intercept_type { --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748716; cv=none; d=zohomail.com; s=zohoarc; b=nBxAAxpgbAGdE5PDZHe0Sbnh2BVgyaNu7+i32wn1Rw22nDANp/TUfcQI783t9APevqC7n1rCSzs6x8TvXka9G2euxnQqk/AF7IKVORPvvLwIvX4fhDaPNriSWvtf3Zv9s7oU9/U0ucLfZejUmDX2UKHvG7AR19TODLviqDRYr0E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748716; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NVGIgw99IFj4HEOzfc5+DvnxduyyMypiBLFSXW+Jamo=; b=ZEeHI5Di/MJsz3DsdM2m00ghensebtFI/8atPKVXOSi21p10U8hCfOL1eX+TjjdwM+3YNIitu3Qy0VM14D1CVU6MMe+X6SUzYMvOVCmzmxzEQ/8ShK9QVz5nMle0A0WTkDN08/GiCAP/va+4go8vFU/Bjmyj9VuR40PU3MeMGXY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748716678530.8485293869912; Tue, 3 Nov 2026 15:31:56 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470819.1679655 (Exim 4.92) (envelope-from ) id 1xONyL-0001Pi-Ie; Tue, 03 Nov 2026 23:31:21 +0000 Received: by outflank-mailman (output) from mailman id 1470819.1679655; Tue, 03 Nov 2026 23:31:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyK-0001JP-3b; Tue, 03 Nov 2026 23:31:20 +0000 Received: by outflank-mailman (input) for mailman id 1470819; Tue, 03 Nov 2026 23:30:41 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxh-0001fH-DV for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:41 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Couq-0h34; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo6-0000000EMFa-0a2k; Tue, 03 Nov 2026 23:20:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=NVGIgw99IFj4HEOzfc5+DvnxduyyMypiBLFSXW+Jamo=; b=u7pXdlPiOO2YZCVs5TKZwm+E7s PXwX5lElsgWzVSkR2TpMNP+x8aoyWlT5eppF2xihBKGFSaeigqO+dVS08ZKfLYyXHRSS2SAVJd2vT 3c0f1S4K6LDYUo6Uwkn/yvuTgUSTvBfgCu7V/4KWr/TADnIgDRSJuBKYv/7If6ZOvaO0=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 31/44] x86: access a vCPU's saved register frame through map/unmap helpers Date: Tue, 3 Nov 2026 17:13:53 -0500 Message-ID: <20261103-asi-hvm-v6-31-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748717844158500 Content-Type: text/plain; charset="utf-8" When a vCPU is switched out, __context_switch() copies its register frame from the stack into struct vcpu, and back when the vCPU is switched in. Code working on a vCPU other than the one running uses that copy: vCPU creation, XEN_DOMCTL_{get,set}vcpucontext, saving and loading the HVM CPU record, VCPUOP_initialise, INIT/SIPI and S3 resets, the result of a hypercall continued on another CPU, PV dom0 construction, and the debug-key dumps. Under Address Space Isolation the frame is to move out of struct vcpu, which is mapped in every context, into memory which other domains' contexts do not map. Its users then need a transient mapping whenever they run in another context. Prepare for that: introduce VCPU_MAP_REGS() and VCPU_UNMAP_REGS(), after VCPU_MAP_XSAVE_AREA(), and convert every user but the context switch, which moves the frame between the stack and its storage. The field becomes _user_regs, so that an access bypassing the helpers fails to build: once the frame can live elsewhere, such an access would silently use a stale copy. vcpu_show_registers() now prints from a copy of the frame for PV vCPUs as well as HVM ones, so that it holds no mapping while printing, and vmx_set_info_guest() reads EFLAGS.TF before engaging the VMCS. No functional change. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/arch/x86/domain.c | 94 ++++++++++++++++++------------- xen/arch/x86/domctl.c | 72 ++++++++++++----------- xen/arch/x86/hvm/domain.c | 6 +- xen/arch/x86/hvm/hvm.c | 89 ++++++++++++++++------------- xen/arch/x86/hvm/vmx/vmcs.c | 9 +-- xen/arch/x86/hvm/vmx/vmx.c | 7 ++- xen/arch/x86/hypercall.c | 4 +- xen/arch/x86/include/asm/domain.h | 28 ++++++++- xen/arch/x86/pv/dom0_build.c | 3 +- xen/arch/x86/traps.c | 20 ++++--- 10 files changed, 204 insertions(+), 128 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 633619b400..143f76babf 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -496,8 +496,12 @@ unsigned int arch_domain_struct_memflags(void) /* Initialise various registers to their architectural INIT/RESET state. */ void arch_vcpu_regs_init(struct vcpu *v) { - memset(&v->arch.user_regs, 0, sizeof(v->arch.user_regs)); - v->arch.user_regs.eflags =3D X86_EFLAGS_MBS; + struct cpu_user_regs *regs =3D VCPU_MAP_REGS(v); + + memset(regs, 0, sizeof(*regs)); + regs->eflags =3D X86_EFLAGS_MBS; + + VCPU_UNMAP_REGS(v, regs); =20 memset(v->arch.dr, 0, sizeof(v->arch.dr)); v->arch.dr6 =3D X86_DR6_DEFAULT; @@ -1152,6 +1156,7 @@ int arch_set_info_guest( { struct domain *d =3D v->domain; const struct cpu_policy *p =3D d->arch.cpu_policy; + struct cpu_user_regs *regs; unsigned int i; unsigned long flags; bool compat; @@ -1279,30 +1284,32 @@ int arch_set_info_guest( else vcpu_reset_fpu(v); =20 - memset(&v->arch.user_regs, 0, sizeof(v->arch.user_regs)); + regs =3D VCPU_MAP_REGS(v); + memset(regs, 0, sizeof(*regs)); =20 if ( !compat ) { - v->arch.user_regs.r15 =3D c.nat->user_regs.r15; - v->arch.user_regs.r14 =3D c.nat->user_regs.r14; - v->arch.user_regs.r13 =3D c.nat->user_regs.r13; - v->arch.user_regs.r12 =3D c.nat->user_regs.r12; - v->arch.user_regs.rbp =3D c.nat->user_regs.rbp; - v->arch.user_regs.rbx =3D c.nat->user_regs.rbx; - v->arch.user_regs.r11 =3D c.nat->user_regs.r11; - v->arch.user_regs.r10 =3D c.nat->user_regs.r10; - v->arch.user_regs.r9 =3D c.nat->user_regs.r9; - v->arch.user_regs.r8 =3D c.nat->user_regs.r8; - v->arch.user_regs.rax =3D c.nat->user_regs.rax; - v->arch.user_regs.rcx =3D c.nat->user_regs.rcx; - v->arch.user_regs.rdx =3D c.nat->user_regs.rdx; - v->arch.user_regs.rsi =3D c.nat->user_regs.rsi; - v->arch.user_regs.rdi =3D c.nat->user_regs.rdi; - v->arch.user_regs.rip =3D c.nat->user_regs.rip; - v->arch.user_regs.cs =3D c.nat->user_regs.cs; - v->arch.user_regs.rflags =3D (c.nat->user_regs.rflags &= X86_EFLAGS_ALL) | X86_EFLAGS_MBS; - v->arch.user_regs.rsp =3D c.nat->user_regs.rsp; - v->arch.user_regs.ss =3D c.nat->user_regs.ss; + regs->r15 =3D c.nat->user_regs.r15; + regs->r14 =3D c.nat->user_regs.r14; + regs->r13 =3D c.nat->user_regs.r13; + regs->r12 =3D c.nat->user_regs.r12; + regs->rbp =3D c.nat->user_regs.rbp; + regs->rbx =3D c.nat->user_regs.rbx; + regs->r11 =3D c.nat->user_regs.r11; + regs->r10 =3D c.nat->user_regs.r10; + regs->r9 =3D c.nat->user_regs.r9; + regs->r8 =3D c.nat->user_regs.r8; + regs->rax =3D c.nat->user_regs.rax; + regs->rcx =3D c.nat->user_regs.rcx; + regs->rdx =3D c.nat->user_regs.rdx; + regs->rsi =3D c.nat->user_regs.rsi; + regs->rdi =3D c.nat->user_regs.rdi; + regs->rip =3D c.nat->user_regs.rip; + regs->cs =3D c.nat->user_regs.cs; + regs->rflags =3D + (c.nat->user_regs.rflags & X86_EFLAGS_ALL) | X86_EFLAGS_MBS; + regs->rsp =3D c.nat->user_regs.rsp; + regs->ss =3D c.nat->user_regs.ss; v->arch.pv.es =3D c.nat->user_regs.es; v->arch.pv.ds =3D c.nat->user_regs.ds; v->arch.pv.fs =3D c.nat->user_regs.fs; @@ -1315,18 +1322,19 @@ int arch_set_info_guest( #ifdef CONFIG_COMPAT else { - v->arch.user_regs.ebx =3D c.cmp->user_regs.ebx; - v->arch.user_regs.ecx =3D c.cmp->user_regs.ecx; - v->arch.user_regs.edx =3D c.cmp->user_regs.edx; - v->arch.user_regs.esi =3D c.cmp->user_regs.esi; - v->arch.user_regs.edi =3D c.cmp->user_regs.edi; - v->arch.user_regs.ebp =3D c.cmp->user_regs.ebp; - v->arch.user_regs.eax =3D c.cmp->user_regs.eax; - v->arch.user_regs.eip =3D c.cmp->user_regs.eip; - v->arch.user_regs.cs =3D c.cmp->user_regs.cs; - v->arch.user_regs.eflags =3D (c.cmp->user_regs.eflags &= X86_EFLAGS_ALL) | X86_EFLAGS_MBS; - v->arch.user_regs.esp =3D c.cmp->user_regs.esp; - v->arch.user_regs.ss =3D c.cmp->user_regs.ss; + regs->ebx =3D c.cmp->user_regs.ebx; + regs->ecx =3D c.cmp->user_regs.ecx; + regs->edx =3D c.cmp->user_regs.edx; + regs->esi =3D c.cmp->user_regs.esi; + regs->edi =3D c.cmp->user_regs.edi; + regs->ebp =3D c.cmp->user_regs.ebp; + regs->eax =3D c.cmp->user_regs.eax; + regs->eip =3D c.cmp->user_regs.eip; + regs->cs =3D c.cmp->user_regs.cs; + regs->eflags =3D + (c.cmp->user_regs.eflags & X86_EFLAGS_ALL) | X86_EFLAGS_MBS; + regs->esp =3D c.cmp->user_regs.esp; + regs->ss =3D c.cmp->user_regs.ss; v->arch.pv.es =3D c.cmp->user_regs.es; v->arch.pv.ds =3D c.cmp->user_regs.ds; v->arch.pv.fs =3D c.cmp->user_regs.fs; @@ -1341,6 +1349,8 @@ int arch_set_info_guest( } #endif =20 + VCPU_UNMAP_REGS(v, regs); + if ( v->vcpu_id =3D=3D 0 && (c(vm_assist) & ~arch_vm_assist_valid_mask= (d)) ) return -EINVAL; =20 @@ -1359,12 +1369,16 @@ int arch_set_info_guest( } =20 #ifdef CONFIG_PV + regs =3D VCPU_MAP_REGS(v); + /* IOPL privileges are virtualised. */ - v->arch.pv.iopl =3D v->arch.user_regs.eflags & X86_EFLAGS_IOPL; - v->arch.user_regs.eflags &=3D ~X86_EFLAGS_IOPL; + v->arch.pv.iopl =3D regs->eflags & X86_EFLAGS_IOPL; + regs->eflags &=3D ~X86_EFLAGS_IOPL; =20 /* Ensure real hardware interrupts are enabled. */ - v->arch.user_regs.eflags |=3D X86_EFLAGS_IF; + regs->eflags |=3D X86_EFLAGS_IF; + + VCPU_UNMAP_REGS(v, regs); =20 nr_gdt_frames =3D DIV_ROUND_UP(c(gdt_ents), 512); if ( nr_gdt_frames > ARRAY_SIZE(v->arch.pv.gdt_frames) ) @@ -2135,7 +2149,7 @@ static void __context_switch(void) if ( !is_idle_domain(pd) ) { ASSERT(read_atomic(&p->dirty_cpu) =3D=3D cpu); - memcpy(&p->arch.user_regs, stack_regs, CTXT_SWITCH_STACK_BYTES); + memcpy(&p->arch._user_regs, stack_regs, CTXT_SWITCH_STACK_BYTES); vcpu_save_fpu(p); pd->arch.ctxt_switch->from(p); } @@ -2151,7 +2165,7 @@ static void __context_switch(void) =20 if ( !is_idle_domain(nd) ) { - memcpy(stack_regs, &n->arch.user_regs, CTXT_SWITCH_STACK_BYTES); + memcpy(stack_regs, &n->arch._user_regs, CTXT_SWITCH_STACK_BYTES); if ( cpu_has_xsave ) { if ( !set_xcr0(n->arch.xcr0 ?: XSTATE_FP_SSE) ) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 1e00510702..04bcde5580 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1461,6 +1461,7 @@ void arch_get_info_guest(struct vcpu *v, vcpu_guest_c= ontext_u c) const struct domain *d =3D v->domain; bool compat =3D is_pv_32bit_domain(d); const struct xsave_struct *xsave_area; + const struct cpu_user_regs *regs; #ifdef CONFIG_COMPAT #define c(fld) (!compat ? (c.nat->fld) : (c.cmp->fld)) #else @@ -1479,29 +1480,32 @@ void arch_get_info_guest(struct vcpu *v, vcpu_guest= _context_u c) c(flags |=3D VGCF_i387_valid); if ( !(v->pause_flags & VPF_down) ) c(flags |=3D VGCF_online); + + regs =3D VCPU_MAP_REGS(v); + if ( !compat ) { /* Backing memory is pre-zeroed. */ - c.nat->user_regs.r15 =3D v->arch.user_regs.r15; - c.nat->user_regs.r14 =3D v->arch.user_regs.r14; - c.nat->user_regs.r13 =3D v->arch.user_regs.r13; - c.nat->user_regs.r12 =3D v->arch.user_regs.r12; - c.nat->user_regs.rbp =3D v->arch.user_regs.rbp; - c.nat->user_regs.rbx =3D v->arch.user_regs.rbx; - c.nat->user_regs.r11 =3D v->arch.user_regs.r11; - c.nat->user_regs.r10 =3D v->arch.user_regs.r10; - c.nat->user_regs.r9 =3D v->arch.user_regs.r9; - c.nat->user_regs.r8 =3D v->arch.user_regs.r8; - c.nat->user_regs.rax =3D v->arch.user_regs.rax; - c.nat->user_regs.rcx =3D v->arch.user_regs.rcx; - c.nat->user_regs.rdx =3D v->arch.user_regs.rdx; - c.nat->user_regs.rsi =3D v->arch.user_regs.rsi; - c.nat->user_regs.rdi =3D v->arch.user_regs.rdi; - c.nat->user_regs.rip =3D v->arch.user_regs.rip; - c.nat->user_regs.cs =3D v->arch.user_regs.cs; - c.nat->user_regs.rflags =3D v->arch.user_regs.rflags; - c.nat->user_regs.rsp =3D v->arch.user_regs.rsp; - c.nat->user_regs.ss =3D v->arch.user_regs.ss; + c.nat->user_regs.r15 =3D regs->r15; + c.nat->user_regs.r14 =3D regs->r14; + c.nat->user_regs.r13 =3D regs->r13; + c.nat->user_regs.r12 =3D regs->r12; + c.nat->user_regs.rbp =3D regs->rbp; + c.nat->user_regs.rbx =3D regs->rbx; + c.nat->user_regs.r11 =3D regs->r11; + c.nat->user_regs.r10 =3D regs->r10; + c.nat->user_regs.r9 =3D regs->r9; + c.nat->user_regs.r8 =3D regs->r8; + c.nat->user_regs.rax =3D regs->rax; + c.nat->user_regs.rcx =3D regs->rcx; + c.nat->user_regs.rdx =3D regs->rdx; + c.nat->user_regs.rsi =3D regs->rsi; + c.nat->user_regs.rdi =3D regs->rdi; + c.nat->user_regs.rip =3D regs->rip; + c.nat->user_regs.cs =3D regs->cs; + c.nat->user_regs.rflags =3D regs->rflags; + c.nat->user_regs.rsp =3D regs->rsp; + c.nat->user_regs.ss =3D regs->ss; c.nat->user_regs.es =3D v->arch.pv.es; c.nat->user_regs.ds =3D v->arch.pv.ds; c.nat->user_regs.fs =3D v->arch.pv.fs; @@ -1515,18 +1519,18 @@ void arch_get_info_guest(struct vcpu *v, vcpu_guest= _context_u c) else { /* Backing memory is pre-zeroed. */ - c.cmp->user_regs.ebp =3D v->arch.user_regs.ebp; - c.cmp->user_regs.ebx =3D v->arch.user_regs.ebx; - c.cmp->user_regs.eax =3D v->arch.user_regs.eax; - c.cmp->user_regs.ecx =3D v->arch.user_regs.ecx; - c.cmp->user_regs.edx =3D v->arch.user_regs.edx; - c.cmp->user_regs.esi =3D v->arch.user_regs.esi; - c.cmp->user_regs.edi =3D v->arch.user_regs.edi; - c.cmp->user_regs.eip =3D v->arch.user_regs.eip; - c.cmp->user_regs.cs =3D v->arch.user_regs.cs; - c.cmp->user_regs.eflags =3D v->arch.user_regs.eflags; - c.cmp->user_regs.esp =3D v->arch.user_regs.esp; - c.cmp->user_regs.ss =3D v->arch.user_regs.ss; + c.cmp->user_regs.ebp =3D regs->ebp; + c.cmp->user_regs.ebx =3D regs->ebx; + c.cmp->user_regs.eax =3D regs->eax; + c.cmp->user_regs.ecx =3D regs->ecx; + c.cmp->user_regs.edx =3D regs->edx; + c.cmp->user_regs.esi =3D regs->esi; + c.cmp->user_regs.edi =3D regs->edi; + c.cmp->user_regs.eip =3D regs->eip; + c.cmp->user_regs.cs =3D regs->cs; + c.cmp->user_regs.eflags =3D regs->eflags; + c.cmp->user_regs.esp =3D regs->esp; + c.cmp->user_regs.ss =3D regs->ss; c.cmp->user_regs.es =3D v->arch.pv.es; c.cmp->user_regs.ds =3D v->arch.pv.ds; c.cmp->user_regs.fs =3D v->arch.pv.fs; @@ -1638,10 +1642,12 @@ void arch_get_info_guest(struct vcpu *v, vcpu_guest= _context_u c) } #endif =20 - if ( guest_kernel_mode(v, &v->arch.user_regs) ) + if ( guest_kernel_mode(v, regs) ) c(flags |=3D VGCF_in_kernel); } =20 + VCPU_UNMAP_REGS(v, regs); + c(vm_assist =3D d->vm_assist); #undef c } diff --git a/xen/arch/x86/hvm/domain.c b/xen/arch/x86/hvm/domain.c index 978d8b2222..71e16e25ba 100644 --- a/xen/arch/x86/hvm/domain.c +++ b/xen/arch/x86/hvm/domain.c @@ -103,7 +103,7 @@ static int check_segment(struct segment_register *reg, = enum x86_segment seg) int arch_set_info_hvm_guest(struct vcpu *v, const struct vcpu_hvm_context = *ctx) { const struct domain *d =3D v->domain; - struct cpu_user_regs *uregs =3D &v->arch.user_regs; + struct cpu_user_regs *uregs; struct segment_register cs, ds, ss, es, tr; const char *errstr; int rc; @@ -188,6 +188,7 @@ int arch_set_info_hvm_guest(struct vcpu *v, const struc= t vcpu_hvm_context *ctx) return -EINVAL; } =20 + uregs =3D VCPU_MAP_REGS(v); uregs->rax =3D regs->eax; uregs->rcx =3D regs->ecx; uregs->rdx =3D regs->edx; @@ -198,6 +199,7 @@ int arch_set_info_hvm_guest(struct vcpu *v, const struc= t vcpu_hvm_context *ctx) uregs->rdi =3D regs->edi; uregs->rip =3D regs->eip; uregs->rflags =3D (regs->eflags & X86_EFLAGS_ALL) | X86_EFLAGS_MBS; + VCPU_UNMAP_REGS(v, uregs); =20 v->arch.hvm.guest_cr[0] =3D regs->cr0; v->arch.hvm.guest_cr[3] =3D regs->cr3; @@ -239,6 +241,7 @@ int arch_set_info_hvm_guest(struct vcpu *v, const struc= t vcpu_hvm_context *ctx) return -EINVAL; } =20 + uregs =3D VCPU_MAP_REGS(v); uregs->rax =3D regs->rax; uregs->rcx =3D regs->rcx; uregs->rdx =3D regs->rdx; @@ -249,6 +252,7 @@ int arch_set_info_hvm_guest(struct vcpu *v, const struc= t vcpu_hvm_context *ctx) uregs->rdi =3D regs->rdi; uregs->rip =3D regs->rip; uregs->rflags =3D (regs->rflags & X86_EFLAGS_ALL) | X86_EFLAGS_MBS; + VCPU_UNMAP_REGS(v, uregs); =20 v->arch.hvm.guest_cr[0] =3D regs->cr0; v->arch.hvm.guest_cr[3] =3D regs->cr3; diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 71da15c0a4..eadccf3d79 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -827,28 +827,29 @@ HVM_REGISTER_SAVE_RESTORE(TSC_ADJUST, hvm_save_tsc_ad= just, NULL, =20 static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t= *h) { + const struct cpu_user_regs *regs =3D VCPU_MAP_REGS(v); struct segment_register seg; struct hvm_hw_cpu ctxt =3D { .tsc =3D hvm_get_guest_tsc_fixed(v, v->domain->arch.hvm.sync_tsc), .msr_tsc_aux =3D v->arch.msrs->tsc_aux, - .rax =3D v->arch.user_regs.rax, - .rbx =3D v->arch.user_regs.rbx, - .rcx =3D v->arch.user_regs.rcx, - .rdx =3D v->arch.user_regs.rdx, - .rbp =3D v->arch.user_regs.rbp, - .rsi =3D v->arch.user_regs.rsi, - .rdi =3D v->arch.user_regs.rdi, - .rsp =3D v->arch.user_regs.rsp, - .rip =3D v->arch.user_regs.rip, - .rflags =3D v->arch.user_regs.rflags, - .r8 =3D v->arch.user_regs.r8, - .r9 =3D v->arch.user_regs.r9, - .r10 =3D v->arch.user_regs.r10, - .r11 =3D v->arch.user_regs.r11, - .r12 =3D v->arch.user_regs.r12, - .r13 =3D v->arch.user_regs.r13, - .r14 =3D v->arch.user_regs.r14, - .r15 =3D v->arch.user_regs.r15, + .rax =3D regs->rax, + .rbx =3D regs->rbx, + .rcx =3D regs->rcx, + .rdx =3D regs->rdx, + .rbp =3D regs->rbp, + .rsi =3D regs->rsi, + .rdi =3D regs->rdi, + .rsp =3D regs->rsp, + .rip =3D regs->rip, + .rflags =3D regs->rflags, + .r8 =3D regs->r8, + .r9 =3D regs->r9, + .r10 =3D regs->r10, + .r11 =3D regs->r11, + .r12 =3D regs->r12, + .r13 =3D regs->r13, + .r14 =3D regs->r14, + .r15 =3D regs->r15, .cr0 =3D v->arch.hvm.guest_cr[0], .cr2 =3D v->arch.hvm.guest_cr[2], .cr3 =3D v->arch.hvm.guest_cr[3], @@ -863,6 +864,8 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, h= vm_domain_context_t *h) }; const struct xsave_struct *xsave_area; =20 + VCPU_UNMAP_REGS(v, regs); + /* * We don't need to save state for a vcpu that is down; the restore * code will leave it down if there is nothing saved. @@ -1020,6 +1023,7 @@ static int cf_check hvm_load_cpu_ctxt(struct domain *= d, hvm_domain_context_t *h) const struct cpu_policy *p =3D d->arch.cpu_policy; unsigned int vcpuid =3D hvm_load_instance(h); struct vcpu *v; + struct cpu_user_regs *regs; struct hvm_hw_cpu ctxt; struct segment_register seg; const char *errstr; @@ -1182,24 +1186,26 @@ static int cf_check hvm_load_cpu_ctxt(struct domain= *d, hvm_domain_context_t *h) else vcpu_reset_fpu(v); =20 - v->arch.user_regs.rax =3D ctxt.rax; - v->arch.user_regs.rbx =3D ctxt.rbx; - v->arch.user_regs.rcx =3D ctxt.rcx; - v->arch.user_regs.rdx =3D ctxt.rdx; - v->arch.user_regs.rbp =3D ctxt.rbp; - v->arch.user_regs.rsi =3D ctxt.rsi; - v->arch.user_regs.rdi =3D ctxt.rdi; - v->arch.user_regs.rsp =3D ctxt.rsp; - v->arch.user_regs.rip =3D ctxt.rip; - v->arch.user_regs.rflags =3D ctxt.rflags | X86_EFLAGS_MBS; - v->arch.user_regs.r8 =3D ctxt.r8; - v->arch.user_regs.r9 =3D ctxt.r9; - v->arch.user_regs.r10 =3D ctxt.r10; - v->arch.user_regs.r11 =3D ctxt.r11; - v->arch.user_regs.r12 =3D ctxt.r12; - v->arch.user_regs.r13 =3D ctxt.r13; - v->arch.user_regs.r14 =3D ctxt.r14; - v->arch.user_regs.r15 =3D ctxt.r15; + regs =3D VCPU_MAP_REGS(v); + regs->rax =3D ctxt.rax; + regs->rbx =3D ctxt.rbx; + regs->rcx =3D ctxt.rcx; + regs->rdx =3D ctxt.rdx; + regs->rbp =3D ctxt.rbp; + regs->rsi =3D ctxt.rsi; + regs->rdi =3D ctxt.rdi; + regs->rsp =3D ctxt.rsp; + regs->rip =3D ctxt.rip; + regs->rflags =3D ctxt.rflags | X86_EFLAGS_MBS; + regs->r8 =3D ctxt.r8; + regs->r9 =3D ctxt.r9; + regs->r10 =3D ctxt.r10; + regs->r11 =3D ctxt.r11; + regs->r12 =3D ctxt.r12; + regs->r13 =3D ctxt.r13; + regs->r14 =3D ctxt.r14; + regs->r15 =3D ctxt.r15; + VCPU_UNMAP_REGS(v, regs); v->arch.dr[0] =3D ctxt.dr0; v->arch.dr[1] =3D ctxt.dr1; v->arch.dr[2] =3D ctxt.dr2; @@ -2563,7 +2569,7 @@ bool hvm_vcpu_virtual_to_linear( { unsigned long addr =3D offset, last_byte; const struct cpu_user_regs *regs =3D v =3D=3D current ? guest_cpu_user= _regs() - : &v->arch.user_regs; + : VCPU_MAP_REGS(v); bool okay =3D false; =20 ASSERT(v =3D=3D current || !vcpu_runnable(v)); @@ -2688,6 +2694,10 @@ bool hvm_vcpu_virtual_to_linear( * failed. The permissions failure is not relevant to some callers. */ *linear_addr =3D addr; + + if ( v !=3D current ) + VCPU_UNMAP_REGS(v, regs); + return okay; } =20 @@ -3956,6 +3966,7 @@ void hvm_init_hypercall_page(struct domain *d, void *= ptr) void hvm_vcpu_reset_state(struct vcpu *v, uint16_t cs, uint16_t ip) { struct domain *d =3D v->domain; + struct cpu_user_regs *regs; struct segment_register reg; =20 domain_lock(d); @@ -3973,7 +3984,9 @@ void hvm_vcpu_reset_state(struct vcpu *v, uint16_t cs= , uint16_t ip) vcpu_reset_fpu(v); =20 arch_vcpu_regs_init(v); - v->arch.user_regs.rip =3D ip; + regs =3D VCPU_MAP_REGS(v); + regs->rip =3D ip; + VCPU_UNMAP_REGS(v, regs); =20 v->arch.hvm.guest_cr[0] =3D X86_CR0_ET; hvm_update_guest_cr(v, 0); diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index 592722f1ca..78b0af29f2 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -2083,15 +2083,13 @@ static void vmx_dump_sel2(const char *name, uint32_= t lim) =20 void vmcs_dump_vcpu(struct vcpu *v) { - struct cpu_user_regs *regs =3D &v->arch.user_regs; + const struct cpu_user_regs *regs =3D v =3D=3D current ? guest_cpu_user= _regs() + : VCPU_MAP_REGS(v); uint32_t vmentry_ctl, vmexit_ctl; unsigned long cr4; uint64_t efer; unsigned int i, n; =20 - if ( v =3D=3D current ) - regs =3D guest_cpu_user_regs(); - vmx_vmcs_enter(v); =20 vmentry_ctl =3D vmr32(VM_ENTRY_CONTROLS), @@ -2236,6 +2234,9 @@ void vmcs_dump_vcpu(struct vcpu *v) vmr16(VIRTUAL_PROCESSOR_ID), vmr(VM_FUNCTION_CONTROL)); =20 vmx_vmcs_exit(v); + + if ( v !=3D current ) + VCPU_UNMAP_REGS(v, regs); } =20 static void cf_check vmcs_dump(unsigned char ch) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 3c7374a9ff..bdbcd0ad4d 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -2159,8 +2159,12 @@ static bool cf_check vmx_event_pending(const struct = vcpu *v) =20 static void cf_check vmx_set_info_guest(struct vcpu *v) { + const struct cpu_user_regs *regs =3D VCPU_MAP_REGS(v); + bool single_step =3D regs->eflags & X86_EFLAGS_TF; unsigned long intr_shadow; =20 + VCPU_UNMAP_REGS(v, regs); + vmx_vmcs_enter(v); =20 __vmwrite(GUEST_DR7, v->arch.dr7); @@ -2174,8 +2178,7 @@ static void cf_check vmx_set_info_guest(struct vcpu *= v) * immediately vmexit and hence make no progress. */ __vmread(GUEST_INTERRUPTIBILITY_INFO, &intr_shadow); - if ( v->domain->debugger_attached && - (v->arch.user_regs.eflags & X86_EFLAGS_TF) && + if ( v->domain->debugger_attached && single_step && (intr_shadow & VMX_INTR_SHADOW_STI) ) { intr_shadow &=3D ~VMX_INTR_SHADOW_STI; diff --git a/xen/arch/x86/hypercall.c b/xen/arch/x86/hypercall.c index dc0a90ca09..bfdc462874 100644 --- a/xen/arch/x86/hypercall.c +++ b/xen/arch/x86/hypercall.c @@ -107,9 +107,11 @@ unsigned long hypercall_create_continuation( =20 void arch_hypercall_tasklet_result(struct vcpu *v, long res) { - struct cpu_user_regs *regs =3D &v->arch.user_regs; + struct cpu_user_regs *regs =3D VCPU_MAP_REGS(v); =20 regs->rax =3D res; + + VCPU_UNMAP_REGS(v, regs); } =20 int hypercall_xlat_continuation(unsigned int *id, unsigned int nr, diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 0e405157b4..457eb69ace 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -632,7 +632,12 @@ struct pv_vcpu =20 struct arch_vcpu { - struct cpu_user_regs user_regs; + /* + * The vCPU's register frame while it is not running: __context_switch= () + * copies it here from the stack when the vCPU is switched out, and ba= ck + * when it is switched in. Everything else goes through VCPU_MAP_REGS= (). + */ + struct cpu_user_regs _user_regs; =20 /* Debug registers. */ unsigned long dr[4]; @@ -741,6 +746,27 @@ static inline void free_vcpu_guest_context(struct vcpu= _guest_context *vgc) vfree(vgc); } =20 +/* + * Fetch a pointer to the register frame a vCPU was last switched out with, + * or which a toolstack, debugger or reset has put there since: the vCPU's + * registers for code working on a vCPU other than the one running (whose + * registers are on the stack, see guest_cpu_user_regs()). + * + * The pointer is valid until VCPU_UNMAP_REGS(), which clears it. A mappi= ng + * must not be held across a scheduling point. + * + * @param v Owner of the register frame + */ +#define VCPU_MAP_REGS(v) (&(v)->arch._user_regs) + +/* + * Drops the mapping of a vCPU's register frame and nullifies its pointer. + * + * @param v Owner of the register frame + * @param r Register frame of v, from VCPU_MAP_REGS() + */ +#define VCPU_UNMAP_REGS(v, r) do { (void)(v); (r) =3D NULL; } while ( fals= e ) + void arch_vcpu_regs_init(struct vcpu *v); =20 struct vcpu_hvm_context; diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index 829eafd67d..a08362df16 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -1007,7 +1007,7 @@ static int __init dom0_construct(const struct boot_do= main *bd) * rSI =3D start_info * [rAX,rBX,rCX,rDX,rDI,rBP,R8-R15 are zero] */ - regs =3D &v->arch.user_regs; + regs =3D VCPU_MAP_REGS(v); v->arch.pv.ds =3D (compat ? FLAT_COMPAT_KERNEL_DS : FLAT_KERNEL_DS); v->arch.pv.es =3D (compat ? FLAT_COMPAT_KERNEL_DS : FLAT_KERNEL_DS); v->arch.pv.fs =3D (compat ? FLAT_COMPAT_KERNEL_DS : FLAT_KERNEL_DS); @@ -1018,6 +1018,7 @@ static int __init dom0_construct(const struct boot_do= main *bd) regs->rsp =3D vstack_end; regs->rsi =3D vstartinfo_start; regs->eflags =3D X86_EFLAGS_IF | X86_EFLAGS_MBS; + VCPU_UNMAP_REGS(v, regs); =20 /* * We don't call arch_set_info_guest(), so some initialisation needs d= oing diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index ac01ba0872..3b4a01ba52 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -273,16 +273,17 @@ void show_registers(const struct cpu_user_regs *regs) =20 void vcpu_show_registers(struct vcpu *v) { - const struct cpu_user_regs *regs =3D &v->arch.user_regs; - struct cpu_user_regs aux_regs; + const struct cpu_user_regs *regs =3D VCPU_MAP_REGS(v); + struct cpu_user_regs aux_regs =3D *regs; struct extra_state state; enum context context; =20 + VCPU_UNMAP_REGS(v, regs); + regs =3D &aux_regs; + if ( is_hvm_vcpu(v) ) { - aux_regs =3D *regs; get_hvm_registers(v, &aux_regs, &state); - regs =3D &aux_regs; context =3D CTXT_hvm_guest; } else @@ -916,6 +917,7 @@ void cf_check show_execution_state(const struct cpu_use= r_regs *regs) =20 void vcpu_show_execution_state(struct vcpu *v) { + const struct cpu_user_regs *regs; unsigned long flags =3D 0; =20 if ( test_bit(_VPF_down, &v->pause_flags) ) @@ -961,12 +963,16 @@ void vcpu_show_execution_state(struct vcpu *v) */ console_unlock_recursive_irqrestore(flags); =20 - show_hvm_stack(v, &v->arch.user_regs); + regs =3D VCPU_MAP_REGS(v); + show_hvm_stack(v, regs); + VCPU_UNMAP_REGS(v, regs); } else { - if ( guest_kernel_mode(v, &v->arch.user_regs) ) - show_guest_stack(v, &v->arch.user_regs); + regs =3D VCPU_MAP_REGS(v); + if ( guest_kernel_mode(v, regs) ) + show_guest_stack(v, regs); + VCPU_UNMAP_REGS(v, regs); =20 console_unlock_recursive_irqrestore(flags); } --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748662; cv=none; d=zohomail.com; s=zohoarc; b=n+Iyfc8Ti1t5o67GIhdJOwRITSwLbwXUx5G7vQvRvuwS5fhXZ+HHuz7bXP94arJCuHaKwEKIeShzEEjVR6sI82VoYGNvJT6TsMJwolcjJ1e3R54dfVo0xCn1vl4IIVV3vfUNQlM/GlDVURQL1/HpdNd1j84LiR5YLcMfP9Ihdcs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748662; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZOf/B1UVKOtb3aCxqsyxbTnFEMbHfy0ZTl7SCHGJvtY=; b=GcjmrYj5bNPzpJ4bUEzD97+K17QuLHBpXof1AjjMzZIHTqKzbb9Mx3Koz1OSG2rjuXUioBCW/nS+OU2ByT63LCOMfWVxZ6bT8qgLa3AqK4wrHH+YxHvbUtrzMiUgFzhMxZ/ZYv4fMkhmzElhQ3fkympn32HhytugqxAbY5ZATpg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748662310867.3441320558828; Tue, 3 Nov 2026 15:31:02 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470792.1679433 (Exim 4.92) (envelope-from ) id 1xONxf-00015b-FC; Tue, 03 Nov 2026 23:30:39 +0000 Received: by outflank-mailman (output) from mailman id 1470792.1679433; Tue, 03 Nov 2026 23:30:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxf-00013D-5m; Tue, 03 Nov 2026 23:30:39 +0000 Received: by outflank-mailman (input) for mailman id 1470792; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxb-0000cR-Ue for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:35 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotE-2nd9; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo6-0000000EMFa-2CES; Tue, 03 Nov 2026 23:20:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=ZOf/B1UVKOtb3aCxqsyxbTnFEMbHfy0ZTl7SCHGJvtY=; b=L3hfN98vo62L+nSTqXOD4KSUXO qNpyPmmtZkQ7eHEjXpQt8TtC5bDbE9Ml6sV8aT41QrolraK8mJk/tPAnnKPfbZrmlhKjLpsxi9s64 uo7x6fk/IbLTo6Ta9yl/7tJ6G0o7QhH6KIcQsssGHP15BuWn6nnCfQjjkkjRWGFOs2V4=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 32/44] x86: move the saved register frame behind the vCPU state window Date: Tue, 3 Nov 2026 17:13:54 -0500 Message-ID: <20261103-asi-hvm-v6-32-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748663811158500 Content-Type: text/plain; charset="utf-8" While a vCPU is not running, its register frame is saved in struct vcpu: a xenheap page, mapped in every context, the sparse directmap view included. Under Address Space Isolation that leaves the general-purpose registers of every descheduled vCPU reachable from other domains' contexts, whether or not the vCPU's own domain is on the sparse view. Keep the frame instead in a domain-heap page of the vCPU's own, mapped for the life of the vCPU at a new slot (index 6) of its state window, like the SVM VMCB and the VMX MSR list; indexes 3-5 are left for the guest XSAVE area, which takes up to three pages. Like the VMCB and MSR list moves, this does not depend on asi=3D, and it applies to every guest vCPU, PV and HVM: struct vcpu no longer holds a copy. The page is allocated in arch_vcpu_create() ahead of arch_vcpu_regs_init(), which initialises the frame, and unmapped before it is freed, on the failure path as at teardown. An HVM vCPU using per-vCPU page tables has its window in its own page tables. For any other vCPU, PV ones included, the window is in the per-domain area that all its domain's vCPUs share, which only that domain's page tables map: a context on the sparse view reaches no other domain's frames. That includes a domain left off the sparse view, such as dom0 under asi=3D1,dom0=3D0, whose vCPUs' frames are no longer mapped in the contexts of the domUs on the sparse view. __context_switch() saves the outgoing vCPU's frame while its page tables are still loaded, and now restores the incoming vCPU's only after write_ptbase(), once its window is mapped. Nothing in between reads the frame: set_xcr0(), the FPU restore, the ctxt_switch->to() hooks, and the PSR and GDT updates. Both copies go through the window address, with interrupts off; a transient mapping would complete the lazy context switch and re-enter __context_switch(). The save is skipped if the outgoing vCPU's page tables are no longer loaded. Only the shadow-mode paths which abandon a PV vCPU's page tables early (update_cr3() failure in new_guest_cr3() and _toggle_guest_pt()) leave them so, loading the idle page tables once they have crashed the domain; anything else reaching that point is a page-table tracking bug, fatal in every build. After such a crash the vCPU's saved frame is the one it was last switched in with, not its registers at the crash, and that is what XEN_DOMCTL_getvcpucontext and the debug keys report for it. VCPU_MAP_REGS() returns the window address when the vCPU is running here on its own page tables (v =3D=3D current and this_cpu(pgtable_vcpu) = =3D=3D v), and otherwise a transient mapping of the page, which, for a single page, cannot fail. Its users all work on a vCPU other than the one running here, normally paused or not yet run: vCPU creation, the toolstack's vCPU context, the HVM CPU record, VCPUOP_initialise, INIT/SIPI and S3 resets, a hypercall continued on another CPU, PV dom0 construction, and the debug-key dumps. The vCPU's own page tables being loaded is not enough: while a CPU holds a vCPU's state lazily they are loaded there, but an IPI syncing that state for another CPU can switch them away at any time. VCPU_UNMAP_REGS() drops a transient mapping with unmap_vcpu_state_page(), which also flushes this CPU's translation. The live frame on the pCPU's stack, which holds a running vCPU's registers, is not affected. Cost: a page per guest vCPU, and a transient mapping on each of the cold paths above. On every switch into a guest vCPU, the restore reaches the frame through a 4KiB, non-global PTE in the per-domain area, just after switch_cr3_cr4() has flushed the TLB, so it takes a page walk, where struct vcpu was reached through a superpage of the full directmap. struct vcpu shrinks from 3072 to 2880 bytes (x86_64 defconfig). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/arch/x86/domain.c | 98 ++++++++++++++++++++++++++++++- xen/arch/x86/include/asm/config.h | 1 + xen/arch/x86/include/asm/domain.h | 26 +++++--- 3 files changed, 116 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 143f76babf..03f6572086 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -493,6 +494,66 @@ unsigned int arch_domain_struct_memflags(void) return MEMF_bits(bits); } =20 +/* + * A vCPU keeps its register frame, while it is not running, in a domain-h= eap + * page of its own, mapped for the life of the vCPU at a fixed slot of its + * state window. + */ +static struct cpu_user_regs *regs_window(const struct vcpu *v) +{ + return (void *)VCPU_STATE_ADDR(v, VCPU_STATE_IDX_REGS); +} + +static int vcpu_regs_alloc(struct vcpu *v) +{ + struct page_info *pg =3D alloc_domheap_page(v->domain, MEMF_no_owner); + mfn_t mfn; + + if ( !pg ) + return -ENOMEM; + + mfn =3D page_to_mfn(pg); + clear_domain_page(mfn); + populate_perdomain_mapping(v, VCPU_STATE_ADDR(v, VCPU_STATE_IDX_REGS), + &mfn, 1, __PAGE_HYPERVISOR_RW); + v->arch.regs_pg =3D pg; + + return 0; +} + +static void vcpu_regs_free(struct vcpu *v) +{ + struct page_info *pg =3D v->arch.regs_pg; + + if ( !pg ) + return; + + unmap_vcpu_state(v, VCPU_STATE_IDX_REGS, 1); + v->arch.regs_pg =3D NULL; + free_domheap_page(pg); +} + +struct cpu_user_regs *vcpu_map_regs(const struct vcpu *v) +{ + /* + * v's page tables being loaded is not enough: while this CPU holds v's + * state lazily, an IPI syncing that state for another CPU can switch + * them away at any time. + */ + if ( v =3D=3D current && this_cpu(pgtable_vcpu) =3D=3D v ) + return regs_window(v); + + return __map_domain_page(v->arch.regs_pg); +} + +void vcpu_unmap_regs(const struct vcpu *v, const struct cpu_user_regs *reg= s) +{ + if ( !regs || regs =3D=3D regs_window(v) ) + return; + + unmap_vcpu_state_page(regs); +} + /* Initialise various registers to their architectural INIT/RESET state. */ void arch_vcpu_regs_init(struct vcpu *v) { @@ -533,6 +594,7 @@ int arch_vcpu_create(struct vcpu *v) BUILD_BUG_ON(VCPU_STATE_IDX_VMCB >=3D VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_N2VMCB >=3D VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_VMX_MSR >=3D VCPU_STATE_NR_PAGES); + BUILD_BUG_ON(VCPU_STATE_IDX_REGS >=3D VCPU_STATE_NR_PAGES); /* Page tables only; torn down with the rest of the per-domain are= a. */ rc =3D create_perdomain_mapping(v, VCPU_STATE_START(v), VCPU_STATE_NR_PAGES, @@ -547,6 +609,10 @@ int arch_vcpu_create(struct vcpu *v) =20 vmce_init_vcpu(v); =20 + /* The frame's storage, ahead of its initialisation. */ + if ( (rc =3D vcpu_regs_alloc(v)) !=3D 0 ) + goto fail; + arch_vcpu_regs_init(v); =20 if ( (rc =3D init_vcpu_msr_policy(v)) ) @@ -589,6 +655,7 @@ int arch_vcpu_create(struct vcpu *v) fail: paging_vcpu_teardown(v); vcpu_destroy_fpu(v); + vcpu_regs_free(v); xfree(v->arch.msrs); v->arch.msrs =3D NULL; free_pervcpu_mappings(v); @@ -613,6 +680,8 @@ void arch_vcpu_destroy(struct vcpu *v) else ASSERT_UNREACHABLE(); =20 + vcpu_regs_free(v); + free_pervcpu_mappings(v); } =20 @@ -2135,6 +2204,19 @@ static void load_default_gdt(unsigned int cpu) per_cpu(full_gdt_loaded, cpu) =3D false; } =20 +/* + * The register frame of p or n, for __context_switch(), which runs with + * interrupts off and with the vCPU's own page tables loaded: the window + * address serves. A transient mapping is not an option from here: it wou= ld + * complete the lazy context switch and re-enter __context_switch(). + */ +static struct cpu_user_regs *vcpu_user_regs(const struct vcpu *v) +{ + BUG_ON(this_cpu(pgtable_vcpu) !=3D v); + + return regs_window(v); +} + static void __context_switch(void) { struct cpu_user_regs *stack_regs =3D guest_cpu_user_regs(); @@ -2149,7 +2231,17 @@ static void __context_switch(void) if ( !is_idle_domain(pd) ) { ASSERT(read_atomic(&p->dirty_cpu) =3D=3D cpu); - memcpy(&p->arch._user_regs, stack_regs, CTXT_SWITCH_STACK_BYTES); + /* + * p's frame is reached through its state window, mapped while p's + * own page tables are loaded, as they normally still are here. N= ot + * so after the shadow-mode paths which abandon a vCPU's tables ea= rly + * (update_cr3() failure), which have crashed the domain: the frame + * is left unsaved. A transient mapping would re-enter this funct= ion. + */ + if ( likely(this_cpu(pgtable_vcpu) =3D=3D p) ) + memcpy(vcpu_user_regs(p), stack_regs, CTXT_SWITCH_STACK_BYTES); + else + BUG_ON(!domain_shutting_down(pd)); vcpu_save_fpu(p); pd->arch.ctxt_switch->from(p); } @@ -2165,7 +2257,6 @@ static void __context_switch(void) =20 if ( !is_idle_domain(nd) ) { - memcpy(stack_regs, &n->arch._user_regs, CTXT_SWITCH_STACK_BYTES); if ( cpu_has_xsave ) { if ( !set_xcr0(n->arch.xcr0 ?: XSTATE_FP_SSE) ) @@ -2189,6 +2280,9 @@ static void __context_switch(void) =20 write_ptbase(n); =20 + if ( !is_idle_domain(nd) ) + memcpy(stack_regs, vcpu_user_regs(n), CTXT_SWITCH_STACK_BYTES); + #ifdef CONFIG_PV /* Prefetch the VMCB if we expect to use it later in the context switc= h */ if ( using_svm() && is_pv_64bit_domain(nd) && !is_idle_domain(nd) ) diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/c= onfig.h index 8a265e25c8..c8d4a8ddc7 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -311,6 +311,7 @@ extern unsigned long xen_phys_start; #define VCPU_STATE_IDX_VMCB 0 #define VCPU_STATE_IDX_N2VMCB 1 #define VCPU_STATE_IDX_VMX_MSR 2 +#define VCPU_STATE_IDX_REGS 6 =20 #define ELFSIZE 64 =20 diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 457eb69ace..6b27af9eb5 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -633,11 +633,12 @@ struct pv_vcpu struct arch_vcpu { /* - * The vCPU's register frame while it is not running: __context_switch= () - * copies it here from the stack when the vCPU is switched out, and ba= ck - * when it is switched in. Everything else goes through VCPU_MAP_REGS= (). + * The page holding the vCPU's register frame while it is not running, + * mapped in its state window: __context_switch() copies the frame the= re + * from the stack when the vCPU is switched out, and back when it is + * switched in. Everything else goes through VCPU_MAP_REGS(). */ - struct cpu_user_regs _user_regs; + struct page_info *regs_pg; =20 /* Debug registers. */ unsigned long dr[4]; @@ -752,12 +753,20 @@ static inline void free_vcpu_guest_context(struct vcp= u_guest_context *vgc) * registers for code working on a vCPU other than the one running (whose * registers are on the stack, see guest_cpu_user_regs()). * + * The frame is in the vCPU's state window: the window address serves while + * the vCPU runs here on its own page tables, and the page is mapped + * transiently otherwise, which cannot fail. + * * The pointer is valid until VCPU_UNMAP_REGS(), which clears it. A mappi= ng - * must not be held across a scheduling point. + * must not be held across a scheduling point: a transient one is only val= id + * on the pCPU which made it. * * @param v Owner of the register frame */ -#define VCPU_MAP_REGS(v) (&(v)->arch._user_regs) +struct cpu_user_regs *vcpu_map_regs(const struct vcpu *v); +void vcpu_unmap_regs(const struct vcpu *v, const struct cpu_user_regs *reg= s); + +#define VCPU_MAP_REGS(v) vcpu_map_regs(v) =20 /* * Drops the mapping of a vCPU's register frame and nullifies its pointer. @@ -765,7 +774,10 @@ static inline void free_vcpu_guest_context(struct vcpu= _guest_context *vgc) * @param v Owner of the register frame * @param r Register frame of v, from VCPU_MAP_REGS() */ -#define VCPU_UNMAP_REGS(v, r) do { (void)(v); (r) =3D NULL; } while ( fals= e ) +#define VCPU_UNMAP_REGS(v, r) do { \ + vcpu_unmap_regs(v, r); \ + (r) =3D NULL; \ +} while ( false ) =20 void arch_vcpu_regs_init(struct vcpu *v); =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748669; cv=none; d=zohomail.com; s=zohoarc; b=aTnP/m3ptebKgIAq7/XLUfYYcuWX0I8CNuYecg9epKED19prSS+OhTndik42OTyEGUOzFpezaOTwbNgcruAKyrkygk/3rm5kDnMCkXJFj5wuRU0CowG8io41yKMCl0fUv++cqGgD9KzqigczVF1Ed52JRQK5R3jCPdFrhGchvYU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748669; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=HhSSqAUO6JoWEvu4Yjy41Bg2ZJEqbHPy2eEHgBHq04s=; b=ngC5T/o84QfiEmvFbdf5m0vCa4TKFlJ/s7MKtP+Sp3wO6cQeiEeKi63eIL0rPzRkFZK1C6v9IErhwoGCAXT7a+ehEDuEiNE8nnSHHwLWG08xyRqz5O4633O0k41ht6mBOIPz0PBAYALCRDCyVYiA5ACNYPIzNrhz24+SJX+7DFU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748669483189.72209709983724; Tue, 3 Nov 2026 15:31:09 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470791.1679424 (Exim 4.92) (envelope-from ) id 1xONxe-0000pu-O1; Tue, 03 Nov 2026 23:30:38 +0000 Received: by outflank-mailman (output) from mailman id 1470791.1679424; Tue, 03 Nov 2026 23:30:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxe-0000mt-Fj; Tue, 03 Nov 2026 23:30:38 +0000 Received: by outflank-mailman (input) for mailman id 1470791; Tue, 03 Nov 2026 23:30:35 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxb-0000cJ-Sv for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:35 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cot6-2B2z; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo7-0000000EMFa-43ya; Tue, 03 Nov 2026 23:20:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=HhSSqAUO6JoWEvu4Yjy41Bg2ZJEqbHPy2eEHgBHq04s=; b=umzSBLXUktyoPhdhok+4W6/JTU PIshuMwNJTTl8KgM7Gkcu57T32wVgNqXfsmP34T1k2ZDgjrZyMw7KwCHkmYOzjutFWNfAPzGVDBD2 Ye+7zBnhfuc3HmyNImgw0LAIaiYXUaTrT9/9n9kqk98wdaoekW94B0W5b+rMQlYp9pzg=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 33/44] x86/hvm: move the emulator's operand caches behind the vCPU state window Date: Tue, 3 Nov 2026 17:13:55 -0500 Message-ID: <20261103-asi-hvm-v6-33-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748671618158500 Content-Type: text/plain; charset="utf-8" The HVM emulator keeps two per-vCPU caches of guest data: the read cache (struct hvmemul_cache), which records every guest-memory read made while an instruction is emulated -- guest page-table entries, instruction bytes and memory operands -- so that a re-execution observes the same values, and the three MMIO caches, which hold the operand bytes of an instruction's MMIO accesses. Both are xvmalloc() allocations, from the xmalloc pool on the xenheap, and so mapped in every context, the sparse directmap view included. Disabling the read cache when emulation completes leaves its entries in place. Put both in a domain-heap page of the vCPU's own, mapped for the life of the vCPU at a new slot (index 7) of its state window: the read cache, counts included, then the MMIO caches, 4032 bytes of the page. As "x86: move the saved register frame behind the vCPU state window" does for the register frame, this applies to every HVM vCPU, whatever asi=3D says; for a vCPU without per-vCPU page tables the window is in the per-domain area that all its domain's vCPUs share, which only that domain's page tables map. The pointers in struct hvm_vcpu_io hold window addresses. hvmemul_cache_init() sets the page up through a transient mapping, as the vCPU is still being built, and hvmemul_cache_destroy(), now out of line, unmaps it before freeing it. The emulator only ever works on the vCPU running, on its own page tables, and reaches the caches through the window address; hvmemul_find_mmio_cache() now asserts as much. The read cache also has users in other contexts: a debugger's guest page walk on a paused vCPU (gdbsx, and the guest stack dump of the debug keys), which reads and extends it if the vCPU was paused mid-emulation, and hvmemul_cancel(), which disables it, on toolstack, VCPUOP_initialise and INIT/SIPI paths. Its accessors therefore map the page transiently unless the vCPU is running on its own page tables, as VCPU_MAP_REGS() does: a single page, so no new failure paths. They drop the mapping with unmap_vcpu_state_page(), which also flushes this CPU's translation. The read and write paths are split into an outer function, which checks and maps the cache once, and a recursive inner one. Cost: a page per HVM vCPU, where the caches took about 4KiB of the xmalloc pool, and the emulator reaches them through a 4KiB mapping in the per-domain area instead of the directmap. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. Notes for reviewers: the sizes before and after, on x86-64 (CONFIG_PAGING_LEVELS 4, MAX_INST_LEN 15). before after read cache ents 16*2*(4+1) + 15*(4+1) (16*2 + 15) * (4+1) =3D 235 (nents) =3D 235 (HVMEMUL_CACHE_ENTS) read cache bytes 8 + 235*16 =3D 3768 the same offsetof(), at 0 MMIO buffer bytes 64 (max_bytes) 64 (MMIO_CACHE_BYTES) MMIO cache bytes 24 + 64 =3D 88, three 88 (MMIO_CACHE_SIZE), three The page: offset bytes contents 0 8 num_ents, max_ents 8 3760 ents[235], 16 bytes each 3768 88 mmio_cache[0]: 24-byte header, buffer[64] 3856 88 mmio_cache[1] 3944 88 mmio_cache[2] 4032 64 unused The four old allocations together asked for 4032 bytes, the span the page now uses. The limits the code checks, max_ents (235) and space (64), are unchanged. The BUILD_BUG_ON() in hvmemul_cache_init() keeps the end within the page. --- xen/arch/x86/domain.c | 2 + xen/arch/x86/hvm/emulate.c | 196 +++++++++++++++++++------ xen/arch/x86/include/asm/config.h | 1 + xen/arch/x86/include/asm/hvm/emulate.h | 9 +- xen/arch/x86/include/asm/hvm/vcpu.h | 7 + 5 files changed, 165 insertions(+), 50 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 03f6572086..66d71ec41b 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -595,6 +595,8 @@ int arch_vcpu_create(struct vcpu *v) BUILD_BUG_ON(VCPU_STATE_IDX_N2VMCB >=3D VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_VMX_MSR >=3D VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_REGS >=3D VCPU_STATE_NR_PAGES); + BUILD_BUG_ON(VCPU_STATE_IDX_EMUL >=3D VCPU_STATE_NR_PAGES); + BUILD_BUG_ON(VCPU_STATE_IDX_EMUL =3D=3D VCPU_STATE_IDX_REGS); /* Page tables only; torn down with the rest of the per-domain are= a. */ rc =3D create_perdomain_mapping(v, VCPU_STATE_START(v), VCPU_STATE_NR_PAGES, diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c index cd2831d34b..e2cf982c25 100644 --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -9,6 +9,7 @@ * Keir Fraser */ =20 +#include #include #include #include @@ -56,6 +57,26 @@ struct hvmemul_cache } ents[]; }; =20 +/* + * AVX512F scatter/gather insns can access up to 16 independent linear + * addresses, up to 8 bytes size. Each such linear range can span a page + * boundary, i.e. may require two page walks. Account for each insn byte + * individually, both for simplicity and to leave some slack space. + */ +#define HVMEMUL_CACHE_ENTS \ + ((16 * 2 + MAX_INST_LEN) * (CONFIG_PAGING_LEVELS + 1)) + +/* Operand bytes per MMIO cache: up to m512 or a tile row. */ +#define MMIO_CACHE_BYTES 64 + +/* Layout of the caches' page: the read cache, then the MMIO caches. */ +#define MMIO_CACHE_SIZE \ + ROUNDUP(offsetof(struct hvm_mmio_cache, buffer[MMIO_CACHE_BYTES]), \ + __alignof__(struct hvm_mmio_cache)) +#define EMUL_PAGE_MMIO_CACHE(i) \ + (ROUNDUP(offsetof(struct hvmemul_cache, ents[HVMEMUL_CACHE_ENTS]), \ + __alignof__(struct hvm_mmio_cache)) + (i) * MMIO_CACHE_SIZE) + static void hvmtrace_io_assist(const ioreq_t *p) { unsigned int size, event; @@ -1130,6 +1151,9 @@ static struct hvm_mmio_cache *hvmemul_find_mmio_cache( unsigned int i; struct hvm_mmio_cache *cache; =20 + ASSERT(hvio =3D=3D ¤t->arch.hvm.hvm_io); + ASSERT(this_cpu(pgtable_vcpu) =3D=3D current); + for ( i =3D 0; i < hvio->mmio_cache_count; i ++ ) { cache =3D hvio->mmio_cache[i]; @@ -3057,77 +3081,133 @@ void hvm_dump_emulation_state(const char *loglvl, = const char *prefix, =20 int hvmemul_cache_init(struct vcpu *v) { - /* - * AVX512F scatter/gather insns can access up to 16 independent linear - * addresses, up to 8 bytes size. Each such linear range can span a pa= ge - * boundary, i.e. may require two page walks. - */ - unsigned int nents =3D 16 * 2 * (CONFIG_PAGING_LEVELS + 1); - unsigned int i, max_bytes =3D 64; + struct hvm_vcpu_io *hvio =3D &v->arch.hvm.hvm_io; + struct page_info *pg =3D alloc_domheap_page(v->domain, MEMF_no_owner); + void *va =3D (void *)VCPU_STATE_ADDR(v, VCPU_STATE_IDX_EMUL), *p; struct hvmemul_cache *cache; + unsigned int i; + mfn_t mfn; =20 - /* - * Account for each insn byte individually, both for simplicity and to - * leave some slack space. - */ - nents +=3D MAX_INST_LEN * (CONFIG_PAGING_LEVELS + 1); + BUILD_BUG_ON(EMUL_PAGE_MMIO_CACHE(ARRAY_SIZE(hvio->mmio_cache)) > + PAGE_SIZE); =20 - cache =3D xvmalloc_flex_struct(struct hvmemul_cache, ents, nents); - if ( !cache ) + if ( !pg ) return -ENOMEM; =20 - /* Cache is disabled initially. */ - cache->num_ents =3D nents + 1; - cache->max_ents =3D nents; + mfn =3D page_to_mfn(pg); + clear_domain_page(mfn); =20 - v->arch.hvm.hvm_io.cache =3D cache; + /* v !=3D current, so we can't use the vCPU state window. */ + p =3D map_domain_page(mfn); =20 - for ( i =3D 0; i < ARRAY_SIZE(v->arch.hvm.hvm_io.mmio_cache); ++i ) + /* Cache is disabled initially. */ + cache =3D p; + cache->num_ents =3D HVMEMUL_CACHE_ENTS + 1; + cache->max_ents =3D HVMEMUL_CACHE_ENTS; + + for ( i =3D 0; i < ARRAY_SIZE(hvio->mmio_cache); ++i ) { - v->arch.hvm.hvm_io.mmio_cache[i] =3D - xvmalloc_flex_struct(struct hvm_mmio_cache, buffer, max_bytes); - if ( !v->arch.hvm.hvm_io.mmio_cache[i] ) - return -ENOMEM; - v->arch.hvm.hvm_io.mmio_cache[i]->space =3D max_bytes; + struct hvm_mmio_cache *mmio =3D p + EMUL_PAGE_MMIO_CACHE(i); + + mmio->space =3D MMIO_CACHE_BYTES; } =20 + unmap_domain_page(p); + + populate_perdomain_mapping(v, (unsigned long)va, &mfn, 1, + __PAGE_HYPERVISOR_RW); + + hvio->emul_pg =3D pg; + hvio->cache =3D va; + for ( i =3D 0; i < ARRAY_SIZE(hvio->mmio_cache); ++i ) + hvio->mmio_cache[i] =3D va + EMUL_PAGE_MMIO_CACHE(i); + return 0; } =20 +void hvmemul_cache_destroy(struct vcpu *v) +{ + struct hvm_vcpu_io *hvio =3D &v->arch.hvm.hvm_io; + unsigned int i; + + if ( !hvio->emul_pg ) + return; + + unmap_vcpu_state(v, VCPU_STATE_IDX_EMUL, 1); + free_domheap_page(hvio->emul_pg); + hvio->emul_pg =3D NULL; + hvio->cache =3D NULL; + for ( i =3D 0; i < ARRAY_SIZE(hvio->mmio_cache); ++i ) + hvio->mmio_cache[i] =3D NULL; +} + +/* + * The read cache of @v, for access from any context: the window address in + * struct hvm_vcpu_io where v runs here on its own page tables, a transient + * mapping of the page otherwise, and the NULL left there once the cache is + * destroyed. v's page tables being loaded is not enough: while this CPU + * holds v's state lazily, an IPI syncing that state for another CPU can + * switch them away at any time. + */ +static struct hvmemul_cache *map_cache(const struct vcpu *v) +{ + const struct hvm_vcpu_io *hvio =3D &v->arch.hvm.hvm_io; + + if ( !hvio->emul_pg || (v =3D=3D current && this_cpu(pgtable_vcpu) =3D= =3D v) ) + return hvio->cache; + + /* The read cache is at the start of the page. */ + return __map_domain_page(hvio->emul_pg); +} + +static void unmap_cache(const struct vcpu *v, + const struct hvmemul_cache *cache) +{ + if ( cache !=3D v->arch.hvm.hvm_io.cache ) + unmap_vcpu_state_page(cache); +} + +#define HVMEMUL_MAP_CACHE(v) map_cache(v) + +/* Drops the mapping of a vCPU's read cache and nullifies its pointer. */ +#define HVMEMUL_UNMAP_CACHE(v, cache) do { \ + unmap_cache(v, cache); \ + (cache) =3D NULL; \ +} while ( false ) + unsigned int hvmemul_cache_disable(struct vcpu *v) { - struct hvmemul_cache *cache =3D v->arch.hvm.hvm_io.cache; + struct hvmemul_cache *cache =3D HVMEMUL_MAP_CACHE(v); unsigned int token =3D cache->num_ents; =20 cache->num_ents =3D cache->max_ents + 1; =20 + HVMEMUL_UNMAP_CACHE(v, cache); + return token; } =20 void hvmemul_cache_restore(struct vcpu *v, unsigned int token) { - struct hvmemul_cache *cache =3D v->arch.hvm.hvm_io.cache; + struct hvmemul_cache *cache =3D HVMEMUL_MAP_CACHE(v); =20 ASSERT(cache->num_ents > cache->max_ents); cache->num_ents =3D token; + + HVMEMUL_UNMAP_CACHE(v, cache); } =20 -bool hvmemul_read_cache(const struct vcpu *v, paddr_t gpa, - void *buffer, unsigned int size) +static bool read_cache(const struct hvmemul_cache *cache, paddr_t gpa, + void *buffer, unsigned int size) { - const struct hvmemul_cache *cache =3D v->arch.hvm.hvm_io.cache; unsigned int i; =20 - /* Cache unavailable? */ - if ( !is_hvm_vcpu(v) || cache->num_ents > cache->max_ents ) - return false; - while ( size > sizeof(cache->ents->data) ) { i =3D gpa & (sizeof(cache->ents->data) - 1) ? -gpa & (sizeof(cache->ents->data) - 1) : sizeof(cache->ents->data); - if ( !hvmemul_read_cache(v, gpa, buffer, i) ) + if ( !read_cache(cache, gpa, buffer, i) ) return false; gpa +=3D i; buffer +=3D i; @@ -3144,22 +3224,37 @@ bool hvmemul_read_cache(const struct vcpu *v, paddr= _t gpa, return false; } =20 -void hvmemul_write_cache(const struct vcpu *v, paddr_t gpa, - const void *buffer, unsigned int size) +bool hvmemul_read_cache(const struct vcpu *v, paddr_t gpa, + void *buffer, unsigned int size) { - struct hvmemul_cache *cache =3D v->arch.hvm.hvm_io.cache; - unsigned int i; + const struct hvmemul_cache *cache; + bool hit =3D false; =20 - /* Cache unavailable? */ - if ( !is_hvm_vcpu(v) || cache->num_ents > cache->max_ents ) - return; + if ( !is_hvm_vcpu(v) ) + return false; + + cache =3D HVMEMUL_MAP_CACHE(v); + + /* Cache available? */ + if ( cache->num_ents <=3D cache->max_ents ) + hit =3D read_cache(cache, gpa, buffer, size); + + HVMEMUL_UNMAP_CACHE(v, cache); + + return hit; +} + +static void write_cache(const struct vcpu *v, struct hvmemul_cache *cache, + paddr_t gpa, const void *buffer, unsigned int size) +{ + unsigned int i; =20 while ( size > sizeof(cache->ents->data) ) { i =3D gpa & (sizeof(cache->ents->data) - 1) ? -gpa & (sizeof(cache->ents->data) - 1) : sizeof(cache->ents->data); - hvmemul_write_cache(v, gpa, buffer, i); + write_cache(v, cache, gpa, buffer, i); gpa +=3D i; buffer +=3D i; size -=3D i; @@ -3186,6 +3281,23 @@ void hvmemul_write_cache(const struct vcpu *v, paddr= _t gpa, cache->num_ents =3D i + 1; } =20 +void hvmemul_write_cache(const struct vcpu *v, paddr_t gpa, + const void *buffer, unsigned int size) +{ + struct hvmemul_cache *cache; + + if ( !is_hvm_vcpu(v) ) + return; + + cache =3D HVMEMUL_MAP_CACHE(v); + + /* Cache available? */ + if ( cache->num_ents <=3D cache->max_ents ) + write_cache(v, cache, gpa, buffer, size); + + HVMEMUL_UNMAP_CACHE(v, cache); +} + /* * Local variables: * mode: C diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/c= onfig.h index c8d4a8ddc7..5bd9970fce 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -312,6 +312,7 @@ extern unsigned long xen_phys_start; #define VCPU_STATE_IDX_N2VMCB 1 #define VCPU_STATE_IDX_VMX_MSR 2 #define VCPU_STATE_IDX_REGS 6 +#define VCPU_STATE_IDX_EMUL 7 =20 #define ELFSIZE 64 =20 diff --git a/xen/arch/x86/include/asm/hvm/emulate.h b/xen/arch/x86/include/= asm/hvm/emulate.h index 178ac32e15..16491a92b1 100644 --- a/xen/arch/x86/include/asm/hvm/emulate.h +++ b/xen/arch/x86/include/asm/hvm/emulate.h @@ -117,14 +117,7 @@ int hvmemul_do_pio_buffer(uint16_t port, * further re-execution. */ int __must_check hvmemul_cache_init(struct vcpu *v); -static inline void hvmemul_cache_destroy(struct vcpu *v) -{ - unsigned int i; - - for ( i =3D 0; i < ARRAY_SIZE(v->arch.hvm.hvm_io.mmio_cache); ++i ) - XVFREE(v->arch.hvm.hvm_io.mmio_cache[i]); - XVFREE(v->arch.hvm.hvm_io.cache); -} +void hvmemul_cache_destroy(struct vcpu *v); bool hvmemul_read_cache(const struct vcpu *v, paddr_t gpa, void *buffer, unsigned int size); void hvmemul_write_cache(const struct vcpu *v, paddr_t gpa, diff --git a/xen/arch/x86/include/asm/hvm/vcpu.h b/xen/arch/x86/include/asm= /hvm/vcpu.h index 876c230784..ee7bc8be00 100644 --- a/xen/arch/x86/include/asm/hvm/vcpu.h +++ b/xen/arch/x86/include/asm/hvm/vcpu.h @@ -46,6 +46,13 @@ struct hvm_vcpu_io { unsigned char mmio_insn[16]; struct hvmemul_cache *cache; =20 + /* + * The page holding the caches above, mapped in the vCPU's state windo= w: + * mmio_cache[] and cache hold addresses in the window, for use while = the + * vCPU's own page tables are loaded. + */ + struct page_info *emul_pg; + /* * For string instruction emulation we need to be able to signal a * necessary retry through other than function return codes. --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748662; cv=none; d=zohomail.com; s=zohoarc; b=HehGKRFx8pCCJYkivecJu6RTLbmoaCAQUlwIFGXyhMABjLDhJPg3vB0EBBwD+9eL2quYZnMD/a0OUwX5PY/SwwMHkXSQntLgVIdhrWzqegEiRO+9qRksNPFlNwEN7spmApD/Iqup34SrgtX+iTs6Oq6r1yE9wS8tEwFG8q9SWPU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748662; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xMTLrvxxwpwmKfoWs2RiIcYg6g+Liai9ytC5nmpdgLQ=; b=ZR5kE+gJpHH24BzCNMDVp4Rq33uE2zfRl+DKdatM9EIkiS4ETYEqS7Z+OvRQqdbwVOwq1BDHbx+Xc/55fdPPi3B9eogTufFM0p0kfkcAY+8v5tMM5rwBLhqJGu+Orl2DXxeAioB5qzX0MdhJbdIY+hboXW+OSfilJ87F8wqyKkg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 179374866274496.33732893575927; Tue, 3 Nov 2026 15:31:02 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470796.1679461 (Exim 4.92) (envelope-from ) id 1xONxh-0001Yn-EP; Tue, 03 Nov 2026 23:30:41 +0000 Received: by outflank-mailman (output) from mailman id 1470796.1679461; Tue, 03 Nov 2026 23:30:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxg-0001VV-VS; Tue, 03 Nov 2026 23:30:40 +0000 Received: by outflank-mailman (input) for mailman id 1470796; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000cf-1O for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotM-3JWU; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo8-0000000EMFa-2DhE; Tue, 03 Nov 2026 23:20:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=xMTLrvxxwpwmKfoWs2RiIcYg6g+Liai9ytC5nmpdgLQ=; b=yWLy8e/IpNDGhWHzoGLtuB2yL2 PWMLKC5wPoo+Ore5hFSM61WQpSUGH3hxeVFrPlXaOTYA8V47i3AFStTeL+FFob2B7+zAS5MkpPKcb JYrftt4kUH1/XCzAx94Sl+P5NF98llk8XuBMToqDdU+uRy2D8HnYplUopop/4zicq7QY=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 34/44] x86/hvm: clear the vCPU's copy of an emulated I/O request once consumed Date: Tue, 3 Nov 2026 17:13:56 -0500 Message-ID: <20261103-asi-hvm-v6-34-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748663781158500 Content-Type: text/plain; charset="utf-8" hvmemul_do_io() keeps the request it is emulating in the vCPU's struct vcpu_io: the data written, or the response once a device model has supplied it, with the address and repeat count. Once the access is done only the state is set back to STATE_IOREQ_NONE, so the operand stays in struct vcpu, a xenheap page mapped in every context, until the next emulated I/O overwrites it. Drop the copy wherever it is done with: when the response has been taken into hvmemul_do_io()'s local copy on the instruction's re-execution, when Xen's own handlers have completed the access, when the request is retried, suspended or not sent, and in hvmemul_cancel(). A request sent to a device model stays while it is in flight: the re-execution checks it, and the port-I/O completion reads its address and size before re-issuing it. One which needs no completion -- a port write, or a request whose data is a guest buffer's address -- loses its address, data and count as soon as it is sent: the device model has its own copy, and the completion path reads only the state, type, direction and data_is_ptr. Nothing reads the copy after STATE_IOREQ_NONE. Only the x86 emulator changes. struct vcpu_io is common code, and Arm's MMIO completion reads the response from it. Moving the in-flight copy out of struct vcpu belongs with narrowing the ioreq server pages, which carry the same operand while a device model handles it. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/arch/x86/hvm/emulate.c | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c index e2cf982c25..da7c08b5d9 100644 --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -167,6 +167,16 @@ static const struct hvm_io_handler ioreq_server_handle= r =3D { .ops =3D &ioreq_server_ops }; =20 +/* + * The vCPU's copy of an emulated I/O request is done with: nothing reads = it + * before the next request overwrites it. Clear it all, so that the opera= nd + * and the address do not stay in struct vcpu. + */ +static void ioreq_done(struct vcpu_io *vio) +{ + vio->req =3D (ioreq_t){ .state =3D STATE_IOREQ_NONE }; +} + /* * Drop all records of in-flight emulation. This is needed whenever a vCPU= 's * register state may have changed behind the emulator's back. @@ -175,7 +185,7 @@ void hvmemul_cancel(struct vcpu *v) { struct hvm_vcpu_io *hvio =3D &v->arch.hvm.hvm_io; =20 - v->io.req.state =3D STATE_IOREQ_NONE; + ioreq_done(&v->io); v->io.completion =3D VIO_no_completion; hvio->mmio_cache_count =3D 0; hvio->mmio_insn_bytes =3D 0; @@ -265,8 +275,9 @@ static int hvmemul_do_io( case STATE_IOREQ_NONE: break; case STATE_IORESP_READY: - vio->req.state =3D STATE_IOREQ_NONE; + /* The local copy carries the response from here. */ p =3D vio->req; + ioreq_done(vio); =20 /* Verify the emulation request has been correctly re-issued */ if ( (p.type !=3D (is_mmio ? IOREQ_TYPE_COPY : IOREQ_TYPE_PIO)) || @@ -330,7 +341,7 @@ static int hvmemul_do_io( { case X86EMUL_OKAY: case X86EMUL_RETRY: - vio->req.state =3D STATE_IOREQ_NONE; + ioreq_done(vio); break; case X86EMUL_UNHANDLEABLE: { @@ -383,7 +394,7 @@ static int hvmemul_do_io( if ( s =3D=3D NULL ) { rc =3D X86EMUL_RETRY; - vio->req.state =3D STATE_IOREQ_NONE; + ioreq_done(vio); break; } =20 @@ -394,7 +405,7 @@ static int hvmemul_do_io( if ( dir =3D=3D IOREQ_READ ) { rc =3D hvm_process_io_intercept(&ioreq_server_handler,= &p); - vio->req.state =3D STATE_IOREQ_NONE; + ioreq_done(vio); break; } } @@ -440,7 +451,7 @@ static int hvmemul_do_io( paddr_to_pfn(addr)); =20 rc =3D X86EMUL_RETRY; - vio->req.state =3D STATE_IOREQ_NONE; + ioreq_done(vio); break; } =20 @@ -455,15 +466,24 @@ static int hvmemul_do_io( dir ? "read from" : "write to", addr, size); } rc =3D hvm_process_io_intercept(&null_handler, &p); - vio->req.state =3D STATE_IOREQ_NONE; + ioreq_done(vio); } else { rc =3D ioreq_send(s, &p, 0); if ( rc !=3D X86EMUL_RETRY || vio->suspended ) - vio->req.state =3D STATE_IOREQ_NONE; + ioreq_done(vio); else if ( !ioreq_needs_completion(&vio->req) ) + { + /* + * Nothing comes back, and the device model has its own co= py + * of the request: keep only what completion checks. + */ + vio->req.addr =3D 0; + vio->req.data =3D 0; + vio->req.count =3D 0; rc =3D X86EMUL_OKAY; + } } break; } --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748665; cv=none; d=zohomail.com; s=zohoarc; b=OEDYXUnikBU2JwgLVi9YCjF2u/lnCUobtA7wGsKGbzOKNRxESKfOk5rNpish6P10iCjQaMfdOKotb7gsEUUkMEMdoB54MnFRjAnVyYK5RpDwojC+TU3xfu3mReH1V8nd/bPaJlxiUg+OyT/ZHn0p6fC5vkLu/oKjBJCdZ4zkY5M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748665; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=7+SfUGkgBYMN0fY336uRRwse3Bcl2E3YDqRetUP0NXs=; b=ZVkKseZLEpaDVzaKcC05ifUwo1fUBJwH1jsbqwQJukWW/V2MnLgkvuWxRcNTRf15hZk+tlnFCHmdmwTsZDLYIMUMun5iYgWzFPiOdxF0rQS5NaJsoN33jSpwlswHw8Q8Z1ZgUGlMifEKqlb5Bb9gFHa1bxDPt+iKDx0RlnNw+lo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748665097893.605675792261; Tue, 3 Nov 2026 15:31:05 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470795.1679448 (Exim 4.92) (envelope-from ) id 1xONxg-0001Lm-EO; Tue, 03 Nov 2026 23:30:40 +0000 Received: by outflank-mailman (output) from mailman id 1470795.1679448; Tue, 03 Nov 2026 23:30:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxg-0001Hu-0T; Tue, 03 Nov 2026 23:30:40 +0000 Received: by outflank-mailman (input) for mailman id 1470795; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000cZ-0p for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cotc-3iEl; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONo9-0000000EMFa-0gYZ; Tue, 03 Nov 2026 23:20:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=7+SfUGkgBYMN0fY336uRRwse3Bcl2E3YDqRetUP0NXs=; b=T0f1XdTxfYHZA9MbSfrp7iuMAM n+2K51ksVMGTYBaK68A/kaQwIXVGnPn+WUePRhQKS4Smbyblb3oymcfdXSFwdceZABLvIVwBKKSYM 9h96ajWFLCjlUCw4N7hQ5gfZk5kC2/p/x+3SfGdrW2w3lya7CbWFPRqqVIlnoOy7TpbQ=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Stefano Stabellini , Teddy Astie Subject: [PATCH v6 35/44] x86/domain_page: introduce map_domain_pages() Date: Tue, 3 Nov 2026 17:13:57 -0500 Message-ID: <20261103-asi-hvm-v6-35-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748665604158500 Content-Type: text/plain; charset="utf-8" map_domain_page() maps a single page, so callers needing a virtually contiguous mapping of several (not necessarily physically contiguous) frames have had to fall back to vmap(), which maps whatever the frames hold in every context, those of domains on the sparse directmap view included, for the lifetime of the mapping. Provide a multipage counterpart which maps them in the calling vCPU's per-domain area instead, which only its domain's page tables carry: map_domain_pages(mfn, nr) / unmap_domain_pages(ptr, nr) Each vCPU has MAPCACHE_MULTIPAGE_ENTRIES slots of its own for such mappings. With per-vCPU page tables, they are just above the vCPU's single-page mapcache. Where a domain's vCPUs share one per-domain area, they are at the end of the mapcache sub-area, where each vCPU has a fixed block of its own. No single-page mapcache is needed: the multipage slots are in the same VA range whether or not the domain has one. The mappings are order-aligned and tracked in a per-vCPU bitmap, and bypass the maphash and the deferred garbage/epoch machinery. At unmap a mapping's PTEs are cleared, and a local flush done. Since each vCPU has its own slots, even with a domain-wide L3, the local flush suffices for correctness: even if a stray TLB entry is speculatively loaded on another processor, no other vCPU will use that mapping, and it will be flushed before the vCPU which owns the slots runs on that CPU. With regard to exposure: in a domain whose vCPUs share one per-domain area, other vCPUs may be able to reach the mapping speculatively while it exists, and afterwards through a stale TLB entry until their CPU next flushes. For guests on the full directmap, this doesn't expose anything new, as the page is already mapped in the directmap anyway (unless it's a PV domain and the page is past the short directmap). For PV guests on the sparse directmap view, it's still an improvement over the status quo, as exposure is limited to the domain, rather than the whole system (as in the case of vmap). An unmap will BUG() when given a pointer or a @nr which indexes outside the vCPU's slots, unless that pointer is in the vmap area (see below). These slots are not available before the first page-table switch in boot, nor with the idle vCPU's page tables loaded. Currently no calls are made during boot, and the only known way to reach map_domain_pages() with the idle page tables loaded is on an error path that will already ultimately crash the host. Nonetheless, for robustness, if called from a context where no mapping is available, it will fall back to using vmap(); and unmap recognizes such an address and vunmap()'s it. This briefly exposes contents to every context, and requires a system-wide TLB shootdown on unmap, but allows execution to continue. The alternative would be to BUG() in such a context. For those who want stricter isolation guarantees, at the risk that more situations might come up in the future, a new sub-option, asi=3Dvmap-strict, off by default, makes the fallback a BUG() instead. The first fallback is logged. The handling of a lazy context switch in mapcache_current_vcpu() moves into pgtable_current_vcpu(), which map_domain_pages() uses whatever the domain: an IPI can switch away page tables held lazily at any time. Eight slots hold the largest mapping a known caller needs, a three-page XSAVE area (four slots, order 2), with room for a two-page mapping beside it. Where a domain's vCPUs share one per-domain area, the multipage area takes 256MiB of address space for MAX_VIRT_CPUS, and one L1 table per 64 vCPUs. Later patches convert the two known in-tree consumers. Moving the XSAVE areas off always-mapped memory needs the primitive too: the XSAVE area exceeds one page on AMX-class hardware, and XSAVE/XRSTOR require it to be virtually contiguous. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- docs/misc/xen-command-line.pandoc | 10 +- xen/arch/x86/domain_page.c | 171 +++++++++++++++++++++++++-- xen/arch/x86/include/asm/config.h | 14 +++ xen/arch/x86/include/asm/domain.h | 3 + xen/arch/x86/include/asm/spec_ctrl.h | 1 + xen/arch/x86/spec_ctrl.c | 7 ++ xen/include/xen/domain_page.h | 21 ++++ 7 files changed, 216 insertions(+), 11 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index 092c858f34..52e10eb393 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -203,7 +203,7 @@ to appropriate auditing by Xen. Argo is disabled by de= fault. buggy or malicious other domain spamming the ring. =20 ### asi (x86) -> `=3D List of [ , {pv,hvm,dom0}=3D, directmap=3D, +> `=3D List of [ , {pv,hvm,dom0}=3D, {directmap,vmap-strict}= =3D, > vcpu-pt=3D | vcpu-pt=3Dhvm=3D ]` =20 > Default: `false` @@ -245,6 +245,14 @@ are meant to be used for debugging purposes only.** and `dom0=3D`) set this too, on builds that support it; on other builds = they leave it off. =20 +* `vmap-strict` crashes the host when a multipage transient mapping + (`map_domain_pages()`) is requested where the loaded page-tables + have no slots for one: in early boot, or on the idle vCPU's + page-tables. By default such a mapping falls back to `vmap()`, + which maps the pages in every context. The first such mapping is + logged. The option is for hosts where a crash is preferable to such + a mapping; none is known to be made in normal operation. + ### asid (x86) > `=3D ` =20 diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 4198ee7c18..8e5ef4a34a 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -6,6 +6,7 @@ * Copyright (c) 2003-2006, Keir Fraser */ =20 +#include #include #include #include @@ -17,6 +18,7 @@ #include #include #include +#include =20 /* Whether the vCPU supports mapcache usage, might not be in use. */ static bool has_mapcache(const struct vcpu *v) @@ -24,20 +26,22 @@ static bool has_mapcache(const struct vcpu *v) return is_pv_vcpu(v) || v->domain->arch.asi.vcpu_pt; } =20 -static inline struct vcpu *mapcache_current_vcpu(void) +/* + * The vCPU whose page tables are loaded here, and stay loaded until the + * caller is done with what it maps in them. + */ +static struct vcpu *pgtable_current_vcpu(void) { - const struct vcpu *v =3D this_cpu(pgtable_vcpu); const struct vcpu *curr =3D current; =20 /* During early boot pgtable_vcpu is not set, callers must handle NULL= . */ - if ( !v || !has_mapcache(v) ) + if ( !this_cpu(pgtable_vcpu) ) return NULL; =20 /* - * If we are in a lazy context-switch state from a vCPU with a mapcach= e do - * a full switch to the idle vCPU now, otherwise an incoming - * FLUSH_VCPU_STATE IPI would change the page tables under our feet and - * invalidate any in-use mapcache entries. + * If we are in a lazy context-switch state do a full switch to the id= le + * vCPU now, otherwise an incoming FLUSH_VCPU_STATE IPI would change t= he + * page tables under our feet and invalidate any in-use mappings in th= em. */ if ( unlikely(this_cpu(curr_vcpu) !=3D curr) ) { @@ -56,6 +60,17 @@ static inline struct vcpu *mapcache_current_vcpu(void) return ACCESS_ONCE(this_cpu(pgtable_vcpu)); } =20 +static inline struct vcpu *mapcache_current_vcpu(void) +{ + const struct vcpu *v =3D this_cpu(pgtable_vcpu); + + /* During early boot pgtable_vcpu is not set, callers must handle NULL= . */ + if ( !v || !has_mapcache(v) ) + return NULL; + + return pgtable_current_vcpu(); +} + /* Whether @v's page-tables carry the sparse view of the directmap. */ static bool on_sparse_dmap(const struct vcpu *v) { @@ -341,6 +356,124 @@ void unmap_domain_page(const void *ptr) local_irq_restore(flags); } =20 +/* + * The mapcache index of @v's first map_domain_pages() slot: above the + * single-page slots of its own mapcache with per-vCPU page tables, otherw= ise + * at its stride of the domain-wide multipage area. + */ +static unsigned int multipage_first_slot(const struct vcpu *v) +{ + if ( v->domain->arch.asi.vcpu_pt ) + return MAPCACHE_PERVCPU_ENTRIES; + + return PFN_DOWN(MAPCACHE_MULTIPAGE_START(v) - MAPCACHE_VIRT_START); +} + +/* + * Multipage mappings come from slots of the vCPU whose page tables are + * loaded, which no other vCPU uses, and bypass the maphash and the + * garbage/epoch machinery: at unmap a mapping's PTEs are zapped, and their + * translations dropped from this CPU's TLB, at their addresses and at the= ir + * aliases in the per-domain mirror. + */ +void *map_domain_pages(const mfn_t *mfn, unsigned int nr) +{ + unsigned long flags; + unsigned int i, first; + struct vcpu *v; + int idx; + + if ( !nr || nr > MAPCACHE_MULTIPAGE_ENTRIES ) + { + ASSERT_UNREACHABLE(); + return NULL; + } + + if ( nr =3D=3D 1 ) + return map_domain_page(mfn[0]); + + /* Early boot, or the idle vCPU's page tables: no multipage slots. */ + v =3D pgtable_current_vcpu(); + if ( !v || is_idle_vcpu(v) ) + { + BUG_ON(opt_vmap_strict); + printk_once(XENLOG_WARNING + "%pS: map_domain_pages() falling back to vmap()\n", + __builtin_return_address(0)); + return vmap(mfn, nr); + } + + first =3D multipage_first_slot(v); + + local_irq_save(flags); + + idx =3D bitmap_find_free_region(&v->arch.mapcache.multipage_inuse, + MAPCACHE_MULTIPAGE_ENTRIES, + get_order_from_pages(nr)); + if ( idx >=3D 0 ) + for ( i =3D 0; i < nr; i++ ) + l1e_write(&MAPCACHE_L1ENT(first + idx + i), + l1e_from_mfn(mfn[i], __PAGE_HYPERVISOR_RW)); + + local_irq_restore(flags); + + if ( idx < 0 ) + return NULL; + + return (void *)MAPCACHE_VIRT_START + pfn_to_paddr(first + idx); +} + +void unmap_domain_pages(const void *ptr, unsigned int nr) +{ + unsigned long va =3D (unsigned long)ptr & PAGE_MASK; + unsigned long flags, slot; + unsigned int idx, first, i; + struct vcpu *v; + + BUG_ON(!nr); + + if ( nr =3D=3D 1 ) + { + unmap_domain_page(ptr); + return; + } + + if ( va >=3D VMAP_VIRT_START && va < VMAP_VIRT_END ) + { + vunmap((void *)va); + return; + } + + v =3D pgtable_current_vcpu(); + BUG_ON(!v || is_idle_vcpu(v) || va < MAPCACHE_VIRT_START); + + first =3D multipage_first_slot(v); + slot =3D PFN_DOWN(va - MAPCACHE_VIRT_START); + BUG_ON(slot < first || slot - first >=3D MAPCACHE_MULTIPAGE_ENTRIES); + + idx =3D slot - first; + BUG_ON(nr > MAPCACHE_MULTIPAGE_ENTRIES - idx); + + local_irq_save(flags); + + for ( i =3D 0; i < nr; i++ ) + { + unsigned long addr =3D va + i * PAGE_SIZE; + + ASSERT(l1e_get_flags(MAPCACHE_L1ENT(slot + i)) & _PAGE_PRESENT); + l1e_write(&MAPCACHE_L1ENT(slot + i), l1e_empty()); + flush_tlb_one_local(addr); + if ( !is_pv_64bit_vcpu(v) ) + flush_tlb_one_local(addr - PERDOMAIN_VIRT_START + + PERDOMAIN_ALT_VIRT_START); + } + + bitmap_release_region(&v->arch.mapcache.multipage_inuse, idx, + get_order_from_pages(nr)); + + local_irq_restore(flags); +} + void mapcache_domain_init(struct domain *d) { struct mapcache_domain *dcache =3D &d->arch.pv.mapcache; @@ -359,7 +492,7 @@ void mapcache_domain_init(struct domain *d) =20 BUILD_BUG_ON(MAPCACHE_VIRT_END + PAGE_SIZE * (3 + 2 * PFN_UP(BITS_TO_LONGS(MAPCACHE_ENTRIES) * sizeof(long)= )) > - MAPCACHE_VIRT_START + (PERDOMAIN_SLOT_MBYTES << 20)); + MAPCACHE_MULTIPAGE_VIRT_START); bitmap_pages =3D PFN_UP(BITS_TO_LONGS(MAPCACHE_ENTRIES) * sizeof(long)= ); dcache->cache.inuse =3D (void *)MAPCACHE_VIRT_END + PAGE_SIZE; dcache->cache.garbage =3D dcache->cache.inuse + @@ -376,12 +509,28 @@ int mapcache_vcpu_init(struct vcpu *v) unsigned long i; unsigned int ents, nr; =20 + BUILD_BUG_ON(MAPCACHE_MULTIPAGE_ENTRIES > BITS_PER_LONG); + + /* Without per-vCPU page tables, the vCPU's stride of the multipage ar= ea. */ + if ( !is_idle_domain(d) && !d->arch.asi.vcpu_pt ) + { + int rc =3D create_perdomain_mapping(v, MAPCACHE_MULTIPAGE_START(v), + MAPCACHE_MULTIPAGE_ENTRIES, + NIL(l1_pgentry_t *), NULL); + + if ( rc ) + return rc; + } + if ( !has_mapcache(v) ) return 0; =20 cache =3D vcpu_mapcache(v, &dcache); if ( !dcache ) { + BUILD_BUG_ON(MAPCACHE_PERVCPU_ENTRIES + MAPCACHE_MULTIPAGE_ENTRIES= > + MAPCACHE_ENTRIES); + /* Per-vCPU bitmaps are an unsigned long each, in the vCPU itself.= */ cache->inuse =3D v->arch.mapcache.inuse; cache->garbage =3D v->arch.mapcache.garbage; @@ -396,8 +545,10 @@ int mapcache_vcpu_init(struct vcpu *v) =20 if ( ents > cache->entries ) { - /* Populate page tables. */ - int rc =3D create_perdomain_mapping(v, MAPCACHE_VIRT_START, ents, + /* Populate page tables, a per-vCPU cache's multipage slots includ= ed. */ + int rc =3D create_perdomain_mapping(v, MAPCACHE_VIRT_START, + ents + (dcache ? 0 + : MAPCACHE_MULTIPAGE_ENT= RIES), NIL(l1_pgentry_t *), NULL); =20 /* Populate the domain-wide bit maps (kept in the per-domain area)= . */ diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/c= onfig.h index 5bd9970fce..52d1ebc0e7 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -281,10 +281,24 @@ extern unsigned long xen_phys_start; #define MAPCACHE_VCPU_ENTRIES (CONFIG_PAGING_LEVELS * CONFIG_PAGING_LEV= ELS) /* Single-page slots of a per-vCPU mapcache (vCPU-PT domains). */ #define MAPCACHE_PERVCPU_ENTRIES 64 +/* + * Multipage slots of each vCPU: above the single-page ones of a per-vCPU + * mapcache, otherwise at the vCPU's stride of the multipage area. + */ +#define MAPCACHE_MULTIPAGE_ORDER 3 +#define MAPCACHE_MULTIPAGE_ENTRIES (1U << MAPCACHE_MULTIPAGE_ORDER) #define MAPCACHE_ENTRIES (MAX_VIRT_CPUS * MAPCACHE_VCPU_ENTRIES) #define MAPCACHE_VIRT_START PERDOMAIN_VIRT_SLOT(1) #define MAPCACHE_VIRT_END (MAPCACHE_VIRT_START + \ MAPCACHE_ENTRIES * PAGE_SIZE) +/* The multipage area, at the end of the sub-area. */ +#define MAPCACHE_MULTIPAGE_VA_SHIFT (MAPCACHE_MULTIPAGE_ORDER + PAGE_SHIFT) +#define MAPCACHE_MULTIPAGE_VIRT_START \ + (MAPCACHE_VIRT_START + (PERDOMAIN_SLOT_MBYTES << 20) - \ + (MAX_VIRT_CPUS << MAPCACHE_MULTIPAGE_VA_SHIFT)) +#define MAPCACHE_MULTIPAGE_START(v) \ + (MAPCACHE_MULTIPAGE_VIRT_START + \ + ((v)->vcpu_id << MAPCACHE_MULTIPAGE_VA_SHIFT)) =20 /* Argument translation area. The third per-domain-mapping sub-area. */ #define ARG_XLAT_VIRT_START PERDOMAIN_VIRT_SLOT(2) diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 6b27af9eb5..3748e417ce 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -84,6 +84,9 @@ struct mapcache_vcpu { uint32_t refcnt; } hash[MAPHASH_ENTRIES]; =20 + /* Which of the vCPU's map_domain_pages() slots are in use. */ + unsigned long multipage_inuse; + /* Accounting of a per-vCPU mapcache (vCPU-PT domains). */ struct mapcache cache; /* Backing for cache.{inuse,garbage}. */ diff --git a/xen/arch/x86/include/asm/spec_ctrl.h b/xen/arch/x86/include/as= m/spec_ctrl.h index b62229ab5d..d9ad94ddb0 100644 --- a/xen/arch/x86/include/asm/spec_ctrl.h +++ b/xen/arch/x86/include/asm/spec_ctrl.h @@ -88,6 +88,7 @@ extern uint8_t default_scf; extern int8_t opt_xpti_hwdom, opt_xpti_domu; =20 extern bool opt_vcpu_pt_hwdom, opt_vcpu_pt_hvm; +extern bool opt_vmap_strict; =20 extern bool cpu_has_bug_l1tf; extern int8_t opt_pv_l1tf_hwdom, opt_pv_l1tf_domu; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index c86f5ff886..3c52916eda 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -86,6 +86,9 @@ bool __ro_after_init opt_bp_spec_reduce =3D true; =20 static bool __initdata opt_ibpb_alt; =20 +/* BUG() where map_domain_pages() would fall back to vmap() (vmap-strict).= */ +bool __ro_after_init opt_vmap_strict; + /* Give each vCPU of an HVM domain its own per-domain area (vCPU-PT). */ bool __ro_after_init opt_vcpu_pt_hvm; bool __ro_after_init opt_vcpu_pt_hwdom; @@ -591,6 +594,8 @@ static int __init cf_check parse_asi(const char *s) rc =3D -EINVAL; } } + else if ( (val =3D parse_boolean("vmap-strict", s, ss)) >=3D 0= ) + opt_vmap_strict =3D val; else if ( (val =3D parse_boolean("vcpu-pt", s, ss)) !=3D -1 ) { switch ( val ) @@ -840,6 +845,8 @@ static void __init print_details(enum ind_thunk thunk) printk(" ASI features for PV VMs:%s\n", sparse_dmap_pv() && !opt_xpti_domu ? " sparse-directmap" : " None"); + printk(" ASI vmap-strict: %s\n", + opt_vmap_strict ? "enabled" : "disable= d"); } } =20 diff --git a/xen/include/xen/domain_page.h b/xen/include/xen/domain_page.h index c89b149e54..bf45a4cf55 100644 --- a/xen/include/xen/domain_page.h +++ b/xen/include/xen/domain_page.h @@ -31,6 +31,27 @@ void *map_domain_page(mfn_t mfn); */ void unmap_domain_page(const void *ptr); =20 +/* + * Map @nr page frames, not necessarily physically contiguous, at + * consecutive virtual addresses. Returns the virtual address of the + * first frame, or NULL when no space is available. As with + * map_domain_page(), the mapping is only valid within the current VCPU, + * and must be unmapped by it; unmap_domain_pages() takes the same @nr, + * and accepts a pointer carrying an in-page offset. Implemented on x86 + * only at present. + * + * @nr has an implementation-defined upper bound (x86: + * MAPCACHE_MULTIPAGE_ENTRIES). Exceeding it is a caller bug rather than a + * transient failure, and fails the mapping; callers needing more pages + * than a per-VCPU window holds want vmap() directly. + * + * unmap_domain_pages() is fatal given a pointer or an @nr which indexes + * outside the current VCPU's multipage slots. A wrong @nr within them + * is not detected: callers must pass the @nr they mapped with. + */ +void *map_domain_pages(const mfn_t *mfn, unsigned int nr); +void unmap_domain_pages(const void *ptr, unsigned int nr); + /* * Given a VA from map_domain_page(), return its underlying MFN. */ --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748667; cv=none; d=zohomail.com; s=zohoarc; b=KBOcMbhY3Uk/evHZyzolv9Hmgf6sSYPyFqc5eybQQf1qGUtNkmFHCGY+4gguI1gOxwv+Xxpu3Ylf3D57/otIcOfYE5uq4pemrna/kSCrRBnaexk7zpAGxBu/WcIm4Fxae6AEfbJVUq/rCBRHpwvcES5rwfjO/GgMm+Ww7o6Vrww= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748667; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nCu/9nxrMBPoeFP8cJd8ihmMLFkJzdeaGH0E1Itmh5E=; b=PFHXVPkBuzyEiAJ1Rx1qohQhkgLb9xB7bfqlGvGTkhWFzms1VsGESq1CSDIOmQ+vX8PIR7fQlbce/DRJElycl2j1BsAT0uIUv8KNSnA1DwbNyh0D4v/ONZuA4COTHBwgFQk0tdMctsdwzkzk6QbTLFbeJ/iYnfAkXiXKuaJzVQs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748667597853.4834638496942; Tue, 3 Nov 2026 15:31:07 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470794.1679453 (Exim 4.92) (envelope-from ) id 1xONxg-0001QN-T7; Tue, 03 Nov 2026 23:30:40 +0000 Received: by outflank-mailman (output) from mailman id 1470794.1679453; Tue, 03 Nov 2026 23:30:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxg-0001P4-Eb; Tue, 03 Nov 2026 23:30:40 +0000 Received: by outflank-mailman (input) for mailman id 1470794; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000cW-09 for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cot2-1ezN; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoB-0000000EMFa-0jBr; Tue, 03 Nov 2026 23:20:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=nCu/9nxrMBPoeFP8cJd8ihmMLFkJzdeaGH0E1Itmh5E=; b=2+Go13xgx3rs6kb+85arVkIJfH PD27N7CAB47+Zq61Ro1SoMwlN6vQJuWTzGt/jSAF/Oi7uV4myISFSEv6xRhhYpQ/90UpiXzSxV3CB kr/U867nGpFzP8EmEPciCwJN1AIX5NQ8qyS3hj2UJJfWTjF8gbe9Sh1QnrORpT2z+tVU=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , Tim Deegan , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 36/44] x86/shadow: use map_domain_pages() for cross-page emulated writes Date: Tue, 3 Nov 2026 17:13:58 -0500 Message-ID: <20261103-asi-hvm-v6-36-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748669504158500 Content-Type: text/plain; charset="utf-8" sh_emulate_map_dest() picks its mapping method purely on whether the write straddles a page boundary: a single page goes through map_domain_page(), two pages through vmap() -- mapping the destination in every context for the duration, those of domains on the sparse directmap view included, whatever the guest's own configuration. map_domain_pages() maps it only in the page tables of the guest's own domain; use it for the two-page case. The NULL return handling is unchanged: the primitive can fail only for want of a free slot, and the emulation then bails out as unhandleable exactly as before. This is one of the two in-tree consumers of the primitive. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/arch/x86/mm/shadow/hvm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c index 0b23326214..b1c3d5e637 100644 --- a/xen/arch/x86/mm/shadow/hvm.c +++ b/xen/arch/x86/mm/shadow/hvm.c @@ -506,7 +506,7 @@ static void *sh_emulate_map_dest(struct vcpu *v, unsign= ed long vaddr, /* Cross-page writes mean probably not a pagetable. */ sh_remove_shadows(d, sh_ctxt->mfn[1], 0, 0 /* Slow, can fail. */ ); =20 - map =3D vmap(sh_ctxt->mfn, 2); + map =3D map_domain_pages(sh_ctxt->mfn, 2); if ( !map ) { put_page(mfn_to_page(sh_ctxt->mfn[0])); @@ -670,7 +670,7 @@ static void sh_emulate_unmap_dest(struct vcpu *v, void = *addr, if ( unlikely(mfn_valid(sh_ctxt->mfn[1])) ) { put_page(mfn_to_page(sh_ctxt->mfn[1])); - vunmap((void *)((unsigned long)addr & PAGE_MASK)); + unmap_domain_pages(addr, 2); } else unmap_domain_page(addr); --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748695; cv=none; d=zohomail.com; s=zohoarc; b=V2235eqCQyI40tV8npKwMOfPPccXTf854NsQDn6OGo9WHTB7HkLaPZZrJzzDpTTnqd8U46COJbO4NF7kCwX7YKS+c8avqp8pHu8o3ldy5HtlzlzWwpkxkwAgY5unfdhgez9ysj5nj47+93uBR+7WG9oC5XbBPLy3Dqw6Ez/JC0s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748695; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tI8rWUCKpztUW6i1B6fFTfydag4KVKVQaRj7AWM+f08=; b=mRmgkSMhMWt7oO0VD4CsJl0Rd8j/rrM2Nhq2E4jf1+dIQtzP/JnSNNUINMn++oBaBbtsIClhsNs4wiAjDD7TgOGrgoougO58vfgJwTm7/vVMSMboLQCX5y4eoJQsxjwneEYidVH25wk8kUaNHxAJjMexiMukXS8kPHie0PJrJhI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748695168777.012581359606; Tue, 3 Nov 2026 15:31:35 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470813.1679600 (Exim 4.92) (envelope-from ) id 1xONy7-0006iZ-K5; Tue, 03 Nov 2026 23:31:07 +0000 Received: by outflank-mailman (output) from mailman id 1470813.1679600; Tue, 03 Nov 2026 23:31:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONy5-0006ZJ-Ls; Tue, 03 Nov 2026 23:31:05 +0000 Received: by outflank-mailman (input) for mailman id 1470813; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000e7-Iz for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CowF-2ALZ; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoB-0000000EMFa-2YLH; Tue, 03 Nov 2026 23:20:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=tI8rWUCKpztUW6i1B6fFTfydag4KVKVQaRj7AWM+f08=; b=PFdYYBSlPazogd+zDW6QBUkiY7 krXZyDtMh2h+WOSWP1a1EFPa1w38RNzW0hp5gW/O75U0cR9bV8/udKImyscxFr7oztXsToMuO2UoV ew3BI2Iu2sVSETl1QmuahdOYfFPWIyUpRz8gvT72MKaMsaUUy1KdmPD73w8akAQp/gCs=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 37/44] x86/hvm: use map_domain_pages() for multi-frame emulated accesses Date: Tue, 3 Nov 2026 17:13:59 -0500 Message-ID: <20261103-asi-hvm-v6-37-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748695617158500 Content-Type: text/plain; charset="utf-8" hvmemul_map_linear_addr() picks its mapping method purely on the frame count: one frame goes through map_domain_page(), more through vmap() -- mapping the guest data in every context for the duration, those of domains on the sparse directmap view included, whatever the guest's own configuration. map_domain_pages() maps them only in the page tables of the guest's own domain; use it for the multi-frame case. The NULL return handling is unchanged: the primitive can fail only for want of a free slot, and the access then bails out as unhandleable exactly as before. This is one of the two in-tree consumers of the primitive. Other long-lived global mappings of guest pages (the ioreq server pages among them) remain in the vmap area. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/arch/x86/hvm/emulate.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c index da7c08b5d9..7e7beccb93 100644 --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -735,7 +735,8 @@ static void *hvmemul_map_linear_addr( =20 /* * The maximum access size depends on the number of adjacent mfns[] wh= ich - * can be vmap()'d, accouting for possible misalignment within the reg= ion. + * can be mapped together, accounting for possible misalignment within + * the region. * The higher level emulation callers are responsible for ensuring that * mfns[] is large enough for the requested access size. */ @@ -824,9 +825,9 @@ static void *hvmemul_map_linear_addr( /* Entire access within a single frame? */ if ( nr_frames =3D=3D 1 ) mapping =3D map_domain_page(hvmemul_ctxt->mfn[0]); - /* Multiple frames? Need to vmap(). */ - else if ( (mapping =3D vmap(hvmemul_ctxt->mfn, - nr_frames)) =3D=3D NULL ) + /* Multiple frames? Need a multi-page mapping. */ + else if ( (mapping =3D map_domain_pages(hvmemul_ctxt->mfn, + nr_frames)) =3D=3D NULL ) goto unhandleable; =20 #ifndef NDEBUG /* Poision unused mfn[]s with INVALID_MFN. */ @@ -867,7 +868,7 @@ static void hvmemul_unmap_linear_addr( if ( nr_frames =3D=3D 1 ) unmap_domain_page(mapping); else - vunmap(mapping); + unmap_domain_pages(mapping, nr_frames); =20 for ( i =3D 0; i < nr_frames; i++ ) { --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748692; cv=none; d=zohomail.com; s=zohoarc; b=Sq+h6TSpmCHtsHUzsIcpmaF40CIjdwZTjL0BMwI2Lg3zfOBd7ypD63xWEJUQV5AdxWqVQs5ydOfbXO/96NF4rlEZCdh83WN37Cn08OVQctGUI/va+X8dkrGst8aK0umDjGdgs8Xvf5qUNIpIIl+SaJ114Wu5sIQNQpa4KhOFjss= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748692; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=FlX8i68BurZZbddP5JFg/suxuQm3LKHJ87QfUKq0fho=; b=SoRj4K+KNfRdBo/vOv5nxehRG3jJc+FiFHejxm2Bf+ZoZOETON0gmgnAedJb32YbZzokexUQ2klQRThcAuRedYmhPjiDteuxXLeBRI8wNUZmbE8zXlSUHK+yE9gL3tHi6CPwTq83GT9ps4QmpT3GIMKMsEO3gz5zvK9ajPSsUN8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748692882134.10359891573125; Tue, 3 Nov 2026 15:31:32 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470816.1679629 (Exim 4.92) (envelope-from ) id 1xONyF-0008VS-Fs; Tue, 03 Nov 2026 23:31:15 +0000 Received: by outflank-mailman (output) from mailman id 1470816.1679629; Tue, 03 Nov 2026 23:31:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONyD-0008OB-RC; Tue, 03 Nov 2026 23:31:13 +0000 Received: by outflank-mailman (input) for mailman id 1470816; Tue, 03 Nov 2026 23:30:41 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxg-0001WI-Tw for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:40 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotS-3RdR; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoC-0000000EMFa-0N4x; Tue, 03 Nov 2026 23:20:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=FlX8i68BurZZbddP5JFg/suxuQm3LKHJ87QfUKq0fho=; b=htbWlk3KJqG5vECGkaAg8TQ5wn 7uXoaZRD1izMpagwbi1PMdwVCkrkUJhA57EeehYboSwoZRIgbCCdD8UU5x6Nhv2vmIM3MuoeLq1+h 9z6DjsG9kaBLG0RdTM0mqMQQ+zAXmfUxe9D5WL5KE6V1TTT9KA6HI49qTdC2vrQ84PoM=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 38/44] x86/emulator: Refactor FXSAVE_AREA to use wrappers Date: Tue, 3 Nov 2026 17:14:00 -0500 Message-ID: <20261103-asi-hvm-v6-38-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748693656158500 Content-Type: text/plain; charset="utf-8" From: Alejandro Vallejo The FXSAVE/FXRSTOR emulation scratch buffer is the calling vCPU's own XSAVE area, reached today through a raw pointer dereference. Route it through the VCPU_MAP_XSAVE_AREA()/VCPU_UNMAP_XSAVE_AREA() seam instead, so the buffer keeps working when guest XSAVE areas move behind per-vCPU mappings. This path always operates on `current`, for which the map has a fastpath; the unmap is a no-op outside Xen builds. The macro pair is named MAP_FXSAVE_AREA()/UNMAP_FXSAVE_AREA() rather than keeping the bare FXSAVE_AREA name, so that the mapping operation is visible at the use sites. blk_fxsave() saves directly into the caller's buffer when it is large enough, so it needs a second variable: the save destination and the mapped area are not the same pointer, and only the latter may be handed to the unmap. Signed-off-by: Alejandro Vallejo Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Revived from "x86: Address Space Isolation FPU preparations" v3 patch 08/12 (<20250110132823.24348-9-alejandro.vallejo@cloud.com>, Acked-by Jan Beulich modulo the rename below; no v4 was ever posted). Changes in v5: - New in this version. Changes since the previously posted version: - Renamed FXSAVE_AREA to MAP_FXSAVE_AREA(), as requested by Jan Beulich on the original thread ("the existing macro better would be adjusted to become MAP_FXSAVE_AREA()"), making the map operation visible at the use sites. - Introduced the separate save-destination variable in blk_fxsave(). The original unmapped `fxsr` after that variable had been repointed at the caller's buffer, so the unmap received a pointer that was never mapped. Harmless while the unmap was a stub, but a latent bug, and exactly the class of mistake Jan predicted in the message quoted above when he objected to the mapping being invisible at the use sites. - Reworded the commit message. - Unmap before raising #GP for a reserved MXCSR bit in blk_fxrstor: generate_exception_if() leaves through the done: label, past the unmap. --- xen/arch/x86/x86_emulate/blk.c | 36 +++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/xen/arch/x86/x86_emulate/blk.c b/xen/arch/x86/x86_emulate/blk.c index 9fddf649ee..b3055c8084 100644 --- a/xen/arch/x86/x86_emulate/blk.c +++ b/xen/arch/x86/x86_emulate/blk.c @@ -11,9 +11,12 @@ !defined(X86EMUL_NO_SIMD) # ifdef __XEN__ # include -# define FXSAVE_AREA ((void *)¤t->arch.xsave_area->fpu_sse) +/* Has a fastpath for `current`, so there's no actual map */ +# define MAP_FXSAVE_AREA() ((void *)VCPU_MAP_XSAVE_AREA(current)) +# define UNMAP_FXSAVE_AREA(x) VCPU_UNMAP_XSAVE_AREA(current, x) # else -# define FXSAVE_AREA get_fpu_save_area() +# define MAP_FXSAVE_AREA() get_fpu_save_area() +# define UNMAP_FXSAVE_AREA(x) ((void)(x)) # endif #endif =20 @@ -254,7 +257,7 @@ int x86_emul_blk( =20 case blk_fxrstor: { - struct x86_fxsr *fxsr =3D FXSAVE_AREA; + struct x86_fxsr *fxsr =3D MAP_FXSAVE_AREA(); =20 ASSERT(!data); ASSERT(bytes =3D=3D sizeof(*fxsr)); @@ -277,18 +280,26 @@ int x86_emul_blk( (unsigned int)offsetof(struct x86_fxsr, rsvd= ))); memset(fxsr->rsvd, 0, sizeof(*fxsr) - offsetof(struct x86_fxsr, rs= vd)); =20 - generate_exception_if(fxsr->mxcsr & ~mxcsr_mask, X86_EXC_GP, 0); + if ( fxsr->mxcsr & ~mxcsr_mask ) + { + UNMAP_FXSAVE_AREA(fxsr); + generate_exception(X86_EXC_GP, 0); + } =20 if ( s->rex_prefix & REX_W ) asm volatile ( "fxrstorq %0" :: "m" (*fxsr) ); else asm volatile ( "fxrstor %0" :: "m" (*fxsr) ); + + UNMAP_FXSAVE_AREA(fxsr); + break; } =20 case blk_fxsave: { - struct x86_fxsr *fxsr =3D FXSAVE_AREA; + struct x86_fxsr *fxsr =3D MAP_FXSAVE_AREA(); + struct x86_fxsr *dst =3D fxsr; =20 ASSERT(!data); ASSERT(bytes =3D=3D sizeof(*fxsr)); @@ -296,17 +307,20 @@ int x86_emul_blk( =20 if ( s->op_bytes < sizeof(*fxsr) ) /* Don't chance consuming uninitialized data. */ - memset(fxsr, 0, s->op_bytes); + memset(dst, 0, s->op_bytes); else - fxsr =3D ptr; + dst =3D ptr; =20 if ( s->rex_prefix & REX_W ) - asm volatile ( "fxsaveq %0" : "=3Dm" (*fxsr) ); + asm volatile ( "fxsaveq %0" : "=3Dm" (*dst) ); else - asm volatile ( "fxsave %0" : "=3Dm" (*fxsr) ); + asm volatile ( "fxsave %0" : "=3Dm" (*dst) ); + + if ( dst !=3D ptr ) /* i.e. s->op_bytes < sizeof(*fxsr) */ + memcpy(ptr, dst, s->op_bytes); + + UNMAP_FXSAVE_AREA(fxsr); =20 - if ( fxsr !=3D ptr ) /* i.e. s->op_bytes < sizeof(*fxsr) */ - memcpy(ptr, fxsr, s->op_bytes); break; } =20 --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748686; cv=none; d=zohomail.com; s=zohoarc; b=Z8DOD+V6l4t3eNhTCqJQmC/ICzWdbeKQ9SILu0qtu82MAjrtOSVpixSO02m28Kh+B2Z4X4RP1VPTIr2svR/9OaW+Ft1EBRxmfeC+XMEQVXHAmIvPiyWD9uiaXWHFdG8k+EUL2QzdNVKrrLFMngEe/uV2kNrYdPRo298XXIcNKOs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748686; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=78UJyNn4VY/h+N1P7RYObuj4ivXfajt4ZYGSI254v8U=; b=mRrjWRtRj8NssZsOAZ0UHUIcUJUqCGPw8pHLpvrIW9aQWXftvYKJhqbhc4EtSCwQ9GOv8znsoI9K5FiinzDMFvtuUWC5/FxQqGEeGLO8iHyOQzt6+D824W0drcwG6BZpdtoZG3s3hj/48N89aqt7iT0PwSpqRuG7vPmPKRF1Sdc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748686744113.68207176454803; Tue, 3 Nov 2026 15:31:26 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470808.1679558 (Exim 4.92) (envelope-from ) id 1xONxw-0004Ri-TO; Tue, 03 Nov 2026 23:30:56 +0000 Received: by outflank-mailman (output) from mailman id 1470808.1679558; Tue, 03 Nov 2026 23:30:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxv-0004O0-CD; Tue, 03 Nov 2026 23:30:55 +0000 Received: by outflank-mailman (input) for mailman id 1470808; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000ds-HC for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Covi-1oFa; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoC-0000000EMFa-2YTw; Tue, 03 Nov 2026 23:20:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=78UJyNn4VY/h+N1P7RYObuj4ivXfajt4ZYGSI254v8U=; b=3AAaRV0jEBDArkkKqQ4+hWBY7M 0fi2JDdjjuWW0a4Tzp1Bz/dVPqGP0vMFXOOf1eBqxMQ8Qs60d3WFcH/SsNUnOiUIzd0N7yd5elt8R tSQqEBAmcF93TWmBQsEMURWMwB2gEEtDr/1yPEk4BKu7ba9F5ZLxHtbyX/H6gooCOjCY=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 39/44] x86/mpx: Map/unmap xsave area in read_bndcfgu() Date: Tue, 3 Nov 2026 17:14:01 -0500 Message-ID: <20261103-asi-hvm-v6-39-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748687642158500 Content-Type: text/plain; charset="utf-8" From: Alejandro Vallejo The BNDCSR read executes XSAVE into the local idle vCPU's save area as scratch space and pulls BNDCFGU out of the result. Fetch that buffer through the VCPU_MAP_XSAVE_AREA() seam rather than a raw dereference, so the function is indifferent to where vCPU save areas live. This was the last raw access to another vCPU's save-area pointer outside the seam. No functional change. Signed-off-by: Alejandro Vallejo Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Revived from "x86: Address Space Isolation FPU preparations" v3 patch 09/12 (<20250110132823.24348-10-alejandro.vallejo@cloud.com>; no v4 was ever posted). Changes in v5: - New in this version. Changes since the previously posted version: - Constified the vcpu pointer, as requested by Jan Beulich on the original thread (the xstate pointer itself stays mutable: the asm()s write through it, as Jan noted in withdrawing that half of the request). - Expanded the commit message. --- xen/arch/x86/xstate.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 8336414c52..2eae3f098d 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -1023,9 +1023,10 @@ int handle_xsetbv(u32 index, u64 new_bv) =20 uint64_t read_bndcfgu(void) { + uint64_t bndcfgu =3D 0; unsigned long cr0 =3D read_cr0(); - struct xsave_struct *xstate - =3D idle_vcpu[smp_processor_id()]->arch.xsave_area; + const struct vcpu *v =3D idle_vcpu[smp_processor_id()]; + struct xsave_struct *xstate =3D VCPU_MAP_XSAVE_AREA(v); const struct xstate_bndcsr *bndcsr; =20 ASSERT(cpu_has_mpx); @@ -1051,7 +1052,12 @@ uint64_t read_bndcfgu(void) if ( cr0 & X86_CR0_TS ) write_cr0(cr0); =20 - return xstate->xsave_hdr.xstate_bv & X86_XCR0_BNDCSR ? bndcsr->bndcfgu= : 0; + if ( xstate->xsave_hdr.xstate_bv & X86_XCR0_BNDCSR ) + bndcfgu =3D bndcsr->bndcfgu; + + VCPU_UNMAP_XSAVE_AREA(v, xstate); + + return bndcfgu; } =20 void xstate_set_init(uint64_t mask) --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748685; cv=none; d=zohomail.com; s=zohoarc; b=DMtbKG1tpSh9HypbDTlJrx68mpE4pvnughgTYFelfXux0xSQ6O6fL7oH3tt6SxeKh7JwWBvlVF0VgSmtGkaW0Q63L7GIqsC1QH8I/WOR5SNt1+b2VghyPRgd9omkE2326qqRIAYo/75s4Jab6L9VzKSf0o3/DYtSF0BPyHSBK3o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748685; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+ze/mUts7B6alTNouyX2ejvqaet9bE/wAgVhHkz5gYM=; b=RIt/r5nqfp8mYBEsbS87+SU6yXEQjap5pS+yzFTB0EXAV9f1ltptTOwocCKrKxNcAuPGvzayxm0G++mGqH9R35ZeJkEwk+olMWkqoHXzdWS6m4unFh4QWF3D0uQ1FcqBp5GuVYzhBULQ7heBY2kUbzxVGME6vXPbqvTAPvuyDH8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748685913265.0157859897174; Tue, 3 Nov 2026 15:31:25 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470814.1679610 (Exim 4.92) (envelope-from ) id 1xONyA-0007Jy-3a; Tue, 03 Nov 2026 23:31:10 +0000 Received: by outflank-mailman (output) from mailman id 1470814.1679610; Tue, 03 Nov 2026 23:31:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONy8-0007BW-BR; Tue, 03 Nov 2026 23:31:08 +0000 Received: by outflank-mailman (input) for mailman id 1470814; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000eU-N8 for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CouI-06Vu; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoD-0000000EMFa-0N4L; Tue, 03 Nov 2026 23:20:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=+ze/mUts7B6alTNouyX2ejvqaet9bE/wAgVhHkz5gYM=; b=vblZ+qDcgbyK7289GtynIGBy6w Y4rD6msZISrkm+/4Gwolo9g7c1S4FaucHuqrKd9mUkx9W+5HXmD0QN+GtM/0YguYxR+/hVQ2FvH0/ FVMmzdRR+3ePCDdRJs+AB+x6LWBCmlcFokQZIuiawZ5OQW49XWWW1hvHyY9KLjCKPJow=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 40/44] x86/fpu: Pass explicit xsave areas to fpu_(f)xsave() Date: Tue, 3 Nov 2026 17:14:02 -0500 Message-ID: <20261103-asi-hvm-v6-40-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748687664158500 Content-Type: text/plain; charset="utf-8" From: Alejandro Vallejo Have the save entry point fetch the vCPU's XSAVE area once, through the VCPU_MAP_XSAVE_AREA() seam, and pass the resulting pointer down explicitly: xsave() gains an area parameter in place of its internal dereference, and fpu_fxsave() takes the legacy region directly. The lower-level helpers no longer know where a vCPU's save area lives, which is the property the coming relocation of guest save areas relies on; the vcpu pointers become const on the way. No functional change. Signed-off-by: Alejandro Vallejo Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Revived from "x86: Address Space Isolation FPU preparations" v3 patch 10/12 (<20250110132823.24348-11-alejandro.vallejo@cloud.com>, Acked-by Jan Beulich; no v4 was ever posted). Changes in v5: - New in this version. Changes since the previously posted version: - Regenerated against the eager-FPU-only tree: the entry point is now vcpu_save_fpu() (vcpu_xsave_mask()/_vcpu_save_fpu() no longer exist), and the mask is always XSTATE_ALL. - Expanded the commit message. --- xen/arch/x86/i387.c | 18 ++++++++++++------ xen/arch/x86/include/asm/xstate.h | 2 +- xen/arch/x86/xstate.c | 3 +-- 3 files changed, 14 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index c6796bc97b..65491b5697 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -109,7 +109,8 @@ static inline void fpu_fxrstor(struct vcpu *v) /*******************************/ =20 /* Save x87 extended state */ -static inline void fpu_xsave(struct vcpu *v) +static inline void fpu_xsave(const struct vcpu *v, + struct xsave_struct *xsave_area) { bool ok; =20 @@ -119,15 +120,14 @@ static inline void fpu_xsave(struct vcpu *v) */ ok =3D set_xcr0(v->arch.xcr0_accum | XSTATE_FP_SSE); ASSERT(ok); - xsave(v, XSTATE_ALL); + xsave(v, xsave_area, XSTATE_ALL); ok =3D set_xcr0(v->arch.xcr0 ?: XSTATE_FP_SSE); ASSERT(ok); } =20 /* Save x87 FPU, MMX, SSE and SSE2 state */ -static inline void fpu_fxsave(struct vcpu *v) +static inline void fpu_fxsave(const struct vcpu *v, fpusse_t *fpu_ctxt) { - fpusse_t *fpu_ctxt =3D &v->arch.xsave_area->fpu_sse; unsigned int fip_width =3D v->domain->arch.x87_fip_width; =20 if ( fip_width !=3D 4 ) @@ -192,15 +192,21 @@ void vcpu_restore_fpu(struct vcpu *v) */ void vcpu_save_fpu(struct vcpu *v) { + struct xsave_struct *xsave_area; + ASSERT(!is_idle_vcpu(v)); =20 /* This can happen, if a paravirtualised guest OS has set its CR0.TS. = */ clts(); =20 + xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + if ( cpu_has_xsave ) - fpu_xsave(v); + fpu_xsave(v, xsave_area); else - fpu_fxsave(v); + fpu_fxsave(v, &xsave_area->fpu_sse); + + VCPU_UNMAP_XSAVE_AREA(v, xsave_area); } =20 /* Initialize FPU's context save area */ diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/x= state.h index 0519379edb..40bb6f152e 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -95,7 +95,7 @@ uint64_t get_xcr0(void); void set_msr_xss(u64 xss); uint64_t get_msr_xss(void); uint64_t read_bndcfgu(void); -void xsave(struct vcpu *v, uint64_t mask); +void xsave(const struct vcpu *v, struct xsave_struct *ptr, uint64_t mask); void xrstor(struct vcpu *v, uint64_t mask); void xstate_set_init(uint64_t mask); bool xsave_enabled(const struct vcpu *v); diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 2eae3f098d..058bc4c0bc 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -304,9 +304,8 @@ void compress_xsave_states(struct vcpu *v, const void *= src, unsigned int size) VCPU_UNMAP_XSAVE_AREA(v, xstate); } =20 -void xsave(struct vcpu *v, uint64_t mask) +void xsave(const struct vcpu *v, struct xsave_struct *ptr, uint64_t mask) { - struct xsave_struct *ptr =3D v->arch.xsave_area; uint32_t hmask =3D mask >> 32; uint32_t lmask =3D mask; unsigned int fip_width =3D v->domain->arch.x87_fip_width; --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748693; cv=none; d=zohomail.com; s=zohoarc; b=jcxEPtKPiBo7DztA2+KD+Blo6VArWB8htYMoqsf3ldc0F8wSA6U+ESkSA6U9aAw6otgNmX0pZA/1CPu0mzeMZvWwEa9fnUbNVGgeQzWgrDwWMLRWFRSDQK6WvBtlO1+JFzVI+TdfZQ+6nXXKmI+rqE2d9yAjAtd+uxslwJR7WP0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748693; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=O3n3UQQODatRF1IOFe5HjC1fiBbAzhiwh2RBel9BdYM=; b=kBuILR3TC/IYT8XlCQ5rHvik9S8hyPZgmDAROiT51GTTbSt7LglbMAA1dyvYax51m2QS4JLUzw2npO+RXooJxFbQJB1I25aqX3wMuxvtGT8hP3QopP+40nJEQafMD2ZdvlH5RZ0yhc27JAo/XDJOUgkxxZo4WglstiQ7uk4USKQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748692998937.7618660130721; Tue, 3 Nov 2026 15:31:32 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470812.1679567 (Exim 4.92) (envelope-from ) id 1xONxy-0004re-Vq; Tue, 03 Nov 2026 23:30:58 +0000 Received: by outflank-mailman (output) from mailman id 1470812.1679567; Tue, 03 Nov 2026 23:30:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxx-0004l3-BU; Tue, 03 Nov 2026 23:30:57 +0000 Received: by outflank-mailman (input) for mailman id 1470812; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000eO-L1 for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000CowL-2IRJ; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoD-0000000EMFa-2Nu9; Tue, 03 Nov 2026 23:20:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=O3n3UQQODatRF1IOFe5HjC1fiBbAzhiwh2RBel9BdYM=; b=Ac3F6HMDN03iYHMp15QDsTkCtG e10Ll0MTW0eMH+z8uX3BQ8VpuWXkWMlHdGnk7BpYQKSC11HmlgSPS++kWp7KjHoTB9QE51hd6r3tR BHViVvKio1DBoCa5uMGlqkzknuOSMxf/Jx8ta+Z9LJrYaW1Y4xr0D7paeoXYYBSTtGAA=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 41/44] x86/fpu: Pass explicit xsave areas to fpu_(f)xrstor() Date: Tue, 3 Nov 2026 17:14:03 -0500 Message-ID: <20261103-asi-hvm-v6-41-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748693644158500 Content-Type: text/plain; charset="utf-8" From: Alejandro Vallejo Mirror of the save-side change: the restore entry point fetches the vCPU's XSAVE area once, through the VCPU_MAP_XSAVE_AREA() seam, and passes the pointer down explicitly. xrstor() gains an area parameter in place of its internal dereference (its fault-recovery writes go through the same mapped pointer), fpu_fxrstor() reduces to taking just the legacy region, and the two direct xrstor() callers -- XSETBV emulation and xstate_set_init() -- pass the pointers they already hold mapped. In handle_xsetbv() this moves the xrstor inside the mapped section, where it belongs. No functional change. Signed-off-by: Alejandro Vallejo Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Revived from "x86: Address Space Isolation FPU preparations" v3 patch 11/12 (<20250110132823.24348-12-alejandro.vallejo@cloud.com>, Acked-by Jan Beulich; no v4 was ever posted). Changes in v5: - New in this version. Changes since the previously posted version: - Regenerated against the eager-FPU-only tree: the entry point is now vcpu_restore_fpu() (the lazy/nonlazy split no longer exists), and the mask is always XSTATE_ALL. - Moved handle_xsetbv()'s xrstor() call inside its map/unmap bracket, which the explicit-area form makes possible. - Expanded the commit message. --- xen/arch/x86/i387.c | 19 ++++++++++++------- xen/arch/x86/include/asm/xstate.h | 2 +- xen/arch/x86/xstate.c | 7 +++---- 3 files changed, 16 insertions(+), 12 deletions(-) diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 65491b5697..b796687eeb 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -20,7 +20,8 @@ /* FPU Restore Functions */ /*******************************/ /* Restore x87 extended state */ -static inline void fpu_xrstor(struct vcpu *v, uint64_t mask) +static inline void fpu_xrstor(const struct vcpu *v, + struct xsave_struct *xsave_area, uint64_t ma= sk) { bool ok; =20 @@ -30,16 +31,14 @@ static inline void fpu_xrstor(struct vcpu *v, uint64_t = mask) */ ok =3D set_xcr0(v->arch.xcr0_accum | XSTATE_FP_SSE); ASSERT(ok); - xrstor(v, mask); + xrstor(v, xsave_area, mask); ok =3D set_xcr0(v->arch.xcr0 ?: XSTATE_FP_SSE); ASSERT(ok); } =20 /* Restore x87 FPU, MMX, SSE and SSE2 state */ -static inline void fpu_fxrstor(struct vcpu *v) +static inline void fpu_fxrstor(const fpusse_t *fpu_ctxt) { - const fpusse_t *fpu_ctxt =3D &v->arch.xsave_area->fpu_sse; - /* * Some CPUs don't save/restore FDP/FIP/FOP unless an exception * is pending. Clear the x87 state here by setting it to fixed @@ -174,12 +173,18 @@ static inline void fpu_fxsave(const struct vcpu *v, f= pusse_t *fpu_ctxt) /* Restore FPU state whenever VCPU is schduled in. */ void vcpu_restore_fpu(struct vcpu *v) { + struct xsave_struct *xsave_area; + ASSERT(!is_idle_vcpu(v)); =20 + xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + if ( cpu_has_xsave ) - fpu_xrstor(v, XSTATE_ALL); + fpu_xrstor(v, xsave_area, XSTATE_ALL); else - fpu_fxrstor(v); + fpu_fxrstor(&xsave_area->fpu_sse); + + VCPU_UNMAP_XSAVE_AREA(v, xsave_area); =20 /* Xen doesn't need TS set, but the guest might. */ if ( is_pv_vcpu(v) && (v->arch.pv.ctrlreg[0] & X86_CR0_TS) ) diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/x= state.h index 40bb6f152e..3d57aaadd8 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -96,7 +96,7 @@ void set_msr_xss(u64 xss); uint64_t get_msr_xss(void); uint64_t read_bndcfgu(void); void xsave(const struct vcpu *v, struct xsave_struct *ptr, uint64_t mask); -void xrstor(struct vcpu *v, uint64_t mask); +void xrstor(const struct vcpu *v, struct xsave_struct *ptr, uint64_t mask); void xstate_set_init(uint64_t mask); bool xsave_enabled(const struct vcpu *v); int __must_check validate_xstate(const struct domain *d, diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 058bc4c0bc..3c9be271a3 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -378,11 +378,10 @@ void xsave(const struct vcpu *v, struct xsave_struct = *ptr, uint64_t mask) ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET] =3D fip_width; } =20 -void xrstor(struct vcpu *v, uint64_t mask) +void xrstor(const struct vcpu *v, struct xsave_struct *ptr, uint64_t mask) { uint32_t hmask =3D mask >> 32; uint32_t lmask =3D mask; - struct xsave_struct *ptr =3D v->arch.xsave_area; unsigned int faults, prev_faults; =20 /* @@ -1011,8 +1010,8 @@ int handle_xsetbv(u32 index, u64 new_bv) clts(); =20 asm ( "stmxcsr %0" : "=3Dm" (xsave_area->fpu_sse.mxcsr) ); + xrstor(curr, xsave_area, mask); VCPU_UNMAP_XSAVE_AREA(curr, xsave_area); - xrstor(curr, mask); if ( cr0 & X86_CR0_TS ) write_cr0(cr0); } @@ -1079,7 +1078,7 @@ void xstate_set_init(uint64_t mask) =20 xstate =3D VCPU_MAP_XSAVE_AREA(v); memset(&xstate->xsave_hdr, 0, sizeof(xstate->xsave_hdr)); - xrstor(v, mask); + xrstor(v, xstate, mask); VCPU_UNMAP_XSAVE_AREA(v, xstate); =20 if ( cr0 & X86_CR0_TS ) --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748668; cv=none; d=zohomail.com; s=zohoarc; b=Me8xcn9SccKgv7C940xG6qKTl8u1lT00kvvHtfD7YnNJ/UKCnjqiS30MqNL0ZwAgLrZ7o1Jjl0czqhdyRCbVJsXF9vVJtmNT5RHK3VGTPJwmn5ckuaKLliKxf5DqdmhZqboCvujBVBMTc/naAT9+zd/ZvuGDuUS4VDrXpshQgF8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748668; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=JymEwuliQGpipSiVxt1K5QHdAnPa509ERWMPUlPlzDM=; b=TjqilBv1S99pYpKLD+/oIxumsW6GdUKJqR6pDpYPo3t8L3xt7MSLcA0VfmLJqJkjHU4kC4lDZ+zWLuibdIOfHK5+ATcmiCoTpL9yBMa4VAyieB7DuoCWPqZ+aRoBOGy061nuimxiYaI2XKMt/i556kK4rIs7wCE1zheSsuvbqC4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748668202290.8295235472008; Tue, 3 Nov 2026 15:31:08 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470789.1679413 (Exim 4.92) (envelope-from ) id 1xONxe-0000hJ-6t; Tue, 03 Nov 2026 23:30:38 +0000 Received: by outflank-mailman (output) from mailman id 1470789.1679413; Tue, 03 Nov 2026 23:30:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxe-0000gz-0Y; Tue, 03 Nov 2026 23:30:38 +0000 Received: by outflank-mailman (input) for mailman id 1470789; Tue, 03 Nov 2026 23:30:35 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxb-0000c8-SD for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:35 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000CotK-3C0h; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoE-0000000EMFa-0N9V; Tue, 03 Nov 2026 23:20:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=JymEwuliQGpipSiVxt1K5QHdAnPa509ERWMPUlPlzDM=; b=zbrOD9foDHs/1ZurYAVqDbKW/M cv64cI8Um2wye+9aXyjo4/52fpfwcqx13OrvxZ04h3V5lKuhl7OZf3U3tBNPNEheICJfC7iMCehPu YJzdPU/DOSd/qgI2B0VMo6+gSQ593yGiBud8OHZ89lJ//R4kkbmb+uMUG4C50K2nmvds=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 42/44] x86/xstate: classify xsave-area seam accesses by failure mode Date: Tue, 3 Nov 2026 17:14:04 -0500 Message-ID: <20261103-asi-hvm-v6-42-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748669601158500 Content-Type: text/plain; charset="utf-8" The VCPU_MAP_XSAVE_AREA() seam currently cannot fail: the map is a pointer fetch. Once guest XSAVE areas move behind per-vCPU mappings, mapping another vCPU's multi-page area becomes an operation that can run out of resources. Split the seam's users along that line ahead of the change: - expand_xsave_states() and compress_xsave_states() address the full area; they gain an error return, which their four callers (the vcpuextstate domctls and the HVM context save/load handlers) propagate. The two deserialising callers (the setvcpuextstate domctl and the HVM xsave-state load handler) restore the previously-set XCR0 fields if the deserialisation fails, so a failed restore leaves the vCPU's state as it was. - Accessors touching only the legacy region and the XSAVE header -- vcpu_reset_fpu(), vcpu_setup_fpu(), hvm_save_cpu_ctxt() and arch_get_info_guest() -- switch to a new VCPU_MAP_XSAVE_AREA_PREFIX() variant. The prefix always fits the area's first page, so that variant can remain infallible, and no error plumbing is needed on the paths (INIT/SIPI, S3 resume) that have no way to fail. No functional change: both variants are still pointer fetches. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/arch/x86/domctl.c | 26 ++++++++++++++++++-------- xen/arch/x86/hvm/hvm.c | 27 ++++++++++++++++++--------- xen/arch/x86/i387.c | 8 ++++---- xen/arch/x86/include/asm/xstate.h | 13 +++++++++++-- xen/arch/x86/xstate.c | 18 +++++++++++++++--- 5 files changed, 66 insertions(+), 26 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 04bcde5580..f0dad47a87 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1010,10 +1010,10 @@ long arch_do_domctl( goto vcpuextstate_out; } =20 - expand_xsave_states(v, xsave_area, xsave_size); + ret =3D expand_xsave_states(v, xsave_area, xsave_size); =20 - if ( copy_to_guest_offset(evc->buffer, offset, xsave_area, - xsave_size) ) + if ( !ret && copy_to_guest_offset(evc->buffer, offset, + xsave_area, xsave_size) ) ret =3D -EFAULT; xfree(xsave_area); } @@ -1077,14 +1077,24 @@ long arch_do_domctl( ret =3D -EINVAL; /* Not legitimate data. */ else { + uint64_t old_xcr0, old_xcr0_accum; + vcpu_pause(v); =20 + old_xcr0 =3D v->arch.xcr0; + old_xcr0_accum =3D v->arch.xcr0_accum; v->arch.xcr0 =3D _xcr0; v->arch.xcr0_accum =3D _xcr0_accum; - compress_xsave_states(v, _xsave_area, - evc->size - PV_XSAVE_HDR_SIZE); + ret =3D compress_xsave_states(v, _xsave_area, + evc->size - PV_XSAVE_HDR_SIZE); =20 - if ( is_hvm_domain(d) ) + if ( ret ) + { + /* Leave the vCPU's state as it was. */ + v->arch.xcr0 =3D old_xcr0; + v->arch.xcr0_accum =3D old_xcr0_accum; + } + else if ( is_hvm_domain(d) ) hvmemul_cancel(v); =20 vcpu_unpause(v); @@ -1468,10 +1478,10 @@ void arch_get_info_guest(struct vcpu *v, vcpu_guest= _context_u c) #define c(fld) (c.nat->fld) #endif =20 - xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + xsave_area =3D VCPU_MAP_XSAVE_AREA_PREFIX(v); BUILD_BUG_ON(sizeof(c.nat->fpu_ctxt) !=3D sizeof(xsave_area->fpu_sse)); memcpy(&c.nat->fpu_ctxt, &xsave_area->fpu_sse, sizeof(c.nat->fpu_ctxt)= ); - VCPU_UNMAP_XSAVE_AREA(v, xsave_area); + VCPU_UNMAP_XSAVE_AREA_PREFIX(v, xsave_area); =20 if ( is_pv_domain(d) ) c(flags =3D v->arch.pv.vgc_flags & ~VGCF_in_kernel); diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index eadccf3d79..42a7d03b3a 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -932,10 +932,10 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v,= hvm_domain_context_t *h) ctxt.ldtr_base =3D seg.base; ctxt.ldtr_arbytes =3D seg.attr; =20 - xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + xsave_area =3D VCPU_MAP_XSAVE_AREA_PREFIX(v); BUILD_BUG_ON(sizeof(ctxt.fpu_regs) !=3D sizeof(xsave_area->fpu_sse)); memcpy(ctxt.fpu_regs, &xsave_area->fpu_sse, sizeof(ctxt.fpu_regs)); - VCPU_UNMAP_XSAVE_AREA(v, xsave_area); + VCPU_UNMAP_XSAVE_AREA_PREFIX(v, xsave_area); ctxt.flags =3D XEN_X86_FPU_INITIALISED; =20 return hvm_save_entry(CPU, v->vcpu_id, h, &ctxt); @@ -1251,10 +1251,8 @@ static int cf_check hvm_save_cpu_xsave_states( ctxt->xcr0 =3D v->arch.xcr0; ctxt->xcr0_accum =3D v->arch.xcr0_accum; =20 - expand_xsave_states(v, &ctxt->save_area, - size - offsetof(typeof(*ctxt), save_area)); - - return 0; + return expand_xsave_states(v, &ctxt->save_area, + size - offsetof(typeof(*ctxt), save_area)); } =20 /* @@ -1279,6 +1277,7 @@ static int cf_check hvm_load_cpu_xsave_states( struct hvm_hw_cpu_xsave *ctxt; const struct hvm_save_descriptor *desc; unsigned int i, desc_start, desc_length; + uint64_t old_xcr0, old_xcr0_accum; =20 /* Which vcpu is this? */ vcpuid =3D hvm_load_instance(h); @@ -1383,12 +1382,22 @@ static int cf_check hvm_load_cpu_xsave_states( } /* Checking finished */ =20 + old_xcr0 =3D v->arch.xcr0; + old_xcr0_accum =3D v->arch.xcr0_accum; v->arch.xcr0 =3D ctxt->xcr0; v->arch.xcr0_accum =3D ctxt->xcr0_accum; - compress_xsave_states(v, &ctxt->save_area, - size - offsetof(struct hvm_hw_cpu_xsave, save_ar= ea)); =20 - return 0; + err =3D compress_xsave_states(v, &ctxt->save_area, + size - offsetof(struct hvm_hw_cpu_xsave, + save_area)); + if ( err ) + { + /* Leave the vCPU's state as it was. */ + v->arch.xcr0 =3D old_xcr0; + v->arch.xcr0_accum =3D old_xcr0_accum; + } + + return err; } =20 #define HVM_CPU_MSR_SIZE(cnt) offsetof(struct hvm_msr, msr[cnt]) diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index b796687eeb..d49e391e99 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -222,7 +222,7 @@ int vcpu_init_fpu(struct vcpu *v) =20 void vcpu_reset_fpu(struct vcpu *v) { - struct xsave_struct *xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + struct xsave_struct *xsave_area =3D VCPU_MAP_XSAVE_AREA_PREFIX(v); =20 *xsave_area =3D (struct xsave_struct) { .xsave_hdr.xstate_bv =3D X86_XCR0_X87, @@ -233,19 +233,19 @@ void vcpu_reset_fpu(struct vcpu *v) xsave_area->fpu_sse.fcw =3D FCW_RESET; xsave_area->fpu_sse.ftw =3D FXSAVE_FTW_RESET; =20 - VCPU_UNMAP_XSAVE_AREA(v, xsave_area); + VCPU_UNMAP_XSAVE_AREA_PREFIX(v, xsave_area); } =20 void vcpu_setup_fpu(struct vcpu *v, const void *data) { - struct xsave_struct *xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + struct xsave_struct *xsave_area =3D VCPU_MAP_XSAVE_AREA_PREFIX(v); =20 *xsave_area =3D (struct xsave_struct) { .fpu_sse =3D *(const fpusse_t*)data, .xsave_hdr.xstate_bv =3D XSTATE_FP_SSE, }; =20 - VCPU_UNMAP_XSAVE_AREA(v, xsave_area); + VCPU_UNMAP_XSAVE_AREA_PREFIX(v, xsave_area); } =20 /* Free FPU's context save area */ diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/x= state.h index 3d57aaadd8..c23308cbe2 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -103,8 +103,8 @@ int __must_check validate_xstate(const struct domain *d, uint64_t xcr0, uint64_t xcr0_accum, const struct xsave_hdr *hdr); int __must_check handle_xsetbv(u32 index, u64 new_bv); -void expand_xsave_states(const struct vcpu *v, void *dest, unsigned int si= ze); -void compress_xsave_states(struct vcpu *v, const void *src, unsigned int s= ize); +int expand_xsave_states(const struct vcpu *v, void *dest, unsigned int siz= e); +int compress_xsave_states(struct vcpu *v, const void *src, unsigned int si= ze); =20 /* extended state init and cleanup functions */ void xstate_free_save_area(struct vcpu *v); @@ -171,4 +171,13 @@ xsave_area_compressed(const struct xsave_struct *xsave= _area) */ #define VCPU_UNMAP_XSAVE_AREA(v, x) do { (void)(v); (x) =3D NULL; } while = (0) =20 +/* + * Variant of VCPU_MAP_XSAVE_AREA() for accessors which touch nothing + * beyond the legacy region and the XSAVE header (the first + * XSTATE_AREA_MIN_SIZE bytes). That prefix always fits within the + * area's first page, so this variant cannot fail. + */ +#define VCPU_MAP_XSAVE_AREA_PREFIX(v) VCPU_MAP_XSAVE_AREA(v) +#define VCPU_UNMAP_XSAVE_AREA_PREFIX(v, x) VCPU_UNMAP_XSAVE_AREA(v, x) + #endif /* __ASM_XSTATE_H */ diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 3c9be271a3..2dd1699aba 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -177,14 +177,19 @@ static void setup_xstate_comp(uint16_t *comp_offsets, * It is the callers responsibility to ensure that there is xsave state to * serialise, and that the provided buffer is exactly the right size. */ -void expand_xsave_states(const struct vcpu *v, void *dest, unsigned int si= ze) +int expand_xsave_states(const struct vcpu *v, void *dest, unsigned int siz= e) { const struct xsave_struct *xstate =3D VCPU_MAP_XSAVE_AREA(v); const void *src; uint16_t comp_offsets[sizeof(xfeature_mask)*8]; - u64 xstate_bv =3D xstate->xsave_hdr.xstate_bv; + u64 xstate_bv; u64 valid; =20 + if ( !xstate ) + return -ENOMEM; + + xstate_bv =3D xstate->xsave_hdr.xstate_bv; + /* Check there is state to serialise (i.e. at least an XSAVE_HDR) */ BUG_ON(!v->arch.xcr0_accum); /* Check there is the correct room to decompress into. */ @@ -233,6 +238,8 @@ void expand_xsave_states(const struct vcpu *v, void *de= st, unsigned int size) =20 out: VCPU_UNMAP_XSAVE_AREA(v, xstate); + + return 0; } =20 /* @@ -245,13 +252,16 @@ void expand_xsave_states(const struct vcpu *v, void *= dest, unsigned int size) * It is the callers responsibility to ensure that the source buffer conta= ins * xsave state, is uncompressed, and is exactly the right size. */ -void compress_xsave_states(struct vcpu *v, const void *src, unsigned int s= ize) +int compress_xsave_states(struct vcpu *v, const void *src, unsigned int si= ze) { struct xsave_struct *xstate =3D VCPU_MAP_XSAVE_AREA(v); void *dest; uint16_t comp_offsets[sizeof(xfeature_mask)*8]; u64 xstate_bv, valid; =20 + if ( !xstate ) + return -ENOMEM; + BUG_ON(!v->arch.xcr0_accum); BUG_ON(size !=3D xstate_uncompressed_size(v->arch.xcr0_accum)); ASSERT(!xsave_area_compressed(src)); @@ -302,6 +312,8 @@ void compress_xsave_states(struct vcpu *v, const void *= src, unsigned int size) =20 out: VCPU_UNMAP_XSAVE_AREA(v, xstate); + + return 0; } =20 void xsave(const struct vcpu *v, struct xsave_struct *ptr, uint64_t mask) --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748677; cv=none; d=zohomail.com; s=zohoarc; b=nBSrPiqk8xhip3IeDUW3zGb2DGvdyS9PNbzL437i9+pHYyBJREcHSzgsW1ojjfxvGm9kLWPt7rTT/gnwKOUyUD6p2VZkCa/+dU6NDaBlF7rE/pn21LxiK2ubSc+WuiBjQw0DApDUpd3uc/IriRL9SpownpHe/8tgQCOIN2RP4KI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748677; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DkoUGAtm2uD3yJT2Of+9zcbopY7hys4Hst7tr895h8s=; b=gmLazWxU6ZnLFNU71hwfjdLtXAxNlxJuhB81WSDXVFJR4pGurj3StxAsPw51iD5li1bI0I1PW569FV3mV169mDbia9FrgwwkzYhaC/JgK/nljiE6I0Wdmn0qpGpWM0vOhmC5w6SWQUHhaBBBfnSp9KGa4xpOp3izUneK64M7trs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748677551314.5290708496748; Tue, 3 Nov 2026 15:31:17 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470807.1679540 (Exim 4.92) (envelope-from ) id 1xONxt-0003xV-Tu; Tue, 03 Nov 2026 23:30:53 +0000 Received: by outflank-mailman (output) from mailman id 1470807.1679540; Tue, 03 Nov 2026 23:30:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxs-0003s8-Gi; Tue, 03 Nov 2026 23:30:52 +0000 Received: by outflank-mailman (input) for mailman id 1470807; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000do-FR for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxc-0000000Covg-1eHq; Tue, 03 Nov 2026 23:30:36 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoG-0000000EMFa-0CcU; Tue, 03 Nov 2026 23:20:56 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=DkoUGAtm2uD3yJT2Of+9zcbopY7hys4Hst7tr895h8s=; b=0zZZVuMbiTzSgULRHdyHIFL1nu JaOS1K6nBf0PCpyp7d5Tig3pUeuBzD94xkenpFTP2LIma4RNkDHkbDI5gVYdGJr497IajtC/7sMhi YiV6okeH+AQbg6PwplJQPcQIt6hy2zNZh725Cbgd501EmdscmeH9oUzcx0n/40cs1+NU=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 43/44] x86/domain: restore FPU state after switching to the new page tables Date: Tue, 3 Nov 2026 17:14:05 -0500 Message-ID: <20261103-asi-hvm-v6-43-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748679534158500 Content-Type: text/plain; charset="utf-8" __context_switch() restores the incoming vCPU's FPU state (vcpu_restore_fpu(n)) while still running on the outgoing vCPU's page tables, before write_ptbase(n). That dereferences n's XSAVE area through whatever address it happens to have in p's address space -- fine today, when the area is globally mapped, but not once guest XSAVE state moves to a per-vCPU mapping that need not exist in another vCPU's address space (upcoming Address Space Isolation work). Move the eager restore to just after write_ptbase(n), so it runs with n's page tables loaded. The XCR0/XSS setup stays where it is (it programs control registers / an MSR, independent of the page tables, and XSS must be set before the supervisor-state xrstor). Nothing between the save and the new restore point touches guest FPU state: the ctxt_switch->to() hooks, psr and GDT handling are all FPU-free, and Xen is built without SSE. No functional change with the current global XSAVE-area mapping. Assisted-by: Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. --- xen/arch/x86/domain.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 66d71ec41b..8222ad9e2a 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -2267,7 +2267,6 @@ static void __context_switch(void) if ( cpu_has_xsaves && is_hvm_vcpu(n) ) set_msr_xss(n->arch.msrs->xss.raw); } - vcpu_restore_fpu(n); nd->arch.ctxt_switch->to(n); } =20 @@ -2283,7 +2282,10 @@ static void __context_switch(void) write_ptbase(n); =20 if ( !is_idle_domain(nd) ) + { memcpy(stack_regs, vcpu_user_regs(n), CTXT_SWITCH_STACK_BYTES); + vcpu_restore_fpu(n); + } =20 #ifdef CONFIG_PV /* Prefetch the VMCB if we expect to use it later in the context switc= h */ --=20 2.56.0 From nobody Thu Nov 5 07:30:49 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793748664; cv=none; d=zohomail.com; s=zohoarc; b=egG6Axjf0/2u05R4S80VucQFcDhv26apeoVluBAQu/RbBTtIWlidcgUcq83Gt+HIh0f3eyWL4ie90E5xMo6w5OrFBzeiAvPPOohLqOs0eqb861FSRk43gMSFywFph0kB/SGLEIJnSgmiTWKiwGr1h85DOxaQCPbxRtgU2+BJOj4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793748664; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=c8tN695W+rtN1RhSPiCaFplmtV0cEomDPq3ZGoUcUC8=; b=hBZtVKFjBFZM2GWpWfLpdflVGJBSgdyduOUtsXzl5b4nVHa3ndU4r+69WpNwej5ubbn51gpACX9Boa0oQU65EIcqjX8j2p54QalIXwPYwoXLSBNUzhMda7KRKZT5dD3tlS4QzCIhuon6sdeRIXNtgB00qO4utXorkyBV6GDMNJE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793748664853362.34079169177164; Tue, 3 Nov 2026 15:31:04 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.1470797.1679478 (Exim 4.92) (envelope-from ) id 1xONxj-0001zZ-JB; Tue, 03 Nov 2026 23:30:43 +0000 Received: by outflank-mailman (output) from mailman id 1470797.1679478; Tue, 03 Nov 2026 23:30:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxi-0001xl-SL; Tue, 03 Nov 2026 23:30:42 +0000 Received: by outflank-mailman (input) for mailman id 1470797; Tue, 03 Nov 2026 23:30:36 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xONxc-0000ck-2I for xen-devel@lists.xenproject.org; Tue, 03 Nov 2026 23:30:36 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xONxb-0000000Cotx-3yqC; Tue, 03 Nov 2026 23:30:35 +0000 Received: from [2607:fb91:4c8c:bd3f:1da5:e92a:c5fb:865a] (helo=localhost.localdomain) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (Exim 4.98.2) (envelope-from ) id 1xONoG-0000000EMFa-2EMz; Tue, 03 Nov 2026 23:20:56 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=c8tN695W+rtN1RhSPiCaFplmtV0cEomDPq3ZGoUcUC8=; b=TBZcP5OrqDD/+cdHzIgjCNzMbb gaN18Y0vXeDJvJxK1h+FGqz/jPAdBEp2QzsyA3az8c639iJiCOWYQrtdTi54/UDHMM99XifdGEugw 0JeOXNP5wk/ia41AOOrTtudbbOta9A7Mo4+NJr8e8ZtbIiRZxhRs3ondG6V55urnRUXw=; From: George Dunlap To: xen-devel@lists.xenproject.org Cc: Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v6 44/44] x86/xstate: move guest XSAVE areas behind the vCPU state window Date: Tue, 3 Nov 2026 17:14:06 -0500 Message-ID: <20261103-asi-hvm-v6-44-59dbc5ce9615@xenproject.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> References: <20261103-asi-hvm-v6-0-59dbc5ce9615@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793748665600158500 Content-Type: text/plain; charset="utf-8" Guest vCPUs' XSAVE areas hold guest-owned register state, yet live in always-mapped memory: xenheap objects when they fit a page, vmap-backed allocations on hardware (AMX-class) where they do not. Under Address Space Isolation both placements leave the state reachable from every context. Move them to individual domain-heap pages, installed once at vCPU creation at a fixed reservation (indexes 3-5) in the per-vCPU state window, following the pattern set by the SVM VMCB and VMX MSR list relocations. With this, all eight slots of the state window are in use. The seam introduced by the VCPU_MAP_XSAVE_AREA() wrappers now gains its real implementation. The window address serves when v runs here on its own page tables: v =3D=3D current, and this_cpu(pgtable_vcpu) =3D=3D= v. Neither test suffices alone. Inside the EFI runtime bracket, and after a shadow-mode update_cr3() failure, `current`'s own page tables are not loaded. While a CPU holds v's state lazily, v's page tables are loaded, but an IPI syncing that state for another CPU can switch them away at any time. The page-table test also structurally rules out one vCPU reaching through a same-vcpu_id window address into another domain's state, which a dispatch on `current` alone would have made possible to get wrong silently. With the owning vCPU running on its tables, or for an idle vCPU, the map is a pointer fetch, so the hot paths map nothing. Otherwise (a paused, never-started or lazily held vCPU) the full-area map uses map_domain_pages() and can fail; those callers gained error handling in "x86/xstate: classify xsave-area seam accesses by failure mode". The prefix variant maps only the area's first page, which the legacy region and header always fit, and hence cannot fail. A mapping of a single page is dropped with unmap_vcpu_state_page(), which also flushes this CPU's translation of it; unmap_domain_pages() flushes a multipage mapping itself. __context_switch() saves the outgoing vCPU's FPU state after `current` has moved to the incoming vCPU, with interrupts off, while the outgoing vCPU's page tables -- and therefore its window -- are normally still loaded. It takes the window address through a path of its own, vcpu_save_fpu_ctxt_switch(): the transient-mapping fallback must not run from there, as it would sync the local execution state and re-enter __context_switch(). The shadow-mode paths which abandon a vCPU's page tables early (update_cr3() failure in new_guest_cr3() and _toggle_guest_pt()) leave the idle tables loaded with the outgoing vCPU's state still in registers. Those paths have already crashed the domain, so there is nothing worth saving, and the save is skipped there, as the register frame's already is: both sit under one test of the loaded page tables. Anything else failing it is a page-table tracking bug rather than a recoverable state, so it is fatal in every build. The window reservation is fixed at three pages, sized for the largest save area current hardware produces (AMX TILEDATA); xstate_init() refuses, with an explanatory panic, hardware whose enabled feature set exceeds it. It does so on every boot, with or without asi=3D: every guest vCPU's area moves. Idle vCPUs keep a globally-reachable area (xenheap, or vmap space where it exceeds a page, as with AMX) deliberately: it is scratch for accesses which execute with a guest's page tables loaded (read_bndcfgu(), xstate_set_init()) and holds no guest register state. On such paths the area a guest's MPX state is transiently saved into remains reachable from other contexts, as today. The backing pages are caller-owned (no _PAGE_AVAIL0): the per-domain teardown reclaims only the page tables, so xstate_free_save_area() unmaps the pages from the window and frees them itself. Without per-vCPU page tables (every PV domain) the window is in the per-domain area that all its domain's vCPUs share, which outlives a vCPU whose creation fails. A failed partial allocation unwinds in place, and xstate_free_save_area() tolerates the resulting half-built state, which arch_vcpu_create()'s failure path relies on. Memory cost: sub-page areas (all non-AMX hardware) round up from a 2.7KiB xenheap object to a full page per vCPU. The move does not depend on asi=3D, and applies to PV vCPUs as to HVM ones. A context switch now calls vcpu_xsave_area_ctxt_switch() for the save, and vcpu_map_xsave_area() and vcpu_unmap_xsave_area() around the restore, out of line, instead of reading v->arch.xsave_area. The XRSTOR following write_ptbase() reaches the area through a 4KiB, non-global PTE in the per-domain area. As switch_cr3_cr4() has just flushed every non-global translation, that takes a page walk on every switch to a guest vCPU, where a sub-page area, a xenheap object, was reached through a superpage of the full directmap; the walk goes through the L1 table which the register frame's restore has just walked. On an Intel NUC, a 64-bit PV guest's vCPU context switch takes 15-30ns longer, with XPTI off or on and against either the tree before the XSAVE work or the one just before this change; code placement alone moves that measurement by up to about 20ns between builds. That was measured while the PV guest's register frames were still in struct vcpu. On AMX-class hardware, where the area takes three pages, a full-area map for a vCPU other than the one running here goes through map_domain_pages(): XEN_DOMCTL_getvcpuextstate and XEN_DOMCTL_setvcpuextstate make one for PV save, restore and migration, as the HVM XSAVE save record does for HVM guests. The area is then mapped in the calling vCPU's multipage slots, which only the calling domain's page tables carry, and dropping the mapping flushes the calling CPU's translations alone. Each of those operations can now fail with -ENOMEM. Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5 Signed-off-by: George Dunlap --- Changes in v5: - New in this version. xstate_init() now panics, with or without asi=3D, if the enabled XSAVE features need more than the three pages the window reserves. No current hardware does. Would you rather this did more, for instance size the window's XSAVE range from the size found at boot, or fall back to xenheap areas on such hardware? --- xen/arch/x86/domain.c | 19 ++-- xen/arch/x86/i387.c | 23 +++- xen/arch/x86/include/asm/config.h | 3 + xen/arch/x86/include/asm/domain.h | 15 ++- xen/arch/x86/include/asm/i387.h | 1 + xen/arch/x86/include/asm/xstate.h | 60 +++++------ xen/arch/x86/xstate.c | 172 +++++++++++++++++++++++++++++- 7 files changed, 243 insertions(+), 50 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 8222ad9e2a..f750ff4e6a 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -594,7 +594,11 @@ int arch_vcpu_create(struct vcpu *v) BUILD_BUG_ON(VCPU_STATE_IDX_VMCB >=3D VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_N2VMCB >=3D VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_VMX_MSR >=3D VCPU_STATE_NR_PAGES); + BUILD_BUG_ON(VCPU_STATE_IDX_XSAVE + VCPU_STATE_XSAVE_PAGES > + VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_REGS >=3D VCPU_STATE_NR_PAGES); + BUILD_BUG_ON(VCPU_STATE_IDX_REGS < VCPU_STATE_IDX_XSAVE + + VCPU_STATE_XSAVE_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_EMUL >=3D VCPU_STATE_NR_PAGES); BUILD_BUG_ON(VCPU_STATE_IDX_EMUL =3D=3D VCPU_STATE_IDX_REGS); /* Page tables only; torn down with the rest of the per-domain are= a. */ @@ -2234,17 +2238,20 @@ static void __context_switch(void) { ASSERT(read_atomic(&p->dirty_cpu) =3D=3D cpu); /* - * p's frame is reached through its state window, mapped while p's - * own page tables are loaded, as they normally still are here. N= ot - * so after the shadow-mode paths which abandon a vCPU's tables ea= rly - * (update_cr3() failure), which have crashed the domain: the frame - * is left unsaved. A transient mapping would re-enter this funct= ion. + * p's frame and XSAVE area are reached through its state window, + * mapped while p's own page tables are loaded, as they normally + * still are here. Not so after the shadow-mode paths which aband= on + * a vCPU's tables early (update_cr3() failure), which have crashed + * the domain: both are left unsaved. A transient mapping would + * re-enter this function. */ if ( likely(this_cpu(pgtable_vcpu) =3D=3D p) ) + { memcpy(vcpu_user_regs(p), stack_regs, CTXT_SWITCH_STACK_BYTES); + vcpu_save_fpu_ctxt_switch(p); + } else BUG_ON(!domain_shutting_down(pd)); - vcpu_save_fpu(p); pd->arch.ctxt_switch->from(p); } =20 diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index d49e391e99..239404b37b 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -177,7 +177,9 @@ void vcpu_restore_fpu(struct vcpu *v) =20 ASSERT(!is_idle_vcpu(v)); =20 + /* v runs here on its own page tables: the map cannot fail. */ xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + BUG_ON(!xsave_area); =20 if ( cpu_has_xsave ) fpu_xrstor(v, xsave_area, XSTATE_ALL); @@ -195,25 +197,36 @@ void vcpu_restore_fpu(struct vcpu *v) * On each context switch, save the necessary FPU info of VCPU being switc= h=20 * out. It dispatches saving operation based on CPU's capability. */ -void vcpu_save_fpu(struct vcpu *v) +static void save_fpu(struct vcpu *v, struct xsave_struct *xsave_area) { - struct xsave_struct *xsave_area; - ASSERT(!is_idle_vcpu(v)); =20 /* This can happen, if a paravirtualised guest OS has set its CR0.TS. = */ clts(); =20 - xsave_area =3D VCPU_MAP_XSAVE_AREA(v); - if ( cpu_has_xsave ) fpu_xsave(v, xsave_area); else fpu_fxsave(v, &xsave_area->fpu_sse); +} =20 +/* Save the FPU state of v, the vCPU running here. */ +void vcpu_save_fpu(struct vcpu *v) +{ + /* As in vcpu_restore_fpu(): the fastpath, which cannot fail. */ + struct xsave_struct *xsave_area =3D VCPU_MAP_XSAVE_AREA(v); + + BUG_ON(!xsave_area); + save_fpu(v, xsave_area); VCPU_UNMAP_XSAVE_AREA(v, xsave_area); } =20 +/* As vcpu_save_fpu(), for __context_switch() switching v out. */ +void vcpu_save_fpu_ctxt_switch(struct vcpu *v) +{ + save_fpu(v, vcpu_xsave_area_ctxt_switch(v)); +} + /* Initialize FPU's context save area */ int vcpu_init_fpu(struct vcpu *v) { diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/c= onfig.h index 52d1ebc0e7..498e3c91fa 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -325,6 +325,9 @@ extern unsigned long xen_phys_start; #define VCPU_STATE_IDX_VMCB 0 #define VCPU_STATE_IDX_N2VMCB 1 #define VCPU_STATE_IDX_VMX_MSR 2 +#define VCPU_STATE_IDX_XSAVE 3 +/* Pages reserved for the XSAVE area; bounded at boot in xstate_init(). */ +#define VCPU_STATE_XSAVE_PAGES 3 #define VCPU_STATE_IDX_REGS 6 #define VCPU_STATE_IDX_EMUL 7 =20 diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 3748e417ce..69ab854177 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -685,12 +685,19 @@ struct arch_vcpu unsigned long cr3; /* (MA) value to install in HW CR3= */ =20 /* - * The save area for Processor Extended States and the bitmask of the - * XSAVE/XRSTOR features. They are used by: 1) when a vcpu (which has - * dirtied FPU/SSE) is scheduled out we XSAVE the states here; 2) in - * #NM handler, we XRSTOR the states we XSAVE-ed; + * The save area for Processor Extended States, XSAVEd on context + * switch out and XRSTORed on context switch in. For guest vCPUs + * the backing pages (xsave_pg[], xsave_pgcnt of them) come from the + * domain heap and are mapped once, for the life of the vCPU, at a + * fixed spot in the per-vCPU state window; xsave_area holds that + * window address, valid whenever the owning vCPU's page tables are + * loaded. Idle vCPUs' areas are ordinary Xen allocations (xenheap, + * or vmap space when larger than a page), reachable from every + * context. Any other access goes through VCPU_MAP_XSAVE_AREA(). */ struct xsave_struct *xsave_area; + struct page_info *xsave_pg[VCPU_STATE_XSAVE_PAGES]; + unsigned int xsave_pgcnt; uint64_t xcr0; /* Accumulated eXtended features mask for using XSAVE/XRESTORE by Xen * itself, as we can never know whether guest OS depends on content diff --git a/xen/arch/x86/include/asm/i387.h b/xen/arch/x86/include/asm/i38= 7.h index 0717005d31..8d28de3ba8 100644 --- a/xen/arch/x86/include/asm/i387.h +++ b/xen/arch/x86/include/asm/i387.h @@ -29,6 +29,7 @@ struct ix87_env { =20 void vcpu_restore_fpu(struct vcpu *v); void vcpu_save_fpu(struct vcpu *v); +void vcpu_save_fpu_ctxt_switch(struct vcpu *v); int vcpu_init_fpu(struct vcpu *v); void vcpu_destroy_fpu(struct vcpu *v); =20 diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/x= state.h index c23308cbe2..a906df11ad 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -130,46 +130,43 @@ xsave_area_compressed(const struct xsave_struct *xsav= e_area) } =20 /* - * Fetch a pointer to a vCPU's XSAVE area + * Fetch a pointer to a vCPU's XSAVE area. * - * TL;DR: If v =3D=3D current, the mapping is guaranteed to already exist. + * Guest vCPUs' areas live on the domain heap, mapped once for the life + * of the vCPU at a fixed spot in the owning vCPU's state window. For a + * vCPU running here on its own page tables, and for an idle vCPU, the + * "map" is a pointer fetch. * - * Despite the name, this macro might not actually map anything. The only = case - * in which a mutation of page tables is strictly required is when ASI=3D= =3Don && - * v!=3Dcurrent. For everything else the mapping already exists and needs = not - * be created nor destroyed. + * Otherwise -- a paused, never-started or lazily held vCPU -- the area + * is genuinely mapped, and mapping the full, possibly multi-page area + * can fail: VCPU_MAP_XSAVE_AREA() may return NULL, and callers must + * handle that. The _PREFIX variant below cannot fail. * - * +-----------------+--------------+ - * | v =3D=3D current | v !=3D current | - * +--------------+-----------------+--------------+ - * | ASI enabled | per-vCPU fixmap | actual map | - * +--------------+-----------------+--------------+ - * | ASI disabled | directmap | - * +--------------+--------------------------------+ - * - * There MUST NOT be outstanding maps of XSAVE areas of the non-current vC= PU - * at the point of context switch. Otherwise, the unmap operation will - * misbehave. - * - * TODO: Expand the macro to the ASI cases after infra to do so is in plac= e. + * There MUST NOT be outstanding maps of another vCPU's XSAVE area at + * the point of context switch: such a mapping is only valid on the + * pCPU that created it, and only until the next reschedule. * * @param v Owner of the XSAVE area */ -#define VCPU_MAP_XSAVE_AREA(v) ((v)->arch.xsave_area) +struct xsave_struct *vcpu_map_xsave_area(const struct vcpu *v); +void vcpu_unmap_xsave_area(const struct vcpu *v, const void *area); +struct xsave_struct *vcpu_map_xsave_area_prefix(const struct vcpu *v); +void vcpu_unmap_xsave_area_prefix(const struct vcpu *v, const void *area); +struct xsave_struct *vcpu_xsave_area_ctxt_switch(const struct vcpu *v); + +#define VCPU_MAP_XSAVE_AREA(v) vcpu_map_xsave_area(v) =20 /* - * Drops the mapping of a vCPU's XSAVE area and nullifies its pointer on e= xit - * - * See VCPU_MAP_XSAVE_AREA() for additional information on the persistence= of - * these mappings. This macro only tears down the mappings in the ASI=3Don= && - * v!=3Dcurrent case. - * - * TODO: Expand the macro to the ASI cases after infra to do so is in plac= e. + * Drops the mapping of a vCPU's XSAVE area and nullifies the local + * pointer on exit. * * @param v Owner of the XSAVE area * @param x XSAVE blob of v */ -#define VCPU_UNMAP_XSAVE_AREA(v, x) do { (void)(v); (x) =3D NULL; } while = (0) +#define VCPU_UNMAP_XSAVE_AREA(v, x) do { \ + vcpu_unmap_xsave_area(v, x); \ + (x) =3D NULL; \ +} while ( false ) =20 /* * Variant of VCPU_MAP_XSAVE_AREA() for accessors which touch nothing @@ -177,7 +174,10 @@ xsave_area_compressed(const struct xsave_struct *xsave= _area) * XSTATE_AREA_MIN_SIZE bytes). That prefix always fits within the * area's first page, so this variant cannot fail. */ -#define VCPU_MAP_XSAVE_AREA_PREFIX(v) VCPU_MAP_XSAVE_AREA(v) -#define VCPU_UNMAP_XSAVE_AREA_PREFIX(v, x) VCPU_UNMAP_XSAVE_AREA(v, x) +#define VCPU_MAP_XSAVE_AREA_PREFIX(v) vcpu_map_xsave_area_prefix(v) +#define VCPU_UNMAP_XSAVE_AREA_PREFIX(v, x) do { \ + vcpu_unmap_xsave_area_prefix(v, x); \ + (x) =3D NULL; \ +} while ( false ) =20 #endif /* __ASM_XSTATE_H */ diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 2dd1699aba..7543edc160 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -5,6 +5,7 @@ * */ =20 +#include #include #include #include @@ -14,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -555,9 +557,61 @@ int xstate_alloc_save_area(struct vcpu *v) =20 /* XSAVE/XRSTOR requires the save area be 64-byte-boundary aligned. */ BUILD_BUG_ON(__alignof(*save_area) < 64); - save_area =3D _xvzalloc(size, __alignof(*save_area)); - if ( save_area =3D=3D NULL ) - return -ENOMEM; + + if ( !is_idle_vcpu(v) ) + { + /* + * The pages are the vCPU's own (no _PAGE_AVAIL0): the area teardo= wn + * leaves them alone, and xstate_free_save_area() frees them. + */ + unsigned int nr =3D PFN_UP(size), i; + mfn_t mfns[VCPU_STATE_XSAVE_PAGES]; + + /* xstate_init() bounds the host size against the reservation. */ + BUG_ON(nr > VCPU_STATE_XSAVE_PAGES); + + for ( i =3D 0; i < nr; i++ ) + { + struct page_info *pg =3D alloc_domheap_page(v->domain, + MEMF_no_owner); + + if ( !pg ) + { + while ( i-- ) + { + free_domheap_page(v->arch.xsave_pg[i]); + v->arch.xsave_pg[i] =3D NULL; + } + return -ENOMEM; + } + + v->arch.xsave_pg[i] =3D pg; + mfns[i] =3D page_to_mfn(pg); + clear_domain_page(mfns[i]); + } + v->arch.xsave_pgcnt =3D nr; + + populate_perdomain_mapping(v, VCPU_STATE_ADDR(v, VCPU_STATE_IDX_XS= AVE), + mfns, nr, __PAGE_HYPERVISOR_RW); + + /* + * The window VA is not reachable from the creating context (v is + * still being built); seed the initial register values through a + * transient mapping of the first page instead. + */ + save_area =3D map_domain_page(mfns[0]); + } + else + { + /* + * Idle vCPUs' areas must be reachable from every context: they are + * scratch for accesses made with a guest's page tables loaded + * (read_bndcfgu(), xstate_set_init()). + */ + save_area =3D _xvzalloc(size, __alignof(*save_area)); + if ( save_area =3D=3D NULL ) + return -ENOMEM; + } =20 /* * We're creating a vCPU, so conceptually we should be choosing the @@ -578,7 +632,15 @@ int xstate_alloc_save_area(struct vcpu *v) save_area->fpu_sse.ftw =3D FXSAVE_FTW_RESET; save_area->fpu_sse.mxcsr =3D MXCSR_DEFAULT; =20 - v->arch.xsave_area =3D save_area; + if ( !is_idle_vcpu(v) ) + { + unmap_domain_page(save_area); + v->arch.xsave_area =3D + (void *)VCPU_STATE_ADDR(v, VCPU_STATE_IDX_XSAVE); + } + else + v->arch.xsave_area =3D save_area; + v->arch.xcr0 =3D 0; v->arch.xcr0_accum =3D 0; =20 @@ -587,7 +649,103 @@ int xstate_alloc_save_area(struct vcpu *v) =20 void xstate_free_save_area(struct vcpu *v) { - XVFREE(v->arch.xsave_area); + if ( !is_idle_vcpu(v) ) + { + unsigned int i, nr =3D v->arch.xsave_pgcnt; + + if ( nr ) + { + unmap_vcpu_state(v, VCPU_STATE_IDX_XSAVE, nr); + for ( i =3D 0; i < nr; i++ ) + { + free_domheap_page(v->arch.xsave_pg[i]); + v->arch.xsave_pg[i] =3D NULL; + } + v->arch.xsave_pgcnt =3D 0; + } + + v->arch.xsave_area =3D NULL; + } + else + XVFREE(v->arch.xsave_area); +} + +struct xsave_struct *vcpu_map_xsave_area(const struct vcpu *v) +{ + mfn_t mfns[VCPU_STATE_XSAVE_PAGES]; + unsigned int i; + + if ( is_idle_vcpu(v) ) + return v->arch.xsave_area; + + /* + * The window VA is valid while v's own page tables are loaded, but th= at + * alone is not enough: while this CPU holds v's state lazily, an IPI + * syncing that state for another CPU can switch v's page tables away + * at any time. Hence the window only for v running here on its own + * page tables, which current's are not inside the EFI runtime bracket + * or after a shadow-mode update_cr3() failure. + */ + if ( v =3D=3D current && this_cpu(pgtable_vcpu) =3D=3D v ) + return v->arch.xsave_area; + + for ( i =3D 0; i < v->arch.xsave_pgcnt; i++ ) + mfns[i] =3D page_to_mfn(v->arch.xsave_pg[i]); + + return map_domain_pages(mfns, v->arch.xsave_pgcnt); +} + +void vcpu_unmap_xsave_area(const struct vcpu *v, const void *area) +{ + if ( !area || is_idle_vcpu(v) || + area =3D=3D (const void *)v->arch.xsave_area ) + return; + + /* + * A single page comes from the single-page mapcache, whose + * domain-wide form defers its flushes; unmap_domain_pages() flushes a + * multipage mapping itself. + */ + if ( v->arch.xsave_pgcnt =3D=3D 1 ) + unmap_vcpu_state_page(area); + else + unmap_domain_pages(area, v->arch.xsave_pgcnt); +} + +struct xsave_struct *vcpu_map_xsave_area_prefix(const struct vcpu *v) +{ + /* The legacy region and header always fit the area's first page. */ + BUILD_BUG_ON(sizeof(struct xsave_struct) > PAGE_SIZE); + + /* The window VA under vcpu_map_xsave_area()'s conditions. */ + if ( is_idle_vcpu(v) || + (v =3D=3D current && this_cpu(pgtable_vcpu) =3D=3D v) ) + return v->arch.xsave_area; + + return map_domain_page(page_to_mfn(v->arch.xsave_pg[0])); +} + +void vcpu_unmap_xsave_area_prefix(const struct vcpu *v, const void *area) +{ + if ( !area || is_idle_vcpu(v) || + area =3D=3D (const void *)v->arch.xsave_area ) + return; + + unmap_vcpu_state_page(area); +} + +/* + * The XSAVE area of @v for __context_switch() saving it: `current` has + * moved on to the incoming vCPU, but @v's page tables are still loaded, a= nd + * interrupts are off, so the window VA serves. A transient mapping is not + * an option from there: it would sync the lazy state and re-enter + * __context_switch(). + */ +struct xsave_struct *vcpu_xsave_area_ctxt_switch(const struct vcpu *v) +{ + BUG_ON(is_idle_vcpu(v) || this_cpu(pgtable_vcpu) !=3D v); + + return v->arch.xsave_area; } =20 static bool valid_xcr0(uint64_t xcr0) @@ -934,6 +1092,10 @@ void xstate_init(struct cpuinfo_x86 *c) xsave_cntxt_size =3D cpuid_count_ebx(0xd, 0); printk("xstate: size: %#x and states: %#"PRIx64"\n", xsave_cntxt_size, xfeature_mask); + + if ( PFN_UP(xsave_cntxt_size) > VCPU_STATE_XSAVE_PAGES ) + panic("XSAVE area needs %lu pages, vCPU state window reserves = %u\n", + PFN_UP(xsave_cntxt_size), VCPU_STATE_XSAVE_PAGES); } else { --=20 2.56.0