From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342290489109.2350270757488; Wed, 2 Sep 2026 02:44:50 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405453.1638952 (Exim 4.92) (envelope-from ) id 1x1hVp-0002Vm-8G; Wed, 02 Sep 2026 09:44:09 +0000 Received: by outflank-mailman (output) from mailman id 1405453.1638952; Wed, 02 Sep 2026 09:44:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVp-0002VY-1d; Wed, 02 Sep 2026 09:44:09 +0000 Received: by outflank-mailman (input) for mailman id 1405453; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net ([195.190.135.20]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVm-0002G4-Vz for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hVl-00B1Tu-V3 for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:44:05 +0200 Received: from [10.42.69.4] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97efe1-e002-0a2a0a5209dd-0a2a4504ec92-22 for ; Wed, 02 Sep 2026 11:44:05 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-ebf023.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe5-b57f-0a2a45040019-d99ba50cef06-1 for ; Wed, 02 Sep 2026 11:44:05 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 698C936928D4; Wed, 2 Sep 2026 10:44:05 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: George Dunlap , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 01/14] x86/domain_page: introduce IRQs-off variants of {,un}map_domain_page() Date: Wed, 2 Sep 2026 10:43:45 +0100 Message-ID: <20260901-asi-part2-1-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ebf023/1788342245-C12DCB50-5B4BB83C/0/0 X-purgate-type: clean X-purgate-size: 10808 X-ZM-MESSAGEID: 1788342294018154100 From: George Dunlap Currently, map_domain_page() cannot be called in the context switch path. However, Xen already needs to update the slot of an incoming PV vcpu's GDT during context switch; and when we soon switch to per-vCPU root pagetables, we'll have to modify two more places. Xen currently solves the problem by special-casing the GDT/LDT L1 tables to be allocated from the xenheap, and stashing a pointer to its address in the xenheap in the domain struct. Rather than add more Xen pagetable pages to the xenheap, introduce a version of map_domain_page which can be called from the context switch path. The reason map_domain_page() cannot be called from the context switch path is x86's lazy context-switch state. Mapcache mappings are created in the page-tables that are loaded on the pCPU. When Xen is in a lazy context-switch state, current is the idle vCPU while the previously-running vCPU's page-tables remain loaded. If in this state, another pcpu wants access to the lazily-swapped-out vcpu's state, it will send a FLUSH_VCPU_STATE IPI to the processor, which will call sync_local_execstate(). sync_local_execstate() is implemented internally by calling a full __context_switch(). In addition to copying the processor state into the vcpu structure, this also switches the loaded pagetables to the idle vcpu's, which would in turn cause mappings created before the IPI to disappear mid-use. Therefore, mappings cannot be held in the mapcache when a FLUSH_VCPU_STATE IPI may execute. To this end, map_domain_page() calls sync_local_execstate() itself proactively when it detects a lazy context-switch state. This guarantees that the pagetables will remain consistent at least until the next context switch. But of course, that synchronization must not be triggered from the context switch path itself: sync_local_execstate() ends up in __context_switch(), so a call made while a context switch is in progress would recurse, and the assertions along that path (current being the idle vCPU) don't hold there either. A full synchronization is sufficient to prevent a FLUSH_VCPU_STATE IPI from switching the pagetables; however, it is not necessary. It suffices to maintain interrupts disabled from before the page is mapped until after it is unmapped. This condition is satisfied for the mappings used on the context switch path. Introduce {,un}map_domain_page_irqoff() variants for callers which guarantee that interrupts remain disabled from the map until the matching unmap. Under that guarantee the synchronization is unnecessary rather than merely inconvenient: no IPI can be delivered while the mapping is in use, so the lazy state cannot change under the caller's feet, and this_cpu(pgtable_vcpu) accurately identifies the mapcache to use (see 622c9a5ba95d "x86/mm: accurately track which vCPU page-tables are loaded"). The variants assert that interrupts are disabled on entry; the rest of the contract remains the caller's responsibility. This will be used by the next patch, which introduces a function which will be used to modify the incoming vCPU's per-domain mappings from within __context_switch(); it will also be used in future ASI patches (tearing down and establishing per-CPU stack mappings during context switch). No functional change for existing callers. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Changes in v2: - New in this version. Replaces "x86/mm: allocate the per-domain page-tables from the xenheap". NB an alternate approach would be to disable the lazy context switch entirely. This simplifies the Xen code in general, and makes a patch like this completely unnecessary, as then map_domain_page would itself be safe to call in a context switch. Tests show, however, that simply removing the lazy context switch measurably hurts wake-heavy workloads: on a wake-paced ping flood, throughput drops 22% (round-trip latency 13=E2=86=9217 =C2=B5s) on my NUC. Another approach is to take up the GDT/LDT L1 technique instead. v1 of the series made all perdomain pagetables allocated out of the xenheap; but this was objected to due to the additional xenheap allocations. An alternate version would allocate from the domheap, and then make permanent mappings in the vmap instead. Another potential performance improvement would be stashing the exact pages we want to map, rather than needing to walk from the L3 each time. We leave both of these for future work. --- xen/arch/x86/domain_page.c | 53 ++++++++++++++++++++++++++++------- xen/include/xen/domain_page.h | 14 +++++++++ 2 files changed, 57 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 72c00194f3..1fc1580e62 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -18,7 +18,7 @@ #include #include =20 -static inline struct vcpu *mapcache_current_vcpu(void) +static inline struct vcpu *mapcache_current_vcpu(bool irqs_off) { struct vcpu *v =3D this_cpu(pgtable_vcpu); struct vcpu *curr =3D current; @@ -36,8 +36,15 @@ static inline struct vcpu *mapcache_current_vcpu(void) * to the idle vCPU now, otherwise an incoming FLUSH_VCPU_STATE IPI wo= uld * change the page tables under our feet an invalidate any in-use mapc= ache * entries. + * + * Callers of the irqs_off variants instead guarantee that interrupts = stay + * disabled until the matching unmap: no IPI can be delivered while the + * mapping is in use, so the lazy state cannot change under our feet a= nd + * pgtable_vcpu identifies the right mapcache directly. This also mak= es + * those variants usable from the context switch path itself, where + * calling sync_local_execstate() would recurse into __context_switch(= ). */ - if ( unlikely(this_cpu(curr_vcpu) !=3D curr) ) + if ( !irqs_off && unlikely(this_cpu(curr_vcpu) !=3D curr) ) { ASSERT(curr =3D=3D idle_vcpu[smp_processor_id()]); sync_local_execstate(); @@ -46,10 +53,12 @@ static inline struct vcpu *mapcache_current_vcpu(void) } =20 /* - * At this point we can guarantee Xen is not in lazy context switch: e= ither - * the code above will have synced the state, or an incoming - * FLUSH_VCPU_STATE IPI has done so behind our back. Use ACCESS_ONCE = to - * ensure the compiler never returns the locally cached pgtable_vcpu v= alue. + * At this point either Xen is not in a lazy context switch (the code + * above will have synced the state, or an incoming FLUSH_VCPU_STATE I= PI + * has done so behind our back), or the caller holds interrupts disabl= ed + * and the state cannot change until it re-enables them. Use ACCESS_O= NCE + * to ensure the compiler never returns the locally cached pgtable_vcpu + * value. */ return ACCESS_ONCE(this_cpu(pgtable_vcpu)); } @@ -59,7 +68,7 @@ static inline struct vcpu *mapcache_current_vcpu(void) #define MAPCACHE_L1ENT(idx) \ __linear_l1_table[l1_linear_offset(MAPCACHE_VIRT_START + pfn_to_paddr(= idx))] =20 -void *map_domain_page(mfn_t mfn) +static void *do_map_domain_page(mfn_t mfn, bool irqs_off) { unsigned long flags; unsigned int idx, i; @@ -73,7 +82,7 @@ void *map_domain_page(mfn_t mfn) return mfn_to_virt(mfn_x(mfn)); #endif =20 - v =3D mapcache_current_vcpu(); + v =3D mapcache_current_vcpu(irqs_off); if ( !v || !is_pv_vcpu(v) ) return mfn_to_virt(mfn_x(mfn)); =20 @@ -165,7 +174,19 @@ void *map_domain_page(mfn_t mfn) return (void *)MAPCACHE_VIRT_START + pfn_to_paddr(idx); } =20 -void unmap_domain_page(const void *ptr) +void *map_domain_page(mfn_t mfn) +{ + return do_map_domain_page(mfn, false); +} + +void *map_domain_page_irqoff(mfn_t mfn) +{ + ASSERT(!local_irq_is_enabled()); + + return do_map_domain_page(mfn, true); +} + +static void do_unmap_domain_page(const void *ptr, bool irqs_off) { unsigned int idx; struct vcpu *v; @@ -178,7 +199,7 @@ void unmap_domain_page(const void *ptr) =20 ASSERT(va >=3D MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); =20 - v =3D mapcache_current_vcpu(); + v =3D mapcache_current_vcpu(irqs_off); ASSERT(v && is_pv_vcpu(v)); =20 dcache =3D &v->domain->arch.pv.mapcache; @@ -223,6 +244,18 @@ void unmap_domain_page(const void *ptr) local_irq_restore(flags); } =20 +void unmap_domain_page(const void *ptr) +{ + do_unmap_domain_page(ptr, false); +} + +void unmap_domain_page_irqoff(const void *ptr) +{ + ASSERT(!local_irq_is_enabled()); + + do_unmap_domain_page(ptr, true); +} + int mapcache_domain_init(struct domain *d) { struct mapcache_domain *dcache =3D &d->arch.pv.mapcache; diff --git a/xen/include/xen/domain_page.h b/xen/include/xen/domain_page.h index c89b149e54..b72dffb4c7 100644 --- a/xen/include/xen/domain_page.h +++ b/xen/include/xen/domain_page.h @@ -31,6 +31,16 @@ void *map_domain_page(mfn_t mfn); */ void unmap_domain_page(const void *ptr); =20 +/* + * Variants of the above for callers which guarantee that interrupts are + * kept disabled from map until the matching unmap. Under that guarantee + * no state synchronization is required to keep the mapping valid, so these + * are safe to use in contexts where such a synchronization must not be + * triggered, in particular from the context switch path itself. + */ +void *map_domain_page_irqoff(mfn_t mfn); +void unmap_domain_page_irqoff(const void *ptr); + /* * Given a VA from map_domain_page(), return its underlying MFN. */ @@ -45,6 +55,7 @@ void *map_domain_page_global(mfn_t mfn); void unmap_domain_page_global(const void *ptr); =20 #define __map_domain_page(pg) map_domain_page(page_to_mfn(pg)) +#define __map_domain_page_irqoff(pg) map_domain_page_irqoff(page_to_mfn(pg= )) =20 static inline void *__map_domain_page_global(const struct page_info *pg) { @@ -54,8 +65,11 @@ static inline void *__map_domain_page_global(const struc= t page_info *pg) #else /* !CONFIG_ARCH_MAP_DOMAIN_PAGE */ =20 #define map_domain_page(mfn) __mfn_to_virt(mfn_x(mfn)) +#define map_domain_page_irqoff(mfn) map_domain_page(mfn) #define __map_domain_page(pg) page_to_virt(pg) +#define __map_domain_page_irqoff(pg) __map_domain_page(pg) #define unmap_domain_page(ptr) ((void)(ptr)) +#define unmap_domain_page_irqoff(ptr) unmap_domain_page(ptr) #define domain_page_map_to_mfn(ptr) _mfn(__virt_to_mfn((unsigned l= ong)(ptr))) =20 static inline void *map_domain_page_global(mfn_t mfn) --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342275755229.15041007398202; Wed, 2 Sep 2026 02:44:35 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405454.1638956 (Exim 4.92) (envelope-from ) id 1x1hVp-0002Xi-Hc; Wed, 02 Sep 2026 09:44:09 +0000 Received: by outflank-mailman (output) from mailman id 1405454.1638956; Wed, 02 Sep 2026 09:44:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVp-0002Wc-7P; Wed, 02 Sep 2026 09:44:09 +0000 Received: by outflank-mailman (input) for mailman id 1405454; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVn-0002G9-Dw for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hVm-00G2d4-AD for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [10.42.69.8] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97efdb-bab6-0a2a0a5309dd-0a2a4508bd48-32 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-c1860d.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe5-f659-0a2a45080019-d99ba50cf561-1 for ; Wed, 02 Sep 2026 11:44:05 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 6DB8836928D7; Wed, 2 Sep 2026 10:44:05 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 02/14] x86/mm: introduce populate_perdomain_mapping() Date: Wed, 2 Sep 2026 10:43:46 +0100 Message-ID: <20260901-asi-part2-2-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-c1860d/1788342245-CD74B87B-3EFE3426/0/0 X-purgate-type: clean X-purgate-size: 12066 X-ZM-MESSAGEID: 1788342278265158500 From: Roger Pau Monn=C3=A9 The per-domain area already has central machinery for building its page-tables and for managing the backing pages it owns itself: create_perdomain_mapping() / destroy_perdomain_mapping(), used by the mapcache bitmaps, the compat argument-translation area, and the GDT/LDT slots alike. What the interface lacks is a way to install a caller's own pages at a chosen address. The PV GDT and LDT code open-codes its own modifications to the per-domain area, capturing aliases of its L1 tables at creation time (create_perdomain_mapping()'s pl1tab argument) and stashing them in d->arch.pv.gdt_ldt_l1tab. Introduce populate_perdomain_mapping(v, va, mfn, nr, flags) to close this gap, giving the perdomain area's rules a single place to live. populate_perdomain_mapping writes the given MFNs, with the given page-table flags, into v's view of the per-domain area: through the recursive linear mappings when v's page-tables are loaded on the current pCPU, or by walking the per-domain page-table structures otherwise. Callers don't need to know where the page-tables live, how the area is structured, or whether it is per-domain or per-vcpu. The fast path is keyed off this_cpu(pgtable_vcpu) rather than current: following 622c9a5ba95d ("x86/mm: accurately track which vCPU page-tables are loaded") that's the accurate way to tell whether the linear mappings reach v's per-domain area, and it copes with the transient states where current doesn't match the loaded page-tables (e.g. the _toggle_guest_pt() error window, or mid context switch). It also removes any need to call sync_local_execstate(): when the vCPU's page-tables aren't loaded, the walk instead maps the per-domain page-table pages with the map_domain_page_irqoff() variants, holding interrupts off for its duration, and so is usable from any context -- including the context switch, before the incoming vcpu's page-tables are loaded. We require the range to already have been populated down to the L1 tables by create_perdomain_mapping(). TLB flushing is left to the caller. A present entry not owned by the area (!_PAGE_AVAIL0) is replaced. A present entry owned by the area (_PAGE_AVAIL0, installed by create_perdomain_mapping() itself) is freed and replaced: such a page is referenced only by the mapping, so displacing it without freeing it would leak it. Nothing in this series replaces area-owned backing, so the free is marked ASSERT_UNREACHABLE(); note that freeing requires a context where the allocator may be entered -- IRQs enabled, not in interrupt context (see ASSERT_ALLOC_CONTEXT()) -- so any future caller replacing area-owned backing must not do so from the context switch path, nor anywhere the slow-path walk (which holds IRQs off) can be taken. Missing page-table structure is a hypervisor bug and BUG_ON(): there is no safe continuation, least of all from the context switch, where the next descriptor fetch through an unmapped GDT slot would be fatal. Subsequent patches convert the users of the stashed L1 tables to this interface, starting with the Xen slots of the full GDT; the stash -- which could in any case not represent per-vcpu mappings without being replicated for every vcpu and slot -- is then removed, leaving create_perdomain_mapping() to manage only the page-table structure and the pages the area owns itself. Later parts of the series use the new interface for their own mappings rather than adding further mechanisms. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Drop the xenheap allocation of the per-domain page-tables (v1's patch 1); the walk instead maps the page-table pages with the new map_domain_page_irqoff() variants, holding interrupts off for the duration. - Re-introduce the linear-map fast path for when the target vCPU's page-tables are loaded on the current pCPU, now keyed off this_cpu(pgtable_vcpu). Changes since the previously posted version: - Split the introduction of populate_perdomain_mapping() from its first user (previously one patch: "x86/pv: introduce function to populate perdomain area and use it to map Xen GDT"). - Drop the linear-map fast path and the sync_local_execstate() call: with the per-domain page-tables in the xenheap (previous patch) the walk needs no mapping, so a single path serves all callers and contexts. - Keep the ASSERT_UNREACHABLE() + free_domheap_page() handling of a replaced area-owned entry, and document the allocation-context requirement it places on callers replacing such entries. BUG_ON() missing page-table structure, instead of domain_crash(). - Take the page-table flags as a parameter (the Xen GDT and guest GDT slots want RW mappings; the zero page backing torn-down GDT slots is mapped read-only, as today). - Document the contract in a header comment. - Make the mfn parameter const and nr unsigned int, matching {create,destroy}_perdomain_mapping(). - Drop the unused cr3_mfn() helper. Considered, but not done to limit churn against the previously posted version: splitting the interface into a "populate" variant (any present entry is a bug) and an "update" variant (replacement expected), so that call sites declare their intent and unexpected collisions become detectable. Of the eventual call sites in the wider series, roughly half are of each kind. Could be done as a follow-up if there is interest. --- xen/arch/x86/include/asm/mm.h | 3 + xen/arch/x86/mm.c | 124 ++++++++++++++++++++++++++++++++++ 2 files changed, 127 insertions(+) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 2254a7e3fe..1888807394 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -606,6 +606,9 @@ int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_= HANDLE_PARAM(void) arg); int create_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr, l1_pgentry_t **pl1tab, struct page_info **ppg); +void populate_perdomain_mapping(const struct vcpu *v, unsigned long va, + const mfn_t *mfn, unsigned int nr, + unsigned int flags); void destroy_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr); void free_perdomain_mappings(struct domain *d); diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index b158742408..552559ecf1 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6334,6 +6334,130 @@ int create_perdomain_mapping(struct domain *d, unsi= gned long va, return rc; } =20 +/* + * Map @nr pages, @mfn[0..nr-1], at consecutive pages from @va in v's view= of + * the per-domain area, with page-table @flags. The range must lie within= a + * single per-domain slot, and must already have been plumbed down to the = L1 + * tables by create_perdomain_mapping(): missing structure is a bug. A + * present entry not owned by the area (no _PAGE_AVAIL0) is silently + * replaced, as that is how callers update their mappings; a present + * area-owned entry is freed and replaced, which constrains the calling + * context (see the comment in the body). No TLB flushing is done: the + * caller decides whether the old translations can still be cached + * anywhere. + * + * When v's page-tables are loaded on this pCPU the L1 entries are reached + * through the recursive linear mappings; otherwise the walk maps the + * per-domain page-table pages transiently with IRQs off, so it needs + * nothing from the current address space and is usable from any context -- + * including the context switch, before the incoming vcpu's page-tables are + * loaded. + */ +void populate_perdomain_mapping(const struct vcpu *v, unsigned long va, + const mfn_t *mfn, unsigned int nr, + unsigned int flags) +{ + l1_pgentry_t *l1tab =3D NULL, *pl1e; + const l3_pgentry_t *l3tab; + const l2_pgentry_t *l2tab; + struct domain *d =3D v->domain; + unsigned long irq_flags; + + ASSERT(va >=3D PERDOMAIN_VIRT_START && + va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); + ASSERT(!nr || !l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1))); + /* Area-owned pages are installed by create_perdomain_mapping() only. = */ + ASSERT(!(flags & _PAGE_AVAIL0)); + + if ( likely(this_cpu(pgtable_vcpu) =3D=3D v) ) + { + unsigned int i; + + /* + * Fast path: v's page-tables are loaded on this pCPU, so the L1 + * entries can be reached using the recursive linear mappings. + */ + pl1e =3D &__linear_l1_table[l1_linear_offset(va)]; + + for ( i =3D 0; i < nr; i++, pl1e++ ) + { + /* + * An area-owned entry (installed by create_perdomain_mapping(= ), + * marked _PAGE_AVAIL0) holds the only reference to its page, = so + * displacing it means freeing it. Nothing in this series + * replaces area-owned backing, hence the ASSERT_UNREACHABLE(); + * any future caller doing so must run where freeing is + * permitted -- IRQs enabled, not in interrupt context (see + * ASSERT_ALLOC_CONTEXT()) -- which the context switch path is + * not. + */ + if ( unlikely(perdomain_l1e_needs_freeing(*pl1e)) ) + { + ASSERT_UNREACHABLE(); + free_domheap_page(l1e_get_page(*pl1e)); + } + l1e_write(pl1e, l1e_from_mfn(mfn[i], flags)); + } + + return; + } + + BUG_ON(!d->arch.perdomain_l3_pg); + + /* + * Slow path: walk v's per-domain page-table pages. All mappings are + * local to this function, so disabling interrupts for the duration of + * the walk satisfies the map_domain_page_irqoff() contract. This in + * turn makes this function usable from the context switch path, where + * a plain map_domain_page() could recurse into __context_switch() via + * sync_local_execstate(). + */ + local_irq_save(irq_flags); + + l3tab =3D __map_domain_page_irqoff(d->arch.perdomain_l3_pg); + + /* + * Missing page-table structure is a hypervisor bug: there is no safe + * continuation, least of all from the context switch, where the next + * descriptor fetch through an unmapped GDT slot would be fatal. + */ + BUG_ON(!(l3e_get_flags(l3tab[l3_table_offset(va)]) & _PAGE_PRESENT)); + + l2tab =3D map_domain_page_irqoff(l3e_get_mfn(l3tab[l3_table_offset(va)= ])); + + for ( ; nr--; va +=3D PAGE_SIZE, mfn++ ) + { + if ( !l1tab || !l1_table_offset(va) ) + { + const l2_pgentry_t *pl2e =3D l2tab + l2_table_offset(va); + + BUG_ON(!(l2e_get_flags(*pl2e) & _PAGE_PRESENT)); + + unmap_domain_page_irqoff(l1tab); + l1tab =3D map_domain_page_irqoff(l2e_get_mfn(*pl2e)); + } + + pl1e =3D &l1tab[l1_table_offset(va)]; + + /* + * As the fast path -- and the slow path holds IRQs off throughout, + * so replacing area-owned backing here is never permitted. + */ + if ( unlikely(perdomain_l1e_needs_freeing(*pl1e)) ) + { + ASSERT_UNREACHABLE(); + free_domheap_page(l1e_get_page(*pl1e)); + } + l1e_write(pl1e, l1e_from_mfn(*mfn, flags)); + } + + unmap_domain_page_irqoff(l1tab); + unmap_domain_page_irqoff(l2tab); + unmap_domain_page_irqoff(l3tab); + + local_irq_restore(irq_flags); +} + void destroy_perdomain_mapping(struct domain *d, unsigned long va, unsigned int nr) { --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342269167271.6830838402018; Wed, 2 Sep 2026 02:44:29 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405456.1638968 (Exim 4.92) (envelope-from ) id 1x1hVq-0002jO-04; Wed, 02 Sep 2026 09:44:10 +0000 Received: by outflank-mailman (output) from mailman id 1405456.1638968; Wed, 02 Sep 2026 09:44:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVp-0002h2-ON; Wed, 02 Sep 2026 09:44:09 +0000 Received: by outflank-mailman (input) for mailman id 1405456; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVn-0002GM-JK for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hVm-00G2d4-QD for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [10.42.69.11] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97efe3-bab6-0a2a0a5309dd-0a2a450ba8c2-8 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-42698a.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe5-b7e8-0a2a450b0019-d99ba50cf2f6-1 for ; Wed, 02 Sep 2026 11:44:05 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 7CC1836928DB; Wed, 2 Sep 2026 10:44:05 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 03/14] x86/pv: use populate_perdomain_mapping() to map the Xen GDT Date: Wed, 2 Sep 2026 10:43:47 +0100 Message-ID: <20260901-asi-part2-3-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-42698a/1788342245-ABAD09EA-76AC3068/0/0 X-purgate-type: clean X-purgate-size: 6038 X-ZM-MESSAGEID: 1788342270556158500 From: Roger Pau Monn=C3=A9 Currently, update_xen_slot_in_full_gdt() uses the stashed direct-map pointer in d->arch.pv.gdt_ldt_l1tab to update the incoming vcpu's page tables with Xen's GDT, by writing a stashed per-cpu copy of a pre-baked L1 entry (either 64-bit or compat version). Switch this to using populate_perdomain_mapping(), which doesn't rely on the stashed address of the l1 page in the direct map. Rather than also stashing a pre-baked value for the payload, compute the mfn from the per-cpu GDT pointer at use: the conversion is a handful of cycles on a path costing thousands, and computing at use removes the parallel {,compat_}gdt_l1e bookkeeping along with its boot-ordering constraint (the cached value could only be generated after Xen's physical relocation, and had to be in place before the first context switch; a use-time lookup is correct by construction). The flags on the final mapping are identical. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Drop the {,compat_}gdt_mfn caching entirely (suggested by Andrew Cooper): compute virt_to_mfn() from the per-cpu GDT pointer at use. The PDX lookup behind it measures ~5-10 cycles warm against a ~1,500-cycle context switch, and this removes the double bookkeeping and the after-relocation caching constraint. The cached-MFN assertion goes with the cache: a use-time computation from a live pointer needs no staleness check. Changes since the previously posted version: - populate_perdomain_mapping() introduction split into the previous patch; this patch is now just the Xen GDT conversion. - Retain the "GDT MFN cached" check as ASSERT(mfn_x(mfn)). --- xen/arch/x86/domain.c | 13 ++++++++----- xen/arch/x86/include/asm/desc.h | 2 -- xen/arch/x86/smpboot.c | 15 --------------- xen/arch/x86/traps.c | 2 -- 4 files changed, 8 insertions(+), 24 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 996b50af7a..d8af06e533 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -2062,11 +2062,14 @@ static always_inline bool need_full_gdt(const struc= t domain *d) =20 static void update_xen_slot_in_full_gdt(const struct vcpu *v, unsigned int= cpu) { - ASSERT(per_cpu(gdt_l1e, cpu).l1); /* Confirm these have been cached. */ - - l1e_write(pv_gdt_ptes(v) + FIRST_RESERVED_GDT_PAGE, - !is_pv_32bit_vcpu(v) ? per_cpu(gdt_l1e, cpu) - : per_cpu(compat_gdt_l1e, cpu)); + mfn_t mfn =3D _mfn(virt_to_mfn(!is_pv_32bit_vcpu(v) + ? per_cpu(gdt, cpu) + : per_cpu(compat_gdt, cpu))); + + populate_perdomain_mapping(v, + GDT_VIRT_START(v) + + (FIRST_RESERVED_GDT_PAGE << PAGE_SHIFT), + &mfn, 1, __PAGE_HYPERVISOR_RW); } =20 static void load_full_gdt(const struct vcpu *v, unsigned int cpu) diff --git a/xen/arch/x86/include/asm/desc.h b/xen/arch/x86/include/asm/des= c.h index dcbdac3ff7..a860134211 100644 --- a/xen/arch/x86/include/asm/desc.h +++ b/xen/arch/x86/include/asm/desc.h @@ -136,10 +136,8 @@ struct __packed desc_ptr { =20 extern seg_desc_t boot_gdt[]; DECLARE_PER_CPU(seg_desc_t *, gdt); -DECLARE_PER_CPU(l1_pgentry_t, gdt_l1e); extern seg_desc_t boot_compat_gdt[]; DECLARE_PER_CPU(seg_desc_t *, compat_gdt); -DECLARE_PER_CPU(l1_pgentry_t, compat_gdt_l1e); DECLARE_PER_CPU(bool, full_gdt_loaded); =20 static inline void lgdt(const struct desc_ptr *gdtr) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 84e9e4beed..9b837a1769 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -1085,8 +1085,6 @@ static int cpu_smpboot_alloc(unsigned int cpu) if ( gdt =3D=3D NULL ) goto out; per_cpu(gdt, cpu) =3D gdt; - per_cpu(gdt_l1e, cpu) =3D - l1e_from_pfn(virt_to_mfn(gdt), __PAGE_HYPERVISOR_RW); memcpy(gdt, boot_gdt, NR_RESERVED_GDT_PAGES * PAGE_SIZE); BUILD_BUG_ON(NR_CPUS > 0x10000); gdt[PER_CPU_GDT_ENTRY - FIRST_RESERVED_GDT_ENTRY].a =3D cpu; @@ -1095,8 +1093,6 @@ static int cpu_smpboot_alloc(unsigned int cpu) per_cpu(compat_gdt, cpu) =3D gdt =3D alloc_xenheap_pages(0, memflags); if ( gdt =3D=3D NULL ) goto out; - per_cpu(compat_gdt_l1e, cpu) =3D - l1e_from_pfn(virt_to_mfn(gdt), __PAGE_HYPERVISOR_RW); memcpy(gdt, boot_compat_gdt, NR_RESERVED_GDT_PAGES * PAGE_SIZE); gdt[PER_CPU_GDT_ENTRY - FIRST_RESERVED_GDT_ENTRY].a =3D cpu; #endif @@ -1173,17 +1169,6 @@ void __init smp_prepare_cpus(void) initialize_cpu_data(0); /* Final full version of the data */ print_cpu_info(0); =20 - /* - * Cache {,compat_}gdt_l1e for the BSP now that physically relocation = is - * done. It must be after physical relocation of Xen, and before the - * first context_switch(). - */ - this_cpu(gdt_l1e) =3D - l1e_from_pfn(virt_to_mfn(boot_gdt), __PAGE_HYPERVISOR_RW); - if ( IS_ENABLED(CONFIG_PV32) ) - this_cpu(compat_gdt_l1e) =3D - l1e_from_pfn(virt_to_mfn(boot_compat_gdt), __PAGE_HYPERVISOR_R= W); - boot_cpu_physical_apicid =3D get_apic_id(); x86_cpu_to_apicid[0] =3D boot_cpu_physical_apicid; =20 diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index 1774966305..2ab61db167 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -71,10 +71,8 @@ DEFINE_PER_CPU(uint64_t, efer); static DEFINE_PER_CPU(unsigned long, last_extable_addr); =20 DEFINE_PER_CPU_READ_MOSTLY(seg_desc_t *, gdt); -DEFINE_PER_CPU_READ_MOSTLY(l1_pgentry_t, gdt_l1e); #ifdef CONFIG_PV32 DEFINE_PER_CPU_READ_MOSTLY(seg_desc_t *, compat_gdt); -DEFINE_PER_CPU_READ_MOSTLY(l1_pgentry_t, compat_gdt_l1e); #endif =20 /* --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342283645624.0966213439067; Wed, 2 Sep 2026 02:44:43 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405455.1638958 (Exim 4.92) (envelope-from ) id 1x1hVp-0002dC-Jf; Wed, 02 Sep 2026 09:44:09 +0000 Received: by outflank-mailman (output) from mailman id 1405455.1638958; Wed, 02 Sep 2026 09:44:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVp-0002c0-Dx; Wed, 02 Sep 2026 09:44:09 +0000 Received: by outflank-mailman (input) for mailman id 1405455; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVn-0002GD-FQ for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hVm-00G2d4-Kq for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [10.42.69.8] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97efdb-bab6-0a2a0a5309dd-0a2a4508bd48-34 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-c1860d.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe5-f659-0a2a45080019-d99ba50ce6fa-1 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 9FFCE36928DD; Wed, 2 Sep 2026 10:44:05 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 04/14] x86/pv: set/clear guest GDT mappings using populate_perdomain_mapping() Date: Wed, 2 Sep 2026 10:43:48 +0100 Message-ID: <20260901-asi-part2-4-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-c1860d/1788342246-D674387B-B33CDC94/0/0 X-purgate-type: clean X-purgate-size: 6510 X-ZM-MESSAGEID: 1788342286461154100 From: Roger Pau Monn=C3=A9 Until the previous patch, update_xen_slot_in_full_gdt() used the stashed pointer in d->arch.pv.gdt_ldt_l1tab to update the incoming vCPU's page tables with Xen's GDT; this was previously necessary because map_domain_page() couldn't be called in a context switch. Having a handy pointer to an always-mapped version of the GDT/LDT L1 table, other sites which modify the table started using it for convenience, even if they weren't called from within a context switch. One example is pv_{set,destroy}_gdt(). The previous patch switched the main user of the stashed reference to use populate_perdomain_mapping() instead. Continue that process by switching both pv_{set,destroy}_gdt() to it as well. pv_destroy_gdt() currently loops over the L1 entries directly, extracting the MFN from each, dropping the type and reference unless it was the zero page, and replacing the entry with a read-only mapping of the zero page. Rather than reading from the stashed L1, drop the references using v->arch.pv.gdt_frames[] instead, and install the zero-page mappings with a single populate_perdomain_mapping() call. This makes gdt_frames[] consistently the source of truth for MFNs. Note that we must maintain the invariant introduced in cf6d39f819 ("x86/PV: properly populate descriptor tables"): pv_destroy_gdt() maps the zero page read-only in torn-down slots rather than unmapping them, so that LAR/LSL/VERR/VERW on a selector beyond the guest's limit clear ZF as on native rather than taking a #PF-converted #GP. (And since pv_set_gdt() tears down the old GDT before installing the new one, guests never see unmapped entries, only zero-page entries.) In the case of pv_set_gdt(), we have a slightly awkward situation with types. The ABI with the guest uses unsigned long[], but populate_perdomain_mapping() wants an array of mfn_t. v->arch.pv.gdt_frames being unsigned long means we can just copy from it across the guest ABI with no conversions. We could in theory convert it to mfn_t[] instead, and then pass v->arch.pv.gdt_frames into populate_perdomain_mapping(); but then we'd need to add a conversion on all the places where frames are copied out. We choose instead to copy frames into a temporary mfn_t array on the stack to pass into populate_perdomain_mapping(). Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Reword commit message Changes since the previously posted version: - Retain the gdt_ents zeroing when tearing down the GDT (its removal was queried by Jan). - Map torn-down slots read-only to the zero page (via the populate_perdomain_mapping() flags parameter) rather than removing the mappings with destroy_perdomain_mapping(): empty slots would be a guest-visible partial revert of cf6d39f819 (see the commit message). With the destroy call gone, its v->arch.cr3 guard -- also queried by Jan -- goes too: the zero-page rewrite runs unconditionally. - Keep gdt_frames[] as unsigned long[] rather than switching it to mfn_t[] as Jan suggested; the commit message explains the trade-off. - Retitle: destroy_perdomain_mapping() is no longer used here. --- xen/arch/x86/pv/descriptor-tables.c | 37 ++++++++++++++++++----------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/xen/arch/x86/pv/descriptor-tables.c b/xen/arch/x86/pv/descript= or-tables.c index 8a32b9ae5c..5dda5bffe3 100644 --- a/xen/arch/x86/pv/descriptor-tables.c +++ b/xen/arch/x86/pv/descriptor-tables.c @@ -49,33 +49,42 @@ bool pv_destroy_ldt(struct vcpu *v) =20 void pv_destroy_gdt(struct vcpu *v) { - l1_pgentry_t *pl1e =3D pv_gdt_ptes(v); - mfn_t zero_mfn =3D _mfn(virt_to_mfn(zero_page)); - l1_pgentry_t zero_l1e =3D l1e_from_mfn(zero_mfn, __PAGE_HYPERVISOR_RO); + const mfn_t zero_mfn =3D _mfn(virt_to_mfn(zero_page)); + mfn_t zero_mfns[ARRAY_SIZE(v->arch.pv.gdt_frames)]; unsigned int i; =20 ASSERT(v =3D=3D current || !vcpu_cpu_dirty(v)); =20 v->arch.pv.gdt_ents =3D 0; - for ( i =3D 0; i < FIRST_RESERVED_GDT_PAGE; i++ ) + + for ( i =3D 0; i < ARRAY_SIZE(zero_mfns); i++ ) { - mfn_t mfn =3D l1e_get_mfn(pl1e[i]); + zero_mfns[i] =3D zero_mfn; =20 - if ( (l1e_get_flags(pl1e[i]) & _PAGE_PRESENT) && - !mfn_eq(mfn, zero_mfn) ) - put_page_and_type(mfn_to_page(mfn)); + /* MFN 0 can never pass get_page_and_type(), so 0 marks unused slo= ts. */ + if ( !v->arch.pv.gdt_frames[i] ) + continue; =20 - l1e_write(&pl1e[i], zero_l1e); + put_page_and_type(mfn_to_page(_mfn(v->arch.pv.gdt_frames[i]))); v->arch.pv.gdt_frames[i] =3D 0; } + + /* + * Point every slot at the zero page, read-only: a descriptor fetch fr= om + * the unused part of the GDT then finds a not-present descriptor rath= er + * than a missing mapping, so LAR/LSL/VERR/VERW on a selector beyond t= he + * guest's limit clear ZF as they do on native, instead of faulting. + */ + populate_perdomain_mapping(v, GDT_VIRT_START(v), zero_mfns, + ARRAY_SIZE(zero_mfns), __PAGE_HYPERVISOR_RO= ); } =20 int pv_set_gdt(struct vcpu *v, const unsigned long frames[], unsigned int entries) { struct domain *d =3D v->domain; - l1_pgentry_t *pl1e; unsigned int i, nr_frames =3D DIV_ROUND_UP(entries, 512); + mfn_t mfns[ARRAY_SIZE(v->arch.pv.gdt_frames)]; =20 ASSERT(v =3D=3D current || !vcpu_cpu_dirty(v)); =20 @@ -90,6 +99,8 @@ int pv_set_gdt(struct vcpu *v, const unsigned long frames= [], if ( !mfn_valid(mfn) || !get_page_and_type(mfn_to_page(mfn), d, PGT_seg_desc_page) ) goto fail; + + mfns[i] =3D mfn; } =20 /* Tear down the old GDT. */ @@ -97,12 +108,10 @@ int pv_set_gdt(struct vcpu *v, const unsigned long fra= mes[], =20 /* Install the new GDT. */ v->arch.pv.gdt_ents =3D entries; - pl1e =3D pv_gdt_ptes(v); for ( i =3D 0; i < nr_frames; i++ ) - { v->arch.pv.gdt_frames[i] =3D frames[i]; - l1e_write(&pl1e[i], l1e_from_pfn(frames[i], __PAGE_HYPERVISOR_RW)); - } + populate_perdomain_mapping(v, GDT_VIRT_START(v), mfns, nr_frames, + __PAGE_HYPERVISOR_RW); =20 return 0; =20 --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342568436635.0846764952688; Wed, 2 Sep 2026 02:49:28 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405529.1639026 (Exim 4.92) (envelope-from ) id 1x1haf-00079g-Rn; Wed, 02 Sep 2026 09:49:09 +0000 Received: by outflank-mailman (output) from mailman id 1405529.1639026; Wed, 02 Sep 2026 09:49:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1haf-00078g-Ou; Wed, 02 Sep 2026 09:49:09 +0000 Received: by outflank-mailman (input) for mailman id 1405529; Wed, 02 Sep 2026 09:49:08 +0000 Received: from mx.expurgate.net ([194.145.224.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1had-00071e-T4 for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:49:08 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1had-00HNoW-9z for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [10.42.69.7] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f112-2eae-0a2a0a5409dd-0a2a4507ac00-8 for ; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-ef75cf.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-b4ea-0a2a45070019-d99ba50cedd2-1 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id CD25936928DF; Wed, 2 Sep 2026 10:44:05 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 05/14] x86/pv: update guest LDT mappings using {populate,destroy}_perdomain_mapping() Date: Wed, 2 Sep 2026 10:43:49 +0100 Message-ID: <20260901-asi-part2-5-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ef75cf/1788342246-3C610AE4-9174E2E5/0/0 X-purgate-type: clean X-purgate-size: 7040 X-ZM-MESSAGEID: 1788342569991158500 From: Roger Pau Monn=C3=A9 Until two patches ago, update_xen_slot_in_full_gdt() used the stashed pointer in d->arch.pv.gdt_ldt_l1tab to update the incoming vCPU's page tables with Xen's GDT; this was previously necessary because map_domain_page() couldn't be called in a context switch. Having a handy pointer to an always-mapped version of the GDT/LDT L1 table, other sites which modify the table started using it for convenience, even if they weren't called from within a context switch. These include pv_map_ldt_shadow_page() and pv_destroy_ldt(). Continue the process of switching users of the stashed reference to use populate_perdomain_mapping() instead. pv_map_ldt_shadow_page() is, by definition, always modifying the currently-running vCPU: it runs from the #PF handler for a descriptor fetch on the guest's behalf, and running the guest implies its page tables are loaded. So it could simply write the linear recursive mappings directly. Go through populate_perdomain_mapping() anyway, to keep a single writer for the per-domain area. For pv_destroy_ldt(), use destroy_perdomain_mapping(). Previously, pv_destroy_ldt() used the L1 LDT entries themselves to determine which MFNs to drop type and count references to. Rather than reading from the stashed L1, keep the MFNs corresponding to L1 slots in an array in the vCPU structure, as we do in the GDT case. (Note that unlike the GDT case, these are not part of a public ABI, so can be mfn_t, avoiding a recast-and-copy.) Note that mappings_dropped (the return value of pv_destroy_ldt()) now reflects the *number of valid MFNs in this array*, not *the number of non-empty L1 entries*. This introduces an invariant we must maintain: pv_map_ldt_shadow_page() writes both the array entry and the mapping, and pv_destroy_ldt() clears both, so the two stay in lockstep. Also note that, unlike pv_destroy_gdt() from the previous patch, pv_destroy_ldt() doesn't fill in the values with zero_l1e (see 61031e64d3), so there's no change here. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Reword the commit message Changes since the previously posted version: - Initialise ldt_frames[] ahead of the first fail-able initialisation step. - Call destroy_perdomain_mapping() with its existing domain parameter; the switch to a vCPU parameter moves to a future patch. - Use populate_perdomain_mapping() in pv_map_ldt_shadow_page() rather than open-coding the linear-map write; retitle accordingly. - Comment the INVALID_MFN skip in pv_destroy_ldt(): the LDT is demand-faulted, so its pages may be sparsely mapped (Alejandro's question on v2). - Rewrite commit message (including describing the ldt_frames[] array's role directly, as Jan asked). --- xen/arch/x86/include/asm/domain.h | 2 ++ xen/arch/x86/pv/descriptor-tables.c | 20 +++++++++++--------- xen/arch/x86/pv/domain.c | 4 ++++ xen/arch/x86/pv/mm.c | 16 ++++++++++++---- 4 files changed, 29 insertions(+), 13 deletions(-) diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 2d0a915410..61a9fe00f0 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -541,6 +541,8 @@ struct pv_vcpu struct trap_info *trap_ctxt; =20 unsigned long gdt_frames[FIRST_RESERVED_GDT_PAGE]; + /* Max LDT entries is 8192, so 8192 * 8 =3D 64KiB (16 pages). */ + mfn_t ldt_frames[16]; unsigned long ldt_base; unsigned int gdt_ents, ldt_ents; =20 diff --git a/xen/arch/x86/pv/descriptor-tables.c b/xen/arch/x86/pv/descript= or-tables.c index 5dda5bffe3..261bf29c90 100644 --- a/xen/arch/x86/pv/descriptor-tables.c +++ b/xen/arch/x86/pv/descriptor-tables.c @@ -20,28 +20,30 @@ */ bool pv_destroy_ldt(struct vcpu *v) { - l1_pgentry_t *pl1e; + const unsigned int nr_frames =3D ARRAY_SIZE(v->arch.pv.ldt_frames); unsigned int i, mappings_dropped =3D 0; - struct page_info *page; =20 ASSERT(!in_irq()); =20 ASSERT(v =3D=3D current || !vcpu_cpu_dirty(v)); =20 - pl1e =3D pv_ldt_ptes(v); + destroy_perdomain_mapping(v->domain, LDT_VIRT_START(v), nr_frames); =20 - for ( i =3D 0; i < 16; i++ ) + for ( i =3D 0; i < nr_frames; i++ ) { - if ( !(l1e_get_flags(pl1e[i]) & _PAGE_PRESENT) ) - continue; + mfn_t mfn =3D v->arch.pv.ldt_frames[i]; + struct page_info *page; =20 - page =3D l1e_get_page(pl1e[i]); - l1e_write(&pl1e[i], l1e_empty()); - mappings_dropped++; + /* The LDT is demand-faulted, so its pages may be sparsely mapped.= */ + if ( mfn_eq(mfn, INVALID_MFN) ) + continue; =20 + v->arch.pv.ldt_frames[i] =3D INVALID_MFN; + page =3D mfn_to_page(mfn); ASSERT_PAGE_IS_TYPE(page, PGT_seg_desc_page); ASSERT_PAGE_IS_DOMAIN(page, v->domain); put_page_and_type(page); + mappings_dropped++; } =20 return mappings_dropped; diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 0c42ae58aa..7ddab1949f 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -340,10 +340,14 @@ void pv_vcpu_destroy(struct vcpu *v) int pv_vcpu_initialise(struct vcpu *v) { struct domain *d =3D v->domain; + unsigned int i; int rc; =20 ASSERT(!is_idle_domain(d)); =20 + for ( i =3D 0; i < ARRAY_SIZE(v->arch.pv.ldt_frames); i++ ) + v->arch.pv.ldt_frames[i] =3D INVALID_MFN; + rc =3D pv_create_gdt_ldt_l1tab(v); if ( rc ) return rc; diff --git a/xen/arch/x86/pv/mm.c b/xen/arch/x86/pv/mm.c index 5378299b8c..da280d7757 100644 --- a/xen/arch/x86/pv/mm.c +++ b/xen/arch/x86/pv/mm.c @@ -53,7 +53,8 @@ bool pv_map_ldt_shadow_page(unsigned int offset) struct vcpu *curr =3D current; struct domain *currd =3D curr->domain; struct page_info *page; - l1_pgentry_t gl1e, *pl1e, nl1e; + l1_pgentry_t gl1e; + mfn_t mfn; unsigned long linear =3D curr->arch.pv.ldt_base + offset; =20 BUG_ON(in_irq()); @@ -87,10 +88,17 @@ bool pv_map_ldt_shadow_page(unsigned int offset) return false; } =20 - pl1e =3D &pv_ldt_ptes(curr)[offset >> PAGE_SHIFT]; - nl1e =3D l1e_from_pfn(l1e_get_pfn(gl1e), __PAGE_HYPERVISOR_RW); + mfn =3D page_to_mfn(page); + curr->arch.pv.ldt_frames[offset >> PAGE_SHIFT] =3D mfn; =20 - l1e_write(pl1e, nl1e); + /* + * Running the guest implies its page-tables are loaded, so the linear + * mappings would do; go through the interface anyway to keep a single + * writer for the per-domain area. + */ + populate_perdomain_mapping(curr, + LDT_VIRT_START(curr) + (offset & PAGE_MASK), + &mfn, 1, __PAGE_HYPERVISOR_RW); =20 return true; } --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342275186299.68502235870596; Wed, 2 Sep 2026 02:44:35 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405452.1638946 (Exim 4.92) (envelope-from ) id 1x1hVo-0002TD-UD; Wed, 02 Sep 2026 09:44:08 +0000 Received: by outflank-mailman (output) from mailman id 1405452.1638946; Wed, 02 Sep 2026 09:44:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVo-0002T6-RA; Wed, 02 Sep 2026 09:44:08 +0000 Received: by outflank-mailman (input) for mailman id 1405452; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net ([194.145.224.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVn-0002GB-2H for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hVm-0093D7-F3 for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [10.42.69.2] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97efe1-8faa-0a2a0a5109dd-0a2a4502e798-16 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-720697.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-6ca4-0a2a45020019-d99ba50ce8c0-1 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id F2A1036928E5; Wed, 2 Sep 2026 10:44:05 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 06/14] x86/pv: remove stashing of GDT/LDT L1 page-tables Date: Wed, 2 Sep 2026 10:43:50 +0100 Message-ID: <20260901-asi-part2-6-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-720697/1788342246-F22A92AC-867215E0/0/0 X-purgate-type: clean X-purgate-size: 4378 X-ZM-MESSAGEID: 1788342276192158500 From: Roger Pau Monn=C3=A9 There are no remaining users of the stashed L1 page-tables in pv_domain.gdt_ldt_l1tab. Remove it, and all helpers. This removes a globally-mapped xenheap allocation, and sets the stage for per-vCPU root page tables. pv_create_gdt_ldt_l1tab() now passes NIL() rather than the stash array. This will cause create_perdomain_mapping() to still eagerly allocate the L1 tables covering the GDT/LDT range; but their addresses are no longer handed back. Doing this is necessary because populate_perdomain_mapping() only fills existing tables, and treats missing structure as a bug. Another side effect of passing NIL() rather than a pointer is that the L1 tables move from the xenheap to the domheap. Residing in the xenheap was only ever a requirement when the stashed pointer had to stay usable; with that requirement dropped, we can relax the allocation requirement as well. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Changes in v2: - With "x86/mm: allocate the per-domain page-tables from the xenheap" dropped from the series, passing NIL() now does move the GDT/LDT L1 tables to the domheap (upstream's allocation for non-capture mode); in v1 they stayed in the xenheap in all modes. Reword the commit message accordingly. Changes since the previously posted version: - Note the implications of changing from pointer to NIL() in pv_create_gdt_ldt_l1tab(). In v2 this also changed where new GDT/LDT L1 tables were allocated from: upstream's capture mode takes them from the xenheap (the stashed pointer has to stay usable), the NIL() mode from the domheap. Here they come from the xenheap in all modes ("x86/mm: allocate the per-domain page-tables from the xenheap"), so the switch only stops the addresses being handed back. --- xen/arch/x86/include/asm/domain.h | 9 --------- xen/arch/x86/pv/domain.c | 10 +--------- 2 files changed, 1 insertion(+), 18 deletions(-) diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 61a9fe00f0..5c7fad26a6 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -287,8 +287,6 @@ struct time_scale { =20 struct pv_domain { - l1_pgentry_t **gdt_ldt_l1tab; - atomic_t nr_l4_pages; =20 /* Is a 32-bit PV guest? */ @@ -524,13 +522,6 @@ struct arch_domain #define has_pirq(d) (!!((d)->arch.emulation_flags & X86_EMU_USE_PIR= Q)) #define has_vpci(d) (!!((d)->arch.emulation_flags & X86_EMU_VPCI)) =20 -#define gdt_ldt_pt_idx(v) \ - ((v)->vcpu_id >> (PAGETABLE_ORDER - GDT_LDT_VCPU_SHIFT)) -#define pv_gdt_ptes(v) \ - ((v)->domain->arch.pv.gdt_ldt_l1tab[gdt_ldt_pt_idx(v)] + \ - (((v)->vcpu_id << GDT_LDT_VCPU_SHIFT) & (L1_PAGETABLE_ENTRIES - 1))) -#define pv_ldt_ptes(v) (pv_gdt_ptes(v) + 16) - struct pv_vcpu { /* map_domain_page() mapping cache. */ diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 7ddab1949f..35d1761c9c 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -315,7 +315,7 @@ static int pv_create_gdt_ldt_l1tab(struct vcpu *v) { return create_perdomain_mapping(v->domain, GDT_VIRT_START(v), 1U << GDT_LDT_VCPU_SHIFT, - v->domain->arch.pv.gdt_ldt_l1tab, + NIL(l1_pgentry_t *), NULL); } =20 @@ -389,8 +389,6 @@ void pv_domain_destroy(struct domain *d) GDT_LDT_MBYTES << (20 - PAGE_SHIFT)); =20 XFREE(d->arch.pv.cpuidmasks); - - FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab); } =20 void noreturn cf_check continue_pv_domain(void); @@ -406,12 +404,6 @@ int pv_domain_initialise(struct domain *d) =20 pv_l1tf_domain_init(d); =20 - d->arch.pv.gdt_ldt_l1tab =3D - alloc_xenheap_pages(0, MEMF_node(domain_to_node(d))); - if ( !d->arch.pv.gdt_ldt_l1tab ) - goto fail; - clear_page(d->arch.pv.gdt_ldt_l1tab); - if ( levelling_caps & ~LCAP_faulting && (d->arch.pv.cpuidmasks =3D xmemdup(&cpuidmask_defaults)) =3D=3D N= ULL ) goto fail; --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342571178262.9996314469987; Wed, 2 Sep 2026 02:49:31 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405530.1639035 (Exim 4.92) (envelope-from ) id 1x1hag-0007D4-71; Wed, 02 Sep 2026 09:49:10 +0000 Received: by outflank-mailman (output) from mailman id 1405530.1639035; Wed, 02 Sep 2026 09:49:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1haf-0007BO-Vk; Wed, 02 Sep 2026 09:49:09 +0000 Received: by outflank-mailman (input) for mailman id 1405530; Wed, 02 Sep 2026 09:49:08 +0000 Received: from mx.expurgate.net ([194.145.224.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1had-00071f-Ue for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:49:08 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1had-00HNoW-BK for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [10.42.69.7] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f112-2eae-0a2a0a5409dd-0a2a4507ac00-10 for ; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-ef75cf.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-b4ea-0a2a45070019-d99ba50cfdf1-1 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 2B0F036928E9; Wed, 2 Sep 2026 10:44:06 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 07/14] x86/mm: simplify create_perdomain_mapping() interface Date: Wed, 2 Sep 2026 10:43:51 +0100 Message-ID: <20260901-asi-part2-7-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ef75cf/1788342246-344CBAE4-F05E644D/0/0 X-purgate-type: clean X-purgate-size: 12132 X-ZM-MESSAGEID: 1788342571873158500 From: Roger Pau Monn=C3=A9 create_perdomain_mapping()'s interface is richer than any caller needs. The paging structure for the requested range is built to a depth selected by the pl1tab and ppg arguments, each of which distinguishes NULL from NIL() from a real pointer: - nr =3D=3D 0: only ensure the per-domain L3 exists; nothing else is allocated, and the other arguments are ignored. - pl1tab =3D=3D a pointer: allocate the L1 tables covering the range from the *xenheap*, and return their (stable, direct-map) addresses in the array -- the mode that existed to build the GDT/LDT stash. - pl1tab =3D=3D NIL(): allocate the L1 tables from the domain heap, and return nothing. - pl1tab =3D=3D NULL: do not plumb L1 tables for their own sake (they are still allocated on demand if data-page population requires them). - ppg =3D=3D a pointer: allocate and install zeroed data pages across the range, and return their struct page_info pointers in the array. - ppg =3D=3D NIL(): allocate and install the zeroed data pages, but hand nothing back; the pages are reachable only through the mapping. - ppg =3D=3D NULL: do not allocate data pages. - both NULL, nr > 0: stop after the slot's L2; do not plumb L1 tables at all. Very few of these modes have users now. The last user of the pl1tab capture mode was removed when we removed the GDT/LDT stash. The ppg capture mode never had any users. Nothing uses the both-NULL L2-only mode with nr !=3D 0. What remains is exactly one bit of information: whether the caller wants the range populated with zeroed, area-owned data pages, or merely plumbed down to the L1 tables, ready for populate_perdomain_mapping() to install caller-owned pages. Replace the two arguments with a boolean expressing that bit. With the stashing mode gone the NIL()/IS_NIL() macros lose their last user, so drop them as well; and document the resulting interface. No caller changes behaviour: every existing call maps onto the boolean exactly. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Changes in v2: - Describe, in the mode enumeration, which heap each pl1tab mode allocates the L1 tables from (capture mode: xenheap; NIL(): the domain heap). With "x86/mm: allocate the per-domain page-tables from the xenheap" dropped from the series, upstream's heap split is back in force at this point, and the previous patch's message refers to it. Changes since the previously posted version: - Drop the now-unused NIL()/IS_NIL() macros as requested during review - Describe the prior interface in the commit message and add a doc comment for the simplified one. --- xen/arch/x86/domain_page.c | 10 +++--- xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/include/asm/mm.h | 6 +--- xen/arch/x86/mm.c | 67 ++++++++++++++++------------------- xen/arch/x86/pv/domain.c | 4 +-- xen/arch/x86/x86_64/mm.c | 3 +- 6 files changed, 39 insertions(+), 53 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 1fc1580e62..b42cf1c8cf 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -279,8 +279,7 @@ int mapcache_domain_init(struct domain *d) spin_lock_init(&dcache->lock); =20 return create_perdomain_mapping(d, (unsigned long)dcache->inuse, - 2 * bitmap_pages + 1, - NIL(l1_pgentry_t *), NULL); + 2 * bitmap_pages + 1, false); } =20 int mapcache_vcpu_init(struct vcpu *v) @@ -297,16 +296,15 @@ int mapcache_vcpu_init(struct vcpu *v) if ( ents > dcache->entries ) { /* Populate page tables. */ - int rc =3D create_perdomain_mapping(d, MAPCACHE_VIRT_START, ents, - NIL(l1_pgentry_t *), NULL); + int rc =3D create_perdomain_mapping(d, MAPCACHE_VIRT_START, ents, = false); =20 /* Populate bit maps. */ if ( !rc ) rc =3D create_perdomain_mapping(d, (unsigned long)dcache->inus= e, - nr, NULL, NIL(struct page_info *= )); + nr, true); if ( !rc ) rc =3D create_perdomain_mapping(d, (unsigned long)dcache->garb= age, - nr, NULL, NIL(struct page_info *= )); + nr, true); =20 if ( rc ) return rc; diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 9a4147b62e..a6e0818468 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -620,7 +620,7 @@ int hvm_domain_initialise(struct domain *d, INIT_LIST_HEAD(&d->arch.hvm.mmcfg_regions); INIT_LIST_HEAD(&d->arch.hvm.msix_tables); =20 - rc =3D create_perdomain_mapping(d, PERDOMAIN_VIRT_START, 0, NULL, NULL= ); + rc =3D create_perdomain_mapping(d, PERDOMAIN_VIRT_START, 0, false); if ( rc ) goto fail; =20 diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 1888807394..30eaec9179 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -600,12 +600,8 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDL= E_PARAM(void) arg); long subarch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg= ); int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) = arg); =20 -#define NIL(type) ((type *)-sizeof(type)) -#define IS_NIL(ptr) (!((uintptr_t)(ptr) + sizeof(*(ptr)))) - int create_perdomain_mapping(struct domain *d, unsigned long va, - unsigned int nr, l1_pgentry_t **pl1tab, - struct page_info **ppg); + unsigned int nr, bool populate); void populate_perdomain_mapping(const struct vcpu *v, unsigned long va, const mfn_t *mfn, unsigned int nr, unsigned int flags); diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 552559ecf1..48d1b427c5 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6211,9 +6211,31 @@ static bool perdomain_l1e_needs_freeing(l1_pgentry_t= l1e) (_PAGE_PRESENT | _PAGE_AVAIL0); } =20 +/* + * Ensure the paging structure for [va, va + nr * PAGE_SIZE) of d's + * per-domain area is in place, allocating whichever levels are missing: + * the (domain-wide) L3 root, the slot's L2, and all L1 tables covering + * the range. All allocations come from the domain heap. The range must + * lie within a single per-domain slot (one L3 entry), and already-present + * levels and entries are left untouched, so calls are idempotent over + * existing ranges. + * + * nr =3D=3D 0: only ensure the per-domain L3 itself exists; populate is + * ignored. Used to set the area up before any sub-range is known. + * + * populate =3D=3D false: stop once the L1 tables are in place. The range= is + * then ready for caller-owned pages to be mapped and unmapped via + * populate_perdomain_mapping() / destroy_perdomain_mapping(), which only + * fill existing tables and treat missing structure as a bug. + * + * populate =3D=3D true: additionally install a freshly allocated, zeroed = page + * at every not-yet-present entry in the range. Such pages are marked + * _PAGE_AVAIL0, "owned by the per-domain area": teardown frees them (see + * perdomain_l1e_needs_freeing()), whereas caller-owned mappings are only + * ever unmapped. + */ int create_perdomain_mapping(struct domain *d, unsigned long va, - unsigned int nr, l1_pgentry_t **pl1tab, - struct page_info **ppg) + unsigned int nr, bool populate) { struct page_info *pg; l3_pgentry_t *l3tab; @@ -6262,55 +6284,32 @@ int create_perdomain_mapping(struct domain *d, unsi= gned long va, =20 unmap_domain_page(l3tab); =20 - if ( !pl1tab && !ppg ) - { - unmap_domain_page(l2tab); - return 0; - } - for ( l1tab =3D NULL; !rc && nr--; ) { l2_pgentry_t *pl2e =3D l2tab + l2_table_offset(va); =20 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { - if ( pl1tab && !IS_NIL(pl1tab) ) - { - l1tab =3D alloc_xenheap_pages(0, MEMF_node(domain_to_node(= d))); - if ( !l1tab ) - { - rc =3D -ENOMEM; - break; - } - ASSERT(!pl1tab[l2_table_offset(va)]); - pl1tab[l2_table_offset(va)] =3D l1tab; - pg =3D virt_to_page(l1tab); - } - else + pg =3D alloc_domheap_page(d, MEMF_no_owner); + if ( !pg ) { - pg =3D alloc_domheap_page(d, MEMF_no_owner); - if ( !pg ) - { - rc =3D -ENOMEM; - break; - } - l1tab =3D __map_domain_page(pg); + rc =3D -ENOMEM; + break; } + l1tab =3D __map_domain_page(pg); clear_page(l1tab); *pl2e =3D l2e_from_page(pg, __PAGE_HYPERVISOR_RW); } else if ( !l1tab ) l1tab =3D map_l1t_from_l2e(*pl2e); =20 - if ( ppg && + if ( populate && !(l1e_get_flags(l1tab[l1_table_offset(va)]) & _PAGE_PRESENT) ) { pg =3D alloc_domheap_page(d, MEMF_no_owner); if ( pg ) { clear_domain_page(page_to_mfn(pg)); - if ( !IS_NIL(ppg) ) - *ppg++ =3D pg; l1tab[l1_table_offset(va)] =3D l1e_from_page(pg, __PAGE_HYPERVISOR_RW | _PAGE_AVAIL0); l2e_add_flags(*pl2e, _PAGE_AVAIL0); @@ -6322,7 +6321,6 @@ int create_perdomain_mapping(struct domain *d, unsign= ed long va, va +=3D PAGE_SIZE; if ( rc || !nr || !l1_table_offset(va) ) { - /* Note that this is a no-op for the alloc_xenheap_page() case= . */ unmap_domain_page(l1tab); l1tab =3D NULL; } @@ -6543,10 +6541,7 @@ void free_perdomain_mappings(struct domain *d) unmap_domain_page(l1tab); } =20 - if ( is_xen_heap_page(l1pg) ) - free_xenheap_page(page_to_virt(l1pg)); - else - free_domheap_page(l1pg); + free_domheap_page(l1pg); } =20 unmap_domain_page(l2tab); diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 35d1761c9c..15a8238aff 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -314,9 +314,7 @@ int switch_compat(struct domain *d) static int pv_create_gdt_ldt_l1tab(struct vcpu *v) { return create_perdomain_mapping(v->domain, GDT_VIRT_START(v), - 1U << GDT_LDT_VCPU_SHIFT, - NIL(l1_pgentry_t *), - NULL); + 1U << GDT_LDT_VCPU_SHIFT, false); } =20 static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v) diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index 8eadab7933..ffeda06e08 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -733,8 +733,7 @@ void __init zap_low_mappings(void) int setup_compat_arg_xlat(struct vcpu *v) { return create_perdomain_mapping(v->domain, ARG_XLAT_START(v), - PFN_UP(COMPAT_ARG_XLAT_SIZE), - NULL, NIL(struct page_info *)); + PFN_UP(COMPAT_ARG_XLAT_SIZE), true); } =20 void free_compat_arg_xlat(struct vcpu *v) --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342273569126.3982032172296; Wed, 2 Sep 2026 02:44:33 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405457.1638971 (Exim 4.92) (envelope-from ) id 1x1hVq-0002oR-6A; Wed, 02 Sep 2026 09:44:10 +0000 Received: by outflank-mailman (output) from mailman id 1405457.1638971; Wed, 02 Sep 2026 09:44:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVq-0002mL-0K; Wed, 02 Sep 2026 09:44:10 +0000 Received: by outflank-mailman (input) for mailman id 1405457; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hVn-0002GC-Nm for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:44:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hVm-00G2eL-JM for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [10.42.69.6] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97efd9-2eae-0a2a0a5409dd-0a2a45069e4a-38 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-16d1c6.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-195a-0a2a45060019-d99ba50cefa3-3 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 5715E36928EB; Wed, 2 Sep 2026 10:44:06 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , Alejandro Vallejo , George Dunlap Subject: [PATCH v2 08/14] x86/mm: purge unneeded destroy_perdomain_mapping() Date: Wed, 2 Sep 2026 10:43:52 +0100 Message-ID: <20260901-asi-part2-8-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-16d1c6/1788342246-FD60A77B-C10AA290/0/0 X-purgate-type: clean X-purgate-size: 2524 X-ZM-MESSAGEID: 1788342276167158500 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that destroy_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. There are two calls made from domain-scoped contexts; both calls turn out to be unnecessary: - destroy_perdomain_mapping() is not logically the undo of create_perdomain_mapping(), as the name and its use in hvm_domain_initialise() suggest. create_ allocates a per-domain L3, but destroy_ tears down mappings without freeing it; and since the call here passes nr =3D=3D 0, it tears down nothing at all. The per-domain L3 page is actually freed by free_perdomain_mappings(), which hvm_domain_initialise()'s caller, arch_domain_create(), already invokes on its failure path. - The call in pv_domain_destroy() is redundant: arch_domain_destroy() unconditionally calls free_perdomain_mappings(), which tears down the same entries and additionally frees the page-table structures. Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Alejandro Vallejo Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Changes in v2: - Added to the series Changes since the previously posted version: - Reworked the commit message to make it more clear how it fits in with the larger series. No functional change. --- xen/arch/x86/hvm/hvm.c | 1 - xen/arch/x86/pv/domain.c | 3 --- 2 files changed, 4 deletions(-) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index a6e0818468..cd425c3342 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -730,7 +730,6 @@ int hvm_domain_initialise(struct domain *d, XFREE(d->arch.hvm.irq); fail0: hvm_destroy_cacheattr_region_list(d); - destroy_perdomain_mapping(d, PERDOMAIN_VIRT_START, 0); fail: hvm_domain_relinquish_resources(d); XFREE(d->arch.hvm.io_handler); diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 15a8238aff..b936ca9b26 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -383,9 +383,6 @@ void pv_domain_destroy(struct domain *d) { pv_l1tf_domain_destroy(d); =20 - destroy_perdomain_mapping(d, GDT_LDT_VIRT_START, - GDT_LDT_MBYTES << (20 - PAGE_SHIFT)); - XFREE(d->arch.pv.cpuidmasks); } =20 --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342573880307.78036480126093; Wed, 2 Sep 2026 02:49:33 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405527.1639017 (Exim 4.92) (envelope-from ) id 1x1haf-000728-EO; Wed, 02 Sep 2026 09:49:09 +0000 Received: by outflank-mailman (output) from mailman id 1405527.1639017; Wed, 02 Sep 2026 09:49:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1haf-000721-Bn; Wed, 02 Sep 2026 09:49:09 +0000 Received: by outflank-mailman (input) for mailman id 1405527; Wed, 02 Sep 2026 09:49:07 +0000 Received: from mx.expurgate.net ([194.145.224.20]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1had-00071X-HH for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:49:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hac-003afr-UC for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:49:06 +0200 Received: from [10.42.69.2] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f10a-bab6-0a2a0a5309dd-0a2a45029afe-20 for ; Wed, 02 Sep 2026 11:49:06 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-720697.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-6ca4-0a2a45020019-d99ba50ce8c0-3 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 7AC9C36928EF; Wed, 2 Sep 2026 10:44:06 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 09/14] x86/mm: prepare destroy_perdomain_mapping() for per-vCPU perdomain areas Date: Wed, 2 Sep 2026 10:43:53 +0100 Message-ID: <20260901-asi-part2-9-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-720697/1788342246-F1CAA2AC-EC0435F5/0/0 X-purgate-type: clean X-purgate-size: 5444 X-ZM-MESSAGEID: 1788342577075154100 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that destroy_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. The remaining callers are already in a vCPU context, so we just need to change the parameter from a domain pointer to a vCPU pointer. Since we now have a specific vCPU in mind, we have the option of using the linear page table mapping rather than map-and-walk. As in populate_perdomain_mapping(), the linear page table fast path is keyed off this_cpu(pgtable_vcpu) matching the target vCPU, which is the conditional that implies "the linear mapping area points to v's per-domain area". Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Added to the series Changes since the previously posted version: - Key the fast path off pgtable_vcpu instead of current, matching populate_perdomain_mapping(), and drop the sync_local_execstate() call. - Also convert the pv_destroy_ldt() call, added by the stash-removal batch. - Reword and retitle for clarity (was: "x86/mm: switch destroy_perdomain_mapping() parameter from domain to vCPU"). --- xen/arch/x86/include/asm/mm.h | 2 +- xen/arch/x86/mm.c | 23 ++++++++++++++++++++++- xen/arch/x86/pv/descriptor-tables.c | 2 +- xen/arch/x86/pv/domain.c | 3 +-- xen/arch/x86/x86_64/mm.c | 2 +- 5 files changed, 26 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 30eaec9179..9a8fda782e 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -605,7 +605,7 @@ int create_perdomain_mapping(struct domain *d, unsigned= long va, void populate_perdomain_mapping(const struct vcpu *v, unsigned long va, const mfn_t *mfn, unsigned int nr, unsigned int flags); -void destroy_perdomain_mapping(struct domain *d, unsigned long va, +void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr); void free_perdomain_mappings(struct domain *d); =20 diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 48d1b427c5..fc524ef0c3 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6456,10 +6456,11 @@ void populate_perdomain_mapping(const struct vcpu *= v, unsigned long va, local_irq_restore(irq_flags); } =20 -void destroy_perdomain_mapping(struct domain *d, unsigned long va, +void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr) { const l3_pgentry_t *l3tab, *pl3e; + const struct domain *d =3D v->domain; =20 ASSERT(va >=3D PERDOMAIN_VIRT_START && va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); @@ -6468,6 +6469,26 @@ void destroy_perdomain_mapping(struct domain *d, uns= igned long va, if ( !d->arch.perdomain_l3_pg ) return; =20 + if ( likely(this_cpu(pgtable_vcpu) =3D=3D v) ) + { + l1_pgentry_t *pl1e; + + /* + * Fast path: v's page-tables are loaded on this pCPU, so the L1 + * entries can be zapped using the recursive linear mappings. + */ + pl1e =3D &__linear_l1_table[l1_linear_offset(va)]; + + for ( ; nr--; pl1e++ ) + { + if ( perdomain_l1e_needs_freeing(*pl1e) ) + free_domheap_page(l1e_get_page(*pl1e)); + l1e_write(pl1e, l1e_empty()); + } + + return; + } + l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); pl3e =3D l3tab + l3_table_offset(va); =20 diff --git a/xen/arch/x86/pv/descriptor-tables.c b/xen/arch/x86/pv/descript= or-tables.c index 261bf29c90..0c1ea4ce3a 100644 --- a/xen/arch/x86/pv/descriptor-tables.c +++ b/xen/arch/x86/pv/descriptor-tables.c @@ -27,7 +27,7 @@ bool pv_destroy_ldt(struct vcpu *v) =20 ASSERT(v =3D=3D current || !vcpu_cpu_dirty(v)); =20 - destroy_perdomain_mapping(v->domain, LDT_VIRT_START(v), nr_frames); + destroy_perdomain_mapping(v, LDT_VIRT_START(v), nr_frames); =20 for ( i =3D 0; i < nr_frames; i++ ) { diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index b936ca9b26..40b834e1a4 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -319,8 +319,7 @@ static int pv_create_gdt_ldt_l1tab(struct vcpu *v) =20 static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v) { - destroy_perdomain_mapping(v->domain, GDT_VIRT_START(v), - 1U << GDT_LDT_VCPU_SHIFT); + destroy_perdomain_mapping(v, GDT_VIRT_START(v), 1U << GDT_LDT_VCPU_SHI= FT); } =20 void pv_vcpu_destroy(struct vcpu *v) diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index ffeda06e08..aa74acec82 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -738,7 +738,7 @@ int setup_compat_arg_xlat(struct vcpu *v) =20 void free_compat_arg_xlat(struct vcpu *v) { - destroy_perdomain_mapping(v->domain, ARG_XLAT_START(v), + destroy_perdomain_mapping(v, ARG_XLAT_START(v), PFN_UP(COMPAT_ARG_XLAT_SIZE)); } =20 --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342561282690.4803556336749; Wed, 2 Sep 2026 02:49:21 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405528.1639022 (Exim 4.92) (envelope-from ) id 1x1haf-00074f-Kh; Wed, 02 Sep 2026 09:49:09 +0000 Received: by outflank-mailman (output) from mailman id 1405528.1639022; Wed, 02 Sep 2026 09:49:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1haf-00074P-Hd; Wed, 02 Sep 2026 09:49:09 +0000 Received: by outflank-mailman (input) for mailman id 1405528; Wed, 02 Sep 2026 09:49:07 +0000 Received: from mx.expurgate.net ([194.145.224.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1had-00071c-Qh for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:49:07 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1had-00HNoW-7e for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [10.42.69.12] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f109-2eae-0a2a0a5409dd-0a2a450c8ef0-38 for ; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-d25034.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-f479-0a2a450c0019-d99ba50cf8a1-3 for ; Wed, 02 Sep 2026 11:44:06 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id A314F36928F3; Wed, 2 Sep 2026 10:44:06 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 10/14] x86/domain_page: drop redundant create_perdomain_mapping() call Date: Wed, 2 Sep 2026 10:43:54 +0100 Message-ID: <20260901-asi-part2-10-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-d25034/1788342246-008CDA5B-4A1DBDB3/0/0 X-purgate-type: clean X-purgate-size: 3821 X-ZM-MESSAGEID: 1788342563090154100 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that create_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. One of the current calls in a domain context turns out to be unnecessary: mapcache_domain_init() pre-plumbs L1 tables over the whole inuse/garbage bitmap range (sized for the full MAPCACHE_ENTRIES capacity), without populating any data pages. The plumbing is redundant: mapcache_vcpu_init() installs the bitmap pages the domain will actually use with populate=3Dtrue calls, which allocate any missing page-table structure on demand -- and since d->max_vcpus is fixed before any vCPU is created, the range those calls cover never grows. The pre-plumbed tail beyond it backs virtual addresses that are never populated at all. Drop the call. With the only fallible operation gone, mapcache_domain_init() becomes void, and arch_domain_create()'s error handling for it goes away. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Added to the series (split out of the following patch). Changes since the previously posted version: - Split out of "x86/mm: switch {create,destroy}_perdomain_mapping() domain parameter to vCPU", where the removal was folded into the parameter switch without its own rationale. --- xen/arch/x86/domain.c | 3 +-- xen/arch/x86/domain_page.c | 7 ++----- xen/arch/x86/include/asm/domain.h | 2 +- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index d8af06e533..efa72cd2f1 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -908,8 +908,7 @@ int arch_domain_create(struct domain *d, } else if ( is_pv_domain(d) ) { - if ( (rc =3D mapcache_domain_init(d)) !=3D 0 ) - goto fail; + mapcache_domain_init(d); =20 if ( (rc =3D pv_domain_initialise(d)) !=3D 0 ) goto fail; diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index b42cf1c8cf..449d4f2a7d 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -256,7 +256,7 @@ void unmap_domain_page_irqoff(const void *ptr) do_unmap_domain_page(ptr, true); } =20 -int mapcache_domain_init(struct domain *d) +void mapcache_domain_init(struct domain *d) { struct mapcache_domain *dcache =3D &d->arch.pv.mapcache; unsigned int bitmap_pages; @@ -265,7 +265,7 @@ int mapcache_domain_init(struct domain *d) =20 #ifdef NDEBUG if ( !mem_hotplug && max_page <=3D PFN_DOWN(__pa(HYPERVISOR_VIRT_END -= 1)) ) - return 0; + return; #endif =20 BUILD_BUG_ON(MAPCACHE_VIRT_END + PAGE_SIZE * (3 + @@ -277,9 +277,6 @@ int mapcache_domain_init(struct domain *d) (bitmap_pages + 1) * PAGE_SIZE / sizeof(long); =20 spin_lock_init(&dcache->lock); - - return create_perdomain_mapping(d, (unsigned long)dcache->inuse, - 2 * bitmap_pages + 1, false); } =20 int mapcache_vcpu_init(struct vcpu *v) diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 5c7fad26a6..38df5c376e 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -89,7 +89,7 @@ struct mapcache_domain { unsigned long *garbage; }; =20 -int mapcache_domain_init(struct domain *d); +void mapcache_domain_init(struct domain *d); int mapcache_vcpu_init(struct vcpu *v); =20 /* x86/64: toggle guest between kernel and user modes. */ --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342576983773.7678957025092; Wed, 2 Sep 2026 02:49:36 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405533.1639069 (Exim 4.92) (envelope-from ) id 1x1hak-0008Be-Dg; Wed, 02 Sep 2026 09:49:14 +0000 Received: by outflank-mailman (output) from mailman id 1405533.1639069; Wed, 02 Sep 2026 09:49:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hak-0008BL-6Y; Wed, 02 Sep 2026 09:49:14 +0000 Received: by outflank-mailman (input) for mailman id 1405533; Wed, 02 Sep 2026 09:49:13 +0000 Received: from mx.expurgate.net ([194.145.224.20]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hai-000869-Vw for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:49:13 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hai-003afr-CK for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:49:12 +0200 Received: from [10.42.69.2] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f10a-bab6-0a2a0a5309dd-0a2a45029afe-40 for ; Wed, 02 Sep 2026 11:49:12 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-720697.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-6ca4-0a2a45020019-d99ba50ce8c0-5 for ; Wed, 02 Sep 2026 11:44:07 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id CF50036928F7; Wed, 2 Sep 2026 10:44:06 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 11/14] x86/mm: prepare create_perdomain_mapping() for per-vCPU perdomain areas Date: Wed, 2 Sep 2026 10:43:55 +0100 Message-ID: <20260901-asi-part2-11-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-720697/1788342247-F34BE2AC-9385394E/0/0 X-purgate-type: clean X-purgate-size: 9636 X-ZM-MESSAGEID: 1788342579198154101 From: Roger Pau Monn=C3=A9 We want to change per-domain mappings to be per-vCPU mappings. In preparation for that, we want to arrange that create_perdomain_mapping() work either with a single perdomain area, or with a per-vCPU perdomain area. Most of the remaining callers are already in a vCPU context. This is no accident: the perdomain area has always been laid out in per-vCPU slices -- each vCPU has its own GDT/LDT window, its own COMPAT_ARG_XLAT pages, its own window of mapcache entries -- and each vCPU's slice is set up as that vCPU is created. For these callers, we just need to change the parameter from a domain pointer to a vCPU pointer. Once the perdomain area itself becomes per-vCPU, the same calls will populate the owning vCPU's own area rather than slices of a shared one. One exception is the call in hvm_domain_initialise(). An HVM vCPU's monitor table is created during vCPU initialisation, and init_xen_l4_slots() stamps the perdomain slot into it at that point -- far earlier than for PV, where the Xen slots are written only once guest page tables are built. hvm_domain_initialise() therefore had an explicit create_perdomain_mapping() call just to make the perdomain root exist ahead of that. Move it to arch_vcpu_create(), covering HVM and PV alike. With a single shared area, the call allocates at most once per domain; but once each vCPU has its own perdomain area, this is the call that will allocate every vCPU's root -- PV included -- before any page tables referencing it are built. vCPU creation is where the call must end up; move it there directly. For PV guests nothing observable changes: the root was already being created during vCPU creation as a side effect (by mapcache_vcpu_init(), or failing that pv_create_gdt_ldt_l1tab()); it now merely becomes explicit. Note that we cannot yet do a parallel movement of free_perdomain_mappings(): the per-domain page-table hierarchy is still a single domain-wide structure shared by all vCPUs, so tearing it down from a per-vCPU path would pull the mappings out from under sibling vCPUs (e.g. on a partially failed, retryable XEN_DOMCTL_max_vcpus), and vCPU-create error paths can rely on domain destruction to free a partially set up hierarchy. Teardown will move to vCPU scope only once the structure itself becomes per-vCPU. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Added to the series Changes since the previously posted version: - Keep free_perdomain_mappings() (and hence perdomain teardown) domain-scoped. - Keep the idle domain without a perdomain area. - Retitle (was: "x86/mm: switch {create,destroy}_perdomain_mapping() domain parameter to vCPU"); destroy_perdomain_mapping() was switched in a separate patch. - Split the removal of mapcache_domain_init()'s redundant create_perdomain_mapping() call into its own (preceding) patch. --- xen/arch/x86/domain.c | 10 ++++++++++ xen/arch/x86/domain_page.c | 6 +++--- xen/arch/x86/hvm/hvm.c | 5 ----- xen/arch/x86/include/asm/mm.h | 2 +- xen/arch/x86/mm.c | 17 +++++++++-------- xen/arch/x86/pv/domain.c | 2 +- xen/arch/x86/x86_64/mm.c | 2 +- 7 files changed, 25 insertions(+), 19 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index efa72cd2f1..1f75d44fe0 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -517,6 +517,16 @@ int arch_vcpu_create(struct vcpu *v) =20 if ( !is_idle_domain(d) ) { + /* + * Make sure the per-domain L3 exists ahead of any consumer (e.g. + * init_xen_l4_slots() for the HVM monitor tables): with + * create_perdomain_mapping() taking a vCPU this can no longer be + * done when creating the domain. + */ + rc =3D create_perdomain_mapping(v, PERDOMAIN_VIRT_START, 0, false); + if ( rc ) + return rc; + paging_vcpu_init(v); =20 if ( (rc =3D vcpu_init_fpu(v)) !=3D 0 ) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 449d4f2a7d..9b375e438a 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -293,14 +293,14 @@ int mapcache_vcpu_init(struct vcpu *v) if ( ents > dcache->entries ) { /* Populate page tables. */ - int rc =3D create_perdomain_mapping(d, MAPCACHE_VIRT_START, ents, = false); + int rc =3D create_perdomain_mapping(v, MAPCACHE_VIRT_START, ents, = false); =20 /* Populate bit maps. */ if ( !rc ) - rc =3D create_perdomain_mapping(d, (unsigned long)dcache->inus= e, + rc =3D create_perdomain_mapping(v, (unsigned long)dcache->inus= e, nr, true); if ( !rc ) - rc =3D create_perdomain_mapping(d, (unsigned long)dcache->garb= age, + rc =3D create_perdomain_mapping(v, (unsigned long)dcache->garb= age, nr, true); =20 if ( rc ) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index cd425c3342..a41ae35374 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -620,10 +620,6 @@ int hvm_domain_initialise(struct domain *d, INIT_LIST_HEAD(&d->arch.hvm.mmcfg_regions); INIT_LIST_HEAD(&d->arch.hvm.msix_tables); =20 - rc =3D create_perdomain_mapping(d, PERDOMAIN_VIRT_START, 0, false); - if ( rc ) - goto fail; - hvm_init_cacheattr_region_list(d); =20 rc =3D paging_enable(d, PG_refcounts|PG_translate|PG_external); @@ -730,7 +726,6 @@ int hvm_domain_initialise(struct domain *d, XFREE(d->arch.hvm.irq); fail0: hvm_destroy_cacheattr_region_list(d); - fail: hvm_domain_relinquish_resources(d); XFREE(d->arch.hvm.io_handler); XFREE(d->arch.hvm.pl_time); diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 9a8fda782e..97924a639b 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -600,7 +600,7 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE= _PARAM(void) arg); long subarch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg= ); int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) = arg); =20 -int create_perdomain_mapping(struct domain *d, unsigned long va, +int create_perdomain_mapping(struct vcpu *v, unsigned long va, unsigned int nr, bool populate); void populate_perdomain_mapping(const struct vcpu *v, unsigned long va, const mfn_t *mfn, unsigned int nr, diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index fc524ef0c3..6dfd75475a 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6212,13 +6212,13 @@ static bool perdomain_l1e_needs_freeing(l1_pgentry_= t l1e) } =20 /* - * Ensure the paging structure for [va, va + nr * PAGE_SIZE) of d's - * per-domain area is in place, allocating whichever levels are missing: - * the (domain-wide) L3 root, the slot's L2, and all L1 tables covering - * the range. All allocations come from the domain heap. The range must - * lie within a single per-domain slot (one L3 entry), and already-present - * levels and entries are left untouched, so calls are idempotent over - * existing ranges. + * Ensure the paging structure for [va, va + nr * PAGE_SIZE) of the + * per-domain area of v's domain is in place, allocating whichever levels + * are missing: the (domain-wide) L3 root, the slot's L2, and all L1 + * tables covering the range. All allocations come from the domain heap. + * The range must lie within a single per-domain slot (one L3 entry), and + * already-present levels and entries are left untouched, so calls are + * idempotent over existing ranges. * * nr =3D=3D 0: only ensure the per-domain L3 itself exists; populate is * ignored. Used to set the area up before any sub-range is known. @@ -6234,9 +6234,10 @@ static bool perdomain_l1e_needs_freeing(l1_pgentry_t= l1e) * perdomain_l1e_needs_freeing()), whereas caller-owned mappings are only * ever unmapped. */ -int create_perdomain_mapping(struct domain *d, unsigned long va, +int create_perdomain_mapping(struct vcpu *v, unsigned long va, unsigned int nr, bool populate) { + struct domain *d =3D v->domain; struct page_info *pg; l3_pgentry_t *l3tab; l2_pgentry_t *l2tab; diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 40b834e1a4..50f2d1284a 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -313,7 +313,7 @@ int switch_compat(struct domain *d) =20 static int pv_create_gdt_ldt_l1tab(struct vcpu *v) { - return create_perdomain_mapping(v->domain, GDT_VIRT_START(v), + return create_perdomain_mapping(v, GDT_VIRT_START(v), 1U << GDT_LDT_VCPU_SHIFT, false); } =20 diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index aa74acec82..bc7e49f4de 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -732,7 +732,7 @@ void __init zap_low_mappings(void) =20 int setup_compat_arg_xlat(struct vcpu *v) { - return create_perdomain_mapping(v->domain, ARG_XLAT_START(v), + return create_perdomain_mapping(v, ARG_XLAT_START(v), PFN_UP(COMPAT_ARG_XLAT_SIZE), true); } =20 --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342571230655.4845705512018; Wed, 2 Sep 2026 02:49:31 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405531.1639040 (Exim 4.92) (envelope-from ) id 1x1hag-0007I6-F5; Wed, 02 Sep 2026 09:49:10 +0000 Received: by outflank-mailman (output) from mailman id 1405531.1639040; Wed, 02 Sep 2026 09:49:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hag-0007Fv-8N; Wed, 02 Sep 2026 09:49:10 +0000 Received: by outflank-mailman (input) for mailman id 1405531; Wed, 02 Sep 2026 09:49:08 +0000 Received: from mx.expurgate.net ([194.145.224.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hae-00071v-Ip for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:49:08 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1had-00HNrp-Vs for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [10.42.69.7] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f112-2eae-0a2a0a5409dd-0a2a4507ac00-14 for ; Wed, 02 Sep 2026 11:49:07 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-ef75cf.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-b4ea-0a2a45070019-d99ba50cfdf1-3 for ; Wed, 02 Sep 2026 11:44:07 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id F407636928FC; Wed, 2 Sep 2026 10:44:06 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 12/14] x86/spec-ctrl: introduce Address Space Isolation command line option Date: Wed, 2 Sep 2026 10:43:56 +0100 Message-ID: <20260901-asi-part2-12-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ef75cf/1788342247-A6EDEAE4-7BC14EE1/0/0 X-purgate-type: clean X-purgate-size: 10768 X-ZM-MESSAGEID: 1788342573891158500 From: Roger Pau Monn=C3=A9 Introduce the `asi=3D` command line option, and the opt_vcpu_pt_{pv,hwdom,h= vm} knobs plus the per-domain d->arch.vcpu_pt setting it controls. The option = is introduced ahead of the functionality it enables, so that the newly added code can be keyed on it from the start; all knobs currently default to off, and enabling any of them taints the boot with a "not functional, development purposes only" warning. XPTI and per-vCPU page-tables are mutually exclusive (they are different answers to the same problem, and the entry paths can only be built for one = of them at a time), so an explicit XPTI request takes precedence over vCPU-PT, per axis: xpti=3Ddom0 clears the hardware domain vCPU-PT knob (when dom0 is PV), and xpti=3Ddomu clears the PV domU one. When XPTI is left to default,= it is turned off for those domain kinds that use vCPU-PT instead. The boot log gains "ASI features for ..." lines for Dom0, HVM and PV domains, so hardware-domain-only configurations remain visible, and the XPTI line is printed unconditionally: users expecting to assert the state of XPTI should not need to derive it from the ASI configuration. Further per-mechanism tokens arrive with their mechanisms in later patches. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Added to the series Changes since the previously posted version: - Make the XPTI exclusion per-axis: an explicit xpti=3Ddom0 previously cleared only the PV domU vCPU-PT knob, leaving dom0 with both XPTI and vCPU-PT enabled. - Include the hardware domain in the development warning and the boot log summary. - Print the XPTI status line unconditionally. - Make the opt_vcpu_pt_* knobs plain booleans preinitialised to false, dropping the late -1 resolution. - Documentation: mention possible protection against unmitigated attacks, use {pv,hvm} notation in the synopsis, and state that pv=3D/hvm=3D do not affect the hardware domain. - Rewrite the commit message. --- docs/misc/xen-command-line.pandoc | 24 ++++++ xen/arch/x86/include/asm/domain.h | 3 + xen/arch/x86/include/asm/spec_ctrl.h | 2 + xen/arch/x86/spec_ctrl.c | 107 ++++++++++++++++++++++++++- 4 files changed, 134 insertions(+), 2 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index 1c711fa980..834f6c57f2 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -202,6 +202,30 @@ to appropriate auditing by Xen. Argo is disabled by d= efault. This option is disabled by default, to protect domains from a DoS by a buggy or malicious other domain spamming the ring. =20 +### asi (x86) +> `=3D List of [ , pv=3D, hvm=3D, +> vcpu-pt=3D | vcpu-pt=3D{pv,hvm}=3D ]` + +> Default: `false` + +Offers control over whether the hypervisor will engage in Address Space +Isolation, by not having potentially sensitive information permanently map= ped +in the VMM page-tables. Using this option might avoid the need to apply +mitigations for certain speculative related attacks, at the cost of mapping +sensitive information on-demand. It might also offer some protection agai= nst +unmitigated speculation-related attacks. + +* `pv=3D` and `hvm=3D` sub-options allow enabling for specific guest types= ; they + do not affect the hardware domain, which follows the whole-feature forms + (the plain boolean, or an un-suffixed `vcpu-pt=3D`). + +**WARNING: manual de-selection of enabled options will invalidate any +protection offered by the feature. The fine grained options provided below +are meant to be used for debugging purposes only.** + +* `vcpu-pt` ensures each vCPU uses a unique top-level page-table and sets = up + a virtual address space region to map memory on a per-vCPU basis. + ### asid (x86) > `=3D ` =20 diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 38df5c376e..fdf7b205ea 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -472,6 +472,9 @@ struct arch_domain /* Don't unconditionally inject #GP for unhandled MSRs. */ bool msr_relaxed; =20 + /* Use a per-vCPU root pt, and switch per-domain slot to per-vCPU. */ + bool vcpu_pt; + /* Emulated devices enabled bitmap. */ uint32_t emulation_flags; } __cacheline_aligned; diff --git a/xen/arch/x86/include/asm/spec_ctrl.h b/xen/arch/x86/include/as= m/spec_ctrl.h index 8f82533c41..770c24f6a0 100644 --- a/xen/arch/x86/include/asm/spec_ctrl.h +++ b/xen/arch/x86/include/asm/spec_ctrl.h @@ -87,6 +87,8 @@ extern uint8_t default_scf; =20 extern int8_t opt_xpti_hwdom, opt_xpti_domu; =20 +extern bool opt_vcpu_pt_pv, opt_vcpu_pt_hwdom, opt_vcpu_pt_hvm; + extern bool cpu_has_bug_l1tf; extern int8_t opt_pv_l1tf_hwdom, opt_pv_l1tf_domu; extern bool opt_bp_spec_reduce; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index bc8538a56f..55a02212a9 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -86,6 +86,14 @@ bool __ro_after_init opt_bp_spec_reduce =3D true; =20 static bool __initdata opt_ibpb_alt; =20 +/* + * Use a per-vCPU root page-table and switch the per-domain slot to per-vC= PU. + * Off by default until the feature is complete. + */ +bool __ro_after_init opt_vcpu_pt_hvm; +bool __ro_after_init opt_vcpu_pt_hwdom; +bool __ro_after_init opt_vcpu_pt_pv; + static int __init cf_check parse_spec_ctrl(const char *s) { const char *ss; @@ -383,6 +391,18 @@ int8_t __ro_after_init opt_xpti_domu =3D -1; =20 static __init void xpti_init_default(void) { + if ( !opt_dom0_pvh && opt_xpti_hwdom =3D=3D 1 && opt_vcpu_pt_hwdom ) + { + printk(XENLOG_ERR + "XPTI incompatible with per-vCPU page-tables, disabling Dom= 0 vCPU-PT\n"); + opt_vcpu_pt_hwdom =3D false; + } + if ( opt_xpti_domu =3D=3D 1 && opt_vcpu_pt_pv ) + { + printk(XENLOG_ERR + "XPTI incompatible with per-vCPU page-tables, disabling PV = DomU vCPU-PT\n"); + opt_vcpu_pt_pv =3D false; + } if ( (boot_cpu_data.vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON)) || cpu_has_rdcl_no ) { @@ -394,9 +414,9 @@ static __init void xpti_init_default(void) else { if ( opt_xpti_hwdom < 0 ) - opt_xpti_hwdom =3D 1; + opt_xpti_hwdom =3D !opt_vcpu_pt_hwdom; if ( opt_xpti_domu < 0 ) - opt_xpti_domu =3D 1; + opt_xpti_domu =3D !opt_vcpu_pt_pv; } } =20 @@ -487,6 +507,66 @@ static int __init cf_check parse_pv_l1tf(const char *s) } custom_param("pv-l1tf", parse_pv_l1tf); =20 +static int __init cf_check parse_asi(const char *s) +{ + const char *ss; + int val, rc =3D 0; + + /* Interpret 'asi' alone in its positive boolean form. */ + if ( *s =3D=3D '\0' ) + opt_vcpu_pt_pv =3D opt_vcpu_pt_hwdom =3D opt_vcpu_pt_hvm =3D true; + + do { + ss =3D strchr(s, ','); + if ( !ss ) + ss =3D strchr(s, '\0'); + + val =3D parse_bool(s, ss); + switch ( val ) + { + case 0: + case 1: + opt_vcpu_pt_pv =3D opt_vcpu_pt_hwdom =3D opt_vcpu_pt_hvm =3D v= al; + break; + + default: + if ( (val =3D parse_boolean("pv", s, ss)) >=3D 0 ) + opt_vcpu_pt_pv =3D val; + else if ( (val =3D parse_boolean("hvm", s, ss)) >=3D 0 ) + opt_vcpu_pt_hvm =3D val; + else if ( (val =3D parse_boolean("vcpu-pt", s, ss)) !=3D -1 ) + { + switch ( val ) + { + case 1: + case 0: + opt_vcpu_pt_pv =3D opt_vcpu_pt_hvm =3D opt_vcpu_pt_hwd= om =3D val; + break; + + case -2: + s +=3D strlen("vcpu-pt=3D"); + if ( (val =3D parse_boolean("pv", s, ss)) >=3D 0 ) + opt_vcpu_pt_pv =3D val; + else if ( (val =3D parse_boolean("hvm", s, ss)) >=3D 0= ) + opt_vcpu_pt_hvm =3D val; + else + default: + rc =3D -EINVAL; + break; + } + } + else if ( *s ) + rc =3D -EINVAL; + break; + } + + s =3D ss + 1; + } while ( *ss ); + + return rc; +} +custom_param("asi", parse_asi); + static void __init print_details(enum ind_thunk thunk) { unsigned int _7d0 =3D 0, _7d2 =3D 0, e8b =3D 0, e21a =3D 0, e21c =3D 0= , max =3D 0, tmp; @@ -680,6 +760,20 @@ static void __init print_details(enum ind_thunk thunk) opt_pv_l1tf_hwdom ? "enabled" : "disabled", opt_pv_l1tf_domu ? "enabled" : "disabled"); #endif + + printk(" ASI features for Dom0:%s%s\n", + opt_vcpu_pt_hwdom ? "" : = " None", + opt_vcpu_pt_hwdom ? " vCPU-PT" : = ""); +#ifdef CONFIG_HVM + printk(" ASI features for HVM VMs:%s%s\n", + opt_vcpu_pt_hvm ? "" : = " None", + opt_vcpu_pt_hvm ? " vCPU-PT" : = ""); +#endif +#ifdef CONFIG_PV + printk(" ASI features for PV VMs:%s%s\n", + opt_vcpu_pt_pv ? "" : = " None", + opt_vcpu_pt_pv ? " vCPU-PT" : = ""); +#endif } =20 static bool __init check_smt_enabled(void) @@ -1866,6 +1960,10 @@ void spec_ctrl_init_domain(struct domain *d) if ( pv ) d->arch.pv.xpti =3D is_hardware_domain(d) ? opt_xpti_hwdom : opt_xpti_domu; + + d->arch.vcpu_pt =3D is_hardware_domain(d) ? opt_vcpu_pt_hwdom + : pv ? opt_vcpu_pt_pv + : opt_vcpu_pt_hvm; } =20 void __init init_speculation_mitigations(void) @@ -2158,6 +2256,11 @@ void __init init_speculation_mitigations(void) hw_smt_enabled && default_xen_spec_ctrl ) setup_force_cpu_cap(X86_FEATURE_SC_MSR_IDLE); =20 + if ( opt_vcpu_pt_pv || opt_vcpu_pt_hwdom || opt_vcpu_pt_hvm ) + warning_add( + "Address Space Isolation is not functional, this option is\n" + "intended to be used only for development purposes.\n"); + xpti_init_default(); =20 l1tf_calculations(); --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342572481768.8529580549855; Wed, 2 Sep 2026 02:49:32 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405532.1639062 (Exim 4.92) (envelope-from ) id 1x1haj-00088O-Vq; Wed, 02 Sep 2026 09:49:13 +0000 Received: by outflank-mailman (output) from mailman id 1405532.1639062; Wed, 02 Sep 2026 09:49:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1haj-00088C-S1; Wed, 02 Sep 2026 09:49:13 +0000 Received: by outflank-mailman (input) for mailman id 1405532; Wed, 02 Sep 2026 09:49:12 +0000 Received: from mx.expurgate.net ([194.145.224.20]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hai-00085T-NF for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:49:12 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hai-003afr-3j for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:49:12 +0200 Received: from [10.42.69.2] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f10a-bab6-0a2a0a5309dd-0a2a45029afe-34 for ; Wed, 02 Sep 2026 11:49:12 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-720697.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-6ca4-0a2a45020019-d99ba50ce8c0-7 for ; Wed, 02 Sep 2026 11:44:07 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 2D56736928FE; Wed, 2 Sep 2026 10:44:07 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: George Dunlap , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 13/14] x86/pv: clear the XPTI root_pgt per-domain slot on context-switch out Date: Wed, 2 Sep 2026 10:43:57 +0100 Message-ID: <20260901-asi-part2-13-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-720697/1788342247-313CC2AC-CB9C2D3F/0/0 X-purgate-type: clean X-purgate-size: 3045 X-ZM-MESSAGEID: 1788342573831158500 Content-Type: text/plain; charset="utf-8" From: George Dunlap XPTI maintains a per-pCPU root page table (root_pgt): a restricted L4, with Xen largely unmapped, that an XPTI domain's guest context actually runs on. Its guest mappings are copied in on the way back to guest context; its per-domain slot is written by paravirt_ctxt_switch_to(), so that it follows whichever domain is scheduled onto the pCPU. Nothing ever clears the slot, however. When the pCPU switches from an XPTI PV vCPU to one that does not refresh the slot -- an HVM vCPU, or the idle vCPU after a lazy state flush -- the last PV domain's per-domain L3 remains referenced from root_pgt. The reference isn't cleared on domain destruction, so could even point to an already freed page. In theory, that slot should never be walked in this state; but it's just generally safer not to leave dangling references around. Consider that cleanup_cpu_root_pgt() frees pagetables by walking root_pgt on CPU offline. Currently it correctly leaves slot 260 alone; but one could easily imagine a mistake in which slot 260 is walked erroneously. Clear the slot in paravirt_ctxt_switch_from(), making the maintenance a pair: cleared on the way out, installed on the way in. The slot is now populated only while the vCPU using it runs, and an erroneous walk in any other state faults cleanly. This change also makes robust behavior simpler when we add per-vCPU areas in a subsequent patch. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: George Dunlap --- Changes in v2: - New patch. --- xen/arch/x86/domain.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 1f75d44fe0..79555e6964 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -2006,8 +2006,18 @@ static void save_segments(struct vcpu *v) =20 void cf_check paravirt_ctxt_switch_from(struct vcpu *v) { + root_pgentry_t *root_pgt =3D this_cpu(root_pgt); + save_segments(v); =20 + /* + * Clear the XPTI per-domain slot: it is installed on the way in by + * paravirt_ctxt_switch_to(), and must not linger while another vCPU + * runs. + */ + if ( root_pgt ) + root_pgt[root_table_offset(PERDOMAIN_VIRT_START)] =3D l4e_empty(); + /* * Disable debug breakpoints. We do this aggressively because if we sw= itch * to an HVM guest we may load DR0-DR3 with values that can cause #DE @@ -2022,6 +2032,12 @@ void cf_check paravirt_ctxt_switch_to(struct vcpu *v) { root_pgentry_t *root_pgt =3D this_cpu(root_pgt); =20 + /* + * If XPTI is active, install the incoming domain's per-domain area + * in the per-domain slot of the L4 we run on while in guest mode. + * The slot was cleared on the way out (see + * paravirt_ctxt_switch_from()). + */ if ( root_pgt ) root_pgt[root_table_offset(PERDOMAIN_VIRT_START)] =3D l4e_from_page(v->domain->arch.perdomain_l3_pg, --=20 2.55.0 From nobody Thu Sep 3 07:03:32 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=umich.edu Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1788342383643831.0156923266923; Wed, 2 Sep 2026 02:46:23 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1405508.1639008 (Exim 4.92) (envelope-from ) id 1x1hXm-0006BC-2B; Wed, 02 Sep 2026 09:46:10 +0000 Received: by outflank-mailman (output) from mailman id 1405508.1639008; Wed, 02 Sep 2026 09:46:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hXl-0006B0-Vp; Wed, 02 Sep 2026 09:46:09 +0000 Received: by outflank-mailman (input) for mailman id 1405508; Wed, 02 Sep 2026 09:46:08 +0000 Received: from mx.expurgate.net ([194.145.224.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1x1hXk-0006AZ-EU for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 09:46:08 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1x1hXj-00HNId-RZ for xen-devel@lists.xenproject.org; Wed, 02 Sep 2026 11:46:07 +0200 Received: from [10.42.69.4] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a97f056-2eae-0a2a0a5409dd-0a2a4504b044-42 for ; Wed, 02 Sep 2026 11:46:07 +0200 Received: from [217.155.165.12] (helo=Georges-MacBook-Pro-2.fritz.box) by tlsNG-ebf023.mxtls.expurgate.net with ESMTP (eXpurgate 4.57.1) (envelope-from ) id 6a97efe6-b57f-0a2a45040019-d99ba50cfba4-3 for ; Wed, 02 Sep 2026 11:44:07 +0200 Received: by Georges-MacBook-Pro-2.fritz.box (Postfix, from userid 501) id 532A23692902; Wed, 2 Sep 2026 10:44:07 +0100 (BST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; none From: George Dunlap To: xen-devel@lists.xenproject.org Cc: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Alejandro Vallejo , Teddy Astie , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , George Dunlap Subject: [PATCH v2 14/14] x86/mm: introduce per-vCPU L3 page-table Date: Wed, 2 Sep 2026 10:43:58 +0100 Message-ID: <20260901-asi-part2-14-ecc269f268b7@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> References: <20260901-asi-part2-0-ecc269f268b7@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-purgate-ID: tlsNG-ebf023/1788342247-C0CDFB50-70616C36/0/0 X-purgate-type: clean X-purgate-size: 25501 X-ZM-MESSAGEID: 1788342385223158500 From: Roger Pau Monn=C3=A9 The per-domain area is currently a single domain-wide structure: one L3, referenced from every root page-table associated with the domain, so every mapping in it is visible to every vCPU of the domain. Its contents are already laid out in per-vCPU slices (each vCPU's GDT/LDT window, COMPAT_ARG_XLAT pages, and mapcache entries), but the visibility is domain-wide. Meanwhile, much of the per-vCPU state Xen maintains -- the VMCB, the VMX MSR load/save areas, FPU/XSAVE state -- lives in always-mapped memory, and so do the pCPU stacks. Allow the "per-domain" area to be per-vCPU instead ("VCPU-PT"). This will immediately isolate the existing per-vCPU mappings from other running vCPUs on HVM domains (PV SMP for VCPU-PT requires further work; see below). We will later build on this, adding per-vCPU mapped areas (into which we can put vCPU state currently in the xenheap, mentioned above); per-vCPU mapcaches (which will eventually allow us to remove domheap pages from the direct map), and finally transient mappings of the pCPU stack on which the vCPU is currently running. Add pervcpu_l3_pg to the arch_vcpu struct, to correspond to the perdomain_l3_pg in the domain struct. (We retain both so that we can switch between per-domain and per-vCPU on a domain-by-domain basis.) In {create,populate,destroy}_perdomain_mapping(), if d->arch.vcpu_pt, use pervcpu_l3_pg as the per-domain L3 (allocating it for a vCPU if it's NULL, just as we allocate for a domain in !vcpu_pt mode); otherwise, use perdomain_l3_pg. Introduce a helper, perdomain_l3(), to consistently choose the correct one. Introduce free_pervcpu_mappings() to free this tree, called from arch_vcpu_destroy() on normal teardown. Since the vcpu structure holds the only reference to pervcpu_l3_pg, arch_vcpu_create() must also call it on its error paths: nothing else records the allocation once the vcpu struct is torn down. The domain-wide free_perdomain_mappings() is unchanged and keeps covering non-vCPU-PT domains. Modify init_xen_l4_slots() to take a vCPU, and use perdomain_l3() to select the value to install in slot 260. Most callers have the specific vCPU in hand (the HVM monitor tables, setup_compat_l4(), PV shadow L4s). Note that this includes dom0_construct(), since at that point we're actually building vCPU 0, so passing in d->vcpu[0] is exactly what we want. In promote_l4_table() we pass in d->vcpu[0]. This is correct without vCPU-PT, where every vCPU selects the same domain-wide L3. With vCPU-PT this is a temporary arrangement: the promoted L4 carries vCPU 0's L3 until the per-pCPU shadow L4 -- which supersedes promoted L4s as what the CPU actually runs on -- arrives in the series (see the SMP note below). Since slot 260 is now keyed off d->vcpu[0], promote_l4_table() refuses (-EINVAL) a domain that has no vCPUs yet, as can happen if a toolstack pins page tables before creating vCPUs. paravirt_ctxt_switch_to() now installs the XPTI root_pgt per-domain slot only when the domain has a domain-wide perdomain area to install. XPTI and vCPU-PT are mutually exclusive (xpti_init_default() disables vCPU-PT if both are explicitly requested, and each defaults off when the other is on) -- so the vCPU-PT slot stays empty, as paravirt_ctxt_switch_from() left it. vCPU-PT is not currently implemented for shadow paging. L4 shadows are currently per-domain objects shared by all vCPUs shadowing the same guest root, just as non-ASI non-shadow PV L4s are. Enabling vCPU-PT for PV shadow guests would require adding vCPU-PT functionality along all the shadow paths, which is outside the scope of the current work. This is guarded on every path that can turn shadow on for a PV domain: paging_domctl() refuses shadow/log-dirty ops (xl save/migrate) for such domains; dom0=3Dshadow is ignored with a warning when dom0 uses vCPU-PT; and shadow_one_bit_enable() refuses the mode with -EOPNOTSUPP. The PV L1TF mitigation cannot be refused up front: it acts at runtime, when a guest installs a not-present PTE whose address is unsafe, by forcing the domain into shadow mode -- which is what a vCPU-PT domain cannot currently have. No special handling is needed, though: pv_l1tf_check_pte() refuses the PTE write and schedules the shadowing tasklet as usual; the tasklet's shadow_one_bit_enable() call fails with -EOPNOTSUPP like any other enable failure; and the tasklet's existing error handling crashes the domain. That is the right disposition -- the entry being installed is precisely what the mitigation exists to catch, so continuing unmitigated is not an option -- and it matches what a build without CONFIG_SHADOW_PAGING does for the same write, with a log trail showing the mitigation was attempted and could not be enabled. Hardware without the erratum is unaffected, the mitigation being off there by default. Note SMP vCPU-PT PV guests are not yet functional at this point in the series: promoted guest L4s embed vCPU#0's L3 in slot 260 for all vCPUs; the per-pCPU shadow L4 that gives each vCPU its own slot 260 arrives with the guest_root_pt and per-pCPU-L4 patches later in the series. HVM vCPU-PT guests are fully functional, SMP included: monitor tables are already per-vCPU, so every HVM vCPU's root carries its own L3 from creation. Signed-off-by: Roger Pau Monn=C3=A9 Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8 Signed-off-by: George Dunlap --- Changes in v2: - Added to the series Changes since the previously posted version: - Expand the commit message with the motivation and the design rationale. - pv-l1tf: let the mitigation's shadowing request fail in shadow_one_bit_enable() and rely on the tasklet's existing failure handling to crash the domain, rather than special-casing vCPU-PT in pv_l1tf_check_pte() or forcing the mitigation off at boot (an earlier revision did the latter, silently withdrawing a protection that is on by default on affected hardware). - Keep domain-wide freeing intact and introduce a vCPU-scoped free_pervcpu_mappings() instead of re-scoping free_perdomain_mappings(); fix the arch_vcpu_create() error-path leaks of a partially built per-vCPU hierarchy. - Exclude PV shadow for vCPU-PT domains on all enable paths: refuse shadow/log-dirty paging_domctl() ops (gate moved here from the later per-pCPU-L4 patch so hazard and gate land together), ignore dom0=3Dshadow with a warning, and refuse the mode (-EOPNOTSUPP) in shadow_one_bit_enable(). - Add a perdomain_l3() helper for the root selection, rather than open-coding the vcpu_pt choice (and testing both root pointers) at each site. - Guard promote_l4_table() against vCPU-less domains. - Install the XPTI root_pgt per-domain slot only when the domain has a perdomain L3; the posted version computed an L4E from the NULL pointer for vCPU-PT domains. (A new preparatory patch pairs the slot's maintenance with a switch-out clear.) - Do not log the refusal for XEN_DOMCTL_SHADOW_OP_OFF. Turning paging off is the de-facto "make sure it is off" interface: the save path issues it unconditionally as best-effort cleanup and discards the result, so every save of a PV domain otherwise printed a hypervisor error for an operation nothing was asking to succeed. --- xen/arch/x86/domain.c | 21 +++++--- xen/arch/x86/include/asm/domain.h | 12 +++++ xen/arch/x86/include/asm/mm.h | 3 +- xen/arch/x86/mm.c | 87 +++++++++++++++++++++++-------- xen/arch/x86/mm/hap/hap.c | 2 +- xen/arch/x86/mm/paging.c | 14 +++++ xen/arch/x86/mm/shadow/common.c | 11 ++++ xen/arch/x86/mm/shadow/hvm.c | 2 +- xen/arch/x86/mm/shadow/multi.c | 2 +- xen/arch/x86/pv/dom0_build.c | 8 ++- xen/arch/x86/pv/domain.c | 2 +- 11 files changed, 126 insertions(+), 38 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 79555e6964..3e571272d8 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -513,7 +513,7 @@ int arch_vcpu_create(struct vcpu *v) =20 rc =3D mapcache_vcpu_init(v); if ( rc ) - return rc; + goto fail_early; =20 if ( !is_idle_domain(d) ) { @@ -525,12 +525,12 @@ int arch_vcpu_create(struct vcpu *v) */ rc =3D create_perdomain_mapping(v, PERDOMAIN_VIRT_START, 0, false); if ( rc ) - return rc; + goto fail_early; =20 paging_vcpu_init(v); =20 if ( (rc =3D vcpu_init_fpu(v)) !=3D 0 ) - return rc; + goto fail_early; =20 vmce_init_vcpu(v); =20 @@ -578,6 +578,8 @@ int arch_vcpu_create(struct vcpu *v) vcpu_destroy_fpu(v); xfree(v->arch.msrs); v->arch.msrs =3D NULL; + fail_early: + free_pervcpu_mappings(v); =20 return rc; } @@ -598,6 +600,8 @@ void arch_vcpu_destroy(struct vcpu *v) pv_vcpu_destroy(v); else ASSERT_UNREACHABLE(); + + free_pervcpu_mappings(v); } =20 int arch_sanitise_domain_config(struct xen_domctl_createdomain *config) @@ -2033,12 +2037,13 @@ void cf_check paravirt_ctxt_switch_to(struct vcpu *= v) root_pgentry_t *root_pgt =3D this_cpu(root_pgt); =20 /* - * If XPTI is active, install the incoming domain's per-domain area - * in the per-domain slot of the L4 we run on while in guest mode. - * The slot was cleared on the way out (see - * paravirt_ctxt_switch_from()). + * If XPTI is active and the domain has a domain-wide perdomain area, + * install it in the per-domain slot of the L4 we run on while in + * guest mode. vCPU-PT domains have none (they don't use the XPTI + * machinery); their slot stays as paravirt_ctxt_switch_from() left + * it: empty. */ - if ( root_pgt ) + if ( root_pgt && v->domain->arch.perdomain_l3_pg ) root_pgt[root_table_offset(PERDOMAIN_VIRT_START)] =3D l4e_from_page(v->domain->arch.perdomain_l3_pg, __PAGE_HYPERVISOR_RW); diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index fdf7b205ea..5c90d7627f 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -330,6 +330,11 @@ struct monitor_write_data { =20 struct arch_domain { + /* + * Domain-wide L3 page-table for the L4 per-domain slot, used when + * the domain does not use per-vCPU page-tables (!d->arch.vcpu_pt). + * NULL otherwise (see v->arch.pervcpu_l3_pg and perdomain_l3()). + */ struct page_info *perdomain_l3_pg; =20 /* I/O-port admin-specified access capabilities. */ @@ -678,6 +683,13 @@ struct arch_vcpu =20 struct vcpu_msrs *msrs; =20 + /* + * Per-vCPU L3 page-table for the L4 per-domain slot, used when the + * domain uses per-vCPU page-tables (d->arch.vcpu_pt). NULL + * otherwise (see d->arch.perdomain_l3_pg and perdomain_l3()). + */ + struct page_info *pervcpu_l3_pg; + struct { bool next_interrupt_enabled; } monitor; diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 97924a639b..acb553df48 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -370,7 +370,7 @@ int devalidate_page(struct page_info *page, unsigned lo= ng type, =20 void init_xen_pae_l2_slots(l2_pgentry_t *l2t, const struct domain *d); void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, - const struct domain *d, mfn_t sl4mfn, bool ro_mpt); + const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt); bool fill_ro_mpt(mfn_t mfn); void zap_ro_mpt(mfn_t mfn); =20 @@ -608,6 +608,7 @@ void populate_perdomain_mapping(const struct vcpu *v, u= nsigned long va, void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va, unsigned int nr); void free_perdomain_mappings(struct domain *d); +void free_pervcpu_mappings(struct vcpu *v); =20 void __iomem *ioremap_wc(paddr_t pa, size_t len); =20 diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 6dfd75475a..48266b1d27 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -1638,20 +1638,32 @@ static int promote_l3_table(struct page_info *page) } #endif /* CONFIG_PV */ =20 +/* + * The root of the per-domain area in use by @v: the vCPU's own L3 for a + * vCPU-PT domain, the domain-wide one otherwise. + */ +static struct page_info *perdomain_l3(const struct vcpu *v) +{ + const struct domain *d =3D v->domain; + + return d->arch.vcpu_pt ? v->arch.pervcpu_l3_pg : d->arch.perdomain_l3_= pg; +} + /* * Fill an L4 with Xen entries. * * This function must write all ROOT_PAGETABLE_PV_XEN_SLOTS, to clobber any * values a guest may have left there from promote_l4_table(). * - * l4t, l4mfn, and d are mandatory, but l4mfn doesn't need to be the mfn u= nder + * l4t, l4mfn, and v are mandatory, but l4mfn doesn't need to be the mfn u= nder * *l4t. All other parameters are optional and will either fill or zero t= he * appropriate slots. Pagetables not shared with guests will gain the * extended directmap. */ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, - const struct domain *d, mfn_t sl4mfn, bool ro_mpt) + const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt) { + const struct domain *d =3D v->domain; /* * PV vcpus need a shortened directmap. HVM and Idle vcpus get the fu= ll * directmap. @@ -1679,7 +1691,7 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn, =20 /* Slot 260: Per-domain mappings. */ l4t[l4_table_offset(PERDOMAIN_VIRT_START)] =3D - l4e_from_page(d->arch.perdomain_l3_pg, __PAGE_HYPERVISOR_RW); + l4e_from_page(perdomain_l3(v), __PAGE_HYPERVISOR_RW); =20 /* Slot 4: Per-domain mappings mirror. */ BUILD_BUG_ON(IS_ENABLED(CONFIG_PV32) && @@ -1755,11 +1767,17 @@ static int promote_l4_table(struct page_info *page) { struct domain *d =3D page_get_owner(page); mfn_t l4mfn =3D page_to_mfn(page); - l4_pgentry_t *pl4e =3D map_domain_page(l4mfn); + l4_pgentry_t *pl4e; unsigned int i; int rc =3D 0; unsigned int partial_flags =3D page->partial_flags; =20 + /* init_xen_l4_slots() needs a vCPU to key the per-domain slot off. */ + if ( unlikely(!d->vcpu || !d->vcpu[0]) ) + return -EINVAL; + + pl4e =3D map_domain_page(l4mfn); + for ( i =3D page->nr_validated_ptes; i < L4_PAGETABLE_ENTRIES; i++, partial_flags =3D 0 ) { @@ -1834,8 +1852,15 @@ static int promote_l4_table(struct page_info *page) =20 if ( !rc ) { + /* + * Use vCPU#0 unconditionally. When not running with ASI enabled = the + * per-domain table is shared between all vCPUs, so it doesn't mat= ter + * which vCPU gets passed to init_xen_l4_slots(). When running wi= th + * ASI enabled this L4 will not be used, as a shadow per-vCPU L4 is + * used instead. + */ init_xen_l4_slots(pl4e, l4mfn, - d, INVALID_MFN, VM_ASSIST(d, m2p_strict)); + d->vcpu[0], INVALID_MFN, VM_ASSIST(d, m2p_strict= )); atomic_inc(&d->arch.pv.nr_l4_pages); } unmap_domain_page(pl4e); @@ -6238,7 +6263,7 @@ int create_perdomain_mapping(struct vcpu *v, unsigned= long va, unsigned int nr, bool populate) { struct domain *d =3D v->domain; - struct page_info *pg; + struct page_info *pg, *l3_pg =3D perdomain_l3(v); l3_pgentry_t *l3tab; l2_pgentry_t *l2tab; l1_pgentry_t *l1tab; @@ -6247,14 +6272,17 @@ int create_perdomain_mapping(struct vcpu *v, unsign= ed long va, ASSERT(va >=3D PERDOMAIN_VIRT_START && va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); =20 - if ( !d->arch.perdomain_l3_pg ) + if ( !l3_pg ) { pg =3D alloc_domheap_page(d, MEMF_no_owner); if ( !pg ) return -ENOMEM; l3tab =3D __map_domain_page(pg); clear_page(l3tab); - d->arch.perdomain_l3_pg =3D pg; + if ( d->arch.vcpu_pt ) + v->arch.pervcpu_l3_pg =3D pg; + else + d->arch.perdomain_l3_pg =3D pg; if ( !nr ) { unmap_domain_page(l3tab); @@ -6264,7 +6292,7 @@ int create_perdomain_mapping(struct vcpu *v, unsigned= long va, else if ( !nr ) return 0; else - l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); + l3tab =3D __map_domain_page(l3_pg); =20 ASSERT(!l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1))); =20 @@ -6359,7 +6387,7 @@ void populate_perdomain_mapping(const struct vcpu *v,= unsigned long va, l1_pgentry_t *l1tab =3D NULL, *pl1e; const l3_pgentry_t *l3tab; const l2_pgentry_t *l2tab; - struct domain *d =3D v->domain; + struct page_info *l3_pg; unsigned long irq_flags; =20 ASSERT(va >=3D PERDOMAIN_VIRT_START && @@ -6401,7 +6429,8 @@ void populate_perdomain_mapping(const struct vcpu *v,= unsigned long va, return; } =20 - BUG_ON(!d->arch.perdomain_l3_pg); + l3_pg =3D perdomain_l3(v); + BUG_ON(!l3_pg); =20 /* * Slow path: walk v's per-domain page-table pages. All mappings are @@ -6413,7 +6442,7 @@ void populate_perdomain_mapping(const struct vcpu *v,= unsigned long va, */ local_irq_save(irq_flags); =20 - l3tab =3D __map_domain_page_irqoff(d->arch.perdomain_l3_pg); + l3tab =3D __map_domain_page_irqoff(l3_pg); =20 /* * Missing page-table structure is a hypervisor bug: there is no safe @@ -6461,13 +6490,13 @@ void destroy_perdomain_mapping(const struct vcpu *v= , unsigned long va, unsigned int nr) { const l3_pgentry_t *l3tab, *pl3e; - const struct domain *d =3D v->domain; + struct page_info *l3_pg =3D perdomain_l3(v); =20 ASSERT(va >=3D PERDOMAIN_VIRT_START && va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS)); ASSERT(!nr || !l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1))); =20 - if ( !d->arch.perdomain_l3_pg ) + if ( !l3_pg ) return; =20 if ( likely(this_cpu(pgtable_vcpu) =3D=3D v) ) @@ -6490,7 +6519,7 @@ void destroy_perdomain_mapping(const struct vcpu *v, = unsigned long va, return; } =20 - l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); + l3tab =3D __map_domain_page(l3_pg); pl3e =3D l3tab + l3_table_offset(va); =20 if ( l3e_get_flags(*pl3e) & _PAGE_PRESENT ) @@ -6529,16 +6558,11 @@ void destroy_perdomain_mapping(const struct vcpu *v= , unsigned long va, unmap_domain_page(l3tab); } =20 -void free_perdomain_mappings(struct domain *d) +static void free_perdomain_l3(struct page_info *l3pg) { - l3_pgentry_t *l3tab; + l3_pgentry_t *l3tab =3D __map_domain_page(l3pg); unsigned int i; =20 - if ( !d->arch.perdomain_l3_pg ) - return; - - l3tab =3D __map_domain_page(d->arch.perdomain_l3_pg); - for ( i =3D 0; i < PERDOMAIN_SLOTS; ++i) if ( l3e_get_flags(l3tab[i]) & _PAGE_PRESENT ) { @@ -6571,10 +6595,27 @@ void free_perdomain_mappings(struct domain *d) } =20 unmap_domain_page(l3tab); - free_domheap_page(d->arch.perdomain_l3_pg); + free_domheap_page(l3pg); +} + +void free_perdomain_mappings(struct domain *d) +{ + if ( !d->arch.perdomain_l3_pg ) + return; + + free_perdomain_l3(d->arch.perdomain_l3_pg); d->arch.perdomain_l3_pg =3D NULL; } =20 +void free_pervcpu_mappings(struct vcpu *v) +{ + if ( !v->arch.pervcpu_l3_pg ) + return; + + free_perdomain_l3(v->arch.pervcpu_l3_pg); + v->arch.pervcpu_l3_pg =3D NULL; +} + static void write_sss_token(unsigned long *ptr) { /* diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c index 0ede4181a0..aba4f77df9 100644 --- a/xen/arch/x86/mm/hap/hap.c +++ b/xen/arch/x86/mm/hap/hap.c @@ -407,7 +407,7 @@ static mfn_t hap_make_monitor_table(struct vcpu *v) m4mfn =3D page_to_mfn(pg); l4e =3D map_domain_page(m4mfn); =20 - init_xen_l4_slots(l4e, m4mfn, d, INVALID_MFN, false); + init_xen_l4_slots(l4e, m4mfn, v, INVALID_MFN, false); unmap_domain_page(l4e); =20 return m4mfn; diff --git a/xen/arch/x86/mm/paging.c b/xen/arch/x86/mm/paging.c index 14ab7defd8..ab68dfa415 100644 --- a/xen/arch/x86/mm/paging.c +++ b/xen/arch/x86/mm/paging.c @@ -675,6 +675,20 @@ int paging_domctl(struct domain *d, struct xen_domctl_= shadow_op *sc, return -EINVAL; } =20 + if ( is_pv_domain(d) && d->arch.vcpu_pt ) + { + /* + * Turning paging off is the de-facto "make sure it is off" + * interface: the save path issues it unconditionally as + * best-effort cleanup and discards the result, so logging an + * error for it is noise on every save of a PV domain. + */ + if ( sc->op !=3D XEN_DOMCTL_SHADOW_OP_OFF ) + gprintk(XENLOG_ERR, + "Paging not supported on PV domains with ASI\n"); + return -EOPNOTSUPP; + } + if ( resuming ? (d->arch.paging.preempt.dom !=3D current->domain || d->arch.paging.preempt.op !=3D sc->op) diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/commo= n.c index e30c6c49e1..b559db84b1 100644 --- a/xen/arch/x86/mm/shadow/common.c +++ b/xen/arch/x86/mm/shadow/common.c @@ -2361,6 +2361,17 @@ static int shadow_one_bit_enable(struct domain *d, u= 32 mode) return -EINVAL; } =20 + /* + * PV shadows embed the (per-vCPU) per-domain slot in L4 shadows shared + * by all vCPUs of the domain, so shadow modes are unavailable to + * domains using per-vCPU page-tables. Toolstack requests are refused + * in paging_domctl(); the pv-l1tf tasklet can still request + * PG_SH_forced at runtime, and crashes the domain when this refusal + * reaches it. + */ + if ( is_pv_domain(d) && d->arch.vcpu_pt ) + return -EOPNOTSUPP; + mode |=3D PG_SH_enable; =20 if ( d->arch.paging.total_pages < sh_min_allocation(d) ) diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c index e6fb97c4b6..0b23326214 100644 --- a/xen/arch/x86/mm/shadow/hvm.c +++ b/xen/arch/x86/mm/shadow/hvm.c @@ -760,7 +760,7 @@ mfn_t sh_make_monitor_table(const struct vcpu *v, unsig= ned int shadow_levels) * shadow-linear mapping will either be inserted below when creating * lower level monitor tables, or later in sh_update_cr3(). */ - init_xen_l4_slots(l4e, m4mfn, d, INVALID_MFN, false); + init_xen_l4_slots(l4e, m4mfn, v, INVALID_MFN, false); =20 if ( shadow_levels < 4 ) { diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c index 1ae1091acd..5b7dbc12dc 100644 --- a/xen/arch/x86/mm/shadow/multi.c +++ b/xen/arch/x86/mm/shadow/multi.c @@ -974,7 +974,7 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_t= ype) =20 BUILD_BUG_ON(sizeof(l4_pgentry_t) !=3D sizeof(shadow_l4e_t)); =20 - init_xen_l4_slots(l4t, gmfn, d, smfn, (!is_pv_32bit_domain(d) = && + init_xen_l4_slots(l4t, gmfn, v, smfn, (!is_pv_32bit_domain(d) = && VM_ASSIST(d, m2p_strict= ))); unmap_domain_page(l4t); } diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index ddeb144b06..52139cffb3 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -726,7 +726,7 @@ static int __init dom0_construct(const struct boot_doma= in *bd) l4start =3D l4tab =3D __va(mpt_alloc); mpt_alloc +=3D PAGE_SIZE; clear_page(l4tab); init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)), - d, INVALID_MFN, true); + d->vcpu[0], INVALID_MFN, true); v->arch.guest_table =3D pagetable_from_paddr(__pa(l4start)); } else @@ -1048,7 +1048,11 @@ static int __init dom0_construct(const struct boot_d= omain *bd) } =20 /* Activate shadow mode, if requested. Reuse the pv_l1tf tasklet. */ - if ( opt_dom0_shadow ) + if ( opt_dom0_shadow && d->arch.vcpu_pt ) + /* Shadow paging is incompatible with per-vCPU page-tables (ASI). = */ + printk(XENLOG_WARNING + "Ignoring dom0=3Dshadow: incompatible with per-vCPU page-ta= bles\n"); + else if ( opt_dom0_shadow ) { printk("Switching dom0 to using shadow paging\n"); tasklet_schedule(&d->arch.paging.shadow.pv_l1tf_tasklet); diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 50f2d1284a..b1d57083f2 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -127,7 +127,7 @@ static int setup_compat_l4(struct vcpu *v) mfn =3D page_to_mfn(pg); l4tab =3D map_domain_page(mfn); clear_page(l4tab); - init_xen_l4_slots(l4tab, mfn, v->domain, INVALID_MFN, false); + init_xen_l4_slots(l4tab, mfn, v, INVALID_MFN, false); unmap_domain_page(l4tab); =20 /* This page needs to look like a pagetable so that it can be shadowed= */ --=20 2.55.0