From nobody Sat Nov 23 23:50:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass header.i=dpsmith@apertussolutions.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=apertussolutions.com) ARC-Seal: i=2; a=rsa-sha256; t=1730568428; cv=pass; d=zohomail.com; s=zohoarc; b=Or1AwhNbH8Y/Yn0KMO7FuxahjYGn21/htNXxogEco6nTBKJLK7QKFnsiIrQQkNBKjCOqqucGwTYbH/kEB/QHQb2hy27lCldcLobCluUE515+4YJPZlyydic/COQSeVvjU1mZhWw7Ay4jfXEnO5JN/9J4WuS5PXUgzZKuftv+F/Y= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1730568428; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=hUGH3RDsDf1IJruHP2Kvrv3CdfGhOfh0xY63XM5AYas=; b=dlqhn1+/G7m+v/r5BEqIB43FyNGtY8yxu0o7rIxg9t6aAzOD7gGef6qsfxMtc86U1ZO6+xnReUIMPgM16XwX2lnkhqBq3oe5RoUArATPN73A95MJhVjbU+SvQxIq/FSpmaaZ1qU9gKJVnrrfxprlepJVUoCBlTM5RHwCn/n9gNQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass header.i=dpsmith@apertussolutions.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=apertussolutions.com) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 173056842865915.594922439743755; Sat, 2 Nov 2024 10:27:08 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.829575.1244584 (Exim 4.92) (envelope-from ) id 1t7Htl-0006nI-Vl; Sat, 02 Nov 2024 17:26:53 +0000 Received: by outflank-mailman (output) from mailman id 829575.1244584; Sat, 02 Nov 2024 17:26:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t7Htl-0006n9-Su; Sat, 02 Nov 2024 17:26:53 +0000 Received: by outflank-mailman (input) for mailman id 829575; Sat, 02 Nov 2024 17:26:52 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t7Htk-0005zO-8V for xen-devel@lists.xenproject.org; Sat, 02 Nov 2024 17:26:52 +0000 Received: from sender3-of-o57.zoho.com (sender3-of-o57.zoho.com [136.143.184.57]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a396f37c-993f-11ef-a0c5-8be0dac302b0; Sat, 02 Nov 2024 18:26:49 +0100 (CET) Received: by mx.zohomail.com with SMTPS id 1730568365486995.6070687004028; Sat, 2 Nov 2024 10:26:05 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a396f37c-993f-11ef-a0c5-8be0dac302b0 X-Custom-Connection: eyJyZW1vdGVpcCI6IjEzNi4xNDMuMTg0LjU3IiwiaGVsbyI6InNlbmRlcjMtb2YtbzU3LnpvaG8uY29tIn0= X-Custom-Transaction: eyJpZCI6ImEzOTZmMzdjLTk5M2YtMTFlZi1hMGM1LThiZTBkYWMzMDJiMCIsInRzIjoxNzMwNTY4NDA5LjYyODEzLCJzZW5kZXIiOiJkcHNtaXRoQGFwZXJ0dXNzb2x1dGlvbnMuY29tIiwicmVjaXBpZW50IjoieGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnIn0= ARC-Seal: i=1; a=rsa-sha256; t=1730568372; cv=none; d=zohomail.com; s=zohoarc; b=h/d0e3JJo5w/7SvCKkTsi3kCw1xMIg8Phnq0Njv6Wl0Q/9BrhL158pOX3T7pZQ+0H2UUPYcYfMZgc2gc268WkKjjkWpxUQXxNoJX2cTzijobcWKoMbeMDSTVGUxNue9K9j4GRWgdCfieDnfbhSfohV9gL4QyA2IKDHFIGg2b2lE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1730568372; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=hUGH3RDsDf1IJruHP2Kvrv3CdfGhOfh0xY63XM5AYas=; b=gZZtJEuxMXMNrVyxcAP6O2chvCiJfrc7gsDUVuXG9MI1lPLJndg5ATz/qck2kl3CaQiS3cK4WX5enpk/8nwFN2cIFk/9kVcqr6z+uGKMSDMJtMpdPBOiBquLCGRSaKmzY9qMdcm9bPurx42zCODzr59yZeghm6ZP+YAE1NTMAcM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=apertussolutions.com; spf=pass smtp.mailfrom=dpsmith@apertussolutions.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1730568372; s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Reply-To; bh=hUGH3RDsDf1IJruHP2Kvrv3CdfGhOfh0xY63XM5AYas=; b=MckSqtGkfpRDMv8MtYbE5UcHK+4ijbl57rLme/js6NhApym2d9OrnAlDLbdpVFZQ dD4mis2Fi5VWLQ/WNfERxuLCVoL+F0mNMPeG4MT1Vt8R+WCefZlrgzeIvGrh2jHQdnd ctZ7oZ9a4DOzO7S4dxVNcJXI2FK513pmX3PTqLsE= From: "Daniel P. Smith" To: xen-devel@lists.xenproject.org Cc: "Daniel P. Smith" , jason.andryuk@amd.com, christopher.w.clark@gmail.com, stefano.stabellini@amd.com, Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 04/12] x86/boot: introduce module release Date: Sat, 2 Nov 2024 13:25:43 -0400 Message-Id: <20241102172551.17233-5-dpsmith@apertussolutions.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241102172551.17233-1-dpsmith@apertussolutions.com> References: <20241102172551.17233-1-dpsmith@apertussolutions.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External X-ZohoMail-DKIM: pass (identity dpsmith@apertussolutions.com) X-ZM-MESSAGEID: 1730568430728116600 Content-Type: text/plain; charset="utf-8" A precarious approach was used to release the pages used to hold a boot mod= ule. The precariousness stemmed from the fact that in the case of PV dom0, the initrd module pages may be either mapped or explicitly copied into the dom0 address space. So to handle this situation, the PV dom0 construction code w= ill set the size of the module to zero, relying on discard_initial_images() to = skip any modules with a size of zero. A function is introduced to release a module when it is no longer needed th= at accepts a boolean parameter, free_mem, to indicate if the corresponding pag= es can be freed. To track that a module has been released, the boot module flag `released` is introduced. The previous release model was a free all at once except those of size zero= s, which would handle any unused modules passed. The new model is one of, free consumed module after usage is complete, thus unconsumed modules do not hav= e a consumer to free them. To address this, the discard_uknown_boot_modules() is introduced and called after the last module identification occurs, initrd, = to free the pages of any boot modules that are identified as not being release= d. After domain construction completes, all modules should be freed. A walk of= the boot modules is added after domain construction to validate and notify if a module is found not to have been released. Signed-off-by: Daniel P. Smith --- Changes since v7: - This is a new approach as an alternative to the `consumed` flag. --- xen/arch/x86/cpu/microcode/core.c | 4 ++ xen/arch/x86/hvm/dom0_build.c | 7 ++-- xen/arch/x86/include/asm/bootinfo.h | 2 + xen/arch/x86/include/asm/setup.h | 3 +- xen/arch/x86/pv/dom0_build.c | 20 ++-------- xen/arch/x86/setup.c | 57 +++++++++++++++++++++++------ xen/xsm/xsm_core.c | 5 +++ 7 files changed, 67 insertions(+), 31 deletions(-) diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode= /core.c index b09cf83249f6..349e857f539a 100644 --- a/xen/arch/x86/cpu/microcode/core.c +++ b/xen/arch/x86/cpu/microcode/core.c @@ -756,6 +756,10 @@ static int __init cf_check microcode_init_cache(void) return rc; } =20 + /* If raw module, we can free it now */ + if ( !opt_scan ) + release_boot_module(&bi->mods[early_mod_idx], true); + if ( !patch ) return -ENOENT; =20 diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c index 3dd913bdb029..a4ac262db463 100644 --- a/xen/arch/x86/hvm/dom0_build.c +++ b/xen/arch/x86/hvm/dom0_build.c @@ -704,6 +704,8 @@ static int __init pvh_load_kernel(struct domain *d, con= st module_t *image, return rc; } =20 + release_module(image, true); + /* * Find a RAM region big enough (and that doesn't overlap with the loa= ded * kernel) in order to load the initrd and the metadata. Note it could= be @@ -751,10 +753,9 @@ static int __init pvh_load_kernel(struct domain *d, co= nst module_t *image, last_addr +=3D len; } last_addr =3D ROUNDUP(last_addr, PAGE_SIZE); - } =20 - /* Free temporary buffers. */ - discard_initial_images(); + release_module(initrd, true); + } =20 if ( cmdline !=3D NULL ) { diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm= /bootinfo.h index 0fbf36739782..b1549d8c8f93 100644 --- a/xen/arch/x86/include/asm/bootinfo.h +++ b/xen/arch/x86/include/asm/bootinfo.h @@ -33,8 +33,10 @@ struct boot_module { /* * Module State Flags: * relocated: indicates module has been relocated in memory. + * released: indicates module's pages have been freed. */ bool relocated:1; + bool released:1; }; =20 /* diff --git a/xen/arch/x86/include/asm/setup.h b/xen/arch/x86/include/asm/se= tup.h index d7ed4f40024c..d68d37a5293b 100644 --- a/xen/arch/x86/include/asm/setup.h +++ b/xen/arch/x86/include/asm/setup.h @@ -36,12 +36,13 @@ void setup_io_bitmap(struct domain *d); extern struct boot_info xen_boot_info; =20 unsigned long initial_images_nrpages(nodeid_t node); -void discard_initial_images(void); +void release_module(const module_t *m, bool mapped); =20 struct boot_module; void *bootstrap_map_bm(const struct boot_module *bm); void *bootstrap_map(const module_t *mod); void bootstrap_unmap(void); +void release_boot_module(struct boot_module *bm, bool mapped); =20 struct rangeset; int remove_xen_ranges(struct rangeset *r); diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index cc882bee61c3..c1f44502a1ac 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -630,9 +630,7 @@ static int __init dom0_construct(struct domain *d, } memcpy(page_to_virt(page), mfn_to_virt(initrd->mod_start), initrd_len); - mpt_alloc =3D (paddr_t)initrd->mod_start << PAGE_SHIFT; - init_domheap_pages(mpt_alloc, - mpt_alloc + PAGE_ALIGN(initrd_len)); + release_module(initrd, true); initrd->mod_start =3D initrd_mfn =3D mfn_x(page_to_mfn(page)); } else @@ -640,18 +638,9 @@ static int __init dom0_construct(struct domain *d, while ( count-- ) if ( assign_pages(mfn_to_page(_mfn(mfn++)), 1, d, 0) ) BUG(); + release_module(initrd, false); } =20 - /* - * We have either: - * - Mapped the initrd directly into dom0, or - * - Copied it and freed the module. - * - * Either way, tell discard_initial_images() to not free it a seco= nd - * time. - */ - initrd->mod_end =3D 0; - iommu_memory_setup(d, "initrd", mfn_to_page(_mfn(initrd_mfn)), PFN_UP(initrd_len), &flush_flags); } @@ -839,7 +828,9 @@ static int __init dom0_construct(struct domain *d, printk("Failed to load the kernel binary\n"); goto out; } + /* All done with kernel, release the module pages */ bootstrap_unmap(); + release_module(image, true); =20 if ( UNSET_ADDR !=3D parms.virt_hypercall ) { @@ -854,9 +845,6 @@ static int __init dom0_construct(struct domain *d, init_hypercall_page(d, _p(parms.virt_hypercall)); } =20 - /* Free temporary buffers. */ - discard_initial_images(); - /* Set up start info area. */ si =3D (start_info_t *)vstartinfo_start; clear_page(si); diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index d061ece0541f..e6d2d25fd038 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -341,27 +341,55 @@ unsigned long __init initial_images_nrpages(nodeid_t = node) return nr; } =20 -void __init discard_initial_images(void) /* a.k.a. Free boot modules */ +void __init release_boot_module(struct boot_module *bm, bool free_mem) +{ + uint64_t start =3D pfn_to_paddr(bm->mod->mod_start); + uint64_t size =3D bm->mod->mod_end; + + if ( bm->released ) + { + printk(XENLOG_WARNING "Attempt second release boot module of type = %d\n", + bm->type); + return; + } + + if ( free_mem ) + init_domheap_pages(start, start + PAGE_ALIGN(size)); + + bm->released =3D true; +} + +void __init release_module(const module_t *m, bool free_mem) { struct boot_info *bi =3D &xen_boot_info; unsigned int i; =20 - for ( i =3D 0; i < bi->nr_modules; ++i ) + for ( i =3D 0; i < bi->nr_modules; i++ ) { - uint64_t start =3D pfn_to_paddr(bi->mods[i].mod->mod_start); - uint64_t size =3D bi->mods[i].mod->mod_end; + if ( bi->mods[i].mod =3D=3D m ) + release_boot_module(&bi->mods[i], free_mem); + } +} =20 - /* - * Sometimes the initrd is mapped, rather than copied, into dom0. - * Size being 0 is how we're instructed to leave the module alone. - */ - if ( size =3D=3D 0 ) +static void __init discard_unknown_boot_modules(void) +{ + struct boot_info *bi =3D &xen_boot_info; + unsigned int i, count =3D 0; + + for_each_boot_module_by_type(i, bi, BOOTMOD_UNKNOWN) + { + struct boot_module *bm =3D &bi->mods[i]; + + if ( bm =3D=3D NULL || bm->released ) continue; =20 - init_domheap_pages(start, start + PAGE_ALIGN(size)); + release_boot_module(bm, true); + count++; } =20 - bi->nr_modules =3D 0; + if ( count ) + printk(XENLOG_DEBUG "Releasing pages for uknown boot module %d\n", + count); } =20 static void __init init_idle_domain(void) @@ -2111,6 +2139,8 @@ void asmlinkage __init noreturn __start_xen(void) initrdidx); } =20 + discard_unknown_boot_modules(); + /* * We're going to setup domain0 using the module(s) that we stashed sa= fely * above our heap. The second module, if present, is an initrd ramdisk. @@ -2122,6 +2152,11 @@ void asmlinkage __init noreturn __start_xen(void) if ( !dom0 ) panic("Could not set up DOM0 guest OS\n"); =20 + /* Check and warn if any modules did not get released */ + for ( i =3D 0; i < bi->nr_modules; i++ ) + if ( !bi->mods[i].released ) + printk(XENLOG_ERR "Boot module %d not released, memory leaked"= , i); + heap_init_late(); =20 init_trace_bufs(); diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c index f255fb63bf6f..d5875599b63a 100644 --- a/xen/xsm/xsm_core.c +++ b/xen/xsm/xsm_core.c @@ -162,6 +162,11 @@ int __init xsm_multiboot_init(struct boot_info *bi) ret =3D xsm_core_init(policy_buffer, policy_size); bootstrap_unmap(); =20 + /* If the policy was loaded from a boot module, release it's memory */ + ret =3D first_boot_module_index(bi, BOOTMOD_XSM_POLICY); + if ( ret >=3D 0 && ret < bi->nr_modules ) + release_boot_module(&bi->mods[ret], true); + return 0; } #endif --=20 2.30.2