From nobody Thu Oct 2 23:48:30 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D06334F490 for ; Tue, 9 Sep 2025 14:44:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757429082; cv=none; b=b9V7dUHtkaTroD7Q6thLrVCAN+c9GNZvVBf2omWoTYecFIJ/e8jDrecxOcYJfXBVa6InzEdjCgcEMLaqSGgTvZkZrcvNAe2AB2d8Wvt0FJTjtcEMg/mvlNmumjIMzKQIekFdPiOFI1K+4784LqZwRE2Ir6cMG1NGggUKNZL/qHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757429082; c=relaxed/simple; bh=9MZqSQvw5qyTZgNHswRXw8wMGyHQiJ8VgjtL5Ns5Ug4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=owoOhyZr5K6/Tc3frs86r7eAtV0QdNgtmH/KVtYjSQLwK/prOObDC3gSP2q0AD6z7W8Pp5vfOXqzoVGtg7j7GYKGc+E/vpzVHFSLD1UmNAjQZOFJLmpHZfuEPA6WnSs6w+vQsyOlh4NAoDU3n45T6Ryxc7X3u0AJlGvlM+IzgHw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hTCMk9Ub; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hTCMk9Ub" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E1801C4CEF4; Tue, 9 Sep 2025 14:44:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1757429082; bh=9MZqSQvw5qyTZgNHswRXw8wMGyHQiJ8VgjtL5Ns5Ug4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hTCMk9Ub9Ng4v+4IqxsqNkL1aWkxpzQIFyf+EXDEwI319MXSJoe6p59xW7F5cMZer juosq20FywgDc+qbSFwXismNwfJE2hPy6oFtTPvDYl0BHz19MJFqdquTnx3V7eKfj3 QTrauj37Pq/l709vyei0pdqxuQnr69m+mdsOk7JV1+zLyX0j6w7A4Mhw6r9PlxPHEb GCV3X6k1uqS+eMqGyTddkfqGpNuljywn0mPJB+gGDW5QSeOABCXFWFMBsgEbGJtaMf HSI+sHNEbu369ei3q/LC0EL7G7DmDx94Jziq5At/SYHfPCI1OutZQS3WELmFc32DSh YhTtId0FXEHzQ== From: Pratyush Yadav To: Alexander Graf , Mike Rapoport , Changyuan Lyu , Andrew Morton , Baoquan He , Pratyush Yadav , Pasha Tatashin , Jason Gunthorpe , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Chris Li , Jason Miu , David Matlack , David Rientjes Cc: linux-kernel@vger.kernel.org, kexec@lists.infradead.org, linux-mm@kvack.org Subject: [RFC PATCH 2/4] kho: use KHO array for preserved memory bitmap serialization Date: Tue, 9 Sep 2025 16:44:22 +0200 Message-ID: <20250909144426.33274-3-pratyush@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250909144426.33274-1-pratyush@kernel.org> References: <20250909144426.33274-1-pratyush@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The preserved memory bitmap preservation creates a linked list of pages to track the bitmaps for preserved memory. Essentially, it is a scattered list of pointers grouped by folio order. Use a KHO array to hold the pointers to the bitmaps instead. This moves the burden of tracking this metadata to the KHO array layer, and makes the KHO core simpler. Currently, the bitmaps are held in chunks, which is a fixed-size array of pointers, plus some metadata including the order of the preserved folios. The KHO array holds only pointers and has no mechanism for grouping. To make the serialization format simpler, move the folio order from struct khoser_mem_chunk to struct khoser_mem_bitmap_ptr. The chunks to hold the bitmaps are not KHO-preserved since they are only used during the scratch-only phase. The same holds true with the KHO array. The pages which track the KHO array metadata are not KHO-preserved and thus are only valid during the scratch phase of the next kernel. After that, they are discarded and freed to buddy. Signed-off-by: Pratyush Yadav --- The diff is a bit hard to read. The final result can be found at https://git.kernel.org/pub/scm/linux/kernel/git/pratyush/linux.git/tree/ker= nel/kexec_handover.c?h=3Dkho-array-rfc-v1#n227 kernel/kexec_handover.c | 148 +++++++++++++++++++--------------------- 1 file changed, 69 insertions(+), 79 deletions(-) diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index ecd1ac210dbd7..26f9f5295f07d 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 #include =20 @@ -80,15 +81,13 @@ struct kho_mem_track { struct xarray orders; }; =20 -struct khoser_mem_chunk; - struct kho_serialization { struct page *fdt; struct list_head fdt_list; struct dentry *sub_fdt_dir; struct kho_mem_track track; - /* First chunk of serialized preserved memory map */ - struct khoser_mem_chunk *preserved_mem_map; + /* Serialized preserved memory map */ + struct kho_array *preserved_mem_map; }; =20 static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size= _t sz) @@ -226,11 +225,11 @@ EXPORT_SYMBOL_GPL(kho_restore_folio); =20 /* Serialize and deserialize struct kho_mem_phys across kexec * - * Record all the bitmaps in a linked list of pages for the next kernel to - * process. Each chunk holds bitmaps of the same order and each block of b= itmaps - * starts at a given physical address. This allows the bitmaps to be spars= e. The - * xarray is used to store them in a tree while building up the data struc= ture, - * but the KHO successor kernel only needs to process them once in order. + * Record all the bitmaps in a KHO array for the next kernel to process. E= ach + * bitmap stores the order of the folios and starts at a given physical ad= dress. + * This allows the bitmaps to be sparse. The xarray is used to store them = in a + * tree while building up the data structure, but the KHO successor kernel= only + * needs to process them once in order. * * All of this memory is normal kmalloc() memory and is not marked for * preservation. The successor kernel will remain isolated to the scratch = space @@ -240,118 +239,107 @@ EXPORT_SYMBOL_GPL(kho_restore_folio); =20 struct khoser_mem_bitmap_ptr { phys_addr_t phys_start; - DECLARE_KHOSER_PTR(bitmap, struct kho_mem_phys_bits *); -}; - -struct khoser_mem_chunk_hdr { - DECLARE_KHOSER_PTR(next, struct khoser_mem_chunk *); unsigned int order; - unsigned int num_elms; -}; - -#define KHOSER_BITMAP_SIZE \ - ((PAGE_SIZE - sizeof(struct khoser_mem_chunk_hdr)) / \ - sizeof(struct khoser_mem_bitmap_ptr)) - -struct khoser_mem_chunk { - struct khoser_mem_chunk_hdr hdr; - struct khoser_mem_bitmap_ptr bitmaps[KHOSER_BITMAP_SIZE]; + unsigned int __reserved; + DECLARE_KHOSER_PTR(bitmap, struct kho_mem_phys_bits *); }; =20 -static_assert(sizeof(struct khoser_mem_chunk) =3D=3D PAGE_SIZE); - -static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chu= nk, - unsigned long order) +static struct khoser_mem_bitmap_ptr *new_bitmap(phys_addr_t start, + struct kho_mem_phys_bits *bits, + unsigned int order) { - struct khoser_mem_chunk *chunk; + struct khoser_mem_bitmap_ptr *bitmap; =20 - chunk =3D kzalloc(PAGE_SIZE, GFP_KERNEL); - if (!chunk) + bitmap =3D kzalloc(sizeof(*bitmap), GFP_KERNEL); + if (!bitmap) return NULL; - chunk->hdr.order =3D order; - if (cur_chunk) - KHOSER_STORE_PTR(cur_chunk->hdr.next, chunk); - return chunk; + + bitmap->phys_start =3D start; + bitmap->order =3D order; + KHOSER_STORE_PTR(bitmap->bitmap, bits); + return bitmap; } =20 -static void kho_mem_ser_free(struct khoser_mem_chunk *first_chunk) +static void kho_mem_ser_free(struct kho_array *ka) { - struct khoser_mem_chunk *chunk =3D first_chunk; + struct khoser_mem_bitmap_ptr *elm; + struct ka_iter iter; =20 - while (chunk) { - struct khoser_mem_chunk *tmp =3D chunk; + if (!ka) + return; =20 - chunk =3D KHOSER_LOAD_PTR(chunk->hdr.next); - kfree(tmp); - } + ka_iter_init_read(&iter, ka); + ka_iter_for_each(&iter, elm) + kfree(elm); + + kho_array_destroy(ka); + kfree(ka); } =20 static int kho_mem_serialize(struct kho_serialization *ser) { - struct khoser_mem_chunk *first_chunk =3D NULL; - struct khoser_mem_chunk *chunk =3D NULL; struct kho_mem_phys *physxa; - unsigned long order; + unsigned long order, pos =3D 0; + struct kho_array *ka =3D NULL; + struct ka_iter iter; + + ka =3D kzalloc(sizeof(*ka), GFP_KERNEL); + if (!ka) + return -ENOMEM; + ka_iter_init_write(&iter, ka); =20 xa_for_each(&ser->track.orders, order, physxa) { struct kho_mem_phys_bits *bits; unsigned long phys; =20 - chunk =3D new_chunk(chunk, order); - if (!chunk) - goto err_free; - - if (!first_chunk) - first_chunk =3D chunk; - xa_for_each(&physxa->phys_bits, phys, bits) { struct khoser_mem_bitmap_ptr *elm; + phys_addr_t start; + + start =3D (phys * PRESERVE_BITS) << (order + PAGE_SHIFT); + elm =3D new_bitmap(start, bits, order); + if (!elm) + goto err_free; =20 - if (chunk->hdr.num_elms =3D=3D ARRAY_SIZE(chunk->bitmaps)) { - chunk =3D new_chunk(chunk, order); - if (!chunk) - goto err_free; - } - - elm =3D &chunk->bitmaps[chunk->hdr.num_elms]; - chunk->hdr.num_elms++; - elm->phys_start =3D (phys * PRESERVE_BITS) - << (order + PAGE_SHIFT); - KHOSER_STORE_PTR(elm->bitmap, bits); + ka_iter_setpos(&iter, pos); + if (ka_iter_setentry(&iter, elm)) + goto err_free; + pos++; } } =20 - ser->preserved_mem_map =3D first_chunk; + ser->preserved_mem_map =3D ka; =20 return 0; =20 err_free: - kho_mem_ser_free(first_chunk); + kho_mem_ser_free(ka); return -ENOMEM; } =20 -static void __init deserialize_bitmap(unsigned int order, - struct khoser_mem_bitmap_ptr *elm) +static void __init deserialize_bitmap(struct khoser_mem_bitmap_ptr *elm) { struct kho_mem_phys_bits *bitmap =3D KHOSER_LOAD_PTR(elm->bitmap); unsigned long bit; =20 for_each_set_bit(bit, bitmap->preserve, PRESERVE_BITS) { - int sz =3D 1 << (order + PAGE_SHIFT); + int sz =3D 1 << (elm->order + PAGE_SHIFT); phys_addr_t phys =3D - elm->phys_start + (bit << (order + PAGE_SHIFT)); + elm->phys_start + (bit << (elm->order + PAGE_SHIFT)); struct page *page =3D phys_to_page(phys); =20 memblock_reserve(phys, sz); memblock_reserved_mark_noinit(phys, sz); - page->private =3D order; + page->private =3D elm->order; } } =20 static void __init kho_mem_deserialize(const void *fdt) { - struct khoser_mem_chunk *chunk; + struct khoser_mem_bitmap_ptr *elm; const phys_addr_t *mem; + struct kho_array *ka; + struct ka_iter iter; int len; =20 mem =3D fdt_getprop(fdt, 0, PROP_PRESERVED_MEMORY_MAP, &len); @@ -361,15 +349,17 @@ static void __init kho_mem_deserialize(const void *fd= t) return; } =20 - chunk =3D *mem ? phys_to_virt(*mem) : NULL; - while (chunk) { - unsigned int i; - - for (i =3D 0; i !=3D chunk->hdr.num_elms; i++) - deserialize_bitmap(chunk->hdr.order, - &chunk->bitmaps[i]); - chunk =3D KHOSER_LOAD_PTR(chunk->hdr.next); + ka =3D *mem ? phys_to_virt(*mem) : NULL; + if (!ka) + return; + if (!kho_array_valid(ka)) { + pr_err("invalid KHO array for preserved memory bitmaps\n"); + return; } + + ka_iter_init_read(&iter, ka); + ka_iter_for_each(&iter, elm) + deserialize_bitmap(elm); } =20 /* --=20 2.47.3