From nobody Mon Feb 9 13:57:34 2026 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B247F27815F for ; Thu, 1 May 2025 22:55:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746140125; cv=none; b=giwQpCmTnoO/gJrvMMD+BENkh43vows4PkTTKemXXdH1XSFop5yqnw0XHK2RKYa0FHUYe6ze3TkGKPkB8rYpIveQK1ahU1Ra2kwjV8dsOHtHFrCzJ9UbI3fsesGzjfI+M867C83BpsxuOGzVTRcsHSQxMbXWxrujPSIrguCbFF4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746140125; c=relaxed/simple; bh=zdWPhrCRGhenM783PYZaKYNEDK7bnm3XtFRg4N5OXU4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cNONYsAIXyJKfVwukF7VTFv7W6LIjgXk8scD47aTIXOiYvUVrRGGtRdKWvv9TkPwWccsxtk+a6sS4VkpGNX59zuEIEiFw9L9LDrprp3oL914r+4rhgok+HtHEZjZ1xYmIgYX/e0T2TbfY1RTgJDLMYWaEWCdh0LVR0M6zcyiMIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--changyuanl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Z29PYda/; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--changyuanl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Z29PYda/" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b16f5365310so907845a12.1 for ; Thu, 01 May 2025 15:55:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746140123; x=1746744923; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ahn0CpwGlow7d1B2oRzig1Hirg8bGDbJkNyQ2rWb7+Y=; b=Z29PYda/e5zrfTN5B5QYgq10nM2gsTQb/ALdV5HP98MRKQSKL59G4/yt+LHB5hAxje equnEZrNetGzk0hH132g/j57rLkcg3R2WFYhZHfkyHAkzrCo3K2RdU4G1ooBDaQ/aTxK 4E2ElhOqUHyTQI3kQr8Q6jGeKQ2ZeKYjpPfbfLiA+/UVCQ3PMK8emXSw/GhfokVO/Ko8 rURx1j4RtL14EqDdJ06bfml7KWVjvEyohuYcfxP+fKs63NZ7dM7mnffVZykLTAvxLXdW bZiBa1z2AiGrPjje9BvY6iqu6sEoYcHZOIdOg8Mq8UobQfxtAWJ+HsOAei/FIc56g3Cq uOmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746140123; x=1746744923; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ahn0CpwGlow7d1B2oRzig1Hirg8bGDbJkNyQ2rWb7+Y=; b=TKMpFR0CYnuO+FFGIwhYej4lHE1ieUCvRks97SA/osIkyJ5sRu3naijXtM/lk+gPXA xuPigPsztmpclP18D+gbe2LLDM/2hZIBVN3MIiXffoN5KnfrSJOYZEVuFfxfa7/z5OCR 9tujUYq2cgxepNCRdtt9eF5hSaaKF4fzKyvUp1NOT54z6OTggxu31iabIrQdeONbNDIJ egql2oJrHWMz3To7Ui8MHpFhGk/p0TPSDfeu9hAd0a1EUJv9/lQG8XWxJOg95eQ4tlG+ MQCMHC2tUhi9KBt3RtxgHkQM7tZncIhfYeN1nKgSostoo8wY6aJQInMsdJ/x9SjAxXnd 1gZg== X-Gm-Message-State: AOJu0Yy9EP1D4+0r9uaD/2gMoulZpEnzOpe1SUaZIHCUj2hTrG6enjMM sRN9w3JSAdazNOva5gguv1R7OawrWXizy2SlqqjaDP3dwEZG8fpGMIYb46FmniiJR2tWbC3EST/ xQR1dBc/wPC/J9J4GNcYgS27j1fRfL2swOauSFkaD+SXnbLv4aQPsFkMMr3U/JY5vy2iaCmZwtw U/uku4v6u5Q4IRQ03q0474bSdNxd3uR1Z1MGuxZ+AnEaFivxpHaEDMlAa3FpCmRQ== X-Google-Smtp-Source: AGHT+IE7Ipcc3Yfu0opFD7T9fLG2qdetuzGT/pPArsEd4Y7NNLnPX6DlCyLOptaPTidbIU/KZquDv6iti8nU9gNz X-Received: from pgq9.prod.google.com ([2002:a63:1049:0:b0:b1f:9d93:19a7]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:9004:b0:1f5:9393:fd4d with SMTP id adf61e73a8af0-20ce04e70cdmr969801637.27.1746140122500; Thu, 01 May 2025 15:55:22 -0700 (PDT) Date: Thu, 1 May 2025 15:54:14 -0700 In-Reply-To: <20250501225425.635167-1-changyuanl@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250501225425.635167-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.906.g1f30a19c02-goog Message-ID: <20250501225425.635167-8-changyuanl@google.com> Subject: [PATCH v7 07/18] kexec: enable KHO support for memory preservation From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: changyuanl@google.com, akpm@linux-foundation.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, corbet@lwn.net, dave.hansen@linux.intel.com, devicetree@vger.kernel.org, dwmw2@infradead.org, ebiederm@xmission.com, graf@amazon.com, hpa@zytor.com, jgowans@amazon.com, kexec@lists.infradead.org, krzk@kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, luto@kernel.org, mark.rutland@arm.com, mingo@redhat.com, pasha.tatashin@soleen.com, pbonzini@redhat.com, peterz@infradead.org, ptyadav@amazon.de, robh@kernel.org, rostedt@goodmis.org, rppt@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, tglx@linutronix.de, thomas.lendacky@amd.com, will@kernel.org, x86@kernel.org, Jason Gunthorpe Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" Introduce APIs allowing KHO users to preserve memory across kexec and get access to that memory after boot of the kexeced kernel kho_preserve_folio() - record a folio to be preserved over kexec kho_restore_folio() - recreates the folio from the preserved memory kho_preserve_phys() - record physically contiguous range to be preserved over kexec. The memory preservations are tracked by two levels of xarrays to manage chunks of per-order 512 byte bitmaps. For instance if PAGE_SIZE =3D 4096, the entire 1G order of a 1TB x86 system would fit inside a single 512 byte bitmap. For order 0 allocations each bitmap will cover 16M of address space. Thus, for 16G of memory at most 512K of bitmap memory will be needed for order 0. At serialization time all bitmaps are recorded in a linked list of pages for the next kernel to process and the physical address of the list is recorded in KHO FDT. The next kernel then processes that list, reserves the memory ranges and later, when a user requests a folio or a physical range, KHO restores corresponding memory map entries. Suggested-by: Jason Gunthorpe Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/kexec_handover.h | 36 +++ kernel/kexec_handover.c | 406 +++++++++++++++++++++++++++++++++ 2 files changed, 442 insertions(+) diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h index 02dcfc8c427e3..348844cffb136 100644 --- a/include/linux/kexec_handover.h +++ b/include/linux/kexec_handover.h @@ -16,13 +16,34 @@ enum kho_event { KEXEC_KHO_ABORT =3D 1, }; =20 +struct folio; struct notifier_block; =20 +#define DECLARE_KHOSER_PTR(name, type) \ + union { \ + phys_addr_t phys; \ + type ptr; \ + } name +#define KHOSER_STORE_PTR(dest, val) \ + ({ \ + typeof(val) v =3D val; \ + typecheck(typeof((dest).ptr), v); \ + (dest).phys =3D virt_to_phys(v); \ + }) +#define KHOSER_LOAD_PTR(src) = \ + ({ \ + typeof(src) s =3D src; \ + (typeof((s).ptr))((s).phys ? phys_to_virt((s).phys) : NULL); \ + }) + struct kho_serialization; =20 #ifdef CONFIG_KEXEC_HANDOVER bool kho_is_enabled(void); =20 +int kho_preserve_folio(struct folio *folio); +int kho_preserve_phys(phys_addr_t phys, size_t size); +struct folio *kho_restore_folio(phys_addr_t phys); int kho_add_subtree(struct kho_serialization *ser, const char *name, void = *fdt); int kho_retrieve_subtree(const char *name, phys_addr_t *phys); =20 @@ -39,6 +60,21 @@ static inline bool kho_is_enabled(void) return false; } =20 +static inline int kho_preserve_folio(struct folio *folio) +{ + return -EOPNOTSUPP; +} + +static inline int kho_preserve_phys(phys_addr_t phys, size_t size) +{ + return -EOPNOTSUPP; +} + +static inline struct folio *kho_restore_folio(phys_addr_t phys) +{ + return NULL; +} + static inline int kho_add_subtree(struct kho_serialization *ser, const char *name, void *fdt) { diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index 59f3cf9557f50..3bf74b4960f84 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -9,6 +9,7 @@ #define pr_fmt(fmt) "KHO: " fmt =20 #include +#include #include #include #include @@ -44,12 +45,307 @@ static int __init kho_parse_enable(char *p) } early_param("kho", kho_parse_enable); =20 +/* + * Keep track of memory that is to be preserved across KHO. + * + * The serializing side uses two levels of xarrays to manage chunks of per= -order + * 512 byte bitmaps. For instance if PAGE_SIZE =3D 4096, the entire 1G ord= er of a + * 1TB system would fit inside a single 512 byte bitmap. For order 0 alloc= ations + * each bitmap will cover 16M of address space. Thus, for 16G of memory at= most + * 512K of bitmap memory will be needed for order 0. + * + * This approach is fully incremental, as the serialization progresses fol= ios + * can continue be aggregated to the tracker. The final step, immediately = prior + * to kexec would serialize the xarray information into a linked list for = the + * successor kernel to parse. + */ + +#define PRESERVE_BITS (512 * 8) + +struct kho_mem_phys_bits { + DECLARE_BITMAP(preserve, PRESERVE_BITS); +}; + +struct kho_mem_phys { + /* + * Points to kho_mem_phys_bits, a sparse bitmap array. Each bit is sized + * to order. + */ + struct xarray phys_bits; +}; + +struct kho_mem_track { + /* Points to kho_mem_phys, each order gets its own bitmap tree */ + struct xarray orders; +}; + +struct khoser_mem_chunk; + struct kho_serialization { struct page *fdt; struct list_head fdt_list; struct dentry *sub_fdt_dir; + struct kho_mem_track track; + /* First chunk of serialized preserved memory map */ + struct khoser_mem_chunk *preserved_mem_map; }; =20 +static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size= _t sz) +{ + void *elm, *res; + + elm =3D xa_load(xa, index); + if (elm) + return elm; + + elm =3D kzalloc(sz, GFP_KERNEL); + if (!elm) + return ERR_PTR(-ENOMEM); + + res =3D xa_cmpxchg(xa, index, NULL, elm, GFP_KERNEL); + if (xa_is_err(res)) + res =3D ERR_PTR(xa_err(res)); + + if (res) { + kfree(elm); + return res; + } + + return elm; +} + +static void __kho_unpreserve(struct kho_mem_track *track, unsigned long pf= n, + unsigned long end_pfn) +{ + struct kho_mem_phys_bits *bits; + struct kho_mem_phys *physxa; + + while (pfn < end_pfn) { + const unsigned int order =3D + min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn)); + const unsigned long pfn_high =3D pfn >> order; + + physxa =3D xa_load(&track->orders, order); + if (!physxa) + continue; + + bits =3D xa_load(&physxa->phys_bits, pfn_high / PRESERVE_BITS); + if (!bits) + continue; + + clear_bit(pfn_high % PRESERVE_BITS, bits->preserve); + + pfn +=3D 1 << order; + } +} + +static int __kho_preserve_order(struct kho_mem_track *track, unsigned long= pfn, + unsigned int order) +{ + struct kho_mem_phys_bits *bits; + struct kho_mem_phys *physxa; + const unsigned long pfn_high =3D pfn >> order; + + might_sleep(); + + physxa =3D xa_load_or_alloc(&track->orders, order, sizeof(*physxa)); + if (IS_ERR(physxa)) + return PTR_ERR(physxa); + + bits =3D xa_load_or_alloc(&physxa->phys_bits, pfn_high / PRESERVE_BITS, + sizeof(*bits)); + if (IS_ERR(bits)) + return PTR_ERR(bits); + + set_bit(pfn_high % PRESERVE_BITS, bits->preserve); + + return 0; +} + +/* almost as free_reserved_page(), just don't free the page */ +static void kho_restore_page(struct page *page) +{ + ClearPageReserved(page); + init_page_count(page); + adjust_managed_page_count(page, 1); +} + +/** + * kho_restore_folio - recreates the folio from the preserved memory. + * @phys: physical address of the folio. + * + * Return: pointer to the struct folio on success, NULL on failure. + */ +struct folio *kho_restore_folio(phys_addr_t phys) +{ + struct page *page =3D pfn_to_online_page(PHYS_PFN(phys)); + unsigned long order; + + if (!page) + return NULL; + + order =3D page->private; + if (order) { + if (order > MAX_PAGE_ORDER) + return NULL; + + prep_compound_page(page, order); + } else { + kho_restore_page(page); + } + + return page_folio(page); +} +EXPORT_SYMBOL_GPL(kho_restore_folio); + +/* Serialize and deserialize struct kho_mem_phys across kexec + * + * Record all the bitmaps in a linked list of pages for the next kernel to + * process. Each chunk holds bitmaps of the same order and each block of b= itmaps + * starts at a given physical address. This allows the bitmaps to be spars= e. The + * xarray is used to store them in a tree while building up the data struc= ture, + * but the KHO successor kernel only needs to process them once in order. + * + * All of this memory is normal kmalloc() memory and is not marked for + * preservation. The successor kernel will remain isolated to the scratch = space + * until it completes processing this list. Once processed all the memory + * storing these ranges will be marked as free. + */ + +struct khoser_mem_bitmap_ptr { + phys_addr_t phys_start; + DECLARE_KHOSER_PTR(bitmap, struct kho_mem_phys_bits *); +}; + +struct khoser_mem_chunk_hdr { + DECLARE_KHOSER_PTR(next, struct khoser_mem_chunk *); + unsigned int order; + unsigned int num_elms; +}; + +#define KHOSER_BITMAP_SIZE \ + ((PAGE_SIZE - sizeof(struct khoser_mem_chunk_hdr)) / \ + sizeof(struct khoser_mem_bitmap_ptr)) + +struct khoser_mem_chunk { + struct khoser_mem_chunk_hdr hdr; + struct khoser_mem_bitmap_ptr bitmaps[KHOSER_BITMAP_SIZE]; +}; + +static_assert(sizeof(struct khoser_mem_chunk) =3D=3D PAGE_SIZE); + +static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chu= nk, + unsigned long order) +{ + struct khoser_mem_chunk *chunk; + + chunk =3D kzalloc(PAGE_SIZE, GFP_KERNEL); + if (!chunk) + return NULL; + chunk->hdr.order =3D order; + if (cur_chunk) + KHOSER_STORE_PTR(cur_chunk->hdr.next, chunk); + return chunk; +} + +static void kho_mem_ser_free(struct khoser_mem_chunk *first_chunk) +{ + struct khoser_mem_chunk *chunk =3D first_chunk; + + while (chunk) { + struct khoser_mem_chunk *tmp =3D chunk; + + chunk =3D KHOSER_LOAD_PTR(chunk->hdr.next); + kfree(tmp); + } +} + +static int kho_mem_serialize(struct kho_serialization *ser) +{ + struct khoser_mem_chunk *first_chunk =3D NULL; + struct khoser_mem_chunk *chunk =3D NULL; + struct kho_mem_phys *physxa; + unsigned long order; + + xa_for_each(&ser->track.orders, order, physxa) { + struct kho_mem_phys_bits *bits; + unsigned long phys; + + chunk =3D new_chunk(chunk, order); + if (!chunk) + goto err_free; + + if (!first_chunk) + first_chunk =3D chunk; + + xa_for_each(&physxa->phys_bits, phys, bits) { + struct khoser_mem_bitmap_ptr *elm; + + if (chunk->hdr.num_elms =3D=3D ARRAY_SIZE(chunk->bitmaps)) { + chunk =3D new_chunk(chunk, order); + if (!chunk) + goto err_free; + } + + elm =3D &chunk->bitmaps[chunk->hdr.num_elms]; + chunk->hdr.num_elms++; + elm->phys_start =3D (phys * PRESERVE_BITS) + << (order + PAGE_SHIFT); + KHOSER_STORE_PTR(elm->bitmap, bits); + } + } + + ser->preserved_mem_map =3D first_chunk; + + return 0; + +err_free: + kho_mem_ser_free(first_chunk); + return -ENOMEM; +} + +static void deserialize_bitmap(unsigned int order, + struct khoser_mem_bitmap_ptr *elm) +{ + struct kho_mem_phys_bits *bitmap =3D KHOSER_LOAD_PTR(elm->bitmap); + unsigned long bit; + + for_each_set_bit(bit, bitmap->preserve, PRESERVE_BITS) { + int sz =3D 1 << (order + PAGE_SHIFT); + phys_addr_t phys =3D + elm->phys_start + (bit << (order + PAGE_SHIFT)); + struct page *page =3D phys_to_page(phys); + + memblock_reserve(phys, sz); + memblock_reserved_mark_noinit(phys, sz); + page->private =3D order; + } +} + +static void __init kho_mem_deserialize(const void *fdt) +{ + struct khoser_mem_chunk *chunk; + const phys_addr_t *mem; + int len; + + mem =3D fdt_getprop(fdt, 0, PROP_PRESERVED_MEMORY_MAP, &len); + + if (!mem || len !=3D sizeof(*mem)) { + pr_err("failed to get preserved memory bitmaps\n"); + return; + } + + chunk =3D *mem ? phys_to_virt(*mem) : NULL; + while (chunk) { + unsigned int i; + + for (i =3D 0; i !=3D chunk->hdr.num_elms; i++) + deserialize_bitmap(chunk->hdr.order, + &chunk->bitmaps[i]); + chunk =3D KHOSER_LOAD_PTR(chunk->hdr.next); + } +} + /* * With KHO enabled, memory can become fragmented because KHO regions may * be anywhere in physical address space. The scratch regions give us a @@ -324,6 +620,9 @@ static struct kho_out kho_out =3D { .lock =3D __MUTEX_INITIALIZER(kho_out.lock), .ser =3D { .fdt_list =3D LIST_HEAD_INIT(kho_out.ser.fdt_list), + .track =3D { + .orders =3D XARRAY_INIT(kho_out.ser.track.orders, 0), + }, }, .finalized =3D false, }; @@ -340,6 +639,73 @@ int unregister_kho_notifier(struct notifier_block *nb) } EXPORT_SYMBOL_GPL(unregister_kho_notifier); =20 +/** + * kho_preserve_folio - preserve a folio across kexec. + * @folio: folio to preserve. + * + * Instructs KHO to preserve the whole folio across kexec. The order + * will be preserved as well. + * + * Return: 0 on success, error code on failure + */ +int kho_preserve_folio(struct folio *folio) +{ + const unsigned long pfn =3D folio_pfn(folio); + const unsigned int order =3D folio_order(folio); + struct kho_mem_track *track =3D &kho_out.ser.track; + + if (kho_out.finalized) + return -EBUSY; + + return __kho_preserve_order(track, pfn, order); +} +EXPORT_SYMBOL_GPL(kho_preserve_folio); + +/** + * kho_preserve_phys - preserve a physically contiguous range across kexec. + * @phys: physical address of the range. + * @size: size of the range. + * + * Instructs KHO to preserve the memory range from @phys to @phys + @size + * across kexec. + * + * Return: 0 on success, error code on failure + */ +int kho_preserve_phys(phys_addr_t phys, size_t size) +{ + unsigned long pfn =3D PHYS_PFN(phys); + unsigned long failed_pfn =3D 0; + const unsigned long start_pfn =3D pfn; + const unsigned long end_pfn =3D PHYS_PFN(phys + size); + int err =3D 0; + struct kho_mem_track *track =3D &kho_out.ser.track; + + if (kho_out.finalized) + return -EBUSY; + + if (!PAGE_ALIGNED(phys) || !PAGE_ALIGNED(size)) + return -EINVAL; + + while (pfn < end_pfn) { + const unsigned int order =3D + min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn)); + + err =3D __kho_preserve_order(track, pfn, order); + if (err) { + failed_pfn =3D pfn; + break; + } + + pfn +=3D 1 << order; + } + + if (err) + __kho_unpreserve(track, start_pfn, failed_pfn); + + return err; +} +EXPORT_SYMBOL_GPL(kho_preserve_phys); + /* Handling for debug/kho/out */ =20 static struct dentry *debugfs_root; @@ -366,6 +732,25 @@ static int kho_out_update_debugfs_fdt(void) static int kho_abort(void) { int err; + unsigned long order; + struct kho_mem_phys *physxa; + + xa_for_each(&kho_out.ser.track.orders, order, physxa) { + struct kho_mem_phys_bits *bits; + unsigned long phys; + + xa_for_each(&physxa->phys_bits, phys, bits) + kfree(bits); + + xa_destroy(&physxa->phys_bits); + kfree(physxa); + } + xa_destroy(&kho_out.ser.track.orders); + + if (kho_out.ser.preserved_mem_map) { + kho_mem_ser_free(kho_out.ser.preserved_mem_map); + kho_out.ser.preserved_mem_map =3D NULL; + } =20 err =3D blocking_notifier_call_chain(&kho_out.chain_head, KEXEC_KHO_ABORT, NULL); @@ -380,12 +765,25 @@ static int kho_abort(void) static int kho_finalize(void) { int err =3D 0; + u64 *preserved_mem_map; void *fdt =3D page_to_virt(kho_out.ser.fdt); =20 err |=3D fdt_create(fdt, PAGE_SIZE); err |=3D fdt_finish_reservemap(fdt); err |=3D fdt_begin_node(fdt, ""); err |=3D fdt_property_string(fdt, "compatible", KHO_FDT_COMPATIBLE); + /** + * Reserve the preserved-memory-map property in the root FDT, so + * that all property definitions will precede subnodes created by + * KHO callers. + */ + err |=3D fdt_property_placeholder(fdt, PROP_PRESERVED_MEMORY_MAP, + sizeof(*preserved_mem_map), + (void **)&preserved_mem_map); + if (err) + goto abort; + + err =3D kho_preserve_folio(page_folio(kho_out.ser.fdt)); if (err) goto abort; =20 @@ -395,6 +793,12 @@ static int kho_finalize(void) if (err) goto abort; =20 + err =3D kho_mem_serialize(&kho_out.ser); + if (err) + goto abort; + + *preserved_mem_map =3D (u64)virt_to_phys(kho_out.ser.preserved_mem_map); + err |=3D fdt_end_node(fdt); err |=3D fdt_finish(fdt); =20 @@ -700,6 +1104,8 @@ void __init kho_memory_init(void) if (kho_in.scratch_phys) { kho_scratch =3D phys_to_virt(kho_in.scratch_phys); kho_release_scratch(); + + kho_mem_deserialize(kho_get_fdt()); } else { kho_reserve_scratch(); } --=20 2.49.0.906.g1f30a19c02-goog