From nobody Mon Feb 9 08:49:39 2026 Received: from mail-vk1-f176.google.com (mail-vk1-f176.google.com [209.85.221.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6E62273F9 for ; Tue, 21 Oct 2025 00:08:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761005340; cv=none; b=ejK32tzKjtz16ALTh6MZaNOu75HPQzprikyMCSGhGN5/EjB8SKQk27AWZ4PdVRdWJ0HcR2dHYHOL7/SpN0GGUv/wwjxl/Lgj3Kz0yd7U6AHiz/kHbBn2zIWdT1mdVmXAww6WVSPA/o5Uw+3XtLEpFL93UCz6jRup+aqlH5hdhHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761005340; c=relaxed/simple; bh=WDnQ7yML0OCi7RtRqHYL6Y4X5Dr6aAb+8kQ+zx1g8z0=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rg5bdSbvW39eghMfrku8yuIcCTPKCgpT0moPtjYSCc+PIBFfh7lX3XkjFHQSrAq/Q191RYLgqzXV56RSZt0l5pg6Hk/9g0yHtg+LL5qSfzhbkxx3NpNvUxVFJWAIEVcQJamXQrXBldbPMYRNoIuDKTlzckw4krBitkaEM9HhndM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com; spf=pass smtp.mailfrom=soleen.com; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b=bWwFgpFH; arc=none smtp.client-ip=209.85.221.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=soleen.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="bWwFgpFH" Received: by mail-vk1-f176.google.com with SMTP id 71dfb90a1353d-554e726e5cfso1659208e0c.3 for ; Mon, 20 Oct 2025 17:08:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1761005338; x=1761610138; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=RrUod6z9AaFkGeIJc9frhWccEivwV7gjE944YavE0C8=; b=bWwFgpFH1pI1SI+GzVHsrS5FPaSI5uSxIiy+lrX4pM3Pl5zPNf/rf4EKq9Zz3jOq0+ /5NrgbtX5C9thIP6LgYKGIiQRIAdbrwwseNE1tmErB/LPyhA9pLtDwQgrPIancVCr9o6 +cEf2X9WixQI1MUdSeai69IsWQznbYpA2V2gPF/R3AW9IBGPzZKKVtpbAMW2+iOXzTtY unDvKoqWpXGduuKljaMT1Yf55B4VSCpdSSZ0nSGnLK+H3K1iiHmpLGB4vpUvJinKd4IC 34iGn3nwh2r8PpsYHUvY7NSx6LyZ4bnYQz5Nineot2T6J5dyT9fIIMiiXncLmi6/V+QS 8aGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761005338; x=1761610138; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RrUod6z9AaFkGeIJc9frhWccEivwV7gjE944YavE0C8=; b=F6xSPoxC8Rv5RJh4N8TLvbuO+tPWd7w3x+pWR5cOE7no9vXk3UrXlGhuwCU7dAR6WW /jfu+KY8N/M8IMLlwnMR6lb7GRl1Obu+DOXarJP+QM4wife51LUTUI8J5ddCX4SMQyWz 957qS8OlZmj9VSfJaon4/CBMzupDoU+AtQi+RZ3zM1CZfmfs0RT7DXMvBp88FxsEbZrj Zz477sq1xrjICHzX0IfVpjnuc8q9vmV6ExeYKtU114L+BBr34ueQWXRwQQ7TrqecaWwT eAY7qjITxFdbR/uMjGtq3oE1VCzsY88hmi2qGiQ4P4lEefhIWcNc24VTETmdRdPvhUR0 Eamw== X-Forwarded-Encrypted: i=1; AJvYcCX+utKHP6jKVa82UkKPz7woeOaUUu4xpGzt88UvHNusANGqzNLnC8vjzIkpASu9btTMHbrLak/DEs8JqKw=@vger.kernel.org X-Gm-Message-State: AOJu0YzlKRVGBeTjPT6l2558vFS9B8iKBqYKQo93fE3y2Y34EpRA39Yz F7MvrmGxzSr3ziXB/tDRjYpmOs8dUZ34PMr3jqy0Sw0Os1CnRN5IG4Bc8ghzzw4stHc= X-Gm-Gg: ASbGncsie6wvatW9TkvFhO1mxXvNIZ746CVA7jUw8eMnc7r048wIAwcApEapHsh8C9k POPzlTV4QkiXuY/n5BywMfQ7kypEowXgw36P/B3Dq3pOG+3xdfmrT6SbdcBrml8OTeD+MDQSQay 3O0pNW8f6DzRibsLa2zGLTG/AuA1864oRkY0xGf1KrxC9hlxGeQDhz5D3XlFjqnwGsiSave2MbD WDoUPrImZMztmOZSEEaEEUe8sosRjSb1FnjtNso3ELLn1/eUAtPMTZ07Z+zxHdeRoCGoupr68my fC05gfwsPdYaLzdQlfl9Ptx3VkKRN7esp4ulNXEKSaQxBIyOQfJMdUiOOpl2MLf4mSLBMdF3YjE KxVmB4OnsJbydLtOHRUjeDpj2v8OmpnADCExMKpZ/u56Vo4jvmszSk7GOHI9IlIkO2i7u0j7mvu HFifDd1hdiwpb/4u7n9N8wvb1BY07+4aSd19w5KGLqC9CP5Oe9WsIzX0j8a+ilLnDsvJHeHNGpf N/bNuVw8demZKZS7c7b+16A/8OO3FSQ X-Google-Smtp-Source: AGHT+IEXSBVMEN9foAesWnzieDCrIXltrdCTVWZ8zY0V+zYO9XF7MLbCmqXOk7hAXlRHd/2MI6iEIA== X-Received: by 2002:a05:6122:1d13:b0:54b:bea6:a226 with SMTP id 71dfb90a1353d-5564ee6bfe3mr3839907e0c.11.1761005337567; Mon, 20 Oct 2025 17:08:57 -0700 (PDT) Received: from soleen.us-east4-b.c.cloudtop-prod-us-east.internal (53.47.86.34.bc.googleusercontent.com. [34.86.47.53]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-55661f6e351sm2822882e0c.4.2025.10.20.17.08.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Oct 2025 17:08:57 -0700 (PDT) From: Pasha Tatashin To: akpm@linux-foundation.org, brauner@kernel.org, corbet@lwn.net, graf@amazon.com, jgg@ziepe.ca, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, masahiroy@kernel.org, ojeda@kernel.org, pasha.tatashin@soleen.com, pratyush@kernel.org, rdunlap@infradead.org, rppt@kernel.org, tj@kernel.org, jasonmiu@google.com, dmatlack@google.com, skhawaja@google.com Subject: [PATCH v3 1/3] liveupdate: kho: warn and fail on metadata or preserved memory in scratch area Date: Mon, 20 Oct 2025 20:08:50 -0400 Message-ID: <20251021000852.2924827-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.51.0.869.ge66316f041-goog In-Reply-To: <20251021000852.2924827-1-pasha.tatashin@soleen.com> References: <20251021000852.2924827-1-pasha.tatashin@soleen.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" It is invalid for KHO metadata or preserved memory regions to be located within the KHO scratch area, as this area is overwritten when the next kernel is loaded, and used early in boot by the next kernel. This can lead to memory corruption. Adds checks to kho_preserve_* and KHO's internal metadata allocators (xa_load_or_alloc, new_chunk) to verify that the physical address of the memory does not overlap with any defined scratch region. If an overlap is detected, the operation will fail and a WARN_ON is triggered. To avoid performance overhead in production kernels, these checks are enabled only when CONFIG_KEXEC_HANDOVER_DEBUG is selected. Signed-off-by: Pasha Tatashin Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Pratyush Yadav --- kernel/Kconfig.kexec | 9 ++++++ kernel/Makefile | 1 + kernel/kexec_handover.c | 53 ++++++++++++++++++++++---------- kernel/kexec_handover_debug.c | 25 +++++++++++++++ kernel/kexec_handover_internal.h | 16 ++++++++++ 5 files changed, 87 insertions(+), 17 deletions(-) create mode 100644 kernel/kexec_handover_debug.c create mode 100644 kernel/kexec_handover_internal.h diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec index 422270d64820..c94d36b5fcd9 100644 --- a/kernel/Kconfig.kexec +++ b/kernel/Kconfig.kexec @@ -109,6 +109,15 @@ config KEXEC_HANDOVER to keep data or state alive across the kexec. For this to work, both source and target kernels need to have this option enabled. =20 +config KEXEC_HANDOVER_DEBUG + bool "Enable Kexec Handover debug checks" + depends on KEXEC_HANDOVER_DEBUGFS + help + This option enables extra sanity checks for the Kexec Handover + subsystem. Since, KHO performance is crucial in live update + scenarios and the extra code might be adding overhead it is + only optionally enabled. + config CRASH_DUMP bool "kernel crash dumps" default ARCH_DEFAULT_CRASH_DUMP diff --git a/kernel/Makefile b/kernel/Makefile index df3dd8291bb6..9fe722305c9b 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -83,6 +83,7 @@ obj-$(CONFIG_KEXEC) +=3D kexec.o obj-$(CONFIG_KEXEC_FILE) +=3D kexec_file.o obj-$(CONFIG_KEXEC_ELF) +=3D kexec_elf.o obj-$(CONFIG_KEXEC_HANDOVER) +=3D kexec_handover.o +obj-$(CONFIG_KEXEC_HANDOVER_DEBUG) +=3D kexec_handover_debug.o obj-$(CONFIG_BACKTRACE_SELF_TEST) +=3D backtracetest.o obj-$(CONFIG_COMPAT) +=3D compat.o obj-$(CONFIG_CGROUPS) +=3D cgroup/ diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index 76f0940fb485..7b460806ef4f 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -8,6 +8,7 @@ =20 #define pr_fmt(fmt) "KHO: " fmt =20 +#include #include #include #include @@ -22,6 +23,7 @@ =20 #include =20 +#include "kexec_handover_internal.h" /* * KHO is tightly coupled with mm init and needs access to some of mm * internal APIs. @@ -133,26 +135,26 @@ static struct kho_out kho_out =3D { =20 static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size= _t sz) { - void *elm, *res; + void *res =3D xa_load(xa, index); =20 - elm =3D xa_load(xa, index); - if (elm) - return elm; + if (res) + return res; + + void *elm __free(kfree) =3D kzalloc(sz, GFP_KERNEL); =20 - elm =3D kzalloc(sz, GFP_KERNEL); if (!elm) return ERR_PTR(-ENOMEM); =20 + if (WARN_ON(kho_scratch_overlap(virt_to_phys(elm), sz))) + return ERR_PTR(-EINVAL); + res =3D xa_cmpxchg(xa, index, NULL, elm, GFP_KERNEL); if (xa_is_err(res)) - res =3D ERR_PTR(xa_err(res)); - - if (res) { - kfree(elm); + return ERR_PTR(xa_err(res)); + else if (res) return res; - } =20 - return elm; + return no_free_ptr(elm); } =20 static void __kho_unpreserve(struct kho_mem_track *track, unsigned long pf= n, @@ -345,15 +347,19 @@ static_assert(sizeof(struct khoser_mem_chunk) =3D=3D = PAGE_SIZE); static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chu= nk, unsigned long order) { - struct khoser_mem_chunk *chunk; + struct khoser_mem_chunk *chunk __free(kfree) =3D NULL; =20 chunk =3D kzalloc(PAGE_SIZE, GFP_KERNEL); if (!chunk) - return NULL; + return ERR_PTR(-ENOMEM); + + if (WARN_ON(kho_scratch_overlap(virt_to_phys(chunk), PAGE_SIZE))) + return ERR_PTR(-EINVAL); + chunk->hdr.order =3D order; if (cur_chunk) KHOSER_STORE_PTR(cur_chunk->hdr.next, chunk); - return chunk; + return no_free_ptr(chunk); } =20 static void kho_mem_ser_free(struct khoser_mem_chunk *first_chunk) @@ -374,14 +380,17 @@ static int kho_mem_serialize(struct kho_serialization= *ser) struct khoser_mem_chunk *chunk =3D NULL; struct kho_mem_phys *physxa; unsigned long order; + int err =3D -ENOMEM; =20 xa_for_each(&ser->track.orders, order, physxa) { struct kho_mem_phys_bits *bits; unsigned long phys; =20 chunk =3D new_chunk(chunk, order); - if (!chunk) + if (IS_ERR(chunk)) { + err =3D PTR_ERR(chunk); goto err_free; + } =20 if (!first_chunk) first_chunk =3D chunk; @@ -391,8 +400,10 @@ static int kho_mem_serialize(struct kho_serialization = *ser) =20 if (chunk->hdr.num_elms =3D=3D ARRAY_SIZE(chunk->bitmaps)) { chunk =3D new_chunk(chunk, order); - if (!chunk) + if (IS_ERR(chunk)) { + err =3D PTR_ERR(chunk); goto err_free; + } } =20 elm =3D &chunk->bitmaps[chunk->hdr.num_elms]; @@ -409,7 +420,7 @@ static int kho_mem_serialize(struct kho_serialization *= ser) =20 err_free: kho_mem_ser_free(first_chunk); - return -ENOMEM; + return err; } =20 static void __init deserialize_bitmap(unsigned int order, @@ -752,6 +763,9 @@ int kho_preserve_folio(struct folio *folio) const unsigned int order =3D folio_order(folio); struct kho_mem_track *track =3D &kho_out.ser.track; =20 + if (WARN_ON(kho_scratch_overlap(pfn << PAGE_SHIFT, PAGE_SIZE << order))) + return -EINVAL; + return __kho_preserve_order(track, pfn, order); } EXPORT_SYMBOL_GPL(kho_preserve_folio); @@ -775,6 +789,11 @@ int kho_preserve_pages(struct page *page, unsigned int= nr_pages) unsigned long failed_pfn =3D 0; int err =3D 0; =20 + if (WARN_ON(kho_scratch_overlap(start_pfn << PAGE_SHIFT, + nr_pages << PAGE_SHIFT))) { + return -EINVAL; + } + while (pfn < end_pfn) { const unsigned int order =3D min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn)); diff --git a/kernel/kexec_handover_debug.c b/kernel/kexec_handover_debug.c new file mode 100644 index 000000000000..6efb696f5426 --- /dev/null +++ b/kernel/kexec_handover_debug.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * kexec_handover_debug.c - kexec handover optional debug functionality + * Copyright (C) 2025 Google LLC, Pasha Tatashin + */ + +#define pr_fmt(fmt) "KHO: " fmt + +#include "kexec_handover_internal.h" + +bool kho_scratch_overlap(phys_addr_t phys, size_t size) +{ + phys_addr_t scratch_start, scratch_end; + unsigned int i; + + for (i =3D 0; i < kho_scratch_cnt; i++) { + scratch_start =3D kho_scratch[i].addr; + scratch_end =3D kho_scratch[i].addr + kho_scratch[i].size; + + if (phys < scratch_end && (phys + size) > scratch_start) + return true; + } + + return false; +} diff --git a/kernel/kexec_handover_internal.h b/kernel/kexec_handover_inter= nal.h new file mode 100644 index 000000000000..05e9720ba7b9 --- /dev/null +++ b/kernel/kexec_handover_internal.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef LINUX_KEXEC_HANDOVER_INTERNAL_H +#define LINUX_KEXEC_HANDOVER_INTERNAL_H + +#include + +#ifdef CONFIG_KEXEC_HANDOVER_DEBUG +bool kho_scratch_overlap(phys_addr_t phys, size_t size); +#else +static inline bool kho_scratch_overlap(phys_addr_t phys, size_t size) +{ + return false; +} +#endif /* CONFIG_KEXEC_HANDOVER_DEBUG */ + +#endif /* LINUX_KEXEC_HANDOVER_INTERNAL_H */ --=20 2.51.0.869.ge66316f041-goog From nobody Mon Feb 9 08:49:39 2026 Received: from mail-vk1-f181.google.com (mail-vk1-f181.google.com [209.85.221.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E8F22A1C7 for ; Tue, 21 Oct 2025 00:09:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761005343; cv=none; b=Z9n0XFWNTT9vJwy1JGkfoEomKXJ5cjnb1g9q1MY2cRkT2y8tm9v0Ro2H4z8cAdj9u4fkkslLqnlF2upS3Zq99o4iwi61nV6dKhxPnmfNIfGCZR4AV/oqjc069UueEjkqV2Thl7gdId7nowxoDBhHiqJR0QCZI8378ncxWnXL9C4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761005343; c=relaxed/simple; bh=xIuMw8XMRWLgufCuYNoWPBcLu0nLGH0NZhBNT+EtTIA=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RwwMWagKA1GdeLv++TopwQfmCTaDPBKjNsDAm0km0IOd4TT+t6Ukpwy4QigsC6ILhjE1hut//tIhUMaKF5dGdeRUk6NRIkx+myeNx2f+TR0CY7gukiQshq7vDYidDbbCCkhqqNKhqlH3NSHggJ53TR6O+FXpu9hVccr0wXKHWJQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com; spf=pass smtp.mailfrom=soleen.com; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b=ehty/tAK; arc=none smtp.client-ip=209.85.221.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=soleen.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="ehty/tAK" Received: by mail-vk1-f181.google.com with SMTP id 71dfb90a1353d-54aa789f9b5so3558330e0c.1 for ; Mon, 20 Oct 2025 17:09:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1761005340; x=1761610140; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=k+GEFK7rnEt3wQnfXqm/bMcNjTmF7K5naCxuLWf6Icw=; b=ehty/tAKFGHks2sbLBfXKyRy3Ea3aFc0DeyOIhbFbeFW5DkZJTUQHhdxdloUiOTExU Zo4O7cmT2FAwnt40WXhUDIxI7E+iaEY+sMTJlDvcbZQr3IoQv9MgDzFdhvI5mMp2posu T6RR1CPDy+M/1DrYCioPXB82XTFlrdxlQSfCk8lRgluKMgTYx8qSlnWiDy9YEYWIjdNC 2T/T7ZZJbDgg476jM2X6VistbNA6MCW2xztM5G00yIIND9If/O4ep8HTRq1cHho7quVm hN3KT/glYqPgbrcN4v08Ng4moIt0D68j8Qt9o1RS8MiSIb6B4mauFuQO4VdDGBJFM2nT dwUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761005340; x=1761610140; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k+GEFK7rnEt3wQnfXqm/bMcNjTmF7K5naCxuLWf6Icw=; b=JgpUpmx/PSER2IbVKLGEQwcM5pnRUnpqvK/D2o2m6XJzrf5Yd6cvmLtm9LKcAdUlkt 6ogdGbAfkLnxGYYYpIrBYuu8rKrpnlUCVaxUXeNPho1L6ZkMJScsQp6LKYA+ePTZUZKN IMnqxzu1hifewGda5X6yBWyaeVZQmUMPfY25JJ5MggH9Z91EJDrzKmf0p+3rJC7ptf7g FsdAglEJCXENZHN/+PpalE4Fec1uUtfG65BwdRRnziNtdKmanNGfT3Nhl5w/uuIyXreI fsGqK7uSak1xuUV+K3OSOjJPoPlCYjTcjOsNL4C6onurbLOsDQa0yIWhYMY+evjB9ewO MQ2Q== X-Forwarded-Encrypted: i=1; AJvYcCXGunHgTaX34+JFLjxuhjCp8ToH+9e6ESyfDm06Ik2KSUST/HiAXsAl7YupJsI3JGu2AUWhjc4VQoEipBs=@vger.kernel.org X-Gm-Message-State: AOJu0Yzyw/Qcv2K/9BWPeHQdM5S03ov/3CE7ug1L/AELsytIsj4Qd/5w Sg4ER3dWMlZd05vZtMu2i08ieTz2wysN/pU7X0JEVTzch4PQqST/OA63OTLg5A7EXP8= X-Gm-Gg: ASbGncv43HOidiI4NL+NI4+Dx9sVYbXgg708JTRRThrON2tkB/iBrN1siEuFPAokzuF TtTDE8uoL1utrXhKw3NVKTYLJWHyGrVcH68WF1thEXlGIyqi4QGI/S2+6aA2H3OTAClczY7mHNz D+nIzQQqq5GhO0HwmOUEj3a23ZYjNT/YUC+Y9hZuj9hNAcLXzdMWGAz5hA6KvQX/APFr3zjrTX8 yCasbqV9Z4fMlm2zRnC+RqkDL2OR6kze1H54QW/TjTROB59rnyIEQJ2BXW8+kAVHw5SVGnU8ya5 iE5C/VdUu+VwY/qW+ep+6njpqNcFWyH+Isa3qv3GY81sTPje16SEoV2tjpWEMkyYUA95hk54b1m n90bFxbM9QEO1j76bAPxv46Ss2raHde+lQpwn0HzT8HS8+TFHuJzFJMey+ZBTyNowWqmcpUddZ+ yQ1lO4V4qHxieL1SyLw76rGdov5pRN2Rwedn9BZMKGmPJprw+QmKxVka7XCYGqH8bPm0RVNFRQm bmA2fsWSpFveoRR08FPo8ofaqGREAQEUV86ZZ9fD/I= X-Google-Smtp-Source: AGHT+IEiKbkHyTIPqxuiQK/s9jE4arlePP4EQ1uFjj4rccnMC3Ytmd/xdu5/e3ZeVXOoj3cegKmXtQ== X-Received: by 2002:a05:6122:2222:b0:54a:9cff:6fe7 with SMTP id 71dfb90a1353d-5564ee53244mr5078357e0c.4.1761005340123; Mon, 20 Oct 2025 17:09:00 -0700 (PDT) Received: from soleen.us-east4-b.c.cloudtop-prod-us-east.internal (53.47.86.34.bc.googleusercontent.com. [34.86.47.53]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-55661f6e351sm2822882e0c.4.2025.10.20.17.08.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Oct 2025 17:08:58 -0700 (PDT) From: Pasha Tatashin To: akpm@linux-foundation.org, brauner@kernel.org, corbet@lwn.net, graf@amazon.com, jgg@ziepe.ca, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, masahiroy@kernel.org, ojeda@kernel.org, pasha.tatashin@soleen.com, pratyush@kernel.org, rdunlap@infradead.org, rppt@kernel.org, tj@kernel.org, jasonmiu@google.com, dmatlack@google.com, skhawaja@google.com Subject: [PATCH v3 2/3] liveupdate: kho: Increase metadata bitmap size to PAGE_SIZE Date: Mon, 20 Oct 2025 20:08:51 -0400 Message-ID: <20251021000852.2924827-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.51.0.869.ge66316f041-goog In-Reply-To: <20251021000852.2924827-1-pasha.tatashin@soleen.com> References: <20251021000852.2924827-1-pasha.tatashin@soleen.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KHO memory preservation metadata is preserved in 512 byte chunks which requires their allocation from slab allocator. Slabs are not safe to be used with KHO because of kfence, and because partial slabs may lead leaks to the next kernel. Change the size to be PAGE_SIZE. The kfence specifically may cause memory corruption, where it randomly provides slab objects that can be within the scratch area. The reason for that is that kfence allocates its objects prior to KHO scratch is marked as CMA region. While this change could potentially increase metadata overhead on systems with sparsely preserved memory, this is being mitigated by ongoing work to reduce sparseness during preservation via 1G guest pages. Furthermore, this change aligns with future work on a stateless KHO, which will also use page-sized bitmaps for its radix tree metadata. Signed-off-by: Pasha Tatashin Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Pratyush Yadav --- kernel/kexec_handover.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index 7b460806ef4f..e5b91761fbfe 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -69,10 +69,10 @@ early_param("kho", kho_parse_enable); * Keep track of memory that is to be preserved across KHO. * * The serializing side uses two levels of xarrays to manage chunks of per= -order - * 512 byte bitmaps. For instance if PAGE_SIZE =3D 4096, the entire 1G ord= er of a - * 1TB system would fit inside a single 512 byte bitmap. For order 0 alloc= ations - * each bitmap will cover 16M of address space. Thus, for 16G of memory at= most - * 512K of bitmap memory will be needed for order 0. + * PAGE_SIZE byte bitmaps. For instance if PAGE_SIZE =3D 4096, the entire = 1G order + * of a 8TB system would fit inside a single 4096 byte bitmap. For order 0 + * allocations each bitmap will cover 128M of address space. Thus, for 16G= of + * memory at most 512K of bitmap memory will be needed for order 0. * * This approach is fully incremental, as the serialization progresses fol= ios * can continue be aggregated to the tracker. The final step, immediately = prior @@ -80,12 +80,14 @@ early_param("kho", kho_parse_enable); * successor kernel to parse. */ =20 -#define PRESERVE_BITS (512 * 8) +#define PRESERVE_BITS (PAGE_SIZE * 8) =20 struct kho_mem_phys_bits { DECLARE_BITMAP(preserve, PRESERVE_BITS); }; =20 +static_assert(sizeof(struct kho_mem_phys_bits) =3D=3D PAGE_SIZE); + struct kho_mem_phys { /* * Points to kho_mem_phys_bits, a sparse bitmap array. Each bit is sized @@ -133,19 +135,19 @@ static struct kho_out kho_out =3D { .finalized =3D false, }; =20 -static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size= _t sz) +static void *xa_load_or_alloc(struct xarray *xa, unsigned long index) { void *res =3D xa_load(xa, index); =20 if (res) return res; =20 - void *elm __free(kfree) =3D kzalloc(sz, GFP_KERNEL); + void *elm __free(kfree) =3D kzalloc(PAGE_SIZE, GFP_KERNEL); =20 if (!elm) return ERR_PTR(-ENOMEM); =20 - if (WARN_ON(kho_scratch_overlap(virt_to_phys(elm), sz))) + if (WARN_ON(kho_scratch_overlap(virt_to_phys(elm), PAGE_SIZE))) return ERR_PTR(-EINVAL); =20 res =3D xa_cmpxchg(xa, index, NULL, elm, GFP_KERNEL); @@ -218,8 +220,7 @@ static int __kho_preserve_order(struct kho_mem_track *t= rack, unsigned long pfn, } } =20 - bits =3D xa_load_or_alloc(&physxa->phys_bits, pfn_high / PRESERVE_BITS, - sizeof(*bits)); + bits =3D xa_load_or_alloc(&physxa->phys_bits, pfn_high / PRESERVE_BITS); if (IS_ERR(bits)) return PTR_ERR(bits); =20 --=20 2.51.0.869.ge66316f041-goog From nobody Mon Feb 9 08:49:39 2026 Received: from mail-ua1-f44.google.com (mail-ua1-f44.google.com [209.85.222.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E98919F40B for ; Tue, 21 Oct 2025 00:09:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761005346; cv=none; b=lz0J9gNFRw+OhXwqYyAkOSljMM/ITXb8eiUoBkOubhZLg4H2qSEBqTuoOeQaPOxeWrvYIM5WDPDknMOu7cKm7bAEvsZhi4968TbCAxQGkJvJoF0x2aOTZK7dfyxGmpcyTvlRwF9s3ZKW48SGJi+pX+i//9fg6Olxx9H0jO1FYYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761005346; c=relaxed/simple; bh=LDHuftkfJUzPNUuh9kXo6Q3FBGbP3xrGn0Wol3ZW4js=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ekCjbGWMqZeMItqu4tYaEwT+XfgnByLOz6N8lJqRkE3LNLeinRPmXYfgZvwFvEjRXI8aSOyW6x+7tTfURm6ErooieW2UF/Tj0KTtIbsf3+zLuciJjVMjCSFp+S0uLqm8vUkMIljxBgiCSn0dvYd+B7F9uUpMxv2ypyc3YXTHeKk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com; spf=pass smtp.mailfrom=soleen.com; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b=KYH69awI; arc=none smtp.client-ip=209.85.222.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=soleen.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="KYH69awI" Received: by mail-ua1-f44.google.com with SMTP id a1e0cc1a2514c-932dfe14b2eso749958241.3 for ; Mon, 20 Oct 2025 17:09:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1761005342; x=1761610142; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=wh0gNpXNnj8TcdXtVkip0MO+OiSr/N3ozqVkyEmZbiA=; b=KYH69awIcsEGRYA+OBfTsfkJ0DRI4p5B88h1ngPGp8Mxa0Uv7Mc3mD6L4F/lB3r/VN O2mPfkv+bPHAGxZ1Hjkid/CkZXR7U9jBX6h2th1NoPaUfu9HQkOJRChzSCCUcvnGQbH0 Cc6XOOF0BKisxg5NGsIhhPuZPdAWmte3RmteuLddcUjtUZbP7RkNlJ+EhbwoL7eupS43 gmM2nn2TWOY5HpmFE8OMQ8tsNY6bUXdByGPQ8Hq2hn6AneprAPN46iWktku9NG0IMSaz lpCVp7g4eojxuft+xiQbmtB5+v/PgqMCWYlZh/txZrzY+1YYUMSWY/DWlROlnlNLkobj xY9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761005342; x=1761610142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wh0gNpXNnj8TcdXtVkip0MO+OiSr/N3ozqVkyEmZbiA=; b=UNPvvgTt7TOviXe1pjrxfDuSbLpZFJZUv90Afe1wXt1+Hfw+axrYXnHRwoSCviMqx2 GQx2CNwjM4OLSYEvk9m9UyXGzuPOlsRm3xzdlyePgMChaEN0Nkvmvnn48wFkobsD5z25 A6kiM2frhYB0lqHtQFu8rO5xLEFYtGuJI0sWXKcjZ9aBeh0eAt3uk/9goh0IO3PxmdUB +K97R5gXNMezn4wIovWA7Iv7T0EpXzyXFc4+6gK9Yt5Jeagk9GLPUW1XfhDBU3I9FD0u BoE2Kot5TjLlRvL8jTudQw2HD6ElHTJYTUvVrcKhwAslDKiw8XIBYAWx12TzCqEnPTlc SpQA== X-Forwarded-Encrypted: i=1; AJvYcCX1G68KTE354vkd29KCZT14lLId4vXm0hXcAGU0yr1Rxx5a183zaHZ8Saxb4CmoGmuU7efAE1R6X0UjsoI=@vger.kernel.org X-Gm-Message-State: AOJu0Ywg9O6x3vYt0+V9FpuFMcnqnMhx/qFr/VCZlNrOqzNkOiRe1BRy U74i8XHjdMd/C3hm2LO18SueS4lt9safQpLGFnwse3I+YbYPHhKa2T+mt3jPWJfpGSw= X-Gm-Gg: ASbGnctlTeWZyB/XsTLXTs8a+tBNjhLsfVn0V1VJnEq4ZyuCAFnmtC2BiKHxiU7YwiX EO1ntrhLAqa6WBoHgcvP5yw3QKentD6XiCtCsdi+2KliyxKeIVEWYH0JafJ9b6/OKnjCR4vFu9J Gc7EEF7dxBsY+6uvz4RvLLhLDVZkFqpXtinLPnm6mOs8s8+DlOdbE1nQX/8FXVRotRi3jnHAu+A MSX9FOyJLw8icbDEWxHdLUYE9HJ749Y4YO4iaSQVWRardSKgUKvM8GsjpF+TZCKtKGZV3K1wOfv 4h20i5bFDcaMxlHoWoQMfpjHxYku/BEouJmIzv4+WU0tkajXha+xmPSfuewTxQdKAJODGRaMPrG DXkF7sp/PP/hmQeipLyHE6GKKT+c3251igeh2eJBEPbZri7EMDs9gbpv2/FHDCMCl9QR21zAXyL 9eVcQsDDy5m0F88PF4j1BOE2SooGc91f4tanG/sSdT4NuPV5cZF/Qbw788G/g/oiJplVVr3Ei+s jxgwDGvQ8h5RRvJfFTVSw== X-Google-Smtp-Source: AGHT+IG0ndYvX9zTAvm2KWn2q1XKGRimjxxOoCLwpUa9MT4euTBOB019JcZ829K5pfY69/fTaACVlg== X-Received: by 2002:a05:6102:3e85:b0:59c:5e29:dd8d with SMTP id ada2fe7eead31-5d7dd6a3fe6mr5404303137.28.1761005342511; Mon, 20 Oct 2025 17:09:02 -0700 (PDT) Received: from soleen.us-east4-b.c.cloudtop-prod-us-east.internal (53.47.86.34.bc.googleusercontent.com. [34.86.47.53]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-55661f6e351sm2822882e0c.4.2025.10.20.17.09.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Oct 2025 17:09:01 -0700 (PDT) From: Pasha Tatashin To: akpm@linux-foundation.org, brauner@kernel.org, corbet@lwn.net, graf@amazon.com, jgg@ziepe.ca, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, masahiroy@kernel.org, ojeda@kernel.org, pasha.tatashin@soleen.com, pratyush@kernel.org, rdunlap@infradead.org, rppt@kernel.org, tj@kernel.org, jasonmiu@google.com, dmatlack@google.com, skhawaja@google.com Subject: [PATCH v3 3/3] liveupdate: kho: allocate metadata directly from the buddy allocator Date: Mon, 20 Oct 2025 20:08:52 -0400 Message-ID: <20251021000852.2924827-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.51.0.869.ge66316f041-goog In-Reply-To: <20251021000852.2924827-1-pasha.tatashin@soleen.com> References: <20251021000852.2924827-1-pasha.tatashin@soleen.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KHO allocates metadata for its preserved memory map using the slab allocator via kzalloc(). This metadata is temporary and is used by the next kernel during early boot to find preserved memory. A problem arises when KFENCE is enabled. kzalloc() calls can be randomly intercepted by kfence_alloc(), which services the allocation from a dedicated KFENCE memory pool. This pool is allocated early in boot via memblock. When booting via KHO, the memblock allocator is restricted to a "scratch area", forcing the KFENCE pool to be allocated within it. This creates a conflict, as the scratch area is expected to be ephemeral and overwriteable by a subsequent kexec. If KHO metadata is placed in this KFENCE pool, it leads to memory corruption when the next kernel is loaded. To fix this, modify KHO to allocate its metadata directly from the buddy allocator instead of slab. Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation") Signed-off-by: Pasha Tatashin Reviewed-by: Pratyush Yadav Reviewed-by: David Matlack Reviewed-by: Mike Rapoport (Microsoft) --- include/linux/gfp.h | 3 +++ kernel/kexec_handover.c | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 0ceb4e09306c..623bee335383 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -7,6 +7,7 @@ #include #include #include +#include #include =20 struct vm_area_struct; @@ -463,4 +464,6 @@ static inline struct folio *folio_alloc_gigantic_noprof= (int order, gfp_t gfp, /* This should be paired with folio_put() rather than free_contig_range().= */ #define folio_alloc_gigantic(...) alloc_hooks(folio_alloc_gigantic_noprof(= __VA_ARGS__)) =20 +DEFINE_FREE(free_page, void *, free_page((unsigned long)_T)) + #endif /* __LINUX_GFP_H */ diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index e5b91761fbfe..de4466b47455 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -142,7 +142,7 @@ static void *xa_load_or_alloc(struct xarray *xa, unsign= ed long index) if (res) return res; =20 - void *elm __free(kfree) =3D kzalloc(PAGE_SIZE, GFP_KERNEL); + void *elm __free(free_page) =3D (void *)get_zeroed_page(GFP_KERNEL); =20 if (!elm) return ERR_PTR(-ENOMEM); @@ -348,9 +348,9 @@ static_assert(sizeof(struct khoser_mem_chunk) =3D=3D PA= GE_SIZE); static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chu= nk, unsigned long order) { - struct khoser_mem_chunk *chunk __free(kfree) =3D NULL; + struct khoser_mem_chunk *chunk __free(free_page) =3D NULL; =20 - chunk =3D kzalloc(PAGE_SIZE, GFP_KERNEL); + chunk =3D (void *)get_zeroed_page(GFP_KERNEL); if (!chunk) return ERR_PTR(-ENOMEM); =20 --=20 2.51.0.869.ge66316f041-goog