From nobody Fri Oct 10 09:39:02 2025 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29F2772608 for ; Fri, 13 Jun 2025 20:10:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845419; cv=none; b=A6nAzgQRzi11w33lXyKTbLM5OMgprXFUHunj2IIiDlkuOBoYrMBmh0iYZQaLGSN4/6F9pYPCz2q1QmNMoUJ47wcm/pGyQQicdB8W91xBmFat7u8MM1FRg7IQTInIfkc8gfI0Omz1QJwWmbdfVs2QGFR2akmqmyDzQ/zMV7jNk6s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845419; c=relaxed/simple; bh=fWgEQj+r02C6XrfVY1RzYmPj4qIMY/NY9bAmjoct/Mg=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=hZJNg5EEc66rko/RX9XLN5zKKvvM5RqWMDUy4dgRNflhkC8pQQHQS87pfvPXxdwaAjnPbKb/IVlcUUxaB322utbcssPZw/8gI2b41/s5x1Ch3qs/MXm51XBRHiR4Tb9Zl5OOw14ZyeN8Hb9cR3ZWKdKsdRrhJFAUGtTkjpHf0Eg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech; spf=pass smtp.mailfrom=neon.tech; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b=XT/pYkqm; arc=none smtp.client-ip=209.85.208.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=neon.tech Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b="XT/pYkqm" Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-606fdbd20afso4870584a12.1 for ; Fri, 13 Jun 2025 13:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neon.tech; s=google; t=1749845415; x=1750450215; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=HdeF68xR6GC/wjGYdz/zqG29CgX3dAqhdfESrVGyzJ8=; b=XT/pYkqml9wmPiQzJTVSkfRwDjHyCNhGCvffMUZSpBnbcy+CT8779zWzXU5SnYCG26 WMeip6KFyglhdZrLloaxF09/qNOW4cTya2xxmb7Nx0JXU62jhpjK+G1sCaL7HQ7EUnPr Zv3EkpxAc3sMQoZtFJaLxVVItWW37jDOdKs/M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749845415; x=1750450215; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HdeF68xR6GC/wjGYdz/zqG29CgX3dAqhdfESrVGyzJ8=; b=JKAFrQaikJuIuGxV17qL1GxNz+LfOYNGtQUFue4kMzQ1Me4c+SnTeZFzF7oLr6Qnr/ IY0VqNLra7//a7K+FCswAv7XqSKojsKG/Lv1vn6XFKwPcqNNcULoioi2hmz3HfocBTiK ftbsVs0acJWnxYV45uuuOA6oS2xnr1tYF3uFdxstMS6lMNoRW7+a4ZydTIzlnRvVhyMY qCNRdltLsBDa60B9/KMdPUgBpkeillJl0ynMMnhkL9GoPfcf6rD21oERTQKk3jZ/n2dy dc18/8HwC5opEW9r8JKFB1dee3CxuDvZa1yKlro6dU9J+HeFTrfcT8sKWAaCXnSl8EA9 Z6bQ== X-Gm-Message-State: AOJu0YxY8oRDRdyF7Z67R+p5Xw07o8Jwv7ofxFapzM8220ImeDS+o9n0 QI88Gn+nGI7pCj8+MnGK63bL1h6TUakadiefVBkAT4LdP6DjfKdIQL9e02SS4QFuKwiuGq/5JoC Rp0rEQ+A= X-Gm-Gg: ASbGncvkYweQqH4Q0xGCYo8mwXz0NvNxzGRJW6x/xNQpqM85cTYHGymfOwuyOzQypls y+7SzrMLLK2xNucImIKSn9HJNzsm/vvkmf6EuGtP2sw0g+DByviBbYYSHg51SMn74/pMngq1dLX Uz7bDJcIsAmm3vPLR+xy1L0jBvtRDTwKtG7WJMjV/zIcy2C++Srkyr2IJIMapZhaKTlDCj9pq62 zUYe0sTcOyFp8fNthqZ5a6J+ljRXseyya4wQY8fmGQiyqRZWs4y+LAd30a1tbSsXG42tjH4nbRr MKgw3Y9yldWtSSa8uSdfsgPRdAUf9+W8O4mYAvaE+NLEhTVKjveTOzkeKt3zz95RcZTtuV5b7g6 p X-Google-Smtp-Source: AGHT+IFEvm0Xpn9ySXG9NpB9ymgE7br5qW9afaQGRT4NGMiC5EQVxt7wMazyNoKPE1tRNxJz70C3bQ== X-Received: by 2002:a05:6402:27c8:b0:608:66a3:fec with SMTP id 4fb4d7f45d1cf-608d088d800mr452063a12.2.1749845415004; Fri, 13 Jun 2025 13:10:15 -0700 (PDT) Received: from [192.168.86.142] ([84.65.228.220]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-608b48cd687sm1663842a12.18.2025.06.13.13.10.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Jun 2025 13:10:14 -0700 (PDT) Message-ID: <3e767aa4-c783-4857-b34e-fdf3f20bd94f@neon.tech> Date: Fri, 13 Jun 2025 21:10:13 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [PATCH v4 1/4] x86/mm: Update mapped addresses in phys_{pmd,pud}_init() From: Em Sharnoff To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org Cc: Ingo Molnar , "H. Peter Anvin" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , "Edgecombe, Rick P" , Oleg Vasilev , Arthur Petukhovsky , Stefan Radig , Misha Sakhnov References: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Language: en-US In-Reply-To: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently kernel_physical_mapping_init() and its dependents return the last physical address mapped ('paddr_last'). This makes it harder to cleanly handle allocation errors in those functions. 'paddr_last' is used to update 'pfn_mapped'/'max_pfn_mapped', so: 1. Introduce add_paddr_range_mapped() to do the update, translating from physical addresses to pfns 2. Call add_paddr_range_mapped() in phys_pud_init() where 'paddr_last' would otherwise be updated due to 1Gi pages. - Note: this includes places where we set 'paddr_last =3D paddr_next', as was added in 20167d3421a0 ("x86-64: Fix accounting in kernel_physical_mapping_init()") add_paddr_range_mapped() is probably too expensive to be called every time a page is updated, so instead, phys_pte_init() continues to return 'paddr_last', and phys_pmd_init() calls add_paddr_range_mapped() only at the end of the loop (should mean it's called every 1Gi). Signed-off-by: Em Sharnoff --- Changelog: - v4: Add this patch --- arch/x86/include/asm/pgtable.h | 3 +- arch/x86/mm/init.c | 23 +++++---- arch/x86/mm/init_32.c | 6 ++- arch/x86/mm/init_64.c | 88 +++++++++++++++++----------------- arch/x86/mm/mm_internal.h | 13 +++-- 5 files changed, 69 insertions(+), 64 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 7bd6bd6df4a1..138d55f48a4f 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1244,8 +1244,7 @@ extern int direct_gbpages; void init_mem_mapping(void); void early_alloc_pgt_buf(void); void __init poking_init(void); -unsigned long init_memory_mapping(unsigned long start, - unsigned long end, pgprot_t prot); +void init_memory_mapping(unsigned long start, unsigned long end, pgprot_t = prot); =20 #ifdef CONFIG_X86_64 extern pgd_t trampoline_pgd_entry; diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index bfa444a7dbb0..1461873b44f1 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -529,16 +529,24 @@ bool pfn_range_is_mapped(unsigned long start_pfn, uns= igned long end_pfn) return false; } =20 +/* + * Update max_pfn_mapped and range_pfn_mapped with the range of physical + * addresses mapped. The range may overlap with previous calls to this fun= ction. + */ +void add_paddr_range_mapped(unsigned long start_paddr, unsigned long end_p= addr) +{ + add_pfn_range_mapped(start_paddr >> PAGE_SHIFT, end_paddr >> PAGE_SHIFT); +} + /* * Setup the direct mapping of the physical memory at PAGE_OFFSET. * This runs before bootmem is initialized and gets pages directly from * the physical memory. To access them they are temporarily mapped. */ -unsigned long __ref init_memory_mapping(unsigned long start, - unsigned long end, pgprot_t prot) +void __ref init_memory_mapping(unsigned long start, + unsigned long end, pgprot_t prot) { struct map_range mr[NR_RANGE_MR]; - unsigned long ret =3D 0; int nr_range, i; =20 pr_debug("init_memory_mapping: [mem %#010lx-%#010lx]\n", @@ -548,13 +556,10 @@ unsigned long __ref init_memory_mapping(unsigned long= start, nr_range =3D split_mem_range(mr, 0, start, end); =20 for (i =3D 0; i < nr_range; i++) - ret =3D kernel_physical_mapping_init(mr[i].start, mr[i].end, - mr[i].page_size_mask, - prot); + kernel_physical_mapping_init(mr[i].start, mr[i].end, + mr[i].page_size_mask, prot); =20 - add_pfn_range_mapped(start >> PAGE_SHIFT, ret >> PAGE_SHIFT); - - return ret >> PAGE_SHIFT; + return; } =20 /* diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c index ad662cc4605c..4427ac433041 100644 --- a/arch/x86/mm/init_32.c +++ b/arch/x86/mm/init_32.c @@ -245,7 +245,7 @@ static inline int is_x86_32_kernel_text(unsigned long a= ddr) * of max_low_pfn pages, by creating page tables starting from address * PAGE_OFFSET: */ -unsigned long __init +void __init kernel_physical_mapping_init(unsigned long start, unsigned long end, unsigned long page_size_mask, @@ -382,7 +382,9 @@ kernel_physical_mapping_init(unsigned long start, mapping_iter =3D 2; goto repeat; } - return last_map_addr; + + add_paddr_range_mapped(start, last_map_addr); + return; } =20 #ifdef CONFIG_HIGHMEM diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 7c4f6f591f2b..e729108bee30 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -502,13 +502,13 @@ phys_pte_init(pte_t *pte_page, unsigned long paddr, u= nsigned long paddr_end, /* * Create PMD level page table mapping for physical addresses. The virtual * and physical address have to be aligned at this level. - * It returns the last physical address mapped. */ -static unsigned long __meminit +static void __meminit phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, unsigned long paddr_en= d, unsigned long page_size_mask, pgprot_t prot, bool init) { unsigned long pages =3D 0, paddr_next; + unsigned long paddr_first =3D paddr; unsigned long paddr_last =3D paddr_end; =20 int i =3D pmd_index(paddr); @@ -579,21 +579,25 @@ phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, u= nsigned long paddr_end, spin_unlock(&init_mm.page_table_lock); } update_page_count(PG_LEVEL_2M, pages); - return paddr_last; + /* + * In case of recovery from previous state, add_paddr_range_mapped() may + * be called with an overlapping range from previous operations. + * It is idempotent, so this is ok. + */ + add_paddr_range_mapped(paddr_first, paddr_last); + return; } =20 /* * Create PUD level page table mapping for physical addresses. The virtual * and physical address do not have to be aligned at this level. KASLR can * randomize virtual addresses up to this level. - * It returns the last physical address mapped. */ -static unsigned long __meminit +static void __meminit phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_en= d, unsigned long page_size_mask, pgprot_t _prot, bool init) { unsigned long pages =3D 0, paddr_next; - unsigned long paddr_last =3D paddr_end; unsigned long vaddr =3D (unsigned long)__va(paddr); int i =3D pud_index(vaddr); =20 @@ -619,10 +623,8 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, un= signed long paddr_end, if (!pud_none(*pud)) { if (!pud_leaf(*pud)) { pmd =3D pmd_offset(pud, 0); - paddr_last =3D phys_pmd_init(pmd, paddr, - paddr_end, - page_size_mask, - prot, init); + phys_pmd_init(pmd, paddr, paddr_end, + page_size_mask, prot, init); continue; } /* @@ -640,7 +642,7 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, uns= igned long paddr_end, if (page_size_mask & (1 << PG_LEVEL_1G)) { if (!after_bootmem) pages++; - paddr_last =3D paddr_next; + add_paddr_range_mapped(paddr, paddr_next); continue; } prot =3D pte_pgprot(pte_clrhuge(*(pte_t *)pud)); @@ -653,13 +655,13 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, u= nsigned long paddr_end, pfn_pud(paddr >> PAGE_SHIFT, prot_sethuge(prot)), init); spin_unlock(&init_mm.page_table_lock); - paddr_last =3D paddr_next; + add_paddr_range_mapped(paddr, paddr_next); continue; } =20 pmd =3D alloc_low_page(); - paddr_last =3D phys_pmd_init(pmd, paddr, paddr_end, - page_size_mask, prot, init); + phys_pmd_init(pmd, paddr, paddr_end, + page_size_mask, prot, init); =20 spin_lock(&init_mm.page_table_lock); pud_populate_init(&init_mm, pud, pmd, init); @@ -668,22 +670,23 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, u= nsigned long paddr_end, =20 update_page_count(PG_LEVEL_1G, pages); =20 - return paddr_last; + return; } =20 -static unsigned long __meminit +static void __meminit phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, unsigned long paddr_en= d, unsigned long page_size_mask, pgprot_t prot, bool init) { - unsigned long vaddr, vaddr_end, vaddr_next, paddr_next, paddr_last; + unsigned long vaddr, vaddr_end, vaddr_next, paddr_next; =20 - paddr_last =3D paddr_end; vaddr =3D (unsigned long)__va(paddr); vaddr_end =3D (unsigned long)__va(paddr_end); =20 - if (!pgtable_l5_enabled()) - return phys_pud_init((pud_t *) p4d_page, paddr, paddr_end, - page_size_mask, prot, init); + if (!pgtable_l5_enabled()) { + phys_pud_init((pud_t *) p4d_page, paddr, paddr_end, + page_size_mask, prot, init); + return; + } =20 for (; vaddr < vaddr_end; vaddr =3D vaddr_next) { p4d_t *p4d =3D p4d_page + p4d_index(vaddr); @@ -705,33 +708,32 @@ phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, u= nsigned long paddr_end, =20 if (!p4d_none(*p4d)) { pud =3D pud_offset(p4d, 0); - paddr_last =3D phys_pud_init(pud, paddr, __pa(vaddr_end), - page_size_mask, prot, init); + phys_pud_init(pud, paddr, __pa(vaddr_end), + page_size_mask, prot, init); continue; } =20 pud =3D alloc_low_page(); - paddr_last =3D phys_pud_init(pud, paddr, __pa(vaddr_end), - page_size_mask, prot, init); + phys_pud_init(pud, paddr, __pa(vaddr_end), + page_size_mask, prot, init); =20 spin_lock(&init_mm.page_table_lock); p4d_populate_init(&init_mm, p4d, pud, init); spin_unlock(&init_mm.page_table_lock); } =20 - return paddr_last; + return; } =20 -static unsigned long __meminit +static void __meminit __kernel_physical_mapping_init(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t prot, bool init) { bool pgd_changed =3D false; - unsigned long vaddr, vaddr_start, vaddr_end, vaddr_next, paddr_last; + unsigned long vaddr, vaddr_start, vaddr_end, vaddr_next; =20 - paddr_last =3D paddr_end; vaddr =3D (unsigned long)__va(paddr_start); vaddr_end =3D (unsigned long)__va(paddr_end); vaddr_start =3D vaddr; @@ -744,16 +746,14 @@ __kernel_physical_mapping_init(unsigned long paddr_st= art, =20 if (pgd_val(*pgd)) { p4d =3D (p4d_t *)pgd_page_vaddr(*pgd); - paddr_last =3D phys_p4d_init(p4d, __pa(vaddr), - __pa(vaddr_end), - page_size_mask, - prot, init); + phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), + page_size_mask, prot, init); continue; } =20 p4d =3D alloc_low_page(); - paddr_last =3D phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), - page_size_mask, prot, init); + phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), + page_size_mask, prot, init); =20 spin_lock(&init_mm.page_table_lock); if (pgtable_l5_enabled()) @@ -769,7 +769,7 @@ __kernel_physical_mapping_init(unsigned long paddr_star= t, if (pgd_changed) sync_global_pgds(vaddr_start, vaddr_end - 1); =20 - return paddr_last; + return; } =20 =20 @@ -777,15 +777,15 @@ __kernel_physical_mapping_init(unsigned long paddr_st= art, * Create page table mapping for the physical memory for specific physical * addresses. Note that it can only be used to populate non-present entrie= s. * The virtual and physical addresses have to be aligned on PMD level - * down. It returns the last physical address mapped. + * down. */ -unsigned long __meminit +void __meminit kernel_physical_mapping_init(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t prot) { - return __kernel_physical_mapping_init(paddr_start, paddr_end, - page_size_mask, prot, true); + __kernel_physical_mapping_init(paddr_start, paddr_end, + page_size_mask, prot, true); } =20 /* @@ -794,14 +794,14 @@ kernel_physical_mapping_init(unsigned long paddr_star= t, * when updating the mapping. The caller is responsible to flush the TLBs = after * the function returns. */ -unsigned long __meminit +void __meminit kernel_physical_mapping_change(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask) { - return __kernel_physical_mapping_init(paddr_start, paddr_end, - page_size_mask, PAGE_KERNEL, - false); + __kernel_physical_mapping_init(paddr_start, paddr_end, + page_size_mask, PAGE_KERNEL, + false); } =20 #ifndef CONFIG_NUMA diff --git a/arch/x86/mm/mm_internal.h b/arch/x86/mm/mm_internal.h index 3f37b5c80bb3..6fea5f7edd48 100644 --- a/arch/x86/mm/mm_internal.h +++ b/arch/x86/mm/mm_internal.h @@ -10,13 +10,12 @@ static inline void *alloc_low_page(void) =20 void early_ioremap_page_table_range_init(void); =20 -unsigned long kernel_physical_mapping_init(unsigned long start, - unsigned long end, - unsigned long page_size_mask, - pgprot_t prot); -unsigned long kernel_physical_mapping_change(unsigned long start, - unsigned long end, - unsigned long page_size_mask); +void add_paddr_range_mapped(unsigned long start_paddr, unsigned long end_p= addr); + +void kernel_physical_mapping_init(unsigned long start, unsigned long end, + unsigned long page_size_mask, pgprot_t prot); +void kernel_physical_mapping_change(unsigned long start, unsigned long end, + unsigned long page_size_mask); void zone_sizes_init(void); =20 extern int after_bootmem; --=20 2.39.5 From nobody Fri Oct 10 09:39:02 2025 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E07B423D28A for ; Fri, 13 Jun 2025 20:11:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845467; cv=none; b=DpA0WBajKkkEtGbhwFoQg/PEXuMs2pprATusXO1TEtz0egJOfzws2V/2VIVAcJqeJ2k8vBfAzDYnqUyNbSxkRQ/lvR4DVPmp4s4vkqyB0Nc/qqU/seT46PWzwSYfEVxnl2QyLDkPuTfR4dza1b2fHUfjtT8Q8WHbXBoWjMgUtBw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845467; c=relaxed/simple; bh=XqpLK8gW6rr2nWZQyZ09pxl4n3XyQrWST5fV0M7NL+o=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=XpeDwq0wpKGK3R0s0yzPixk5UrQatAs3xSpqjl+XrEI/tz66YCtmWnMN0LcehAoq0swY9J1JZbNf+ojG+GOWpdwaVBvGU3kjsyC5zinswu1py4LVoGFtWYAKciuTGOJMr5t8NrFgwhtRWr+uOH+cjeSCB0+DRPHOLkmWevukGq0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech; spf=pass smtp.mailfrom=neon.tech; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b=Qfi3FEJY; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=neon.tech Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b="Qfi3FEJY" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-ade5ca8bc69so366184766b.0 for ; Fri, 13 Jun 2025 13:11:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neon.tech; s=google; t=1749845462; x=1750450262; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=l3rzndryGKj8FfYnmFJ89LSS6fcKwQtrFfmMlORfHhY=; b=Qfi3FEJY+Bur1KtgMRV8dx/JgKdS/I+7Ii7l1/OlnGCXoOY1CBtzWqOi0Xiq7hZjwt TSvDeDawR3iaie5KSK0lSh4RWaljLOI1jK2YMX4bB5TSrT/1FsPz15G7Kj1MbOp240w0 Zh7W/Q0bRsVD6RfwEyQcZ2IjmNYG7YurRiKA8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749845462; x=1750450262; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=l3rzndryGKj8FfYnmFJ89LSS6fcKwQtrFfmMlORfHhY=; b=HMR45rO5mO97IDgx9ZEhE5kFvumc/O+17k2Nws6ym9AJuyob/C7q0wEo5rsIWTN5Lw HUvgkG5fMp1/qNLPajJihL486yhvv4i9s0CmDx8xs+a2yvkZM7C6L6l8t+Xc/HSGhTBu YqZfGcQYxiWCMDFqCvkZBcK3sCgb4QKraepGh7FafuljZzYC+P2v/2BXMrjhYz2g1j2C BQehcJxffUik8IX3dxjjcfmtM1GtvB5kC6pFXljCdD3NjPmz3iGHp1eYVLQKc5nIrMZG sZl0l3Rs309b/8zcuSyvUEEH6Hz+MVC7kjD++ZaPc9Y0SfCC7C6qF+uHLaeZxCfJP84L RMyw== X-Gm-Message-State: AOJu0YzkmJpTK3ZGwRtTXbEwrdXT+qFUG8FmiOeTxiTpRA197VaMnoQZ cOmXH0lDcZOOQR7M5CZdIo9JkYAZ74xXomTRpyetAugKPbUvU+8ucUegXF2BljbKc4NF8mTL9k8 hJ9Pkn8E= X-Gm-Gg: ASbGnctjuJXrPE9QRMGpGCIOKz8vhBegiz8tlzR86OS4pLui+g+W5JKkUCWhU1auCMi cqrArGKzeqDxKUIC/VH0I6yE7LgpkzmgJH9iqN0YnOiMmkH9Sorbea47/46vjIeHsosPO+t5hKH q1BAYfnRp2n747fvjmiid5Xec2OyCv92DPnpisnD+1oBnDMZ05IB9XtRsEfZTns/bMqlcxvqNcq T2MebhX3zyALdfLi19anQkZ+1J/hdb8rgenrGiAYxA4kHK1C1klZoKJ6RJZbl1yfRrYVDHx4i3i p2Z0YcIDzX4UyzYoEV6mJdyMXA20ztm7VNctaWL6yElKB/8uBYtIJnQkRMgyIbHVAA== X-Google-Smtp-Source: AGHT+IHEmyRU5vN+dhA+MrNmjXjScKNcFhj4rTx+ZBf6G3WYchF0YtJ7ycBplglcHfOlSP7d5PWLzQ== X-Received: by 2002:a17:907:1b18:b0:ad2:2fe3:7074 with SMTP id a640c23a62f3a-adfad39c00amr46953066b.14.1749845461793; Fri, 13 Jun 2025 13:11:01 -0700 (PDT) Received: from [192.168.86.142] ([84.65.228.220]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-adfad4505a7sm22175266b.51.2025.06.13.13.11.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Jun 2025 13:11:01 -0700 (PDT) Message-ID: <11b78ad4-9fdf-462c-90e5-469f06064376@neon.tech> Date: Fri, 13 Jun 2025 21:11:00 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [PATCH v4 2/4] x86/mm: Allow error returns from phys_*_init() From: Em Sharnoff To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org Cc: Ingo Molnar , "H. Peter Anvin" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , "Edgecombe, Rick P" , Oleg Vasilev , Arthur Petukhovsky , Stefan Radig , Misha Sakhnov References: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Language: en-US In-Reply-To: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Preparation for returning errors when alloc_low_page() fails. phys_pte_init() is excluded because it can't fail, and it's useful for it to return 'paddr_last' instead. This patch depends on the previous patch ("x86/mm: Update mapped addresses in phys_{pmd,pud}_init()"). Signed-off-by: Em Sharnoff --- Changleog: - v2: Switch from special-casing zero value to using ERR_PTR() - v3: Fix -Wint-conversion errors - v4: Switch return type to int, split alloc handling into separate patch. --- arch/x86/include/asm/pgtable.h | 2 +- arch/x86/mm/init.c | 14 +++-- arch/x86/mm/init_32.c | 4 +- arch/x86/mm/init_64.c | 100 ++++++++++++++++++++++----------- arch/x86/mm/mem_encrypt_amd.c | 8 ++- arch/x86/mm/mm_internal.h | 8 +-- 6 files changed, 87 insertions(+), 49 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 138d55f48a4f..b09194c42688 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1244,7 +1244,7 @@ extern int direct_gbpages; void init_mem_mapping(void); void early_alloc_pgt_buf(void); void __init poking_init(void); -void init_memory_mapping(unsigned long start, unsigned long end, pgprot_t = prot); +int init_memory_mapping(unsigned long start, unsigned long end, pgprot_t p= rot); =20 #ifdef CONFIG_X86_64 extern pgd_t trampoline_pgd_entry; diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 1461873b44f1..1dc8fbd8cb63 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -543,11 +543,12 @@ void add_paddr_range_mapped(unsigned long start_paddr= , unsigned long end_paddr) * This runs before bootmem is initialized and gets pages directly from * the physical memory. To access them they are temporarily mapped. */ -void __ref init_memory_mapping(unsigned long start, +int __ref init_memory_mapping(unsigned long start, unsigned long end, pgprot_t prot) { struct map_range mr[NR_RANGE_MR]; int nr_range, i; + int ret; =20 pr_debug("init_memory_mapping: [mem %#010lx-%#010lx]\n", start, end - 1); @@ -555,11 +556,14 @@ void __ref init_memory_mapping(unsigned long start, memset(mr, 0, sizeof(mr)); nr_range =3D split_mem_range(mr, 0, start, end); =20 - for (i =3D 0; i < nr_range; i++) - kernel_physical_mapping_init(mr[i].start, mr[i].end, - mr[i].page_size_mask, prot); + for (i =3D 0; i < nr_range; i++) { + ret =3D kernel_physical_mapping_init(mr[i].start, mr[i].end, + mr[i].page_size_mask, prot); + if (ret) + return ret; + } =20 - return; + return 0; } =20 /* diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c index 4427ac433041..57bd154c206d 100644 --- a/arch/x86/mm/init_32.c +++ b/arch/x86/mm/init_32.c @@ -245,7 +245,7 @@ static inline int is_x86_32_kernel_text(unsigned long a= ddr) * of max_low_pfn pages, by creating page tables starting from address * PAGE_OFFSET: */ -void __init +int __init kernel_physical_mapping_init(unsigned long start, unsigned long end, unsigned long page_size_mask, @@ -384,7 +384,7 @@ kernel_physical_mapping_init(unsigned long start, } =20 add_paddr_range_mapped(start, last_map_addr); - return; + return 0; } =20 #ifdef CONFIG_HIGHMEM diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index e729108bee30..b18ab2dcc799 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -503,7 +503,7 @@ phys_pte_init(pte_t *pte_page, unsigned long paddr, uns= igned long paddr_end, * Create PMD level page table mapping for physical addresses. The virtual * and physical address have to be aligned at this level. */ -static void __meminit +static int __meminit phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, unsigned long paddr_en= d, unsigned long page_size_mask, pgprot_t prot, bool init) { @@ -585,7 +585,7 @@ phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, uns= igned long paddr_end, * It is idempotent, so this is ok. */ add_paddr_range_mapped(paddr_first, paddr_last); - return; + return 0; } =20 /* @@ -593,12 +593,14 @@ phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, u= nsigned long paddr_end, * and physical address do not have to be aligned at this level. KASLR can * randomize virtual addresses up to this level. */ -static void __meminit +static int __meminit phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_en= d, unsigned long page_size_mask, pgprot_t _prot, bool init) { unsigned long pages =3D 0, paddr_next; unsigned long vaddr =3D (unsigned long)__va(paddr); + int ret; + int i =3D pud_index(vaddr); =20 for (; i < PTRS_PER_PUD; i++, paddr =3D paddr_next) { @@ -623,8 +625,10 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, un= signed long paddr_end, if (!pud_none(*pud)) { if (!pud_leaf(*pud)) { pmd =3D pmd_offset(pud, 0); - phys_pmd_init(pmd, paddr, paddr_end, - page_size_mask, prot, init); + ret =3D phys_pmd_init(pmd, paddr, paddr_end, + page_size_mask, prot, init); + if (ret) + return ret; continue; } /* @@ -660,33 +664,39 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, u= nsigned long paddr_end, } =20 pmd =3D alloc_low_page(); - phys_pmd_init(pmd, paddr, paddr_end, - page_size_mask, prot, init); + ret =3D phys_pmd_init(pmd, paddr, paddr_end, + page_size_mask, prot, init); =20 spin_lock(&init_mm.page_table_lock); pud_populate_init(&init_mm, pud, pmd, init); spin_unlock(&init_mm.page_table_lock); + + /* + * Bail only after updating pud to keep progress from pmd across + * retries. + */ + if (ret) + return ret; } =20 update_page_count(PG_LEVEL_1G, pages); =20 - return; + return 0; } =20 -static void __meminit +static int __meminit phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, unsigned long paddr_en= d, unsigned long page_size_mask, pgprot_t prot, bool init) { unsigned long vaddr, vaddr_end, vaddr_next, paddr_next; + int ret; =20 vaddr =3D (unsigned long)__va(paddr); vaddr_end =3D (unsigned long)__va(paddr_end); =20 - if (!pgtable_l5_enabled()) { - phys_pud_init((pud_t *) p4d_page, paddr, paddr_end, - page_size_mask, prot, init); - return; - } + if (!pgtable_l5_enabled()) + return phys_pud_init((pud_t *) p4d_page, paddr, paddr_end, + page_size_mask, prot, init); =20 for (; vaddr < vaddr_end; vaddr =3D vaddr_next) { p4d_t *p4d =3D p4d_page + p4d_index(vaddr); @@ -708,24 +718,33 @@ phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, u= nsigned long paddr_end, =20 if (!p4d_none(*p4d)) { pud =3D pud_offset(p4d, 0); - phys_pud_init(pud, paddr, __pa(vaddr_end), - page_size_mask, prot, init); + ret =3D phys_pud_init(pud, paddr, __pa(vaddr_end), + page_size_mask, prot, init); + if (ret) + return ret; continue; } =20 pud =3D alloc_low_page(); - phys_pud_init(pud, paddr, __pa(vaddr_end), - page_size_mask, prot, init); + ret =3D phys_pud_init(pud, paddr, __pa(vaddr_end), + page_size_mask, prot, init); =20 spin_lock(&init_mm.page_table_lock); p4d_populate_init(&init_mm, p4d, pud, init); spin_unlock(&init_mm.page_table_lock); + + /* + * Bail only after updating p4d to keep progress from pud across + * retries. + */ + if (ret) + return ret; } =20 - return; + return 0; } =20 -static void __meminit +static int __meminit __kernel_physical_mapping_init(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask, @@ -733,6 +752,7 @@ __kernel_physical_mapping_init(unsigned long paddr_star= t, { bool pgd_changed =3D false; unsigned long vaddr, vaddr_start, vaddr_end, vaddr_next; + int ret; =20 vaddr =3D (unsigned long)__va(paddr_start); vaddr_end =3D (unsigned long)__va(paddr_end); @@ -746,14 +766,16 @@ __kernel_physical_mapping_init(unsigned long paddr_st= art, =20 if (pgd_val(*pgd)) { p4d =3D (p4d_t *)pgd_page_vaddr(*pgd); - phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), - page_size_mask, prot, init); + ret =3D phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), + page_size_mask, prot, init); + if (ret) + return ret; continue; } =20 p4d =3D alloc_low_page(); - phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), - page_size_mask, prot, init); + ret =3D phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), + page_size_mask, prot, init); =20 spin_lock(&init_mm.page_table_lock); if (pgtable_l5_enabled()) @@ -761,15 +783,22 @@ __kernel_physical_mapping_init(unsigned long paddr_st= art, else p4d_populate_init(&init_mm, p4d_offset(pgd, vaddr), (pud_t *) p4d, init); - spin_unlock(&init_mm.page_table_lock); + + /* + * Bail only after updating pgd/p4d to keep progress from p4d + * across retries. + */ + if (ret) + return ret; + pgd_changed =3D true; } =20 if (pgd_changed) sync_global_pgds(vaddr_start, vaddr_end - 1); =20 - return; + return 0; } =20 =20 @@ -779,13 +808,13 @@ __kernel_physical_mapping_init(unsigned long paddr_st= art, * The virtual and physical addresses have to be aligned on PMD level * down. */ -void __meminit +int __meminit kernel_physical_mapping_init(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t prot) { - __kernel_physical_mapping_init(paddr_start, paddr_end, - page_size_mask, prot, true); + return __kernel_physical_mapping_init(paddr_start, paddr_end, + page_size_mask, prot, true); } =20 /* @@ -794,14 +823,14 @@ kernel_physical_mapping_init(unsigned long paddr_star= t, * when updating the mapping. The caller is responsible to flush the TLBs = after * the function returns. */ -void __meminit +int __meminit kernel_physical_mapping_change(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask) { - __kernel_physical_mapping_init(paddr_start, paddr_end, - page_size_mask, PAGE_KERNEL, - false); + return __kernel_physical_mapping_init(paddr_start, paddr_end, + page_size_mask, PAGE_KERNEL, + false); } =20 #ifndef CONFIG_NUMA @@ -980,8 +1009,11 @@ int arch_add_memory(int nid, u64 start, u64 size, { unsigned long start_pfn =3D start >> PAGE_SHIFT; unsigned long nr_pages =3D size >> PAGE_SHIFT; + int ret; =20 - init_memory_mapping(start, start + size, params->pgprot); + ret =3D init_memory_mapping(start, start + size, params->pgprot); + if (ret) + return ret; =20 return add_pages(nid, start_pfn, nr_pages, params); } diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c index 7490ff6d83b1..04e0b92eb9ad 100644 --- a/arch/x86/mm/mem_encrypt_amd.c +++ b/arch/x86/mm/mem_encrypt_amd.c @@ -444,9 +444,11 @@ static int __init early_set_memory_enc_dec(unsigned lo= ng vaddr, * kernel_physical_mapping_change() does not flush the TLBs, so * a TLB flush is required after we exit from the for loop. */ - kernel_physical_mapping_change(__pa(vaddr & pmask), - __pa((vaddr_end & pmask) + psize), - split_page_size_mask); + ret =3D kernel_physical_mapping_change(__pa(vaddr & pmask), + __pa((vaddr_end & pmask) + psize), + split_page_size_mask); + if (ret) + return ret; } =20 ret =3D 0; diff --git a/arch/x86/mm/mm_internal.h b/arch/x86/mm/mm_internal.h index 6fea5f7edd48..dacf3c924fbd 100644 --- a/arch/x86/mm/mm_internal.h +++ b/arch/x86/mm/mm_internal.h @@ -12,10 +12,10 @@ void early_ioremap_page_table_range_init(void); =20 void add_paddr_range_mapped(unsigned long start_paddr, unsigned long end_p= addr); =20 -void kernel_physical_mapping_init(unsigned long start, unsigned long end, - unsigned long page_size_mask, pgprot_t prot); -void kernel_physical_mapping_change(unsigned long start, unsigned long end, - unsigned long page_size_mask); +int kernel_physical_mapping_init(unsigned long start, unsigned long end, + unsigned long page_size_mask, pgprot_t prot); +int kernel_physical_mapping_change(unsigned long start, unsigned long end, + unsigned long page_size_mask); void zone_sizes_init(void); =20 extern int after_bootmem; --=20 2.39.5 From nobody Fri Oct 10 09:39:02 2025 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 108E023C4F8 for ; Fri, 13 Jun 2025 20:11:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845513; cv=none; b=MfHi/98YrlJm9OIHH0wWMh5OeZcREMsM6ClHd8YrkaR6h+88Itg8HXdJyRMGox79NHtSIBPpsqDbTZ6GEJLGub/gTKirZsLj/VU8qaMo4teMkwqy451vWdZCM4pBH+X+y7wUoYB1Sifv2vA4KNAjfQMAIxwu/bItdDY/9RE3his= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845513; c=relaxed/simple; bh=W87Z/HpxToGGkm2QWkvPkaACSn8wTCoH/hM+Wwmd+T4=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=Q0x6XShw9Wcq0MJrP2lcvA1QJ944R7dCLbFFj+Mfg+DKgLfGdZ6cA5OaUmgiDY2qY9V6Qcqo0cLqxOBuLbhxamFrowhtdKn/WOZRIaCglN2tBQbmgCfJGTycpHrMI/LnbLZRCjVj3zWRuNxY2d7PoiD9V0JQqycYAzm2ZQMq79A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech; spf=pass smtp.mailfrom=neon.tech; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b=svaf5b/m; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=neon.tech Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b="svaf5b/m" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-adeaa4f3d07so462646066b.0 for ; Fri, 13 Jun 2025 13:11:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neon.tech; s=google; t=1749845510; x=1750450310; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=SZyPhDp3clF+2nAcuBvrsAf8X3aSjPmMQy6JJ8napoM=; b=svaf5b/mwGj4NuXdOHCqeU5F8+XIP2tEA83Zc+QEY4flCRGVbvX2Wk+Sb9kiPuoMU/ NjNufawFQGhq/YMjrg9ANj/7CYSlQJ/aQam0iD2F0zMBBO63i2G7kFx0FAPLPxF6Ck6d wWlem3RZ0NSIr/AHCeSnjBM2xSY98ZMIkjLSA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749845510; x=1750450310; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SZyPhDp3clF+2nAcuBvrsAf8X3aSjPmMQy6JJ8napoM=; b=jAJXjpzFLgooL+pleY8/yu2IZ9IBPMEUJXBHUrFhMfOaDoGjX3VPW+IgKtr9l4Niqm TKZDmEI6Rxg/fcxd999nxKGdz0+fktCJsf/TEh1xpnCZzOiRunCZsU/MxWprypzsFMYe b1xU3ougx299DGTcNBP8eB7nLTTFCjbUDX5PGnvwR2VdRbs4dilUGy0rT9DWBMeiykU0 pG6C2orP9FGzj2PfSjg9tA6HQ6mUIwuVqe9uBfmCpqzmJlgwdwawZbSfQFnnPHu9AYMQ eR7RiWup4xJ80nt2UwiqzN3OVTdT7N9VMD0kvG3hVdmBVKc3ODYuhIb+fFiFnOWEsJ0C f6JA== X-Gm-Message-State: AOJu0YzwV/oKENTNhG9xiH+tIOF0Aif+cPKsWT26b/vehx/mdJN4MhGA p3Go/R5vyH/dkzBbjbOdmJIec404ffO5ChZo4afjOLXTg1pcsATnNa6xu0BufgMjq+N/40Y+vgO 2mCd0K+0= X-Gm-Gg: ASbGncuLqHVka7yxGumh/CBQ/l35xN4CtfdCzJGWjX2S8I5DcTwKnUsw/sZo00wLldk J1PkPy2xwv1ld80pvN0RIZc1nyYyRZnzkKAmi6ftRQPy7IlntM4PMfXUiAZNIto7eSenICg6UwT hoGRUZavzAnqb4QGtKhpCNiYGBFZGyhzs3vIQkt3C9R/ksZWqRciEpTE8HyXvhaH7BUqzb0yNG7 DIlnvHsa2ehzJ9+OgnsXiQRfTpZ+P6m7/ybYqVuqKAkbIS/HoUs6+IqWuajkXRUxwlu/tpk6mov ZRrqWKYQ8potA37aerN0QoZQfBPNG2wtr769rWX9CyfTc8RtGp/wWkOKwErqKmQymQ== X-Google-Smtp-Source: AGHT+IEPRO9S3H+OVVnpMZsQTJUxr/PZCZ8mq86dBJISkt9MMHMssWxJk3y2H9thIDYzD1Iua45FeA== X-Received: by 2002:a17:907:1c0c:b0:ade:3b16:faa9 with SMTP id a640c23a62f3a-adfad5c771bmr52408266b.29.1749845509990; Fri, 13 Jun 2025 13:11:49 -0700 (PDT) Received: from [192.168.86.142] ([84.65.228.220]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-adec88fed6esm178736366b.103.2025.06.13.13.11.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Jun 2025 13:11:49 -0700 (PDT) Message-ID: <062402c3-e640-4578-b1b1-318a0a728a93@neon.tech> Date: Fri, 13 Jun 2025 21:11:48 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [PATCH v4 3/4] x86/mm: Handle alloc failure in phys_*_init() From: Em Sharnoff To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org Cc: Ingo Molnar , "H. Peter Anvin" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , "Edgecombe, Rick P" , Oleg Vasilev , Arthur Petukhovsky , Stefan Radig , Misha Sakhnov References: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Language: en-US In-Reply-To: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" During memory hotplug, allocation failures in phys_*_init() aren't handled, which results in a null pointer dereference if they occur. This patch depends on the previous patch ("x86/mm: Allow error returns from phys_*_init()"). Signed-off-by: Em Sharnoff --- Changelog: - v4: Split this patch out from the error handling changes --- arch/x86/mm/init_64.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index b18ab2dcc799..a585daa76d69 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -572,6 +572,8 @@ phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, uns= igned long paddr_end, } =20 pte =3D alloc_low_page(); + if (!pte) + return -ENOMEM; paddr_last =3D phys_pte_init(pte, paddr, paddr_end, new_prot, init); =20 spin_lock(&init_mm.page_table_lock); @@ -664,6 +666,8 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, uns= igned long paddr_end, } =20 pmd =3D alloc_low_page(); + if (!pmd) + return -ENOMEM; ret =3D phys_pmd_init(pmd, paddr, paddr_end, page_size_mask, prot, init); =20 @@ -726,6 +730,8 @@ phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, uns= igned long paddr_end, } =20 pud =3D alloc_low_page(); + if (!pud) + return -ENOMEM; ret =3D phys_pud_init(pud, paddr, __pa(vaddr_end), page_size_mask, prot, init); =20 @@ -774,6 +780,8 @@ __kernel_physical_mapping_init(unsigned long paddr_star= t, } =20 p4d =3D alloc_low_page(); + if (!p4d) + return -ENOMEM; ret =3D phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), page_size_mask, prot, init); =20 --=20 2.39.5 From nobody Fri Oct 10 09:39:02 2025 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1354C2343B6 for ; Fri, 13 Jun 2025 20:12:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845561; cv=none; b=fd/RcctUxzEAMOpc8yOet2Boqnov65Ln0zeHFi0kr/u3GGJFM8ArHriqWFOLfkgjefDluxlC0qLj1ohGN4Q+FZvtfWg7Ua+zkGDBGhluZvT8BsYja0TJOi4Y0/cikulkefFRShIqkYOAJgmpvw2Il1PCqWcVrMg9kW+PTd6924w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749845561; c=relaxed/simple; bh=iyGFDId6gNGGAgQ3Tspa6TixUr1kWvUB+RXOYO05TZE=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=SyZYbceamqbSj0XbYp5+zwjMCo+DlywaUECOQJpACXCuftszN8f3EoDoJgZ12un2QD833Pu3eXruxWHO1wF/bTwMgPgftEdaLk9i8iDGN/AQViMnnDYrHxSVzoqk/9b9c8x0IxPZQiqohGtW5uqgzsAavtTIBdLIjOQ3mrCefsw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech; spf=pass smtp.mailfrom=neon.tech; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b=OIgDRmNG; arc=none smtp.client-ip=209.85.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=neon.tech Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=neon.tech Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=neon.tech header.i=@neon.tech header.b="OIgDRmNG" Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-606b58241c9so4175939a12.3 for ; Fri, 13 Jun 2025 13:12:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neon.tech; s=google; t=1749845558; x=1750450358; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=NweKF8SVH3injbfSiS2fNVqMZQ6C6K10QpkWTAAVIKo=; b=OIgDRmNGeKzw/3vBMT+6UjAK1VTHfoRvaT5mpKXyfe5ocdDMkLOMXV/MJHelMhGeUG AuBUUUMRWGtZdgLafMcwDjajuMtTcagxyk0F8dPFFNG9aCX4I759BLk2HdIWx6GJl2Xw koijXseWKn64X3y75S2epVvU6ulVeEjQVfQW8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749845558; x=1750450358; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NweKF8SVH3injbfSiS2fNVqMZQ6C6K10QpkWTAAVIKo=; b=lbnWx8JvnRd9BNCt833ZhEpxKIqvj/ZNH898R3ut9iROO+8QtgHRdJqGE/gJncjrsC Xw3KPwwgPpHdRBabCBsRcL2lNQjbpcCW0UT+LWvVRnn7xtWym+AJCrI9fojt78Q1sVPu Jg7MpXIX3jI7uu9RnVjYIKdCpHRLzIXBTJ0Fjj9bKmt8lU4xvp1l4XGrVv6s7JhfqQC4 ydgu6JU974jH5zmg1QTkzNUI3iaD7op81Lan1wHrKqAeWOP9xeDwyAjgd8v/WSxswPx9 uJC/QNklDg8LPssoORzdV1pcyudVexq2KQpaV1Tb3CKvgt+kP3SGbTHrMl0XM5MBhveV 2N8A== X-Gm-Message-State: AOJu0YypeBkge9EG17qyezJXY5iUR/NFJeQVu7w2bXgyYuIMTPL5mn4Z vybO5T4oaujhNQzB9L6sDip63blBjENle14v2XwxazWVdZM2YBTmZCLot1381NssbrXyXVCoDfU MD6bOoYc= X-Gm-Gg: ASbGncvZetVtCpfa/MVy/FluTYZfkou/BVVGHs6hu6l7KzCSX8Tp0a51go26s+dQKI3 Z56+Ziu3tm6of4oYaMLdBztY6dvQmed4KdTCtRQNPFu3Z0Tuf8k+bF5JwUuG0qek3DpJLFqIxu+ WIalEy09LDWbU7eJlEO7ZA1tYeZ+fXijzyVEhiq5yMmD312gKb8Go5/aax7f3rjQz+ahV/hQ5jt b5GEz/wRmNtHlEdWowNvV16PNBuDRSxfs0+u1BKKuNGHeVHS5cbCY4oRP/2cg8g8ZKgEbfMBdcj K0jr8LD+pRtY/aA/nWynwc7QfdX/XS5IGLPfm0axi1unuqJ8x9rycvHFbHZilPQhBA== X-Google-Smtp-Source: AGHT+IHZOIdE98Xx/meIwe4RQMdXHRB+vSVgcGbiS6IiDa0l13qNjjNb8DTbd8V27rMfCiXEpAIELg== X-Received: by 2002:a05:6402:d0b:b0:607:3344:6ef1 with SMTP id 4fb4d7f45d1cf-608d0979c1emr379552a12.29.1749845557982; Fri, 13 Jun 2025 13:12:37 -0700 (PDT) Received: from [192.168.86.142] ([84.65.228.220]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-608b48a8bf8sm1640985a12.11.2025.06.13.13.12.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Jun 2025 13:12:37 -0700 (PDT) Message-ID: Date: Fri, 13 Jun 2025 21:12:36 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [PATCH v4 4/4] x86/mm: Use GFP_KERNEL for alloc_low_pages() after boot From: Em Sharnoff To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org Cc: Ingo Molnar , "H. Peter Anvin" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Borislav Petkov , "Edgecombe, Rick P" , Oleg Vasilev , Arthur Petukhovsky , Stefan Radig , Misha Sakhnov References: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Language: en-US In-Reply-To: <7d0d307d-71eb-4913-8023-bccc7a8a4a3d@neon.tech> Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently it's GFP_ATOMIC. GFP_KERNEL seems more correct. From Ingo M. [1] > There's no real reason why it should be GFP_ATOMIC AFAICS, other than > some historic inertia that nobody bothered to fix. and previously Mike R. [2] > The few callers that effectively use page allocator for the direct map > updates are gart_iommu_init() and memory hotplug. Neither of them > happen in an atomic context so there is no reason to use GFP_ATOMIC > for these allocations. > > Replace GFP_ATOMIC with GFP_KERNEL to avoid using atomic reserves for > allocations that do not require that. [1]: https://lore.kernel.org/all/aEE6_S2a-1tk1dtI@gmail.com/ [2]: https://lore.kernel.org/all/20211111110241.25968-5-rppt@kernel.org/ Signed-off-by: Em Sharnoff --- Changelog: - v2: Add this patch - v3: No changes - v4: No changes --- arch/x86/mm/init.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 1dc8fbd8cb63..80eddcef992b 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -131,7 +131,7 @@ __ref void *alloc_low_pages(unsigned int num) unsigned int order; =20 order =3D get_order((unsigned long)num << PAGE_SHIFT); - return (void *)__get_free_pages(GFP_ATOMIC | __GFP_ZERO, order); + return (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, order); } =20 if ((pgt_buf_end + num) > pgt_buf_top || !can_use_brk_pgt) { --=20 2.39.5