From nobody Fri Dec 19 18:42:51 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1924175D2C for ; Mon, 19 Aug 2024 14:56:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724079368; cv=none; b=Cns2VMFxahDg3TF5piWNYIHhhl16NBvFA8SeNtIB0/vJa829hBezWn7WTtBx6FfO77EQNIA7mM1yfWrrb+c1f+HCkLdGop8277aDV1WaE42beU9YeaLrZWyDI+ZojgEo70/vlFc1FOsD5PgB8/244gXnFr/085+I7+h14PR0UVM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724079368; c=relaxed/simple; bh=+dthhNZKDtKKFU7kbV8zoxuDYOoD9TPphYxpvoesauY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HOATFsCepGqeuinLDp3mDWnT/ewFzTzADJo+CfJyz1trBwphr3b1/iC4lBVndOA0+u9w2VsNv4E/Ih2Al+VesTZXDojl0fgp1mLD9Hnp/ZeCZlu5fzLEewzlwC2oHLdL41fs6L9vLpKkQhVd5G1ZWq5MFty7GAGReof9FhUJSJs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cGvwVfAT; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cGvwVfAT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1724079366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uVVz1zO8BEoWzJpfVYvNY32S88JW7SsI5g78gkxTnig=; b=cGvwVfATJkRNy0e4LLmf3gGx+rmjWjewD1D7WyAyFmHK79lxD+efB/eYyzmE83NSSw1bca Qfd8xQpYhkNe9jk9Tj7G9HD8Hu5mXPyjuEjCYpZHhI20ch0ob9DpPjlGAnMfkMa9QzNzaH 7eyuErRhucZUxXFR/UqSMvpwPoQSA1k= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-635-L0cX0TIwN3KmO02ATSssbA-1; Mon, 19 Aug 2024 10:56:00 -0400 X-MC-Unique: L0cX0TIwN3KmO02ATSssbA-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D9C761955BF7; Mon, 19 Aug 2024 14:55:58 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.72.116.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 281A419560A3; Mon, 19 Aug 2024 14:55:49 +0000 (UTC) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Ard Biesheuvel , Jan Hendrik Farr , Philipp Rudo , Lennart Poettering , Jarkko Sakkinen , Eric Biederman , Baoquan He , Dave Young , Mark Rutland , Will Deacon , Catalin Marinas , kexec@lists.infradead.org, linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFCv2 7/9] arm64: mm: Change to prototype of Date: Mon, 19 Aug 2024 22:53:40 +0800 Message-ID: <20240819145417.23367-8-piliu@redhat.com> In-Reply-To: <20240819145417.23367-1-piliu@redhat.com> References: <20240819145417.23367-1-piliu@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" The current allocator of __create_pgd_mapping_locked() takes only one parameter, which reflects the current level of page table. But kexec_page_alloc() should extra control information from the parameter kimage. In order to make kexec_page_alloc() as an allocator, re-shape the prototype of the allocator and hence of the __create_pgd_mapping_locked(). Signed-off-by: Pingfan Liu Cc: Ard Biesheuvel Cc: Mark Rutland Cc: Will Deacon Cc: Catalin Marinas To: linux-arm-kernel@lists.infradead.org --- arch/arm64/include/asm/mmu.h | 6 ++++ arch/arm64/mm/mmu.c | 67 ++++++++++++++++++++---------------- 2 files changed, 44 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 65977c7783c58..4b908e0650105 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -64,6 +64,12 @@ extern void arm64_memblock_init(void); extern void paging_init(void); extern void bootmem_init(void); extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt); +extern void __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys, + unsigned long virt, phys_addr_t size, + pgprot_t prot, + phys_addr_t (*pgtable_alloc)(int, void *), + void *arg, + int flags); extern void create_mapping_noalloc(phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot); extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index c927e9312f102..875e2bc0cfb39 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -106,7 +106,7 @@ pgprot_t phys_mem_access_prot(struct file *file, unsign= ed long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); =20 -static phys_addr_t __init early_pgtable_alloc(int shift) +static phys_addr_t __init early_pgtable_alloc(int shift, void *unused) { phys_addr_t phys; =20 @@ -190,7 +190,8 @@ static void init_pte(pte_t *ptep, unsigned long addr, u= nsigned long end, static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(int, void *arg), + void *arg, int flags) { unsigned long next; @@ -205,7 +206,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned l= ong addr, if (flags & NO_EXEC_MAPPINGS) pmdval |=3D PMD_TABLE_PXN; BUG_ON(!pgtable_alloc); - pte_phys =3D pgtable_alloc(PAGE_SHIFT); + pte_phys =3D pgtable_alloc(PAGE_SHIFT, arg); ptep =3D pte_set_fixmap(pte_phys); init_clear_pgtable(ptep); ptep +=3D pte_index(addr); @@ -241,7 +242,9 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned l= ong addr, =20 static void init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + phys_addr_t (*pgtable_alloc)(int, void *), + void *arg, + int flags) { unsigned long next; =20 @@ -263,7 +266,7 @@ static void init_pmd(pmd_t *pmdp, unsigned long addr, u= nsigned long end, READ_ONCE(pmd_val(*pmdp)))); } else { alloc_init_cont_pte(pmdp, addr, next, phys, prot, - pgtable_alloc, flags); + pgtable_alloc, arg, flags); =20 BUG_ON(pmd_val(old_pmd) !=3D 0 && pmd_val(old_pmd) !=3D READ_ONCE(pmd_val(*pmdp))); @@ -275,7 +278,9 @@ static void init_pmd(pmd_t *pmdp, unsigned long addr, u= nsigned long end, static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + phys_addr_t (*pgtable_alloc)(int, void *), + void * arg, + int flags) { unsigned long next; pud_t pud =3D READ_ONCE(*pudp); @@ -292,7 +297,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned l= ong addr, if (flags & NO_EXEC_MAPPINGS) pudval |=3D PUD_TABLE_PXN; BUG_ON(!pgtable_alloc); - pmd_phys =3D pgtable_alloc(PMD_SHIFT); + pmd_phys =3D pgtable_alloc(PMD_SHIFT, arg); pmdp =3D pmd_set_fixmap(pmd_phys); init_clear_pgtable(pmdp); pmdp +=3D pmd_index(addr); @@ -312,7 +317,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned l= ong addr, (flags & NO_CONT_MAPPINGS) =3D=3D 0) __prot =3D __pgprot(pgprot_val(prot) | PTE_CONT); =20 - init_pmd(pmdp, addr, next, phys, __prot, pgtable_alloc, flags); + init_pmd(pmdp, addr, next, phys, __prot, pgtable_alloc, arg, flags); =20 pmdp +=3D pmd_index(next) - pmd_index(addr); phys +=3D next - addr; @@ -323,7 +328,8 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned l= ong addr, =20 static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long = end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(int, void *), + void *arg, int flags) { unsigned long next; @@ -337,7 +343,7 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long a= ddr, unsigned long end, if (flags & NO_EXEC_MAPPINGS) p4dval |=3D P4D_TABLE_PXN; BUG_ON(!pgtable_alloc); - pud_phys =3D pgtable_alloc(PUD_SHIFT); + pud_phys =3D pgtable_alloc(PUD_SHIFT, arg); pudp =3D pud_set_fixmap(pud_phys); init_clear_pgtable(pudp); pudp +=3D pud_index(addr); @@ -368,7 +374,7 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long a= ddr, unsigned long end, READ_ONCE(pud_val(*pudp)))); } else { alloc_init_cont_pmd(pudp, addr, next, phys, prot, - pgtable_alloc, flags); + pgtable_alloc, arg, flags); =20 BUG_ON(pud_val(old_pud) !=3D 0 && pud_val(old_pud) !=3D READ_ONCE(pud_val(*pudp))); @@ -381,7 +387,8 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long a= ddr, unsigned long end, =20 static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long = end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(int, void *), + void *arg, int flags) { unsigned long next; @@ -395,7 +402,7 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long a= ddr, unsigned long end, if (flags & NO_EXEC_MAPPINGS) pgdval |=3D PGD_TABLE_PXN; BUG_ON(!pgtable_alloc); - p4d_phys =3D pgtable_alloc(P4D_SHIFT); + p4d_phys =3D pgtable_alloc(P4D_SHIFT, arg); p4dp =3D p4d_set_fixmap(p4d_phys); init_clear_pgtable(p4dp); p4dp +=3D p4d_index(addr); @@ -411,7 +418,7 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long a= ddr, unsigned long end, next =3D p4d_addr_end(addr, end); =20 alloc_init_pud(p4dp, addr, next, phys, prot, - pgtable_alloc, flags); + pgtable_alloc, arg, flags); =20 BUG_ON(p4d_val(old_p4d) !=3D 0 && p4d_val(old_p4d) !=3D READ_ONCE(p4d_val(*p4dp))); @@ -422,10 +429,11 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long= addr, unsigned long end, p4d_clear_fixmap(); } =20 -static void __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys, +void __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(int, void *), + void *arg, int flags) { unsigned long addr, end, next; @@ -445,7 +453,7 @@ static void __create_pgd_mapping_locked(pgd_t *pgdir, p= hys_addr_t phys, do { next =3D pgd_addr_end(addr, end); alloc_init_p4d(pgdp, addr, next, phys, prot, pgtable_alloc, - flags); + arg, flags); phys +=3D next - addr; } while (pgdp++, addr =3D next, addr !=3D end); } @@ -453,12 +461,13 @@ static void __create_pgd_mapping_locked(pgd_t *pgdir,= phys_addr_t phys, static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(int, void *), + void *arg, int flags) { mutex_lock(&fixmap_lock); __create_pgd_mapping_locked(pgdir, phys, virt, size, prot, - pgtable_alloc, flags); + pgtable_alloc, arg, flags); mutex_unlock(&fixmap_lock); } =20 @@ -466,10 +475,10 @@ static void __create_pgd_mapping(pgd_t *pgdir, phys_a= ddr_t phys, extern __alias(__create_pgd_mapping_locked) void create_kpti_ng_temp_pgd(pgd_t *pgdir, phys_addr_t phys, unsigned long= virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags); + phys_addr_t (*pgtable_alloc)(int, void *), void *arg, int flags); #endif =20 -static phys_addr_t __pgd_pgtable_alloc(int shift) +static phys_addr_t __pgd_pgtable_alloc(int shift, void *unused) { /* Page is zeroed by init_clear_pgtable() so don't duplicate effort. */ void *ptr =3D (void *)__get_free_page(GFP_PGTABLE_KERNEL & ~__GFP_ZERO); @@ -478,9 +487,9 @@ static phys_addr_t __pgd_pgtable_alloc(int shift) return __pa(ptr); } =20 -static phys_addr_t pgd_pgtable_alloc(int shift) +static phys_addr_t pgd_pgtable_alloc(int shift, void *unused) { - phys_addr_t pa =3D __pgd_pgtable_alloc(shift); + phys_addr_t pa =3D __pgd_pgtable_alloc(shift, unused); struct ptdesc *ptdesc =3D page_ptdesc(phys_to_page(pa)); =20 /* @@ -512,7 +521,7 @@ void __init create_mapping_noalloc(phys_addr_t phys, un= signed long virt, &phys, virt); return; } - __create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, + __create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, NULL, NO_CONT_MAPPINGS); } =20 @@ -528,7 +537,7 @@ void __init create_pgd_mapping(struct mm_struct *mm, ph= ys_addr_t phys, flags =3D NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; =20 __create_pgd_mapping(mm->pgd, phys, virt, size, prot, - pgd_pgtable_alloc, flags); + pgd_pgtable_alloc, NULL, flags); } =20 static void update_mapping_prot(phys_addr_t phys, unsigned long virt, @@ -540,7 +549,7 @@ static void update_mapping_prot(phys_addr_t phys, unsig= ned long virt, return; } =20 - __create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, + __create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, NULL, NO_CONT_MAPPINGS); =20 /* flush the TLBs after updating live kernel mappings */ @@ -551,7 +560,7 @@ static void __init __map_memblock(pgd_t *pgdp, phys_add= r_t start, phys_addr_t end, pgprot_t prot, int flags) { __create_pgd_mapping(pgdp, start, __phys_to_virt(start), end - start, - prot, early_pgtable_alloc, flags); + prot, early_pgtable_alloc, NULL, flags); } =20 void __init mark_linear_text_alias_ro(void) @@ -742,7 +751,7 @@ static int __init map_entry_trampoline(void) memset(tramp_pg_dir, 0, PGD_SIZE); __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, entry_tramp_text_size(), prot, - __pgd_pgtable_alloc, NO_BLOCK_MAPPINGS); + __pgd_pgtable_alloc, NULL, NO_BLOCK_MAPPINGS); =20 /* Map both the text and data into the kernel page table */ for (i =3D 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) @@ -1344,7 +1353,7 @@ int arch_add_memory(int nid, u64 start, u64 size, flags |=3D NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; =20 __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), - size, params->pgprot, __pgd_pgtable_alloc, + size, params->pgprot, __pgd_pgtable_alloc, NULL, flags); =20 memblock_clear_nomap(start, size); --=20 2.41.0