From nobody Wed May 8 05:46:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1705406139; cv=none; d=zohomail.com; s=zohoarc; b=VMTCyaYuKEwyFRE6XMAhzwvXlsn//1p2M7xbDsWVty/VDdlkuKG95mIK/dToia2OVSXFtDXY1dvA1skeH6T/85VhihPxFaUBOJ4hFGBB/MENsOy8/V7sWX9COdnr7n0QCzmGG5pZwfP9NhnrbI6KFBX5cJbWtRhWzUeuojeHf18= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1705406139; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Vg9wt2C8p9IZLdVpC0c8HQykip9na6+1kKIeLfSU34o=; b=H1fVdtm5QxUCNy2DvasuWMQDEZbGdG99S411fy221lju+2Xg6gNEg+HHkIIq7eLv66eJ7yDG3x1UTPcgHdMvLtaehjSyoIfGopNfF+LUqi2ELzFe5o/4uf7MrmhjrYLj8TWaoddJLyX3xZLRzZ9NVwKyyTJ9pUdD47vZYff9WdQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1705406139434763.4883561331692; Tue, 16 Jan 2024 03:55:39 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.667714.1039245 (Exim 4.92) (envelope-from ) id 1rPi2H-0000ME-Ht; Tue, 16 Jan 2024 11:55:17 +0000 Received: by outflank-mailman (output) from mailman id 667714.1039245; Tue, 16 Jan 2024 11:55:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2H-0000M7-Ee; Tue, 16 Jan 2024 11:55:17 +0000 Received: by outflank-mailman (input) for mailman id 667714; Tue, 16 Jan 2024 11:55:16 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2G-00009j-Ej for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 11:55:16 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2G-00087M-2C; Tue, 16 Jan 2024 11:55:16 +0000 Received: from 54-240-197-224.amazon.com ([54.240.197.224] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1rPi2F-0007ED-Oq; Tue, 16 Jan 2024 11:55:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=Vg9wt2C8p9IZLdVpC0c8HQykip9na6+1kKIeLfSU34o=; b=t/gab+thHGymI9zQJ2+Z+jkSZU cJtDcFNZ1nytiSYeTr7FHIJaAFIMfJWK+BWiMb1+jjHqXyy1X/NV5hW0/FLa9cIdRRoLX72ElZOBT ny3nbj7CaPmaWoFegJQIrjikPQ+ESEL7BZR1QobX8FRKmqBFigclpPzf/xeC/3Kg4bTo=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: carlo.nonato@minervasys.tech, Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH 1/3] arm/smpboot: Move smp_up_cpu to a new section .data.idmap Date: Tue, 16 Jan 2024 11:55:07 +0000 Message-Id: <20240116115509.77545-2-julien@xen.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116115509.77545-1-julien@xen.org> References: <20240116115509.77545-1-julien@xen.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xen.org) X-ZM-MESSAGEID: 1705406141236100004 Content-Type: text/plain; charset="utf-8" From: Julien Grall With the upcoming work to color Xen, the binary will not be anymore physically contiguous. This will be a problem during boot as the assembly code will need to work out where each piece of Xen reside. An easy way to solve the issue is to have all code/data accessed by the secondary CPUs while the MMU is off within a single page. Right now, smp_up_cpu is used by secondary CPUs to wait there turn for booting before the MMU is on. Yet it is currently in .data which is unlikely to be within the same page as the rest of the idmap. Move smp_up_cpu to the recently create section .data.idmap. The idmap is currently part of the text section and therefore will be mapped read-onl executable. This means that we need to temporarily remap smp_up_cpu in order to update it. Introduce a new function set_smp_up_cpu() for this purpose so the code is not duplicated between when opening and closing the gate. Signed-off-by: Julien Grall --- xen/arch/arm/smpboot.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 7110bc11fc05..8d508a1bb258 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -29,6 +29,10 @@ #include #include =20 +/* Override macros from asm/page.h to make them work with mfn_t */ +#undef virt_to_mfn +#define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) + cpumask_t cpu_online_map; cpumask_t cpu_present_map; cpumask_t cpu_possible_map; @@ -56,7 +60,7 @@ struct init_info init_data =3D }; =20 /* Shared state for coordinating CPU bringup */ -unsigned long smp_up_cpu =3D MPIDR_INVALID; +unsigned long __section(".data.idmap") smp_up_cpu =3D MPIDR_INVALID; /* Shared state for coordinating CPU teardown */ static bool cpu_is_dead; =20 @@ -429,6 +433,28 @@ void stop_cpu(void) wfi(); } =20 +static void set_smp_up_cpu(unsigned long mpidr) +{ + /* + * smp_up_cpu is part of the identity mapping which is read-only. So + * We need to re-map the region so it can be updated. + */ + void *ptr =3D map_domain_page(virt_to_mfn(&smp_up_cpu)); + + ptr +=3D PAGE_OFFSET(&smp_up_cpu); + + *(unsigned long *)ptr =3D mpidr; + + /* + * init_ttbr will be accessed with the MMU off, so ensure the update + * is visible by cleaning the cache. + */ + clean_dcache(ptr); + + unmap_domain_page(ptr); + +} + int __init cpu_up_send_sgi(int cpu) { /* We don't know the GIC ID of the CPU until it has woken up, so just @@ -460,8 +486,7 @@ int __cpu_up(unsigned int cpu) init_data.cpuid =3D cpu; =20 /* Open the gate for this CPU */ - smp_up_cpu =3D cpu_logical_map(cpu); - clean_dcache(smp_up_cpu); + set_smp_up_cpu(cpu_logical_map(cpu)); =20 rc =3D arch_cpu_up(cpu); =20 @@ -497,8 +522,9 @@ int __cpu_up(unsigned int cpu) */ init_data.stack =3D NULL; init_data.cpuid =3D ~0; - smp_up_cpu =3D MPIDR_INVALID; - clean_dcache(smp_up_cpu); + + set_smp_up_cpu(MPIDR_INVALID); + arch_cpu_up_finish(); =20 if ( !cpu_online(cpu) ) --=20 2.40.1 From nobody Wed May 8 05:46:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1705406143; cv=none; d=zohomail.com; s=zohoarc; b=VrgTCdj9A53SBZOxbIsgroVAZH+4HiQW83CdqQDjc89AwE9acj522exT5M0o8PbxUyCa/FnxD5YjqyHdMIFKdpQs8CYWj86EDKjeg78sIoJE60qToNxh1WvGgTjN/uK5w5ZBRrFqBiMdJ8An/VIq7IeK7Gg53KaDfRfZ0w1/1pI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1705406143; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QB4GZQCd//Kzo6y8zaEONm7rNVnOunv9HV6HbpL0NS0=; b=EYm9GfvoG/fwXzG5QcLe13ReenAiKaLTpeMArTJmvbr1voPlWMIh18M69LXb0ScZXTQFKHacR0qdi+WSfUAEcUyTD3UDkWQkxqQXh64dbxMpv/RxM7gGmbillAFwfroW/pWxlTdBqUSsONdUqS+EN8jZ0Lb+STTRWLfXOT4dhvs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1705406143380226.40239784954815; Tue, 16 Jan 2024 03:55:43 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.667715.1039255 (Exim 4.92) (envelope-from ) id 1rPi2I-0000av-PM; Tue, 16 Jan 2024 11:55:18 +0000 Received: by outflank-mailman (output) from mailman id 667715.1039255; Tue, 16 Jan 2024 11:55:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2I-0000ao-LM; Tue, 16 Jan 2024 11:55:18 +0000 Received: by outflank-mailman (input) for mailman id 667715; Tue, 16 Jan 2024 11:55:17 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2H-0000PP-LJ for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 11:55:17 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2H-00087a-6t; Tue, 16 Jan 2024 11:55:17 +0000 Received: from 54-240-197-224.amazon.com ([54.240.197.224] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1rPi2G-0007ED-UI; Tue, 16 Jan 2024 11:55:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=QB4GZQCd//Kzo6y8zaEONm7rNVnOunv9HV6HbpL0NS0=; b=42VSouxcLSxioR0z6ZwNN96j40 CM2a58oaWBHVmF09hokFO9uFmfrcjZc4yqqrZIjb3ua75YOPY36z2pOuNsGtdb94OpXUEspQAq7nh PsdsdRJ2td2ZwYkaYcxSLLx/Vc6xOPjzCqkEePQlehs2p6fgjFkOw7tryF0VCxJcpAsw=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: carlo.nonato@minervasys.tech, Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH 2/3] xen/arm64: head: Use PRINT_ID() for secondary CPU MMU-off boot code Date: Tue, 16 Jan 2024 11:55:08 +0000 Message-Id: <20240116115509.77545-3-julien@xen.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116115509.77545-1-julien@xen.org> References: <20240116115509.77545-1-julien@xen.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xen.org) X-ZM-MESSAGEID: 1705406145079100001 Content-Type: text/plain; charset="utf-8" From: Julien Grall With the upcoming work to color Xen, the binary will not be anymore physically contiguous. This will be a problem during boot as the assembly code will need to work out where each piece of Xen reside. An easy way to solve the issue is to have all code/data accessed by the secondary CPUs while the MMU is off within a single page. Right now, most of the early printk messages are using PRINT() which will add the message in .rodata. This is unlikely to be within the same page as the rest of the idmap. So replace all the PRINT() that can be reachable by the secondary CPU with MMU-off with PRINT_ID(). Signed-off-by: Julien Grall --- xen/arch/arm/arm64/head.S | 14 +++++++------- xen/arch/arm/arm64/mmu/head.S | 2 +- xen/arch/arm/include/asm/arm64/macros.h | 9 ++++++--- 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index cfc04c755400..fa8b00b6f1db 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -289,9 +289,9 @@ GLOBAL(init_secondary) =20 #ifdef CONFIG_EARLY_PRINTK ldr x23, =3DCONFIG_EARLY_UART_BASE_ADDRESS /* x23 :=3D UART base= address */ - PRINT("- CPU ") + PRINT_ID("- CPU ") print_reg x24 - PRINT(" booting -\r\n") + PRINT_ID(" booting -\r\n") #endif bl check_cpu_mode bl cpu_init @@ -314,10 +314,10 @@ ENDPROC(init_secondary) * Clobbers x0 - x5 */ check_cpu_mode: - PRINT("- Current EL ") + PRINT_ID("- Current EL ") mrs x5, CurrentEL print_reg x5 - PRINT(" -\r\n") + PRINT_ID(" -\r\n") =20 /* Are we in EL2 */ cmp x5, #PSR_MODE_EL2t @@ -326,8 +326,8 @@ check_cpu_mode: ret 1: /* OK, we're boned. */ - PRINT("- Xen must be entered in NS EL2 mode -\r\n") - PRINT("- Please update the bootloader -\r\n") + PRINT_ID("- Xen must be entered in NS EL2 mode -\r\n") + PRINT_ID("- Please update the bootloader -\r\n") b fail ENDPROC(check_cpu_mode) =20 @@ -361,7 +361,7 @@ ENDPROC(zero_bss) * Clobbers x0 - x3 */ cpu_init: - PRINT("- Initialize CPU -\r\n") + PRINT_ID("- Initialize CPU -\r\n") =20 /* Set up memory attribute type tables */ ldr x0, =3DMAIRVAL diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S index 92b62ae94ce5..fa40b696ddc8 100644 --- a/xen/arch/arm/arm64/mmu/head.S +++ b/xen/arch/arm/arm64/mmu/head.S @@ -276,7 +276,7 @@ ENDPROC(create_page_tables) enable_mmu: mov x4, x0 mov x5, x1 - PRINT("- Turning on paging -\r\n") + PRINT_ID("- Turning on paging -\r\n") =20 /* * The state of the TLBs is unknown before turning on the MMU. diff --git a/xen/arch/arm/include/asm/arm64/macros.h b/xen/arch/arm/include= /asm/arm64/macros.h index 10e652041f57..6a0108f778a2 100644 --- a/xen/arch/arm/include/asm/arm64/macros.h +++ b/xen/arch/arm/include/asm/arm64/macros.h @@ -39,9 +39,12 @@ * There are multiple flavors: * - PRINT_SECT(section, string): The @string will be located in @section * - PRINT(): The string will be located in .rodata.str. - * - PRINT_ID(): When Xen is running on the Identity Mapping, it is - * only possible to have a limited amount of Xen. This will create - * the string in .rodata.idmap which will always be mapped. + * - PRINT_ID(): This will create the string in .rodata.idmap which + * will always be accessible. This is used when: + * - Xen is running on the identity mapping because not all of Xen is= mapped + * - Running with the MMU-off on secondary boots as Xen may not be + * physically contiguous in memory (e.g. in the case of cache + * coloring). * * Clobbers x0 - x3 */ --=20 2.40.1 From nobody Wed May 8 05:46:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1705406138; cv=none; d=zohomail.com; s=zohoarc; b=NcmGY8TO8bYwrAzx65ElejUsK3Q8o7Py0lmbQK9bC0/wtHzn004i8PdbQ6zaTZqi/2sHbXsw/0M7gledQkv2TPkUPNGTMPCBqjx5s0FYI79uucvpX4+q5zlwIdoKCI/FEXOITIlP7IWKHbRODJrGjyo47l0wSskGe7iSLQ5e7wk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1705406138; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nSJb8H4AUwBfcQwQar1U/80V0XEGtzBfpVJPiZPhupE=; b=PaJC7jI+6pChbCNF7zBfx9MhNVfuQSkbm3xV35V75lAeiRnhzv/ZpljcENFDMqn0sOJ5LfhcuAd3QpYr/YnhZ/GAw2DssRXXldk1tLUud/YZRfSAN2GsEJWqgO22RakiB2tnN402ZVEXGVYI5XUec6jMtBXV6VgNUNe1TTCcEDc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1705406138184665.8714048117155; Tue, 16 Jan 2024 03:55:38 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.667716.1039265 (Exim 4.92) (envelope-from ) id 1rPi2K-0000q1-1Y; Tue, 16 Jan 2024 11:55:20 +0000 Received: by outflank-mailman (output) from mailman id 667716.1039265; Tue, 16 Jan 2024 11:55:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2J-0000ps-TA; Tue, 16 Jan 2024 11:55:19 +0000 Received: by outflank-mailman (input) for mailman id 667716; Tue, 16 Jan 2024 11:55:18 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2I-0000cK-RP for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 11:55:18 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPi2I-00087o-B7; Tue, 16 Jan 2024 11:55:18 +0000 Received: from 54-240-197-224.amazon.com ([54.240.197.224] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1rPi2I-0007ED-3X; Tue, 16 Jan 2024 11:55:18 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=nSJb8H4AUwBfcQwQar1U/80V0XEGtzBfpVJPiZPhupE=; b=IMbW36yI0eeJYIVSzC4dbp8DxA 8udP0YwppNl+SE2F+mTppP10JsR2iHvtvZB/driKQwvS0QwPUleaPFR4Zc4GzK1f7NfIUkpjmpMFr zIkdlBu2YJBNCfgoMOW5bgcDywin4m6Ayn5nF+HY21EBYgVUX2wrghQIoReFltwKBJRE=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: carlo.nonato@minervasys.tech, Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH 3/3] [DO NOT COMMIT] xen/arm: Create a trampoline for secondary boot CPUs Date: Tue, 16 Jan 2024 11:55:09 +0000 Message-Id: <20240116115509.77545-4-julien@xen.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116115509.77545-1-julien@xen.org> References: <20240116115509.77545-1-julien@xen.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xen.org) X-ZM-MESSAGEID: 1705406139375100001 Content-Type: text/plain; charset="utf-8" From: Julien Grall In order to confirm the early boot code is self-contained, allocate a separate trampoline region for secondary to boot from it. Signed-off-by: Julien Grall --- xen/arch/arm/arm64/mmu/mm.c | 7 +++++++ xen/arch/arm/mmu/smpboot.c | 4 +++- xen/arch/arm/psci.c | 5 ++++- xen/arch/arm/smpboot.c | 15 ++++++++++++++- 4 files changed, 28 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c index d2651c948698..3c4988dc75d1 100644 --- a/xen/arch/arm/arm64/mmu/mm.c +++ b/xen/arch/arm/arm64/mmu/mm.c @@ -110,11 +110,18 @@ void __init arch_setup_page_tables(void) prepare_runtime_identity_mapping(); } =20 +extern mfn_t trampoline_start; + void update_identity_mapping(bool enable) { paddr_t id_addr =3D virt_to_maddr(_start); int rc; =20 + if ( !mfn_eq(trampoline_start, INVALID_MFN) ) + { + id_addr =3D mfn_to_maddr(trampoline_start); + } + if ( enable ) rc =3D map_pages_to_xen(id_addr, maddr_to_mfn(id_addr), 1, PAGE_HYPERVISOR_RX); diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index f1cf9252710c..d768dfe065a5 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -72,13 +72,15 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } =20 +extern mfn_t trampoline_start; + static void set_init_ttbr(lpae_t *root) { /* * init_ttbr is part of the identity mapping which is read-only. So * We need to re-map the region so it can be updated */ - void *ptr =3D map_domain_page(virt_to_mfn(&init_ttbr)); + void *ptr =3D map_domain_page(trampoline_start); =20 ptr +=3D PAGE_OFFSET(&init_ttbr); =20 diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index 695d2fa1f1b5..a00574d559d6 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -36,11 +36,14 @@ static uint32_t psci_cpu_on_nr; =20 #define PSCI_RET(res) ((int32_t)(res).a0) =20 +extern mfn_t trampoline_start; + int call_psci_cpu_on(int cpu) { struct arm_smccc_res res; =20 - arm_smccc_smc(psci_cpu_on_nr, cpu_logical_map(cpu), __pa(init_secondar= y), + arm_smccc_smc(psci_cpu_on_nr, cpu_logical_map(cpu), + mfn_to_maddr(trampoline_start) + PAGE_OFFSET(init_second= ary), &res); =20 return PSCI_RET(res); diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 8d508a1bb258..ef84b73ebd46 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -293,10 +293,13 @@ unsigned int __init smp_get_max_cpus(void) return cpus; } =20 +mfn_t trampoline_start =3D INVALID_MFN_INITIALIZER; + void __init smp_prepare_cpus(void) { int rc; + void *trampoline; =20 cpumask_copy(&cpu_present_map, &cpu_possible_map); =20 @@ -304,6 +307,16 @@ smp_prepare_cpus(void) if ( rc ) panic("Unable to allocate CPU sibling/core maps\n"); =20 + /* Create a trampoline to confirm early boot code is self-contained */ + trampoline =3D alloc_xenheap_page(); + BUG_ON(!trampoline); + + memcpy(trampoline, _start, PAGE_SIZE); + clean_dcache_va_range(trampoline, PAGE_SIZE); + invalidate_icache(); + + printk("Trampoline 0x%lx\n", virt_to_maddr(trampoline)); + trampoline_start =3D virt_to_mfn(trampoline); } =20 /* Boot the current CPU */ @@ -439,7 +452,7 @@ static void set_smp_up_cpu(unsigned long mpidr) * smp_up_cpu is part of the identity mapping which is read-only. So * We need to re-map the region so it can be updated. */ - void *ptr =3D map_domain_page(virt_to_mfn(&smp_up_cpu)); + void *ptr =3D map_domain_page(trampoline_start); =20 ptr +=3D PAGE_OFFSET(&smp_up_cpu); =20 --=20 2.40.1