From nobody Thu May 9 11:40:35 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass header.i=@amazon.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=amazon.com ARC-Seal: i=1; a=rsa-sha256; t=1705432402; cv=none; d=zohomail.com; s=zohoarc; b=UXuUd9uyI4z30pj00Xr6yAxinVrgZ3eaBk0/tHKlUtB/dmv8PQRiMTvs4QrbtndLYRgtWcU63umTy0a/48sBMIT05T92ejRTmwxt7tUpZCUtffZ4r5ovvbjcomSTriiAuvR6E+OMwfv50oLDtfC0aQ7DzPJpWAfMgsMyq2zEp+8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1705432402; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fVoEyNUTrAzw7YTY+W9724IaRNR5eJ6bcUYtN3Z0rYg=; b=lLiYz2eA3arHA3tvGhJUflOG0/HM+exIet2vHq2H2DJ5ttEe4FzHdG2kfa1BdIQfOhSa3PWc1FbJ4ulyxU0yolKKNC2BUeTz6/UJxRt1ZDblJrePUPJNb8ST3BXtSjvZDEE2OLl63TZ2AHSeV0szsjfjAjiL+GH/CIhp0+vrlko= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@amazon.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1705432402419929.1942315752045; Tue, 16 Jan 2024 11:13:22 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.668060.1039931 (Exim 4.92) (envelope-from ) id 1rPorw-0004qx-BA; Tue, 16 Jan 2024 19:13:04 +0000 Received: by outflank-mailman (output) from mailman id 668060.1039931; Tue, 16 Jan 2024 19:13:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPorw-0004qq-7m; Tue, 16 Jan 2024 19:13:04 +0000 Received: by outflank-mailman (input) for mailman id 668060; Tue, 16 Jan 2024 19:13:02 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPoXu-0002UD-Ne for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 18:52:22 +0000 Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 6111cb44-b4a0-11ee-98f1-6d05b1d4d9a1; Tue, 16 Jan 2024 19:52:22 +0100 (CET) Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-617e30c2.us-east-1.amazon.com) ([10.43.8.2]) by smtp-border-fw-6001.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jan 2024 18:52:21 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan2.iad.amazon.com [10.32.235.34]) by email-inbound-relay-iad-1a-m6i4x-617e30c2.us-east-1.amazon.com (Postfix) with ESMTPS id 9AAA1693DD; Tue, 16 Jan 2024 18:52:18 +0000 (UTC) Received: from EX19MTAUEB001.ant.amazon.com [10.0.44.209:29814] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.57.22:2525] with esmtp (Farcaster) id 180acf2c-dc78-4ccd-a737-249d135ec91b; Tue, 16 Jan 2024 18:52:18 +0000 (UTC) Received: from EX19D008UEC004.ant.amazon.com (10.252.135.170) by EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Tue, 16 Jan 2024 18:52:12 +0000 Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by EX19D008UEC004.ant.amazon.com (10.252.135.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Tue, 16 Jan 2024 18:52:11 +0000 Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40 via Frontend Transport; Tue, 16 Jan 2024 18:52:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Inumbo-ID: 6111cb44-b4a0-11ee-98f1-6d05b1d4d9a1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1705431143; x=1736967143; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fVoEyNUTrAzw7YTY+W9724IaRNR5eJ6bcUYtN3Z0rYg=; b=YeUeEJfJDbGgM+/EUyvvDZftEyBzIAtb31AgL7NYA9PeOwa5P0Bup5Dz GiVitK+4jGMofgply68Lb5F3J9N44MrkOg/WLJwRHUpYO7QjCQ7WSJMzS k/qQC+zls5EW9FOm6Tl1FUOZsCrGxnL2aRsziHfI2berWCibhSC/khddG Q=; X-IronPort-AV: E=Sophos;i="6.05,200,1701129600"; d="scan'208";a="382569755" X-Farcaster-Flow-ID: 180acf2c-dc78-4ccd-a737-249d135ec91b From: Elias El Yandouzi To: CC: , , , Hongyan Xia , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu , Julien Grall , Elias El Yandouzi Subject: [PATCH v2] x86/setup: Leave early boot slightly earlier Date: Tue, 16 Jan 2024 18:50:48 +0000 Message-ID: <20240116185056.15000-20-eliasely@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116185056.15000-1-eliasely@amazon.com> References: <20240116185056.15000-1-eliasely@amazon.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: Bulk X-ZohoMail-DKIM: pass (identity @amazon.com) X-ZM-MESSAGEID: 1705432403104100001 Content-Type: text/plain; charset="utf-8" From: Hongyan Xia When we do not have a direct map, memory for metadata of heap nodes in init_node_heap() is allocated from xenheap, which needs to be mapped and unmapped on demand. However, we cannot just take memory from the boot allocator to create the PTEs while we are passing memory to the heap allocator. To solve this race, we leave early boot slightly sooner so that Xen PTE pages are allocated from the heap instead of the boot allocator. We can do this because the metadata for the 1st node is statically allocated, and by the time we need memory to create mappings for the 2nd node, we already have enough memory in the heap allocator in the 1st node. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index b813ea75b5..3b698c8c41 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1746,6 +1746,22 @@ void asmlinkage __init noreturn __start_xen(unsigned= long mbi_p) =20 numa_initmem_init(0, raw_max_page); =20 + /* + * When we do not have a direct map, memory for metadata of heap nodes= in + * init_node_heap() is allocated from xenheap, which needs to be mappe= d and + * unmapped on demand. However, we cannot just take memory from the bo= ot + * allocator to create the PTEs while we are passing memory to the heap + * allocator during end_boot_allocator(). + * + * To solve this race, we need to leave early boot before + * end_boot_allocator() so that Xen PTE pages are allocated from the h= eap + * instead of the boot allocator. We can do this because the metadata = for + * the 1st node is statically allocated, and by the time we need memor= y to + * create mappings for the 2nd node, we already have enough memory in = the + * heap allocator in the 1st node. + */ + system_state =3D SYS_STATE_boot; + if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) ) { unsigned long lo =3D virt_to_mfn(HYPERVISOR_VIRT_END - 1); @@ -1777,8 +1793,6 @@ void asmlinkage __init noreturn __start_xen(unsigned = long mbi_p) else end_boot_allocator(); =20 - system_state =3D SYS_STATE_boot; - bsp_stack =3D cpu_alloc_stack(0); if ( !bsp_stack ) panic("No memory for BSP stack\n"); --=20 2.40.1