From nobody Sat Jun 13 07:32:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=amd.com); dmarc=pass(p=quarantine dis=none) header.from=amd.com ARC-Seal: i=2; a=rsa-sha256; t=1780963785; cv=pass; d=zohomail.com; s=zohoarc; b=FCY0HUlS88XTgmrl1nPnMNQ3Uf7pv956FaZ+ZtmCabgxX3G4VB1+ZXMHJS153/M7mnzAA0F5paYfRNVuuIvkim3HwFx9Ewsk02ECQp7mSZdydoFZrTDpvjQexOgioBzqzuNVIX6G9IlfiDsUCEkJj3fyeqjpaQwqpLKXB27BGR4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780963785; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=h69F0/HRgHoEeubvo+fbrRYs/KCcGjkSw3JqSU3qEyg=; b=Koh7qi8RJz0WTGQrnnauHgnvEyP9gq7Yi/XWP2nVvJBerpd1mu7HxWIP1Bqss3cLPPigfA5UgUETUQSAzBbaSeHAI7is9WqQCtqbQs78SKC6Y9slkD/KpsfICtavQ7V/ibDWYK7nm80vtq0yzulJuox8ctjIakkaEDi3A2EC9kM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=amd.com); dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1780963785263637.1719560922028; Mon, 8 Jun 2026 17:09:45 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1332155.1594805 (Exim 4.92) (envelope-from ) id 1wWk1t-0003XH-JU; Tue, 09 Jun 2026 00:09:17 +0000 Received: by outflank-mailman (output) from mailman id 1332155.1594805; Tue, 09 Jun 2026 00:09:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wWk1t-0003XA-GG; Tue, 09 Jun 2026 00:09:17 +0000 Received: by outflank-mailman (input) for mailman id 1332155; Tue, 09 Jun 2026 00:09:17 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wWk1s-0003ND-TQ for xen-devel@lists.xenproject.org; Tue, 09 Jun 2026 00:09:17 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wWk1s-002NXg-2P for xen-devel@lists.xenproject.org; Tue, 09 Jun 2026 02:09:16 +0200 Received: from [10.42.69.2] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a2759ab-bab6-0a2a0a5309dd-0a2a4502da1c-0 for ; Tue, 09 Jun 2026 02:09:15 +0200 Received: from [40.93.196.70] (helo=SA9PR02CU001.outbound.protection.outlook.com) by tlsNG-720697.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a2759aa-af86-0a2a45020019-285dc44694b0-4 for ; Tue, 09 Jun 2026 02:09:15 +0200 Received: from SJ0PR03CA0278.namprd03.prod.outlook.com (2603:10b6:a03:39e::13) by IA1PR12MB6234.namprd12.prod.outlook.com (2603:10b6:208:3e6::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.12; Tue, 9 Jun 2026 00:09:10 +0000 Received: from BY1PEPF0001AE1B.namprd04.prod.outlook.com (2603:10b6:a03:39e:cafe::24) by SJ0PR03CA0278.outlook.office365.com (2603:10b6:a03:39e::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.92.14 via Frontend Transport; Tue, 9 Jun 2026 00:09:10 +0000 Received: from satlexmb07.amd.com (165.204.84.17) by BY1PEPF0001AE1B.mail.protection.outlook.com (10.167.242.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 00:09:09 +0000 Received: from satlexmb10.amd.com (10.181.42.219) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Mon, 8 Jun 2026 19:09:09 -0500 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb10.amd.com (10.181.42.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Mon, 8 Jun 2026 19:09:09 -0500 Received: from fedora.mshome.net (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Mon, 8 Jun 2026 19:09:08 -0500 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=selector1 header.d=amd.com header.i="@amd.com" header.h="From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GcCUSv5WBrwk8MjikeVDPMwe3Z4dI9k9Cildit95pqZxcNMUMg3bFgvgJFfXOCPdJHqD7uyJbzHWtaSbEmm2AdJjvNgA1lUTuW3L3oKVd+PfEg8ZEK1fGLZ159JzIcLlAkKYzdzT90x3r0tOzEXP22vOmQHKFyfmMpF15Xp1DTAgsr0WT1z/pC6ERy0rSE9Ry8LpTgNnEfwCz/5gHpihF9wLut1j/gZODqOO1Wd00y7OLAizVd6uaax6XDLfhAx/WjhfxRjDc35DTi/eNvnQPMjhOzzABmenDOraJYAzb0Chp8CK7zeWqhYiFlKNgFMgLeczs5KO0v+Q+ojAQHaEtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h69F0/HRgHoEeubvo+fbrRYs/KCcGjkSw3JqSU3qEyg=; b=mQYBQ9EAEhx4QpUR/a2oLmdEgGaNpjG0xEXdaq7QnAii8OX6S6hcT+3/Nrvd4rMlyBYS57oVUWk0swhTjtInMsavIBS/l7Z4nZJk7w6v1NkuTrg1HkG1z7hdwmU/UYK5x7IoCpY/JMPuNBblp7hkQ3dc3m8y00yhpFDtg8El4ZWYfnxVfMJI58x5TmzvDm6FjmQVlha6t7+Z0T8gwM5E+3OK+n7JxdBeKbmC/D689JMxIkuv+g2eQA0nXrePFf9bAschcARiiMC3IV0ojip7cNzPxRjuqdDcrbrqqf54dPo4c+9sxJMrgvqzsczVaokv4rd6NwaPc7dEOBaI5p0faQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h69F0/HRgHoEeubvo+fbrRYs/KCcGjkSw3JqSU3qEyg=; b=0r++qOH5ulIlD4JI9esDNRtQn1B2p65xfldio2xDD7JwxIMI6mSAjSWnhwfYwm8XmKZUtrFD5jCrsMMGfGKNQAxyXDQ4xtnEdj/jFKzuvRLG3t99IjyiC50jlVdYieW7Wp/FM4wPZD7/2iaGkp6in16QzFhitW0HDnPQ/KUEpDg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C From: Jason Andryuk To: CC: Jason Andryuk , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v3 1/2] xen/x86: Return virtual address from alloc_stub_page() Date: Mon, 8 Jun 2026 20:06:37 -0400 Message-ID: <20260609000638.121027-2-jason.andryuk@amd.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260609000638.121027-1-jason.andryuk@amd.com> References: <20260609000638.121027-1-jason.andryuk@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY1PEPF0001AE1B:EE_|IA1PR12MB6234:EE_ X-MS-Office365-Filtering-Correlation-Id: ac84d4e8-94ef-4071-0080-08dec5bb5464 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|376014|36860700016|56012099006|11063799006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: Gl5xzHopvMHxUSd7/zlcohws4IjshTajQNZKIKdHeeWj4IObcW/H261GWB8+KN2g1btlmLWskksJlAFgxZvLXLZa/cya+Z1p1xIl/jdcMiMiv2adcZaIGR1xYSfe3LwBsz/+/6sNBvwfanKDUnYUsN3/OiPlGpvim4UQLqIf6443VSr6ppj2Rt9r1vJA5in1FdY95xwFO1EbZVdsUv8xufh1vs+ePj5sNDMz0XH6CfJbqZahQYnMXCM2y0wmYLlQuEb9VWM5m4is4f0GyTtjBPCwovNh1CvPQGxcuYi8Py4mefMgZZeGsqIEG7+oMjvIhG6zYoSSlm2j7NgqEycz9AOznXJu75ztMAzl3Iz7SoXJwKf1W3ney7G4HwBq8xgEpaBnRUmjV6B2BNmEuZppvjQj6sRns9xC8Zca9E5nod3NdvkSYainjuVP50KBvM6yvcmrDwOukaDZS3zhwbR4c9by2TciaB/YJu2jQAwikk+iWsGckJgjUFDfZLLRAxaVaECHwz1ui8suvZ8f44otjSD0erFMwRk+inofD9+ciIuq+mStUcVuF8AeWRizp1iqJgLwwPnXcx8deMXKZT1OVxJkaNTOmFDHShhtB359STf2eIB9zq9C9rH032hu/ADIly8CHQkb8yNXRRxfILF75erViMDzDdMdeu26JUN8zddMY3dsnR2qT8mq1LSkTUGDzQ54suGWwW05XGNZzAF0NzFN97iggM/eLK0cmOj8JXg= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(376014)(36860700016)(56012099006)(11063799006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Ejt5EeMU/qHSFPCBCyROXVV5/y0npbmDMKMzT4wZxmXk9AR/MXW44y7QNMGCjmEo5RVWcKtgaqejZUb6oKG6WjuWS5K9S1zY3n9ROtbDLLAe6pMVfD43FPzNu7HKYfUhhMu1AavgM2sOHi2xfp+6ywr9LRwB5lqCpe2yRZj5/imy+Tb15KmnXtVg5ReehOvsGNUd3Y3HmmB+UNGatXJKpJr3KXvnWOSYmHqq07TngykIuTNfS6+ZGvOzEVATOJs9cnVUTvACJEAhE/DG7LeJCws+V1AAOTL2mQpSw2FPKkyOWefqB/K7CPDLZXD7F6PV8x+Vu28gcpX0uKpJWZUXG/JEAzhUrDpqiNz1yRhfF6+w4NGcClGtJvLWzAmdX2bJZkfVZShlZJ5v0jJz3asawkO83yTSRZ6hJK3xx6rlzXVuYkYv5LZ1AG6/bXe5XOaF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 00:09:09.7908 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ac84d4e8-94ef-4071-0080-08dec5bb5464 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BY1PEPF0001AE1B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6234 X-purgate-ID: tlsNG-720697/1780963755-7BF66161-117588A9/0/0 X-purgate-type: clean X-purgate-size: 1869 X-ZohoMail-DKIM: pass (identity @amd.com) X-ZM-MESSAGEID: 1780963786756158500 Content-Type: text/plain; charset="utf-8" Currently alloc_stub_page() returns the virtual address of the mapped stubs page, and the caller adds the per-CPU offset. Make alloc_stub_page() return the final address. This is in preparation for changing the stubs allocation where the offset will not be tied to the CPU number. The call to alloc_stub_page() in setup.c:start_xen() did not add the offset as it is assumed to run on CPU0. Change the local variable stub_page to stub_va to reflect the value. Signed-off-by: Jason Andryuk --- xen/arch/x86/smpboot.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index ff05955bae..d7619f534b 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -675,7 +675,7 @@ unsigned long alloc_stub_page(unsigned int cpu, unsigne= d long *mfn) else if ( !*mfn ) *mfn =3D mfn_x(page_to_mfn(pg)); =20 - return stub_va; + return stub_va ? stub_va + STUB_BUF_CPU_OFFS(cpu) : 0; } =20 void cpu_exit_clear(unsigned int cpu) @@ -1044,7 +1044,7 @@ static int cpu_smpboot_alloc(unsigned int cpu) unsigned int i, memflags =3D 0; nodeid_t node =3D cpu_to_node(cpu); seg_desc_t *gdt; - unsigned long stub_page; + unsigned long stub_va; int rc =3D -ENOMEM; =20 if ( node !=3D NUMA_NO_NODE ) @@ -1099,10 +1099,10 @@ static int cpu_smpboot_alloc(unsigned int cpu) break; } BUG_ON(i =3D=3D cpu); - stub_page =3D alloc_stub_page(cpu, &per_cpu(stubs.mfn, cpu)); - if ( !stub_page ) + stub_va =3D alloc_stub_page(cpu, &per_cpu(stubs.mfn, cpu)); + if ( !stub_va ) goto out; - per_cpu(stubs.addr, cpu) =3D stub_page + STUB_BUF_CPU_OFFS(cpu); + per_cpu(stubs.addr, cpu) =3D stub_va; =20 rc =3D setup_cpu_root_pgt(cpu); if ( rc ) --=20 2.54.0 From nobody Sat Jun 13 07:32:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=amd.com); dmarc=pass(p=quarantine dis=none) header.from=amd.com ARC-Seal: i=2; a=rsa-sha256; t=1780963787; cv=pass; d=zohomail.com; s=zohoarc; b=h216AY+vntcZ9LlAGcsKAn2lnMJUJ8Sof249sUMJpoyf7RwbmHCRi3k/WdUHioMf4oRZ0BEOJ6bJT8fBCDiE5rGKWeqXg5IHSys/4bg4FvviWIa92Ro6CM/MeUcKy6egBk68Qm41g4DrCUky914V6xUuFCXJylJUxanYtcT5d9c= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1780963787; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=yTwb8ekE7mBmp1WJz2KP3LjX28ggFVhtoF/cTlRbrXY=; b=gxwaf9PUPnYFKXTNmsbJ+0s9F8h20m1i8E/fEQsxXS/IyEQdpca3ZvqGwp5LIb8amhP8YzitpWavKn4Vd88HG6ysGTLEOOT2MG4bxxNtz+9KV4WtWrKHwTWc5WfsK8+WszqcVmFbJ/nFHna0SYs5GKzDHqqh/C9+ry5OM9xUqcE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=amd.com); dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1780963787188865.3853190274107; Mon, 8 Jun 2026 17:09:47 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1332156.1594814 (Exim 4.92) (envelope-from ) id 1wWk1v-0003kb-Tj; Tue, 09 Jun 2026 00:09:19 +0000 Received: by outflank-mailman (output) from mailman id 1332156.1594814; Tue, 09 Jun 2026 00:09:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wWk1v-0003kU-Qb; Tue, 09 Jun 2026 00:09:19 +0000 Received: by outflank-mailman (input) for mailman id 1332156; Tue, 09 Jun 2026 00:09:19 +0000 Received: from mx.expurgate.net ([195.190.135.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wWk1u-0003k0-Uo for xen-devel@lists.xenproject.org; Tue, 09 Jun 2026 00:09:19 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wWk1u-003tJu-BF for xen-devel@lists.xenproject.org; Tue, 09 Jun 2026 02:09:18 +0200 Received: from [10.42.69.6] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from ) id 6a27597b-e002-0a2a0a5209dd-0a2a450695b6-40 for ; Tue, 09 Jun 2026 02:09:18 +0200 Received: from [52.101.46.11] (helo=CO1PR03CU002.outbound.protection.outlook.com) by tlsNG-16d1c6.mxtls.expurgate.net with ESMTPS (eXpurgate 4.56.1) (envelope-from ) id 6a2759ab-7371-0a2a45060019-34652e0bea5a-4 for ; Tue, 09 Jun 2026 02:09:17 +0200 Received: from DS7P221CA0026.NAMP221.PROD.OUTLOOK.COM (2603:10b6:8:25c::13) by BL1PR12MB5900.namprd12.prod.outlook.com (2603:10b6:208:398::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.13; Tue, 9 Jun 2026 00:09:11 +0000 Received: from DS1PEPF00017097.namprd05.prod.outlook.com (2603:10b6:8:25c:cafe::94) by DS7P221CA0026.outlook.office365.com (2603:10b6:8:25c::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.92.14 via Frontend Transport; Tue, 9 Jun 2026 00:09:11 +0000 Received: from satlexmb08.amd.com (165.204.84.17) by DS1PEPF00017097.mail.protection.outlook.com (10.167.18.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.113.7 via Frontend Transport; Tue, 9 Jun 2026 00:09:11 +0000 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Mon, 8 Jun 2026 19:09:10 -0500 Received: from fedora.mshome.net (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Mon, 8 Jun 2026 19:09:10 -0500 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=selector1 header.d=amd.com header.i="@amd.com" header.h="From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nn4xTzilSUUyPxo9EYBK1kyL8PH96khgDf5KkXISvlRmAzSMy+wG8y8n/6kuFxsag1+LU56B/ePs15rl2um4eZAqVlQA1rYA2pJKMPqwRGctIlm1UEMf2ikWI4LRUm9Um8uZ1W8ahOGtHze7ur6YVeFbMYzL82/VxmrF5UI2P0aiHKa1jJlVnlifAtpj+PYcg9lomZfC8N9n1tUlumnum6PmN/20aUZ0SLHO4eU4eZq8jE/vOqWsHsQEEX44ALpmVFWZ5DWeOaSE/6nMo3mpFFV2Geu4oMEKrWL++Z4GHZGPUkXUwqHLHC61P3dDrKw1w5vKAKwvWsGS24dndXUgVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yTwb8ekE7mBmp1WJz2KP3LjX28ggFVhtoF/cTlRbrXY=; b=nPG2dmPiAf3w8VJsZ/9wkW2FuG2yq0Z0HBlt9j0Az0uA5fErROFRvZjpq1IaSC6xwUWJOrEjHORr+Dq/AwBhtfEfFUBPTKF+A/s7/yHdTVw6ZYexqAwL4trGadqwJRvi3pF9M+C+776B/qcq3EHT0LHrwi0W+eCD1j9C3TsbIlQikNIryLmycriFCdG5OGPdew0+XMc0YFUIT/36TeaLhARcpiK83fIQnMr1JWwvjN3s3PxNh7X0IdTl6q2peCYQ3xHXLAuOEjt7av3YllIejz2EmAkKI0BMHvPf1l5xQ/IT5bZurF3Iwyv+oevMavWa3TthBXIWgpLHDZIe4SfDrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yTwb8ekE7mBmp1WJz2KP3LjX28ggFVhtoF/cTlRbrXY=; b=DsrLNKAXPUeLQsDPGW1g3LTGJqTovsPhWhCZOzTPNUjx7UJHQYfIIk9y/mQ8BQgblsvQjPTLvUsOdiqdoWVLuEW+IhTHttTo8xqozi0BTuTu6161PVJfS4vq8GtiZMbwOvoaaC4ojDinG0qg4wnB5EUAfhmSIqZbY2RO7bwIRAs= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C From: Jason Andryuk To: CC: Jason Andryuk , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Teddy Astie Subject: [PATCH v3 2/2] xen/x86: Change stub page allocation/free Date: Mon, 8 Jun 2026 20:06:38 -0400 Message-ID: <20260609000638.121027-3-jason.andryuk@amd.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260609000638.121027-1-jason.andryuk@amd.com> References: <20260609000638.121027-1-jason.andryuk@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF00017097:EE_|BL1PR12MB5900:EE_ X-MS-Office365-Filtering-Correlation-Id: c39c88da-e084-43d2-82ff-08dec5bb553b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700016|376014|18002099003|22082099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: UwH7UNOMkXHopqWUtV7lCPEDT8Bixb+C6ufhaj4udoBHnT+7KUPZT4w20y1spR5LJ6hVEFddJVB3BvPYpm3765PJlS1AZ/KXOZ0mXaOvKzK96ZFWPwloV2ky2KA+KCzcYkULNgEKtuoXwaNlhl8/Rcq/jZrnCtsN8SkZplvkbzYxmBZDolctH/H8W6Gc6X6Fnz/ILPOiuYrBv4CSHIGCZtToZwLT91Gql1R++mj4GauPkEOoz6HC1Xmxtv4ROLGCM+RAAWQX+gBzermMoImx00Fk4twAoG2i+uwpjtBTT9z+WLbadRil/NTm0cfufEjI9E/6uJUDJI15qHHzuG34Yqv4QQvSh0qbR2XGgYSMhD15gBoCDNIzLi691/+w9C/yluaSBMd20NN6OL4RqZ61qOVsEFbymEX741fXRWltWr7NkULTyujWP3dkSVs0cjmEm9tBMVvh7C3rA1BaMiMPFCcGvQ2Te2Fq24Q5ESUwdEIX9V/uJS9pzRIuMawyAUDXdKNExDX7Egnai0i0e7qRT7UrtEmildK0u5Pt4PJno4jHyqjpFMzLtN7ckwzbnYcWlt4+MkUtTREl+9uCQQmOyhnAwKQUuI3GMCPQk14rq0cQX2hlokazeTn++Sj3q0ZPfV/0urDDZbHUxOpODZ/vM/a2TM8ddMVOEbGo8dV7z0cLRBfLMZPMO9MKQU0ukyEQwqMo+T0LBlS2+XKZrv+q6zhHDXqfy3z/a17KAEVb3y0= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700016)(376014)(18002099003)(22082099003)(11063799006)(56012099006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JiKznbqkzD3rEV1ZG2cjNXihkm+2i09z3NbdkNcqsh3C5nvmETIskB9izFEVQu/hUdr7JuEEh5rGB7cNVWz9U9OOF4mO5I0zMadSD5O4/QxTt0vv01BSVLEloV6hD7d6tDBQ6xhLJ+BdzK+AYQaqPePAiEqMDlWEEyHYWnianzjb0+6klv+Sua4kS3mrQGC1uQuZ+vEsz+CcsKzLiVtNdSgOB+V4jT2yWygQdPqRn8euzWrITuesXIbPjhwxMoZfaVJnkwqSYoGk2voex8i7xET8SdwDRb/tCwcv2rWFl5MHhx6yxL/x9ANqRuJk7eQ/2XqFUvNJiLeWJpEDCR7p2wCQv/vScmVHCv4NH7D+tJ1FSq2Tzz3zC9VTVZLlXS9d/sF+zG4CkjEv56B73bqOFyv/RsWCUgsNTA7NZGv/uvJR5i7SfFtG4P5LPQQmsCYL X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2026 00:09:11.2260 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c39c88da-e084-43d2-82ff-08dec5bb553b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF00017097.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5900 X-purgate-ID: tlsNG-16d1c6/1780963758-8D186D75-030DC01D/0/0 X-purgate-type: clean X-purgate-size: 8330 X-ZohoMail-DKIM: pass (identity @amd.com) X-ZM-MESSAGEID: 1780963790054154100 Content-Type: text/plain; charset="utf-8" Today the inline tracking of the stub page is problematic. 0xcc is used to indicate unused, but it is also a "clear value." A !CONFIG_PV build with smt=3D0 will bring up CPU0, bring up CPU1, bring down CPU1, and free the in-use stub page. Subsequent CPU onlining can write to the re-used page. The new approach uses a global, CPU-indexed array of stub pages. However, to handle NUMA aware allocations, we cannot allocate all the pages in advance because the NUMA information is not available. Keep track of 1 current page for each NUMA node, allocated on demand, and allocate the stub buffers out of those pages. The current NUMA allocation approach is opportunistic sharing among the groups of 32 processors. The new approach will allocate buffers densely in a NUMA node. stub pages are no longer freed. They remain referenced in the global CPU-indexed array and are re-used if the CPU is re-onlined. stubs and node_stubs don't have an explicit lock. During boot they are accessed single threaded. During runtime, &cpu_add_remove_lock serializes access. Fixes: 7a66ac8d1633 ("x86: move syscall trampolines off the stack") Signed-off-by: Jason Andryuk --- I'm not sure how to test the NUMA part - I don't have an NUMA system. Also, if NUMA is active, is a cpu node of NUMA_NO_NODE still possible? I used the MAX_NUMNODES + 1 array sizing to handle that, but it's not obvious to me if that is necessary. Roger mentioned removing the per-cpu stubs.mfn. We'd need to replace that with exposing the stubs array for traps and the emulator. I have no idea if that will be an improvement and am looking for agreement on this patch before attempting. --- xen/arch/x86/include/asm/stubs.h | 2 +- xen/arch/x86/setup.c | 3 +- xen/arch/x86/smpboot.c | 110 +++++++++++++++++++++---------- 3 files changed, 77 insertions(+), 38 deletions(-) diff --git a/xen/arch/x86/include/asm/stubs.h b/xen/arch/x86/include/asm/st= ubs.h index a520928e9a..9d776f81dd 100644 --- a/xen/arch/x86/include/asm/stubs.h +++ b/xen/arch/x86/include/asm/stubs.h @@ -32,6 +32,6 @@ struct stubs { }; =20 DECLARE_PER_CPU(struct stubs, stubs); -unsigned long alloc_stub_page(unsigned int cpu, unsigned long *mfn); +unsigned long assign_stub_page(unsigned int cpu); =20 #endif /* X86_ASM_STUBS_H */ diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 19ee857abf..0cac94cbdb 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -2089,8 +2089,7 @@ void asmlinkage __init noreturn __start_xen(void) =20 init_idle_domain(); =20 - this_cpu(stubs.addr) =3D alloc_stub_page(smp_processor_id(), - &this_cpu(stubs).mfn); + this_cpu(stubs.addr) =3D assign_stub_page(0); BUG_ON(!this_cpu(stubs.addr)); =20 bsp_traps_reinit(); /* Needs stubs allocated, must be before presmp_in= itcalls. */ diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index d7619f534b..d9cd90389d 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -641,41 +641,96 @@ static int do_boot_cpu(int apicid, int cpu) return rc; } =20 -#define STUB_BUF_CPU_OFFS(cpu) (((cpu) & (STUBS_PER_PAGE - 1)) * STUB_BUF_= SIZE) +/* + * Indexed by CPU. `pg` may be shared by up to STUBS_PER_PAGE CPUs. Offs= et + * is the byte offset into the stub page for the CPU's stub buffer. + */ +struct stub_info { + struct page_info *pg; + unsigned int offset; +}; +struct stub_info __read_mostly stubs[NR_CPUS]; =20 -unsigned long alloc_stub_page(unsigned int cpu, unsigned long *mfn) +/* + * Index by NUMA node. + * + * `pg` is the current stub page for the node. + * `next` is the next available stub index (STUBS_PER_PAGE available). + * + * if `pg` is NULL, allocate a new one. + * if `pg` is !NULL, use `pg` and stub `next` + * When STUBS_PER_PAGE are all assigned, clear `pg` and `next`. + */ +struct stub_node { + struct page_info *pg; + unsigned int next; +}; +struct stub_node stub_nodes[MAX_NUMNODES + 1]; + +nodeid_t cpu_to_stub_node(unsigned int cpu) { - unsigned long stub_va; + nodeid_t node =3D cpu_to_node(cpu); + + return node =3D=3D NUMA_NO_NODE ? MAX_NUMNODES : node; +} + +static struct page_info *alloc_stub_page(unsigned int cpu) +{ + nodeid_t node =3D cpu_to_stub_node(cpu); + unsigned int stub_idx; struct page_info *pg; =20 BUILD_BUG_ON(STUBS_PER_PAGE & (STUBS_PER_PAGE - 1)); =20 - if ( *mfn ) - pg =3D mfn_to_page(_mfn(*mfn)); - else + if ( !stub_nodes[node].pg ) { - nodeid_t node =3D cpu_to_node(cpu); unsigned int memflags =3D node !=3D NUMA_NO_NODE ? MEMF_node(node)= : 0; =20 - pg =3D alloc_domheap_page(NULL, memflags); - if ( !pg ) - return 0; + stub_nodes[node].pg =3D alloc_domheap_page(NULL, memflags); + stub_nodes[node].next =3D 0; + + if ( !stub_nodes[node].pg ) + return NULL; =20 - unmap_domain_page(memset(__map_domain_page(pg), 0xcc, PAGE_SIZE)); + unmap_domain_page(memset(__map_domain_page(stub_nodes[node].pg), + 0xcc, PAGE_SIZE)); } =20 + stub_idx =3D stub_nodes[node].next++; + pg =3D stub_nodes[node].pg; + stubs[cpu].pg =3D stub_nodes[node].pg; + stubs[cpu].offset =3D stub_idx * STUB_BUF_SIZE; + if ( stub_nodes[node].next =3D=3D STUBS_PER_PAGE ) + { + stub_nodes[node].pg =3D NULL; + stub_nodes[node].next =3D 0; + } + + return pg; +} + +unsigned long assign_stub_page(unsigned int cpu) +{ + unsigned long stub_va; + struct page_info *pg =3D stubs[cpu].pg; + + if ( !pg ) + pg =3D alloc_stub_page(cpu); + + if ( !pg ) + return 0; + stub_va =3D XEN_VIRT_END - FIXADDR_X_SIZE - (cpu + 1) * PAGE_SIZE; if ( map_pages_to_xen(stub_va, page_to_mfn(pg), 1, PAGE_HYPERVISOR_RX | MAP_SMALL_PAGES) ) - { - if ( !*mfn ) - free_domheap_page(pg); stub_va =3D 0; + else + { + per_cpu(stubs.mfn, cpu) =3D mfn_x(page_to_mfn(pg)); + stub_va +=3D stubs[cpu].offset; } - else if ( !*mfn ) - *mfn =3D mfn_x(page_to_mfn(pg)); =20 - return stub_va ? stub_va + STUB_BUF_CPU_OFFS(cpu) : 0; + return stub_va; } =20 void cpu_exit_clear(unsigned int cpu) @@ -990,19 +1045,12 @@ static void cpu_smpboot_free(unsigned int cpu, bool = remove) { mfn_t mfn =3D _mfn(per_cpu(stubs.mfn, cpu)); unsigned char *stub_page =3D map_domain_page(mfn); - unsigned int i; =20 - memset(stub_page + STUB_BUF_CPU_OFFS(cpu), 0xcc, STUB_BUF_SIZE); - for ( i =3D 0; i < STUBS_PER_PAGE; ++i ) - if ( stub_page[i * STUB_BUF_SIZE] !=3D 0xcc ) - break; + memset(stub_page + stubs[cpu].offset, 0xcc, STUB_BUF_SIZE); unmap_domain_page(stub_page); destroy_xen_mappings(per_cpu(stubs.addr, cpu) & PAGE_MASK, (per_cpu(stubs.addr, cpu) | ~PAGE_MASK) + 1); per_cpu(stubs.addr, cpu) =3D 0; - per_cpu(stubs.mfn, cpu) =3D 0; - if ( i =3D=3D STUBS_PER_PAGE ) - free_domheap_page(mfn_to_page(mfn)); } =20 if ( IS_ENABLED(CONFIG_PV32) ) @@ -1041,7 +1089,7 @@ void *cpu_alloc_stack(unsigned int cpu) static int cpu_smpboot_alloc(unsigned int cpu) { struct cpu_info *info; - unsigned int i, memflags =3D 0; + unsigned int memflags =3D 0; nodeid_t node =3D cpu_to_node(cpu); seg_desc_t *gdt; unsigned long stub_va; @@ -1091,15 +1139,7 @@ static int cpu_smpboot_alloc(unsigned int cpu) memcpy(per_cpu(idt, cpu), bsp_idt, sizeof(bsp_idt)); disable_each_ist(per_cpu(idt, cpu)); =20 - for ( stub_page =3D 0, i =3D cpu & ~(STUBS_PER_PAGE - 1); - i < nr_cpu_ids && i <=3D (cpu | (STUBS_PER_PAGE - 1)); ++i ) - if ( cpu_online(i) && cpu_to_node(i) =3D=3D node ) - { - per_cpu(stubs.mfn, cpu) =3D per_cpu(stubs.mfn, i); - break; - } - BUG_ON(i =3D=3D cpu); - stub_va =3D alloc_stub_page(cpu, &per_cpu(stubs.mfn, cpu)); + stub_va =3D assign_stub_page(cpu); if ( !stub_va ) goto out; per_cpu(stubs.addr, cpu) =3D stub_va; --=20 2.54.0