From nobody Thu Sep 11 00:11:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 462DBC61DA4 for ; Wed, 22 Feb 2023 22:13:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232674AbjBVWNm (ORCPT ); Wed, 22 Feb 2023 17:13:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232484AbjBVWNi (ORCPT ); Wed, 22 Feb 2023 17:13:38 -0500 Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com [IPv6:2607:f8b0:4864:20::832]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A0FA43915 for ; Wed, 22 Feb 2023 14:13:20 -0800 (PST) Received: by mail-qt1-x832.google.com with SMTP id w23so9165420qtn.6 for ; Wed, 22 Feb 2023 14:13:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bKnaLUvz03dzH+Nhj/4CLV2CqW7kem3zsIr+xE7cWbY=; b=AONcsGL41TxiV4/KM1Lw+I8z3tLBaN6k8vDTCGI67ULKsXkkIpp8kCvU0dB+BdMMJ7 SW7DoLHR17TNCN/1kaxclzarB29h1EkSJsRHu4P+OuW0cT8nXrY8aSN11AYI5E2lFKy5 FxbLfKgD7BV409f/JMdV9vV6CpH+VfC/eHyFTtn6fWhPxbOlUa0VkBHI3yr7g6WWbvaI 1/gusrGcYubP+PhFw68RDc5HbjzhJUTNQHiLg76WjU71ApLE57O1ODFlnpjfy5LE+EEn nLRMpbnwI5cs0LQ2F/U5rx4NraeDIFY8ng/Y4Z11duInSTaN9XzsHI1sT+rsdxuj+f1K e8nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bKnaLUvz03dzH+Nhj/4CLV2CqW7kem3zsIr+xE7cWbY=; b=m65+zuAEIa/rWL5GmRDf4BB+QY0PP0ManAsJ/YHDdU59QQnvFpFSdljIDz9Nl1MxCI KKA7jkYj/WC2g1bdokqeoOxMHToLZSLfifHrnLcI/YkwmcbtqvyunEzWBA9509vL67v8 NImBRClljaC3eADXBq33vE1te58NEIxVel8ee7NHzZL6m+fDXlpWqMija2DkNBVfcY/v S6szTUTGZe9SYdrV23xupySypBMiZPwsqFYYtWy72RoIhUzqxEvZJMAoWjAhU/iqVWKn hcCUqhC40B0vHCURCcyIvTS/ffP7i4F5irVYEFWyLaBVOB8208yR01GFRkKJWsKcm0Q4 ylJw== X-Gm-Message-State: AO0yUKXWxPXD6auL+VkFxNJ+6LydXJ8UiJv5QR3VTsS89LJZWqtSda7L NNl+PECKUbV6Sy3ggnc4uuUtmcs4rg== X-Google-Smtp-Source: AK7set8x+RkH/L+AxiVbtTj+82I3i49ILT32qjT9hzgsdtOg8ieeGbtootox4fxrFSemgif9Rjmu9A== X-Received: by 2002:ac8:5fc7:0:b0:3b9:a60f:b2bb with SMTP id k7-20020ac85fc7000000b003b9a60fb2bbmr18333081qta.56.1677103998855; Wed, 22 Feb 2023 14:13:18 -0800 (PST) Received: from citadel.. (075-129-116-198.res.spectrum.com. [75.129.116.198]) by smtp.gmail.com with ESMTPSA id ey17-20020a05622a4c1100b003b86b088755sm4902460qtb.15.2023.02.22.14.13.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 14:13:18 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: David Woodhouse , Usama Arif , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Andy Lutomirski , Ingo Molnar , Brian Gerst Subject: [PATCH 1/6] x86/smpboot: Use CPU number instead of APIC ID for single CPU startup Date: Wed, 22 Feb 2023 17:12:56 -0500 Message-Id: <20230222221301.245890-2-brgerst@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230222221301.245890-1-brgerst@gmail.com> References: <20230222221301.245890-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Encoding the CPU number directly in smpboot_control skips the APIC ID lookup when booting a single CPU. This will enable the boot CPU to use the same code as secondaries, since the APIC ID array is not populated during early boot. Signed-off-by: Brian Gerst Reviewed-by: David Woodhouse --- arch/x86/kernel/head_64.S | 12 +++++++----- arch/x86/kernel/smpboot.c | 2 +- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 0e4e53d231db..c1253aa737ca 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -252,20 +252,22 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L= _GLOBAL) jz .Lsetup_cpu =20 /* - * Secondary CPUs find out the offsets via the APIC ID. For parallel - * boot the APIC ID is retrieved from CPUID, otherwise it's encoded - * in smpboot_control: + * For parallel boot, the APIC ID is retrieved from CPUID, and then + * used to look up the CPU number. For booting a single CPU, the + * CPU number is encoded in smpboot_control. + * * Bit 31 STARTUP_SECONDARY flag (checked above) * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b) * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01) - * Bit 0-24 APIC ID if STARTUP_APICID_CPUID_xx flags are not set + * Bit 0-24 CPU# if STARTUP_APICID_CPUID_xx flags are not set */ testl $STARTUP_APICID_CPUID_0B, %edx jnz .Luse_cpuid_0b testl $STARTUP_APICID_CPUID_01, %edx jnz .Luse_cpuid_01 andl $0x0FFFFFFF, %edx - jmp .Lsetup_AP + movl %edx, %ecx + jmp .Linit_cpu_data =20 .Luse_cpuid_01: mov $0x01, %eax diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 5b6d72b3d14b..e1a2843c2841 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1140,7 +1140,7 @@ static int do_boot_cpu(int apicid, int cpu, struct ta= sk_struct *idle, early_gdt_descr.address =3D (unsigned long)get_cpu_gdt_rw(cpu); initial_stack =3D idle->thread.sp; } else if (!do_parallel_bringup) { - smpboot_control =3D STARTUP_SECONDARY | apicid; + smpboot_control =3D STARTUP_SECONDARY | cpu; } =20 /* Enable the espfix hack for this CPU */ --=20 2.39.2 From nobody Thu Sep 11 00:11:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DEC6C61DA4 for ; Wed, 22 Feb 2023 22:13:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232768AbjBVWNu (ORCPT ); Wed, 22 Feb 2023 17:13:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232500AbjBVWNi (ORCPT ); Wed, 22 Feb 2023 17:13:38 -0500 Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com [IPv6:2607:f8b0:4864:20::82b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AE0A43918 for ; Wed, 22 Feb 2023 14:13:21 -0800 (PST) Received: by mail-qt1-x82b.google.com with SMTP id s12so9171653qtq.11 for ; Wed, 22 Feb 2023 14:13:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iNsRaJl4d7YqZipc65JwWHSPo1qnkcWEe8GKRqSMg+M=; b=p+blT5c7UnqpDedKTJuaegL1xV7yBrj4SNpVyJiV5ClFgm62zbf0U4JcZYroD3+yna kauHCJSdSFg6+mbCOLzNn6rUbrZ4ArOA87BLXPSEoZk22VjR9Fq17STuHPO2OvAtB/M2 HxT39jqgo+CPMRhiRziypJXvHMi3CMdju9U5E+WvI4ek5Xt8PSAwnCCkE112fvVtyTuc wxDekild1aWJArG4mF4o2QaM+Vv2V4aoqFWBn7BovANvDJCFQNOfsQ2C2ajafOLcFep1 Mg8szLLDW/LtRqKHFLkyFeKjC36P5rMK0I0a770zvcFWbqzGqn349RrQ164qs3o5SB1f K5tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iNsRaJl4d7YqZipc65JwWHSPo1qnkcWEe8GKRqSMg+M=; b=ubnj52I/Oc0UV3g/R7MNXmhOVdaaILhxAkOrheiuov+hyfp0N58QoBmGdygm+T7J3W 8/Il5wA1CPTgop+qhVy1sSd4pdn5HDi0CpyxcFYpWr+SFsPri0O+MaCYiT9rkYKKeExS 6aSQIE6Ca1YzRDnh1OCCbayeyWDY8Q6MMujqnHgafzpgT5xsfZVrrG1Bic6tIG4rD62c D8uLByEC4EOdmIKkNO+KGMUVnoBPhdrBiHGuMGvs4SbPZ4d3vqOX0PZtp/Dbxv7g/9G5 znAPMJxiSGcZlkIh45VdrPdkxZM1JJBszm0kIl8vRl/Ov2sgJzvtj+lb7fRUSo+AxAaF Gxmw== X-Gm-Message-State: AO0yUKWcAfrcW2DHICdQsXBq3t5j8wueGiX+vHg7KqtGgJczsDYwuBKZ tueLfEeXpmkfxu6jCO+JBtfJBLtHTA== X-Google-Smtp-Source: AK7set/Q+P3G4wNM4qm7CNiRH01c4Y/gAoXOPpr4ceI8RVBHNiXP3dxMPuirIpKWyRa6c3CHgHNvBw== X-Received: by 2002:ac8:5952:0:b0:3b8:6c68:e6d with SMTP id 18-20020ac85952000000b003b86c680e6dmr17469894qtz.13.1677104000049; Wed, 22 Feb 2023 14:13:20 -0800 (PST) Received: from citadel.. (075-129-116-198.res.spectrum.com. [75.129.116.198]) by smtp.gmail.com with ESMTPSA id ey17-20020a05622a4c1100b003b86b088755sm4902460qtb.15.2023.02.22.14.13.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 14:13:19 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: David Woodhouse , Usama Arif , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Andy Lutomirski , Ingo Molnar , Brian Gerst Subject: [PATCH 2/6] x86/smpboot: Use current_task to get idle thread Date: Wed, 22 Feb 2023 17:12:57 -0500 Message-Id: <20230222221301.245890-3-brgerst@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230222221301.245890-1-brgerst@gmail.com> References: <20230222221301.245890-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The idle_threads array is not populated during early boot. Use current_task instead, which is initialized to init_task for the boot CPU. Also simplify start_cpu0(). Since the boot CPU never really goes offline, the GSBASE is still set up and can be used for per-cpu accesses. Signed-off-by: Brian Gerst Reviewed-by: David Woodhouse --- arch/x86/kernel/asm-offsets.c | 1 + arch/x86/kernel/head_64.S | 7 ++----- kernel/smpboot.c | 2 +- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c index 8650f29387e0..445bce086717 100644 --- a/arch/x86/kernel/asm-offsets.c +++ b/arch/x86/kernel/asm-offsets.c @@ -114,6 +114,7 @@ static void __used common(void) OFFSET(TSS_sp1, tss_struct, x86_tss.sp1); OFFSET(TSS_sp2, tss_struct, x86_tss.sp2); OFFSET(X86_top_of_stack, pcpu_hot, top_of_stack); + OFFSET(X86_current_task, pcpu_hot, current_task); #ifdef CONFIG_CALL_DEPTH_TRACKING OFFSET(X86_call_depth, pcpu_hot, call_depth); #endif diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index c1253aa737ca..c32e5b06a9ce 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -315,7 +315,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_G= LOBAL) movq %rcx, early_gdt_descr_base(%rip) =20 /* Find the idle task stack */ - movq idle_threads(%rbx), %rcx + movq pcpu_hot + X86_current_task(%rbx), %rcx movq TASK_threadsp(%rcx), %rcx movq %rcx, initial_stack(%rip) #endif /* CONFIG_SMP */ @@ -460,12 +460,9 @@ SYM_CODE_END(secondary_startup_64) SYM_CODE_START(start_cpu0) ANNOTATE_NOENDBR UNWIND_HINT_EMPTY - /* Load the per-cpu base for CPU#0 */ - leaq __per_cpu_offset(%rip), %rbx - movq (%rbx), %rbx =20 /* Find the idle task stack */ - movq idle_threads(%rbx), %rcx + movq PER_CPU_VAR(pcpu_hot) + X86_current_task, %rcx movq TASK_threadsp(%rcx), %rsp =20 jmp .Ljump_to_C_code diff --git a/kernel/smpboot.c b/kernel/smpboot.c index a18a21dff9bc..2c7396da470c 100644 --- a/kernel/smpboot.c +++ b/kernel/smpboot.c @@ -25,7 +25,7 @@ * For the hotplug case we keep the task structs around and reuse * them. */ -DEFINE_PER_CPU(struct task_struct *, idle_threads); +static DEFINE_PER_CPU(struct task_struct *, idle_threads); =20 struct task_struct *idle_thread_get(unsigned int cpu) { --=20 2.39.2 From nobody Thu Sep 11 00:11:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2217C61DA4 for ; Wed, 22 Feb 2023 22:13:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232798AbjBVWN5 (ORCPT ); Wed, 22 Feb 2023 17:13:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232457AbjBVWNj (ORCPT ); Wed, 22 Feb 2023 17:13:39 -0500 Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69F4291 for ; Wed, 22 Feb 2023 14:13:22 -0800 (PST) Received: by mail-qt1-x834.google.com with SMTP id b6so2958961qtb.1 for ; Wed, 22 Feb 2023 14:13:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aTZtaHujjzboIBxRswwzZg+kJeIkZN07DRxgFsYXLxo=; b=V/Mk3ffQteBbNJV3TTjZlY2uoV9u5vqGBg9SrvRB6qP0XE7IgPIyqTNImL2lw0q8in XP/vGzUzZ85JuLFSxAn7McTz/vGaXDDngNLCuQuBhMxyuHSM37z3ZhoSvbXN9VeZDK6x hAg2Z6OO59lv0p62k/SFHfl2CPu4i0h8ZWdfawP8Hun75JcYhxyy8kzdKJIXWypMc44s CwRDhmExAcAvYqkQGqcJlBhnLcEJLtlxx7z934iZh18VKVjq68eSyONDcS+faioD8RjO gA5mpFeDIVzVTGsLWSYjrp9W+jsJ4UF6NKYTxMbeWxvu5uxaTdLhn5nfiyLlT910nNJL oRaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aTZtaHujjzboIBxRswwzZg+kJeIkZN07DRxgFsYXLxo=; b=Fw2lRzBM71tLwRSRns2AjS82PQlROHUo1TpiotLohaEzamx+mBn2eldhaqh2zZ8OIc x+AnOye69mkKPrz2YlnASHsDv+l2z0cnsKD//F5kiccISBSTOY2Xn9Pkl8Tqp47Gr7lp BmFMA49F6JogTE4NKfuD5xBf02SnHnvKcmCf0a0G7mupzSwDaV+lWedof4AXOvrmQKh8 hz4sWHs30eIgKOICcyXmeotylVun9EF9h4gTkrwttK6ReXiqmovIJJp5+AKbJYTQrAkC HmVPDA4Nf7zPQGjGJ/kxC1qtZmr1i/H3p6Npx7EfHCJpWCOYolUlhWzWtEJrPMMen6i0 hbyw== X-Gm-Message-State: AO0yUKW7MlAXs+bGsm9iTY+LW9vNlKB4bxtgPHdX1sV78p0BHAgwxNJS 6flZ9kKTFb1n3GYAIoYzuWEV80wsKQ== X-Google-Smtp-Source: AK7set8lNvS9WvOjXR8PkYKgn+VaKLp8tEnXjAgPLtOybsWh1FUKc8plyT/GBKsPLEl2HFn4pnhJFg== X-Received: by 2002:a05:622a:118a:b0:3b2:1003:37e5 with SMTP id m10-20020a05622a118a00b003b2100337e5mr18518163qtk.55.1677104001194; Wed, 22 Feb 2023 14:13:21 -0800 (PST) Received: from citadel.. (075-129-116-198.res.spectrum.com. [75.129.116.198]) by smtp.gmail.com with ESMTPSA id ey17-20020a05622a4c1100b003b86b088755sm4902460qtb.15.2023.02.22.14.13.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 14:13:20 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: David Woodhouse , Usama Arif , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Andy Lutomirski , Ingo Molnar , Brian Gerst Subject: [PATCH 3/6] x86/smpboot: Remove initial_stack on 64-bit Date: Wed, 22 Feb 2023 17:12:58 -0500 Message-Id: <20230222221301.245890-4-brgerst@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230222221301.245890-1-brgerst@gmail.com> References: <20230222221301.245890-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Load RSP from current_task->thread.sp instead. Signed-off-by: Brian Gerst Reviewed-by: David Woodhouse --- arch/x86/include/asm/processor.h | 6 +++++- arch/x86/kernel/acpi/sleep.c | 2 +- arch/x86/kernel/head_64.S | 35 ++++++++++++++++++-------------- arch/x86/xen/xen-head.S | 2 +- 4 files changed, 27 insertions(+), 18 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 8d73004e4cac..7f64b69c2b0e 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -647,7 +647,11 @@ static inline void spin_lock_prefetch(const void *x) #define KSTK_ESP(task) (task_pt_regs(task)->sp) =20 #else -#define INIT_THREAD { } +extern unsigned long __end_init_task[]; + +#define INIT_THREAD { \ + .sp =3D (unsigned long)&__end_init_task - PTREGS_SIZE, \ +} =20 extern unsigned long KSTK_ESP(struct task_struct *task); =20 diff --git a/arch/x86/kernel/acpi/sleep.c b/arch/x86/kernel/acpi/sleep.c index 47e75c056cb5..008fda8b1982 100644 --- a/arch/x86/kernel/acpi/sleep.c +++ b/arch/x86/kernel/acpi/sleep.c @@ -113,7 +113,7 @@ int x86_acpi_suspend_lowlevel(void) saved_magic =3D 0x12345678; #else /* CONFIG_64BIT */ #ifdef CONFIG_SMP - initial_stack =3D (unsigned long)temp_stack + sizeof(temp_stack); + current->thread.sp =3D (unsigned long)temp_stack + sizeof(temp_stack); early_gdt_descr.address =3D (unsigned long)get_cpu_gdt_rw(smp_processor_id()); initial_gs =3D per_cpu_offset(smp_processor_id()); diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index c32e5b06a9ce..f7905ba4b992 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -62,8 +62,8 @@ SYM_CODE_START_NOALIGN(startup_64) * tables and then reload them. */ =20 - /* Set up the stack for verify_cpu(), similar to initial_stack below */ - leaq (__end_init_task - FRAME_SIZE)(%rip), %rsp + /* Set up the stack for verify_cpu() */ + leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp =20 leaq _text(%rip), %rdi =20 @@ -245,11 +245,11 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L= _GLOBAL) #ifdef CONFIG_SMP /* * Is this the boot CPU coming up? If so everything is available - * in initial_gs, initial_stack and early_gdt_descr. + * in initial_gs and early_gdt_descr. */ movl smpboot_control(%rip), %edx testl $STARTUP_SECONDARY, %edx - jz .Lsetup_cpu + jz .Linit_cpu0_data =20 /* * For parallel boot, the APIC ID is retrieved from CPUID, and then @@ -302,6 +302,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_= GLOBAL) hlt jmp 1b =20 +.Linit_cpu0_data: + movq __per_cpu_offset(%rip), %rdx + jmp .Lsetup_cpu + .Linit_cpu_data: /* Get the per cpu offset for the given CPU# which is in ECX */ leaq __per_cpu_offset(%rip), %rbx @@ -314,13 +318,21 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L= _GLOBAL) addq %rbx, %rcx movq %rcx, early_gdt_descr_base(%rip) =20 - /* Find the idle task stack */ - movq pcpu_hot + X86_current_task(%rbx), %rcx - movq TASK_threadsp(%rcx), %rcx - movq %rcx, initial_stack(%rip) + movq %rbx, %rdx +#else + xorl %edx, %edx #endif /* CONFIG_SMP */ =20 .Lsetup_cpu: + /* + * Setup a boot time stack - Any secondary CPU will have lost its stack + * by now because the cr3-switch above unmaps the real-mode stack + * + * RDX contains the per-cpu offset + */ + movq pcpu_hot + X86_current_task(%rdx), %rax + movq TASK_threadsp(%rax), %rsp + /* * We must switch to a new descriptor in kernel space for the GDT * because soon the kernel won't have access anymore to the userspace @@ -355,12 +367,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_= GLOBAL) movl initial_gs+4(%rip),%edx wrmsr =20 - /* - * Setup a boot time stack - Any secondary CPU will have lost its stack - * by now because the cr3-switch above unmaps the real-mode stack - */ - movq initial_stack(%rip), %rsp - /* Drop the realmode protection. For the boot CPU the pointer is NULL! */ movq trampoline_lock(%rip), %rax testq %rax, %rax @@ -517,7 +523,6 @@ SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb) * The FRAME_SIZE gap is a convention which helps the in-kernel unwinder * reliably detect the end of the stack. */ -SYM_DATA(initial_stack, .quad init_thread_union + THREAD_SIZE - FRAME_SIZE) SYM_DATA(trampoline_lock, .quad 0); __FINITDATA =20 diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index e36ea4268bd2..91f7a53519a7 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -49,7 +49,7 @@ SYM_CODE_START(startup_xen) ANNOTATE_NOENDBR cld =20 - mov initial_stack(%rip), %rsp + leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp =20 /* Set up %gs. * --=20 2.39.2 From nobody Thu Sep 11 00:11:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 277C9C61DA4 for ; Wed, 22 Feb 2023 22:13:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232781AbjBVWNx (ORCPT ); Wed, 22 Feb 2023 17:13:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232523AbjBVWNi (ORCPT ); Wed, 22 Feb 2023 17:13:38 -0500 Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7916269E for ; Wed, 22 Feb 2023 14:13:23 -0800 (PST) Received: by mail-qt1-x82a.google.com with SMTP id w23so9165558qtn.6 for ; Wed, 22 Feb 2023 14:13:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q/4x9pKiWHQd4E9PHb0AStQCoTUIAgCrnRCTs6gjCBg=; b=bq5W5DCO6zteSItMu7IDxWg0jtdy8N6ZOUsZFYI4Vr05QFSPAVgMMRnA6zUY+tOrAF GVWiTHmEkbq38U5+QK9+aWiuuuPkT7npWFu1ZXam+cwCnSmDQqfJ4AYzh8MfTp6q6bO6 v767B9ashF+IX6xrk//Pw5X5srWJn0H2F7Kwcd+nn+Cr1RFxjbgE3xVw4nFkLMVwjpeq xx1R30EJ7OKV8xZmuu7Dj6oCfR7qKq1SzCaKKXnAf/x7wcX8tn7EXFVWVl7E3L+pzrE4 1TGDCzFEU/ROJBP/HrKEQA34whLuXfeTaa05ilWfAUuedpMiPMzuk9AZ8X9ae/+xjhP9 FPXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q/4x9pKiWHQd4E9PHb0AStQCoTUIAgCrnRCTs6gjCBg=; b=HRsJOlWRp5KJ/HBab1okzvIQhqkTJnvoOiuEAa9mb2hrlvqiVHL980spirqlKlpD7t OuAUoxIKGzjiGBExJGr6Be/ycKvC/w58Gok4Jx8KNfhISnUdXlEtA4BvEjtkE82MX1e+ O7qUGEf0dR0E14o+Q2eQ8uDMSA/yCdrrVUVSsk7P0eiSBi4UZcXHrpR+I+5/pc7gF0tT x7TtHQsHNvU/FUFvh/cO1yJXR6bDiL8dQqtXc5RC9mNYwtszWBBi+8HQqOgEeUmCK+2t PeAAj6m4M3JuQ9egpNif/czMTrRaYQOFnvkBOZUvJrnKPU/tIFCy6GKzxvqcedpQyFzZ eQgw== X-Gm-Message-State: AO0yUKUMyFbK3ravVCNoXcGgYcZlap1Df3wq1dpwguXg7EOzKHMQT0C3 Y7UUrCcXckmA0yCiawMCStuhtqZqMw== X-Google-Smtp-Source: AK7set8VhrBFVUDZAsuO6MKEQN7b3zUGUuRcxvvKJs/rfLseKH1Y8j57Lr2lm7q4FztP3nQ82ww+YA== X-Received: by 2002:ac8:5850:0:b0:3b3:b92f:3650 with SMTP id h16-20020ac85850000000b003b3b92f3650mr2410995qth.62.1677104002589; Wed, 22 Feb 2023 14:13:22 -0800 (PST) Received: from citadel.. (075-129-116-198.res.spectrum.com. [75.129.116.198]) by smtp.gmail.com with ESMTPSA id ey17-20020a05622a4c1100b003b86b088755sm4902460qtb.15.2023.02.22.14.13.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 14:13:22 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: David Woodhouse , Usama Arif , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Andy Lutomirski , Ingo Molnar , Brian Gerst Subject: [PATCH 4/6] x86/smpbppt: Remove early_gdt_descr on 64-bit Date: Wed, 22 Feb 2023 17:12:59 -0500 Message-Id: <20230222221301.245890-5-brgerst@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230222221301.245890-1-brgerst@gmail.com> References: <20230222221301.245890-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Build the GDT descriptor on the stack instead. Signed-off-by: Brian Gerst Reviewed-by: David Woodhouse --- arch/x86/kernel/acpi/sleep.c | 2 -- arch/x86/kernel/head_64.S | 19 +++++++------------ 2 files changed, 7 insertions(+), 14 deletions(-) diff --git a/arch/x86/kernel/acpi/sleep.c b/arch/x86/kernel/acpi/sleep.c index 008fda8b1982..6538ddb55f28 100644 --- a/arch/x86/kernel/acpi/sleep.c +++ b/arch/x86/kernel/acpi/sleep.c @@ -114,8 +114,6 @@ int x86_acpi_suspend_lowlevel(void) #else /* CONFIG_64BIT */ #ifdef CONFIG_SMP current->thread.sp =3D (unsigned long)temp_stack + sizeof(temp_stack); - early_gdt_descr.address =3D - (unsigned long)get_cpu_gdt_rw(smp_processor_id()); initial_gs =3D per_cpu_offset(smp_processor_id()); /* Force the startup into boot mode */ saved_smpboot_ctrl =3D xchg(&smpboot_control, 0); diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index f7905ba4b992..0dd57d573a0e 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -245,7 +245,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_G= LOBAL) #ifdef CONFIG_SMP /* * Is this the boot CPU coming up? If so everything is available - * in initial_gs and early_gdt_descr. + * in initial_gs. */ movl smpboot_control(%rip), %edx testl $STARTUP_SECONDARY, %edx @@ -313,11 +313,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_= GLOBAL) /* Save it for GS BASE setup */ movq %rbx, initial_gs(%rip) =20 - /* Calculate the GDT address */ - movq $gdt_page, %rcx - addq %rbx, %rcx - movq %rcx, early_gdt_descr_base(%rip) - movq %rbx, %rdx #else xorl %edx, %edx @@ -339,7 +334,12 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_= GLOBAL) * addresses where we're currently running on. We have to do that here * because in 32bit we couldn't load a 64bit linear address. */ - lgdt early_gdt_descr(%rip) + subq $16, %rsp + movw $(GDT_SIZE-1), (%rsp) + leaq gdt_page(%rdx), %rax + movq %rax, 2(%rsp) + lgdt (%rsp) + addq $16, %rsp =20 /* set up data segments */ xorl %eax,%eax @@ -754,11 +754,6 @@ SYM_DATA_END(level1_fixmap_pgt) =20 .data .align 16 - -SYM_DATA(early_gdt_descr, .word GDT_ENTRIES*8-1) -SYM_DATA_LOCAL(early_gdt_descr_base, .quad INIT_PER_CPU_VAR(gdt_page)) - - .align 16 SYM_DATA(smpboot_control, .long 0) =20 .align 16 --=20 2.39.2 From nobody Thu Sep 11 00:11:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03BD2C61DA4 for ; Wed, 22 Feb 2023 22:14:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232858AbjBVWOD (ORCPT ); Wed, 22 Feb 2023 17:14:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232553AbjBVWNj (ORCPT ); Wed, 22 Feb 2023 17:13:39 -0500 Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 014162713 for ; Wed, 22 Feb 2023 14:13:24 -0800 (PST) Received: by mail-qt1-x834.google.com with SMTP id x1so9804191qtw.3 for ; Wed, 22 Feb 2023 14:13:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UZl87VAowiKqRTZnAo8837ZE4CLgK5LvrG7fNO1ZVqU=; b=jYzbenw1o4WXN7UaxcFUzN7LT/X3NE/lyfzPyQvKp/a3COZO15lUWzwS6saNXInRTi Ou/YHN0RnMFNntznqTHHshDdc292dsv+JpqZRmGXuiKAVA/eFRD6p59akg3shOdNz42O bR6wfBlbVwNk9FADxavCZWou7CQhd0FbFeDE9fCpnFeVD0dUrre5BcsWAmpe4Rpn402O A5oH2CnBkxtnrUUTZII/NdTnxrF1K9Pz7VbDisC3k1nE5lAeTUFwPduhlbnNEqzpHZ7Z PX5jKT9GoHtqkLXk0smZ8U6iuCPvy6OJO/CFHESfdjoH/71UwltmwW+shllu5Tqe2pvG 3ZMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UZl87VAowiKqRTZnAo8837ZE4CLgK5LvrG7fNO1ZVqU=; b=DebcNascAt8gTb0eB6YB8bhg7+JjbAGTeF8AfFG1hykhIUv1qgzUX7TGoiiLZKDWLZ 9/T956GhSuCImDKuk5x84h3WDMnOy5v/LoiYbGiFPQOmni9+FAcnrBcWjhfSrwaUGDnO d0aM0QzzB2qha9ja/pTde103OljJGdhBcia9wRrrKumRKotrhtVBjgeuAqN/FSMD+PTd gJbd7YOC4LfFBMnQ2cytLF+Qiim2gn3QBCfqY/Olv6PTb6XuequdwnOpYoUp4shUdBjt VLATPqMRUCEXNXHJ5lujH5rnAogsUJJdsBAGiumMfb7fhXbaimB85WrNOYWQdMLB/NEX yKlw== X-Gm-Message-State: AO0yUKUvXcst6rf2q4qUR7z4YA/zz5+GCUjatv1XdVP+OM/EhTOgz3DZ VZmtHlH9uARrfpZ/P5boGBDdrVIrAA== X-Google-Smtp-Source: AK7set8pjyOZJpyw4AK24+y/4GfR1klo7ua0e3MNGMI1Rn4vmgQWqIMgSqsR493uhVD5Dwd79T7+Sg== X-Received: by 2002:a05:622a:1483:b0:3b8:6b4c:aba8 with SMTP id t3-20020a05622a148300b003b86b4caba8mr19772710qtx.39.1677104003866; Wed, 22 Feb 2023 14:13:23 -0800 (PST) Received: from citadel.. (075-129-116-198.res.spectrum.com. [75.129.116.198]) by smtp.gmail.com with ESMTPSA id ey17-20020a05622a4c1100b003b86b088755sm4902460qtb.15.2023.02.22.14.13.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 14:13:23 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: David Woodhouse , Usama Arif , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Andy Lutomirski , Ingo Molnar , Brian Gerst Subject: [PATCH 5/6] x86/smpboot: Remove initial_gs Date: Wed, 22 Feb 2023 17:13:00 -0500 Message-Id: <20230222221301.245890-6-brgerst@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230222221301.245890-1-brgerst@gmail.com> References: <20230222221301.245890-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use the percpu offset directly to set GSBASE. Signed-off-by: Brian Gerst Reviewed-by: David Woodhouse --- arch/x86/include/asm/realmode.h | 1 - arch/x86/kernel/acpi/sleep.c | 1 - arch/x86/kernel/head_64.S | 34 ++++++++++----------------------- 3 files changed, 10 insertions(+), 26 deletions(-) diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmod= e.h index f0357cfe2fb0..87e5482acd0d 100644 --- a/arch/x86/include/asm/realmode.h +++ b/arch/x86/include/asm/realmode.h @@ -60,7 +60,6 @@ extern struct real_mode_header *real_mode_header; extern unsigned char real_mode_blob_end[]; =20 extern unsigned long initial_code; -extern unsigned long initial_gs; extern unsigned long initial_stack; #ifdef CONFIG_AMD_MEM_ENCRYPT extern unsigned long initial_vc_handler; diff --git a/arch/x86/kernel/acpi/sleep.c b/arch/x86/kernel/acpi/sleep.c index 6538ddb55f28..214dd4a79860 100644 --- a/arch/x86/kernel/acpi/sleep.c +++ b/arch/x86/kernel/acpi/sleep.c @@ -114,7 +114,6 @@ int x86_acpi_suspend_lowlevel(void) #else /* CONFIG_64BIT */ #ifdef CONFIG_SMP current->thread.sp =3D (unsigned long)temp_stack + sizeof(temp_stack); - initial_gs =3D per_cpu_offset(smp_processor_id()); /* Force the startup into boot mode */ saved_smpboot_ctrl =3D xchg(&smpboot_control, 0); #endif diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 0dd57d573a0e..9ed87ba0609f 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -67,18 +67,10 @@ SYM_CODE_START_NOALIGN(startup_64) =20 leaq _text(%rip), %rdi =20 - /* - * initial_gs points to initial fixed_percpu_data struct with storage for - * the stack protector canary. Global pointer fixups are needed at this - * stage, so apply them as is done in fixup_pointer(), and initialize %gs - * such that the canary can be accessed at %gs:40 for subsequent C calls. - */ + /* Setup GSBASE to allow stack canary access for C code */ movl $MSR_GS_BASE, %ecx - movq initial_gs(%rip), %rax - movq $_text, %rdx - subq %rdx, %rax - addq %rdi, %rax - movq %rax, %rdx + leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx + movl %edx, %eax shrq $32, %rdx wrmsr =20 @@ -243,10 +235,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_= GLOBAL) ANNOTATE_NOENDBR // above =20 #ifdef CONFIG_SMP - /* - * Is this the boot CPU coming up? If so everything is available - * in initial_gs. - */ + /* Is this the boot CPU coming up? */ movl smpboot_control(%rip), %edx testl $STARTUP_SECONDARY, %edx jz .Linit_cpu0_data @@ -308,12 +297,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_= GLOBAL) =20 .Linit_cpu_data: /* Get the per cpu offset for the given CPU# which is in ECX */ - leaq __per_cpu_offset(%rip), %rbx - movq (%rbx,%rcx,8), %rbx - /* Save it for GS BASE setup */ - movq %rbx, initial_gs(%rip) - - movq %rbx, %rdx + movq __per_cpu_offset(,%rcx,8), %rdx #else xorl %edx, %edx #endif /* CONFIG_SMP */ @@ -363,8 +347,11 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_= GLOBAL) * the per cpu areas are set up. */ movl $MSR_GS_BASE,%ecx - movl initial_gs(%rip),%eax - movl initial_gs+4(%rip),%edx +#ifndef CONFIG_SMP + leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx +#endif + movl %edx, %eax + shrq $32, %rdx wrmsr =20 /* Drop the realmode protection. For the boot CPU the pointer is NULL! */ @@ -514,7 +501,6 @@ SYM_CODE_END(vc_boot_ghcb) __REFDATA .balign 8 SYM_DATA(initial_code, .quad x86_64_start_kernel) -SYM_DATA(initial_gs, .quad INIT_PER_CPU_VAR(fixed_percpu_data)) #ifdef CONFIG_AMD_MEM_ENCRYPT SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb) #endif --=20 2.39.2 From nobody Thu Sep 11 00:11:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B5C6C636D6 for ; Wed, 22 Feb 2023 22:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232815AbjBVWOA (ORCPT ); Wed, 22 Feb 2023 17:14:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232524AbjBVWNj (ORCPT ); Wed, 22 Feb 2023 17:13:39 -0500 Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B4B92D68 for ; Wed, 22 Feb 2023 14:13:26 -0800 (PST) Received: by mail-qt1-x82d.google.com with SMTP id b6so2959134qtb.1 for ; Wed, 22 Feb 2023 14:13:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MKWSB76bpGFHfSMuRjbeX5BE0c7lf8Zs7C04cRyFq44=; b=SNEmnY6SbsIYNsIbyiBD4bfLXQZarAQFjf1PgWn08FxeWwnvPuspqhFkaTJd1+LpGs TTWkSFwf6h4emTlR7H3WRaqhiPm5khBQgmo7NuvAGYED8WJPcw6T7Vpwp+P/Zb+ozsXd XVuqdwaoRl/OQsUm0o409j7MR6cnh50XSCEzZnsn6Uf8Tts9tiwZO9RVdYYNPKcb99HG CWxO7Ye67f/R4M4zqDzn2+NaT6WOitGJsoPKc9vckemunHnSGUXNRzFxvjecbVCTe3cD hc7lzfgTNMCGiR7ou27YISwyCI4nU8PubrBOI6BDPtJeyV/TbbQXK7B4AT+JQSvaqr/D gG2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MKWSB76bpGFHfSMuRjbeX5BE0c7lf8Zs7C04cRyFq44=; b=rbRfVMcpReFxsFJ03qP+ONBF19wF9aMHqfV2c5sgteesC1WoES4yYZtV2R2wE74imy nPR93HaW4YBnLXczp0JxHbyAdxedaoM0aqxYiQUtaovAAhy4FEJkGAtrXGEf1LSCk9z9 fiLrmdBfuOtuQ0sB8J1SFGTV/HZoc0YP/KToNO6w9yfqK43u/kiQZeY/OAL+a8o6gFbT RiJTQWFbr8N2K4uQdj2qgp3zM7cAPqLcYhjyrEs0D6HUXCMPywGK7wrJZibVqdAdejRN 3pxKsvpVzHsL94G1ssVBT3WESrvJ6I8XIfOzrVjdf0GNDxC6X9wpelgFiWI2cs0yO/qA KtMg== X-Gm-Message-State: AO0yUKWagj6rW4JWfQ0uM48qod4lm/AOIQMTm8ByQuut4ZS08gE5+yHJ DLu2hwbLkYtuVyMSZ+9D2AaQIQyhxQ== X-Google-Smtp-Source: AK7set+WqWfCTFNa8+rLCgae4hcHWf16A8wYvxx+r5KTVl8ANeRkhImZhwopwzcNlYRh6/XlecxcSA== X-Received: by 2002:ac8:5ad5:0:b0:3bb:7649:bd89 with SMTP id d21-20020ac85ad5000000b003bb7649bd89mr18887315qtd.61.1677104004977; Wed, 22 Feb 2023 14:13:24 -0800 (PST) Received: from citadel.. (075-129-116-198.res.spectrum.com. [75.129.116.198]) by smtp.gmail.com with ESMTPSA id ey17-20020a05622a4c1100b003b86b088755sm4902460qtb.15.2023.02.22.14.13.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 14:13:24 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: David Woodhouse , Usama Arif , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Andy Lutomirski , Ingo Molnar , Brian Gerst Subject: [PATCH 6/6] x86/smpboot: Simplify boot CPU setup Date: Wed, 22 Feb 2023 17:13:01 -0500 Message-Id: <20230222221301.245890-7-brgerst@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230222221301.245890-1-brgerst@gmail.com> References: <20230222221301.245890-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Now that the per-cpu GSBASE, stack, and GDT descriptor can be derived dynammically by CPU number, the boot CPU can use a fixed CPU number and take the same path as secondary CPUs. Signed-off-by: Brian Gerst Reviewed-by: David Woodhouse --- arch/x86/include/asm/smp.h | 1 - arch/x86/kernel/head_64.S | 25 +++++++------------------ arch/x86/kernel/smpboot.c | 6 +++--- 3 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index b4b29e052b6e..32c9157238c0 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -202,7 +202,6 @@ extern unsigned int smpboot_control; #endif /* !__ASSEMBLY__ */ =20 /* Control bits for startup_64 */ -#define STARTUP_SECONDARY 0x80000000 #define STARTUP_APICID_CPUID_0B 0x40000000 #define STARTUP_APICID_CPUID_01 0x20000000 =20 diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 9ed87ba0609f..949c13b26811 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -235,28 +235,22 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L= _GLOBAL) ANNOTATE_NOENDBR // above =20 #ifdef CONFIG_SMP - /* Is this the boot CPU coming up? */ - movl smpboot_control(%rip), %edx - testl $STARTUP_SECONDARY, %edx - jz .Linit_cpu0_data - /* * For parallel boot, the APIC ID is retrieved from CPUID, and then * used to look up the CPU number. For booting a single CPU, the * CPU number is encoded in smpboot_control. * - * Bit 31 STARTUP_SECONDARY flag (checked above) * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b) * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01) * Bit 0-24 CPU# if STARTUP_APICID_CPUID_xx flags are not set */ - testl $STARTUP_APICID_CPUID_0B, %edx + movl smpboot_control(%rip), %ecx + testl $STARTUP_APICID_CPUID_0B, %ecx jnz .Luse_cpuid_0b - testl $STARTUP_APICID_CPUID_01, %edx + testl $STARTUP_APICID_CPUID_01, %ecx jnz .Luse_cpuid_01 - andl $0x0FFFFFFF, %edx - movl %edx, %ecx - jmp .Linit_cpu_data + andl $0x0FFFFFFF, %ecx + jmp .Lsetup_cpu =20 .Luse_cpuid_01: mov $0x01, %eax @@ -277,7 +271,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_G= LOBAL) =20 .Lfind_cpunr: cmpl (%rbx,%rcx,4), %edx - jz .Linit_cpu_data + jz .Lsetup_cpu inc %ecx cmpl nr_cpu_ids(%rip), %ecx jb .Lfind_cpunr @@ -291,18 +285,13 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L= _GLOBAL) hlt jmp 1b =20 -.Linit_cpu0_data: - movq __per_cpu_offset(%rip), %rdx - jmp .Lsetup_cpu - -.Linit_cpu_data: +.Lsetup_cpu: /* Get the per cpu offset for the given CPU# which is in ECX */ movq __per_cpu_offset(,%rcx,8), %rdx #else xorl %edx, %edx #endif /* CONFIG_SMP */ =20 -.Lsetup_cpu: /* * Setup a boot time stack - Any secondary CPU will have lost its stack * by now because the cr3-switch above unmaps the real-mode stack diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index e1a2843c2841..c159a5c2df9f 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1140,7 +1140,7 @@ static int do_boot_cpu(int apicid, int cpu, struct ta= sk_struct *idle, early_gdt_descr.address =3D (unsigned long)get_cpu_gdt_rw(cpu); initial_stack =3D idle->thread.sp; } else if (!do_parallel_bringup) { - smpboot_control =3D STARTUP_SECONDARY | cpu; + smpboot_control =3D cpu; } =20 /* Enable the espfix hack for this CPU */ @@ -1580,7 +1580,7 @@ void __init native_smp_prepare_cpus(unsigned int max_= cpus) */ if (eax) { pr_debug("Using CPUID 0xb for parallel CPU startup\n"); - smpboot_control =3D STARTUP_SECONDARY | STARTUP_APICID_CPUID_0B; + smpboot_control =3D STARTUP_APICID_CPUID_0B; } else { pr_info("Disabling parallel bringup because CPUID 0xb looks untrustwort= hy\n"); do_parallel_bringup =3D false; @@ -1588,7 +1588,7 @@ void __init native_smp_prepare_cpus(unsigned int max_= cpus) } else if (do_parallel_bringup) { /* Without X2APIC, what's in CPUID 0x01 should suffice. */ pr_debug("Using CPUID 0x1 for parallel CPU startup\n"); - smpboot_control =3D STARTUP_SECONDARY | STARTUP_APICID_CPUID_01; + smpboot_control =3D STARTUP_APICID_CPUID_01; } =20 if (do_parallel_bringup) { --=20 2.39.2