From nobody Thu Dec 18 18:47:40 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=linutronix.de ARC-Seal: i=1; a=rsa-sha256; t=1681515897; cv=none; d=zohomail.com; s=zohoarc; b=ZyxWfSaFAwWfaHxHvcVF4p3IYW5y+5YjCUvfLK75ELlhsvJjf2Mn0IB1LUUcU5noywafY2vW7LkiZJ4EXdrbRIXIPsHJkDkfPJ2ShvG72EJsvzvTJu3Iy8Yrp+FpS9lLOwXdTu+/B0s9ocBOzYo2ekLIN7hsfABZ2Op3S3pCAlg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1681515897; h=Content-Type:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vYiQOFtP2lP/XNgK/P8qTVjY7wF3a2V1zLIb771O4ZM=; b=TtYZcgb7JHacrC98Bm+ycS6V6khz6xAFDBvWVUPIgACc7Kqj0CypnLDjvbkjWW5ygh28efj41zZIe0dAwvoJwuZD3RDKcj55CwnXaObMz690UTUEX4JPrMma2HC/J4GDfKftiy+Lb9DENK+j8rNYlwll37NQ+sDgQhG2ZviQW7E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 168151589778679.55037492774875; Fri, 14 Apr 2023 16:44:57 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.521273.809840 (Exim 4.92) (envelope-from ) id 1pnT5n-0004KA-R7; Fri, 14 Apr 2023 23:44:35 +0000 Received: by outflank-mailman (output) from mailman id 521273.809840; Fri, 14 Apr 2023 23:44:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pnT5n-0004HW-GB; Fri, 14 Apr 2023 23:44:35 +0000 Received: by outflank-mailman (input) for mailman id 521273; Fri, 14 Apr 2023 23:44:34 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pnT5m-0001Th-CM for xen-devel@lists.xenproject.org; Fri, 14 Apr 2023 23:44:34 +0000 Received: from galois.linutronix.de (galois.linutronix.de [193.142.43.55]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4edf5eb0-db1e-11ed-b21e-6b7b168915f2; Sat, 15 Apr 2023 01:44:33 +0200 (CEST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4edf5eb0-db1e-11ed-b21e-6b7b168915f2 Message-ID: <20230414232309.948211096@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1681515873; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=vYiQOFtP2lP/XNgK/P8qTVjY7wF3a2V1zLIb771O4ZM=; b=ozP/whFRJCNtzWtZf47EwOD3ctF5Yl7nBhhPFmfGOp3vNuB6ZYseZErnaJuxpU20wGHlVt aTZzyJkMmm5NdQRvyG+n9diEY1Th1N+/hNKcN5FC+5wvr+HXD2qTAouaqdTP1xuZf9HF0k M58CetJX/We25l+wDbpp+iFsAladFY6A+8V0DeXEQPBkTYzbCTvagMj54yMvSeCmqTBY8B NuJdxqnKhTdSvfyUvbfD7TzIVkuVpe2zLY9+oBadLQdJiEqE/pGfpKmP3r5+H23H6uRJJs iN3tSKES7XyjbX1WXywuTqJ797O7g2NsRouRI92YsdfZkjPlG9hwc3S8b4FTCw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1681515873; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=vYiQOFtP2lP/XNgK/P8qTVjY7wF3a2V1zLIb771O4ZM=; b=OXd3Ubv1MGKAClArDXPNKshLMJqlxJq7sB82Sa9cgrlhWYCSXIxBkFLh2ALnT54B63KCKg PsqQBV4ALo0t5yBQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, David Woodhouse , Andrew Cooper , Brian Gerst , "Arjan van de Veen" , Paolo Bonzini , Paul McKenney , Tom Lendacky , Sean Christopherson , Oleksandr Natalenko , Paul Menzel , "Guilherme G. Piccoli" , Piotr Gorski , David Woodhouse , Usama Arif , Juergen Gross , Boris Ostrovsky , xen-devel@lists.xenproject.org, Russell King , Arnd Bergmann , linux-arm-kernel@lists.infradead.org, Catalin Marinas , Will Deacon , Guo Ren , linux-csky@vger.kernel.org, Thomas Bogendoerfer , linux-mips@vger.kernel.org, "James E.J. Bottomley" , Helge Deller , linux-parisc@vger.kernel.org, Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, Mark Rutland , Sabin Rapan Subject: [patch 12/37] x86/smpboot: Make TSC synchronization function call based References: <20230414225551.858160935@linutronix.de> MIME-Version: 1.0 Date: Sat, 15 Apr 2023 01:44:32 +0200 (CEST) X-ZohoMail-DKIM: pass (identity @linutronix.de) X-ZM-MESSAGEID: 1681515899287100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Spin-waiting on the control CPU until the AP reaches the TSC synchronization is just a waste especially in the case that there is no synchronization required. As the synchronization has to run with interrupts disabled the control CPU part can just be done from a SMP function call. The upcoming AP issues that call async only in the case that synchronization is required. Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/tsc.h | 2 -- arch/x86/kernel/smpboot.c | 20 +++----------------- arch/x86/kernel/tsc_sync.c | 36 +++++++++++------------------------- 3 files changed, 14 insertions(+), 44 deletions(-) --- a/arch/x86/include/asm/tsc.h +++ b/arch/x86/include/asm/tsc.h @@ -55,12 +55,10 @@ extern bool tsc_async_resets; #ifdef CONFIG_X86_TSC extern bool tsc_store_and_check_tsc_adjust(bool bootcpu); extern void tsc_verify_tsc_adjust(bool resume); -extern void check_tsc_sync_source(int cpu); extern void check_tsc_sync_target(void); #else static inline bool tsc_store_and_check_tsc_adjust(bool bootcpu) { return f= alse; } static inline void tsc_verify_tsc_adjust(bool resume) { } -static inline void check_tsc_sync_source(int cpu) { } static inline void check_tsc_sync_target(void) { } #endif =20 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -278,11 +278,7 @@ static void notrace start_secondary(void /* Otherwise gcc will move up smp_processor_id() before cpu_init() */ barrier(); =20 - /* - * Check TSC synchronization with the control CPU, which will do - * its part of this from wait_cpu_online(), making it an implicit - * synchronization point. - */ + /* Check TSC synchronization with the control CPU. */ check_tsc_sync_target(); =20 /* @@ -1144,21 +1140,11 @@ static void wait_cpu_callin(unsigned int } =20 /* - * Bringup step four: Synchronize the TSC and wait for the target AP - * to reach set_cpu_online() in start_secondary(). + * Bringup step four: Wait for the target AP to reach set_cpu_online() in + * start_secondary(). */ static void wait_cpu_online(unsigned int cpu) { - unsigned long flags; - - /* - * Check TSC synchronization with the AP (keep irqs disabled - * while doing so): - */ - local_irq_save(flags); - check_tsc_sync_source(cpu); - local_irq_restore(flags); - /* * Wait for the AP to mark itself online, so the core caller * can drop sparse_irq_lock. --- a/arch/x86/kernel/tsc_sync.c +++ b/arch/x86/kernel/tsc_sync.c @@ -245,7 +245,6 @@ bool tsc_store_and_check_tsc_adjust(bool */ static atomic_t start_count; static atomic_t stop_count; -static atomic_t skip_test; static atomic_t test_runs; =20 /* @@ -344,21 +343,14 @@ static inline unsigned int loop_timeout( } =20 /* - * Source CPU calls into this - it waits for the freshly booted - * target CPU to arrive and then starts the measurement: + * The freshly booted CPU initiates this via an async SMP function call. */ -void check_tsc_sync_source(int cpu) +static void check_tsc_sync_source(void *__cpu) { + unsigned int cpu =3D (unsigned long)__cpu; int cpus =3D 2; =20 /* - * No need to check if we already know that the TSC is not - * synchronized or if we have no TSC. - */ - if (unsynchronized_tsc()) - return; - - /* * Set the maximum number of test runs to * 1 if the CPU does not provide the TSC_ADJUST MSR * 3 if the MSR is available, so the target can try to adjust @@ -368,16 +360,9 @@ void check_tsc_sync_source(int cpu) else atomic_set(&test_runs, 3); retry: - /* - * Wait for the target to start or to skip the test: - */ - while (atomic_read(&start_count) !=3D cpus - 1) { - if (atomic_read(&skip_test) > 0) { - atomic_set(&skip_test, 0); - return; - } + /* Wait for the target to start. */ + while (atomic_read(&start_count) !=3D cpus - 1) cpu_relax(); - } =20 /* * Trigger the target to continue into the measurement too: @@ -397,14 +382,14 @@ void check_tsc_sync_source(int cpu) if (!nr_warps) { atomic_set(&test_runs, 0); =20 - pr_debug("TSC synchronization [CPU#%d -> CPU#%d]: passed\n", + pr_debug("TSC synchronization [CPU#%d -> CPU#%u]: passed\n", smp_processor_id(), cpu); =20 } else if (atomic_dec_and_test(&test_runs) || random_warps) { /* Force it to 0 if random warps brought us here */ atomic_set(&test_runs, 0); =20 - pr_warn("TSC synchronization [CPU#%d -> CPU#%d]:\n", + pr_warn("TSC synchronization [CPU#%d -> CPU#%u]:\n", smp_processor_id(), cpu); pr_warn("Measured %Ld cycles TSC warp between CPUs, " "turning off TSC clock.\n", max_warp); @@ -457,11 +442,12 @@ void check_tsc_sync_target(void) * SoCs the TSC is frequency synchronized, but still the TSC ADJUST * register might have been wreckaged by the BIOS.. */ - if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable) { - atomic_inc(&skip_test); + if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable) return; - } =20 + /* Kick the control CPU into the TSC synchronization function */ + smp_call_function_single(cpumask_first(cpu_online_mask), check_tsc_sync_s= ource, + (unsigned long *)(unsigned long)cpu, 0); retry: /* * Register this CPU's participation and wait for the