From nobody Mon Feb 9 22:39:11 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987573; cv=none; d=zohomail.com; s=zohoarc; b=XVEP4IgB9MR7hpmWvIng3SEHQE0AErMoZuB+lC4mOYZWRvQckNXWJOJukgr0Av4Ra5zjtRRstwZgiPVDAfooQk3ATz756CAhy4f0u0kyn5LjCtmsg63G9zpY8vK0DFqDJq8f7XsNpFREi0VKG6NyrGg2k3SZNsNUMNXkAz2iJgk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987573; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JUJm8AzApP3BXUfz/8kYW4X5+H9Ehhz9tOTxTyy5q78=; b=EfFEJtruwhJWByjf9vZNz1BbOvsts2FHTcexIhwAKXlw9sKhqVhUn+UGQ3JxxpfdyuvyVVnLjFmWrqwI2EUZENN+By+6M79cFG8l12y1Sw4/MU8Icxe5Qs8TF61p8NWMfoVxa1YbvyGHA7+pASFILJ5xST0UVqcBIh9lVaddzdw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987573111400.6117840870629; Sat, 20 Aug 2022 02:26:13 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390687.628273 (Exim 4.92) (envelope-from ) id 1oPKjj-0000FF-Nw; Sat, 20 Aug 2022 09:25:47 +0000 Received: by outflank-mailman (output) from mailman id 390687.628273; Sat, 20 Aug 2022 09:25:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjj-0000C7-ED; Sat, 20 Aug 2022 09:25:47 +0000 Received: by outflank-mailman (input) for mailman id 390687; Sat, 20 Aug 2022 09:25:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjg-0006NK-KD for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:44 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0e5d84f3-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:39 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AF1872005B; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5933213440; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id KAScFJOoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0e5d84f3-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987539; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JUJm8AzApP3BXUfz/8kYW4X5+H9Ehhz9tOTxTyy5q78=; b=pLwDNZdYw2eOdoTT+7tU43E4Jb9JnNXc6u/4Ne/QAtm6XHwsyV2Fd9Ul8cM8HsqGgbKKbP bJTrtLPgUbNpMM20fcnP4tOowZ4b2qR7nc8zd8Q9h8+omZJH7S28LBFKap+aBkpIK0QCoy G2ypbxWHcgr96ClzCmZzKbr93DnKD24= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , "Rafael J. Wysocki" , Pavel Machek Subject: [PATCH v2 09/10] x86/mtrr: add a stop_machine() handler calling only cache_cpu_init() Date: Sat, 20 Aug 2022 11:25:32 +0200 Message-Id: <20220820092533.29420-10-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987573709100006 Content-Type: text/plain; charset="utf-8" Instead of having a stop_machine() handler for either a specific MTRR register or all state at once, add a handler just for calling cache_cpu_init() if appropriate. Add functions for calling stop_machine() with this handler as well. Add a generic replacements for mtrr_bp_restore() and a wrapper for mtrr_bp_init(). Signed-off-by: Juergen Gross --- V2: - completely new replacement of former patch 2 --- arch/x86/include/asm/cacheinfo.h | 5 +- arch/x86/include/asm/mtrr.h | 4 -- arch/x86/kernel/cpu/cacheinfo.c | 59 +++++++++++++++++++++- arch/x86/kernel/cpu/common.c | 3 +- arch/x86/kernel/cpu/mtrr/mtrr.c | 87 +------------------------------- arch/x86/kernel/setup.c | 3 +- arch/x86/kernel/smpboot.c | 4 +- arch/x86/power/cpu.c | 3 +- 8 files changed, 72 insertions(+), 96 deletions(-) diff --git a/arch/x86/include/asm/cacheinfo.h b/arch/x86/include/asm/cachei= nfo.h index e80ed3c523c8..a122a1aad936 100644 --- a/arch/x86/include/asm/cacheinfo.h +++ b/arch/x86/include/asm/cacheinfo.h @@ -14,6 +14,9 @@ void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, i= nt cpu); =20 void cache_disable(void); void cache_enable(void); -void cache_cpu_init(void); +void cache_bp_init(void); +void cache_bp_restore(void); +void cache_ap_init(void); +void cache_aps_init(void); =20 #endif /* _ASM_X86_CACHEINFO_H */ diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index 5d31219c8529..ec73d1e5bafb 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -42,8 +42,6 @@ extern int mtrr_add_page(unsigned long base, unsigned lon= g size, extern int mtrr_del(int reg, unsigned long base, unsigned long size); extern int mtrr_del_page(int reg, unsigned long base, unsigned long size); extern void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi); -extern void mtrr_ap_init(void); -extern void mtrr_aps_init(void); extern void mtrr_bp_restore(void); extern int mtrr_trim_uncached_memory(unsigned long end_pfn); extern int amd_special_default_mtrr(void); @@ -85,8 +83,6 @@ static inline int mtrr_trim_uncached_memory(unsigned long= end_pfn) static inline void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi) { } -#define mtrr_ap_init() do {} while (0) -#define mtrr_aps_init() do {} while (0) #define mtrr_bp_restore() do {} while (0) #define mtrr_disable() do {} while (0) #define mtrr_enable() do {} while (0) diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index c6e7c93e45e8..4946f93eb16f 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -15,6 +15,7 @@ #include #include #include +#include =20 #include #include @@ -1121,7 +1122,7 @@ void cache_enable(void) __releases(cache_disable_lock) raw_spin_unlock(&cache_disable_lock); } =20 -void cache_cpu_init(void) +static void cache_cpu_init(void) { unsigned long flags; =20 @@ -1141,3 +1142,59 @@ void cache_cpu_init(void) } =20 bool cache_aps_delayed_init; + +static int cache_rendezvous_handler(void *unused) +{ + if (cache_aps_delayed_init || !cpu_online(smp_processor_id())) + cache_cpu_init(); + + return 0; +} + +void __init cache_bp_init(void) +{ + mtrr_bp_init(); + + if (cache_generic) + cache_cpu_init(); +} + +void cache_bp_restore(void) +{ + if (cache_generic) + cache_cpu_init(); +} + +void cache_ap_init(void) +{ + if (!cache_generic || cache_aps_delayed_init) + return; + + /* + * Ideally we should hold mtrr_mutex here to avoid mtrr entries + * changed, but this routine will be called in cpu boot time, + * holding the lock breaks it. + * + * This routine is called in two cases: + * + * 1. very early time of software resume, when there absolutely + * isn't mtrr entry changes; + * + * 2. cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug + * lock to prevent mtrr entry changes + */ + stop_machine_from_inactive_cpu(cache_rendezvous_handler, NULL, + cpu_callout_mask); +} + +/* + * Delayed cache initialization for all AP's + */ +void cache_aps_init(void) +{ + if (!cache_generic || !cache_aps_delayed_init) + return; + + stop_machine(cache_rendezvous_handler, NULL, cpu_online_mask); + cache_aps_delayed_init =3D false; +} diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 64a73f415f03..5c122dd17ec2 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -1936,7 +1937,7 @@ void identify_secondary_cpu(struct cpuinfo_x86 *c) #ifdef CONFIG_X86_32 enable_sep_cpu(); #endif - mtrr_ap_init(); + cache_ap_init(); validate_apic_and_package_id(c); x86_spec_ctrl_setup_ap(); update_srbds_msr(); diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtr= r.c index b68ec43e33a2..38531e021581 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -70,9 +70,6 @@ static const struct mtrr_ops *mtrr_ops[X86_VENDOR_NUM] __= ro_after_init; =20 const struct mtrr_ops *mtrr_if; =20 -static void set_mtrr(unsigned int reg, unsigned long base, - unsigned long size, mtrr_type type); - void __init set_mtrr_ops(const struct mtrr_ops *ops) { if (ops->vendor && ops->vendor < X86_VENDOR_NUM) @@ -155,25 +152,8 @@ static int mtrr_rendezvous_handler(void *info) { struct set_mtrr_data *data =3D info; =20 - /* - * We use this same function to initialize the mtrrs during boot, - * resume, runtime cpu online and on an explicit request to set a - * specific MTRR. - * - * During boot or suspend, the state of the boot cpu's mtrrs has been - * saved, and we want to replicate that across all the cpus that come - * online (either at the end of boot or resume or during a runtime cpu - * online). If we're doing that, @reg is set to something special and on - * all the cpu's we do cache_cpu_init() (On the logical cpu that - * started the boot/resume sequence, this might be a duplicate - * cache_cpu_init()). - */ - if (data->smp_reg !=3D ~0U) { - mtrr_if->set(data->smp_reg, data->smp_base, - data->smp_size, data->smp_type); - } else if (cache_aps_delayed_init || !cpu_online(smp_processor_id())) { - cache_cpu_init(); - } + mtrr_if->set(data->smp_reg, data->smp_base, + data->smp_size, data->smp_type); return 0; } =20 @@ -243,19 +223,6 @@ static void set_mtrr_cpuslocked(unsigned int reg, unsi= gned long base, stop_machine_cpuslocked(mtrr_rendezvous_handler, &data, cpu_online_mask); } =20 -static void set_mtrr_from_inactive_cpu(unsigned int reg, unsigned long bas= e, - unsigned long size, mtrr_type type) -{ - struct set_mtrr_data data =3D { .smp_reg =3D reg, - .smp_base =3D base, - .smp_size =3D size, - .smp_type =3D type - }; - - stop_machine_from_inactive_cpu(mtrr_rendezvous_handler, &data, - cpu_callout_mask); -} - /** * mtrr_add_page - Add a memory type region * @base: Physical base address of region in pages (in units of 4 kB!) @@ -764,7 +731,6 @@ void __init mtrr_bp_init(void) CACHE_GENERIC_PAT; changed_by_mtrr_cleanup =3D mtrr_cleanup(phys_addr); - cache_cpu_init(); } } } @@ -781,27 +747,6 @@ void __init mtrr_bp_init(void) } } =20 -void mtrr_ap_init(void) -{ - if (!cache_generic || cache_aps_delayed_init) - return; - - /* - * Ideally we should hold mtrr_mutex here to avoid mtrr entries - * changed, but this routine will be called in cpu boot time, - * holding the lock breaks it. - * - * This routine is called in two cases: - * - * 1. very early time of software resume, when there absolutely - * isn't mtrr entry changes; - * - * 2. cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug - * lock to prevent mtrr entry changes - */ - set_mtrr_from_inactive_cpu(~0U, 0, 0, 0); -} - /** * mtrr_save_state - Save current fixed-range MTRR state of the first * cpu in cpu_online_mask. @@ -817,34 +762,6 @@ void mtrr_save_state(void) smp_call_function_single(first_cpu, mtrr_save_fixed_ranges, NULL, 1); } =20 -/* - * Delayed MTRR initialization for all AP's - */ -void mtrr_aps_init(void) -{ - if (!cache_generic) - return; - - /* - * Check if someone has requested the delay of AP MTRR initialization, - * by doing set_mtrr_aps_delayed_init(), prior to this point. If not, - * then we are done. - */ - if (!cache_aps_delayed_init) - return; - - set_mtrr(~0U, 0, 0, 0); - cache_aps_delayed_init =3D false; -} - -void mtrr_bp_restore(void) -{ - if (!cache_generic) - return; - - cache_cpu_init(); -} - static int __init mtrr_init_finialize(void) { if (!mtrr_enabled) diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 216fee7144ee..e0e185ee0229 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -1075,7 +1076,7 @@ void __init setup_arch(char **cmdline_p) =20 /* update e820 for memory not covered by WB MTRRs */ if (IS_ENABLED(CONFIG_MTRR)) - mtrr_bp_init(); + cache_bp_init(); else pat_disable("PAT support disabled because CONFIG_MTRR is disabled in the= kernel."); =20 diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index ef7bce21cbe8..ff793f436904 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1445,7 +1445,7 @@ void arch_thaw_secondary_cpus_begin(void) =20 void arch_thaw_secondary_cpus_end(void) { - mtrr_aps_init(); + cache_aps_init(); } =20 /* @@ -1488,7 +1488,7 @@ void __init native_smp_cpus_done(unsigned int max_cpu= s) =20 nmi_selftest(); impress_friends(); - mtrr_aps_init(); + cache_aps_init(); } =20 static int __initdata setup_possible_cpus =3D -1; diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c index bb176c72891c..754221c9a1c3 100644 --- a/arch/x86/power/cpu.c +++ b/arch/x86/power/cpu.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -261,7 +262,7 @@ static void notrace __restore_processor_state(struct sa= ved_context *ctxt) do_fpu_end(); tsc_verify_tsc_adjust(true); x86_platform.restore_sched_clock_state(); - mtrr_bp_restore(); + cache_bp_restore(); perf_restore_debug_store(); =20 c =3D &cpu_data(smp_processor_id()); --=20 2.35.3