From nobody Mon Feb 9 21:28:51 2026 Received: from e3i714.smtp2go.com (e3i714.smtp2go.com [158.120.86.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDA5E2D061B for ; Wed, 4 Feb 2026 16:30:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=158.120.86.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770222641; cv=none; b=Ikn99DgbQBlmBED9jXj/c5V67Cueb8ZMyHf8nzje+c+2gUoyu51b4JqTFE989OahsT7yV38OYbK++dr0TJmBlrnbO1/P3BbUuS29I6/CUn2g6S38NIX3iiF6NoOLX9L76wHSNtelgqa5ixZHUr9zVnNegD6yIRciJVbxyCIkhks= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770222641; c=relaxed/simple; bh=Mi2mTNkUtDcvKyyCCEu5+HRMrcoi1nB+HuF3sBMcF4I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=uVEpBTGWw8vhuDI4Aeh36BP0WC8THXwv81438b7GsyXzpzB+zjYnxAEnxJ4a7GlLu+94r2mI00Bd9HCh/GjjMtrWuyZya/oHMgS5ZBjDJKTB1pvXLu89BYI6RU6lSNwsfTCI/QIa5qr6CY8jAEiC/uxShO/7VyvG94msU4Y/vV4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=paulson-ellis.org; spf=pass smtp.mailfrom=em992990.paulson-ellis.org; dkim=pass (2048-bit key) header.d=paulson-ellis.org header.i=@paulson-ellis.org header.b=gtTaYNW3; arc=none smtp.client-ip=158.120.86.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=paulson-ellis.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=em992990.paulson-ellis.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=paulson-ellis.org header.i=@paulson-ellis.org header.b="gtTaYNW3" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paulson-ellis.org; i=@paulson-ellis.org; q=dns/txt; s=s992990; t=1770221731; h=from : subject : to : message-id : date; bh=RuDrz5E6bZ7YuRiAAtBDHGgz3uFLFzdTomkbqjKrkq4=; b=gtTaYNW3/uPz2OBAzndjx8A7AnggPwgGJ/XSC7HYyHLIiKd5YJ/8sM4CjLc8gdAHM+CIQ JSBCwhlwQWUVFkbOAskmLtZhJQCPE494bZgR47bxBislbeqNv4uQLK6/+VHbZ6caQXIzn66 LT+h2H+jdGRbLY8NV5QEDQbUQdoSaWv05s+ZyUYBiHP3pahC3aKTo+n/tImZDXQw0bA0/X1 GkIrQecbNfU7RWEPkOl375M8Pc3o/Ck0GbdFDHk9MSqjZxb5T/ETiIcpI7MzHzOTdj5hne8 AZzUQUm2un/97Em/U6eJjOocKwLRkKlMvyuRSJKgdSm8A7TD7RIOf3WBMIIA== Received: from [10.133.246.224] (helo=macbookpro.fritz.box) by smtpcorp.com with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.99.1-S2G) (envelope-from ) id 1vnfXG-4o5NDgrvSlP-rqRo; Wed, 04 Feb 2026 16:15:22 +0000 From: Chris Paulson-Ellis Date: Wed, 04 Feb 2026 16:15:18 +0000 Subject: [PATCH] x86/mtrr: Expedite cache init if MTRRs change on resume Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260204-macbook-pro-mtrr-resume-fix-v1-1-25fc4326d06a@paulson-ellis.org> X-B4-Tracking: v=1; b=H4sIAJVwg2kC/x3MQQqDMBBA0avIrDuQjtbaXqW4sMmoQ4kJk1aEk Lsbunx/8TMkVuEEzyaD8i5JwlZxvTRg12lbGMVVAxnqDZkO/WTfIXwwakD/VUXl9POMsxxIrR0 eRLe7GwzUQ1Su+X9/jaWchAqmOG0AAAA= To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org, Juergen Gross , Chris Paulson-Ellis X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1770221721; l=9113; i=chris@paulson-ellis.org; s=20260204; h=from:subject:message-id; bh=Mi2mTNkUtDcvKyyCCEu5+HRMrcoi1nB+HuF3sBMcF4I=; b=MW6T8dUnUN37trglYS9FKTr9WvHa6M/aAZGta4FjYtbeW1k40eX7G/DqYxh5DLb3uxcKEg3oO NMR+SfbIykzDxhjxCkeMdTZaAcVnrzI+TlEVNO4yKICjTKatTxjM0+c X-Developer-Key: i=chris@paulson-ellis.org; a=ed25519; pk=PcJVcGbMcVgdFMzEsrG+yObMbx9eeF4JFd0TrIx9QsE= X-Report-Abuse: Please forward a copy of this message, including all headers, to Feedback-ID: 992990m:992990a9tohFO:992990sdZBql9ULa X-smtpcorp-track: K3OGD9mDSyq9.m20Z-VYYaagU.3ZUriwExJiN An Apple MacBook Pro mid-2015 almost always takes ~20s to resume from suspend, measured from lid open to display of the desktop lock screen. This is due to the MTRRs being different between suspend and resume on 4 out of the 8 CPUs, including the boot CPU. The CPUs execute very slowly until arch_thaw_secondary_cpus_end() calls cache_aps_init(), which restores the MTRR settings to their pre-suspend values. To obtain a reasonable resume time, we need to minimise the time the CPUs execute with inconsistent MTRR settings. We do this by detecting the unexpected restore of the MTRRs on the boot CPU in cache_bp_restore(), and use this to override the delay of cache initialisation in the cache_ap_online() CPU hotplug handler, and skip the delayed cache initialisation in cache_aps_init(). With this fix, the system in question resumes in ~3s. --- As per the commit message, this patch is about making my mid-2015 MacBook Pro resume from suspend in a reasonable amount of time (3s rather that 20s). My whole investigative journey is documented in [1]. There is a lot there that will not be of interest, but some if the details may be, such as the observed MTRR values on resume [2]. The Apple BIOS seems to be buggy with respect to CPU MTRR settings. Here is what I've observed: * The MTRRs are always sane & consistent on boot. * The MTRRs are unchanged by any of the Linux software I run. * After suspending, on resume the boot processor MTRRs have usually been modified [2] - I'm assuming by the BIOS. * The MTRR modifications are the same on every suspend/resume cycle. * Both the fixed & variable MTRRs change. * When Linux brings up the secondary processors on resume, 3 of them with have the same modified MTRRs as the BP, and 4 will have the original MTRRs. There are 8 CPUs in total, so half are OK, and half have changed. * The AP MTRRs that are modified have the same fixed & variable MTRR changes as on the BP. * On the BP only, the default MTTR type is changed from writeback to uncachable. * For a single boot, every suspend/resume cycle will see the same MTRR modifications on the same list of APs. * After rebooting, the list of which specific 3/7 APs have the MTRR modifications may change. * I've never observed modified MTRRs on APs without also observing it on the BP. * On some (rare) boots the problem is not observed at all, regardless of how many suspend/resume cycles are performed. * FWIW I never observed extended resume times running MacOS X for many years. I only switched to running Linux natively when it went EOL. The original MTRRs settings are restored by Linux and the system is fine, but this doesn't happen until after all the CPUs are brought up. While this is happening the CPUs have inconsistent MTRR settings, and the system runs very slowly, causing the long resume times. I have observed variation in the extended resume times [3], which is probaby caused not by which Linux version is running (which is what I was looking for), but by the (apparently random) differences in which specific APs have modified MTRRs, and therefore how long the system runs with inconsistent MTRR settings. My solution to the problem is to detect the MTRR modifications on the BP, and then expedite restoration of the MTRRs on the APs. A downside of this solution is that many more synchronised MTRR checks are performed [4], which could impact systems with many CPUs. I therefore have some questions about my fix: Are there systems were is it normal for the BP to have different MTRR settings between suspend and resume? If so, the new behaviour would be triggered on these systems, and resume times could be affected if they have many CPUs. Should I therefore add some additional condition on the new behaviour? Such as: * Somehow identifying my specific BIOS. * Limiting it to systems with <=3D n CPUs (where n is 8 for my system). * Requiring some runtime config to enable it, such as a boot parameter. [1] https://macbook-pro-resume-delay.pages.dev/ [2] https://macbook-pro-resume-delay.pages.dev/journey#changes-after-resume [3] https://macbook-pro-resume-delay.pages.dev/journey#id3 [4] https://macbook-pro-resume-delay.pages.dev/journey#cost-of-extra-checks Signed-off-by: Chris Paulson-Ellis --- arch/x86/include/asm/mtrr.h | 7 +++++-- arch/x86/kernel/cpu/cacheinfo.c | 33 ++++++++++++++++++++++++++------- arch/x86/kernel/cpu/mtrr/generic.c | 6 +++++- 3 files changed, 36 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index 76b95bd1a405..bc5872e57f3c 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -60,7 +60,7 @@ extern int mtrr_trim_uncached_memory(unsigned long end_pf= n); extern int amd_special_default_mtrr(void); void mtrr_disable(void); void mtrr_enable(void); -void mtrr_generic_set_state(void); +bool mtrr_generic_set_state(void); # else static inline void guest_force_mtrr_state(struct mtrr_var_range *var, unsigned int num_var, @@ -105,7 +105,10 @@ static inline int mtrr_trim_uncached_memory(unsigned l= ong end_pfn) #define mtrr_bp_init() do {} while (0) #define mtrr_disable() do {} while (0) #define mtrr_enable() do {} while (0) -#define mtrr_generic_set_state() do {} while (0) +static inline bool mtrr_generic_set_state(void) +{ + return false; +} # endif =20 #ifdef CONFIG_COMPAT diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index 51a95b07831f..7911beffc339 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -710,15 +710,16 @@ void cache_enable(void) __releases(cache_disable_lock) raw_spin_unlock(&cache_disable_lock); } =20 -static void cache_cpu_init(void) +static bool cache_cpu_init(void) { unsigned long flags; + bool changed =3D false; =20 local_irq_save(flags); =20 if (memory_caching_control & CACHE_MTRR) { cache_disable(); - mtrr_generic_set_state(); + changed =3D mtrr_generic_set_state(); cache_enable(); } =20 @@ -726,6 +727,8 @@ static void cache_cpu_init(void) pat_cpu_init(); =20 local_irq_restore(flags); + + return changed; } =20 static bool cache_aps_delayed_init =3D true; @@ -743,7 +746,7 @@ bool get_cache_aps_delayed_init(void) static int cache_rendezvous_handler(void *unused) { if (get_cache_aps_delayed_init() || !cpu_online(smp_processor_id())) - cache_cpu_init(); + (void)cache_cpu_init(); =20 return 0; } @@ -754,20 +757,31 @@ void __init cache_bp_init(void) pat_bp_init(); =20 if (memory_caching_control) - cache_cpu_init(); + (void)cache_cpu_init(); } =20 +static bool cache_bp_changed_on_restore; + void cache_bp_restore(void) { if (memory_caching_control) - cache_cpu_init(); + cache_bp_changed_on_restore =3D cache_cpu_init(); } =20 static int cache_ap_online(unsigned int cpu) { cpumask_set_cpu(cpu, cpu_cacheinfo_mask); =20 - if (!memory_caching_control || get_cache_aps_delayed_init()) + if (!memory_caching_control) + return 0; + + /* + * Normally we delay MTRR (and PAT) init until cache_aps_init(), but if + * the MTRRs had to be restored on the boot processor on resume, then + * delaying any required MTTR restore on the APs can lead to very slow + * CPU execution during the period when the MTRRs are inconsistent. + */ + if (get_cache_aps_delayed_init() && !cache_bp_changed_on_restore) return 0; =20 /* @@ -803,8 +817,13 @@ void cache_aps_init(void) if (!memory_caching_control || !get_cache_aps_delayed_init()) return; =20 - stop_machine(cache_rendezvous_handler, NULL, cpu_online_mask); + if (cache_bp_changed_on_restore) + pr_warn("mtrr: your CPUs had unexpected MTRR settings on resume\n"); + else + stop_machine(cache_rendezvous_handler, NULL, cpu_online_mask); + set_cache_aps_delayed_init(false); + cache_bp_changed_on_restore =3D false; } =20 static int __init cache_ap_register(void) diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/= generic.c index 0863733858dc..6d96fde84275 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -960,12 +960,14 @@ void mtrr_enable(void) mtrr_wrmsr(MSR_MTRRdefType, deftype_lo, deftype_hi); } =20 -void mtrr_generic_set_state(void) +bool mtrr_generic_set_state(void) { unsigned long mask, count; + bool changed; =20 /* Actually set the state */ mask =3D set_mtrr_state(); + changed =3D mask !=3D 0; =20 /* Use the atomic bitops to update the global mask */ for (count =3D 0; count < sizeof(mask) * 8; ++count) { @@ -973,6 +975,8 @@ void mtrr_generic_set_state(void) set_bit(count, &smp_changes_mask); mask >>=3D 1; } + + return changed; } =20 /** --- base-commit: 18f7fcd5e69a04df57b563360b88be72471d6b62 change-id: 20260204-macbook-pro-mtrr-resume-fix-23c892257d80 Best regards, --=20 Chris Paulson-Ellis