From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987565; cv=none; d=zohomail.com; s=zohoarc; b=eNgSGPHSFtzTVCiQdu4dxIfZ7Lfa0OgBOhqq6Z+UtxUu9vbKz3jXRe+IOtFRZ10jI8Y90rbz/mTuXnH/znv8A7/p6K8I8XO79UwmTyj3OAry4LovkTHAGZD0OG+xhPdZtF7UGGfOMsCzeVoFs4UHw60+7tfzD2PfKifb23esdaQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987565; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dNonIY2eSN2qALXA1xwqca8n1uiC2OZmHsAoG2ku6S4=; b=P70OxdJDmaOjewLaHDEk4zK/paOg6g5P88kbJWoO0v/KJekf5QPy4wSs9ND0mxSbfUuJ88ST4t3ZEgpwK/KH99MluraP+PZ5RQf5Z6YtoCCrF9KXR3NtYrbvaBDyS5jrX5yCsaevrPFuePr5pMaD3YMCoXjj/gvmshao4haj8PU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987565608559.7371938940984; Sat, 20 Aug 2022 02:26:05 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390681.628201 (Exim 4.92) (envelope-from ) id 1oPKjd-0006p5-L2; Sat, 20 Aug 2022 09:25:41 +0000 Received: by outflank-mailman (output) from mailman id 390681.628201; Sat, 20 Aug 2022 09:25:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjd-0006oJ-FJ; Sat, 20 Aug 2022 09:25:41 +0000 Received: by outflank-mailman (input) for mailman id 390681; Sat, 20 Aug 2022 09:25:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjc-0006NK-Aq for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:40 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0cdff878-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:37 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 25C021FA75; Sat, 20 Aug 2022 09:25:37 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D0E1B13440; Sat, 20 Aug 2022 09:25:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6DubMZCoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0cdff878-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987537; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dNonIY2eSN2qALXA1xwqca8n1uiC2OZmHsAoG2ku6S4=; b=V/DSldnei2N0b/QjoZVHWD91fOusf/NBo0/8KqkWOsov44+Zf25qNDINpw+6cJHaFuWV2n sIcrQI04yUJyzSgBFXyN9sl+qQyIuu1oi2bh6+M/3sIkYTYNpNHAkgh4E1QpuICJVDdhgt BKvehrmm4SOtEce3Ucp+OllxKvMFKgI= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , stable@vger.kernel.org Subject: [PATCH v2 01/10] x86/mtrr: fix MTRR fixup on APs Date: Sat, 20 Aug 2022 11:25:24 +0200 Message-Id: <20220820092533.29420-2-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987568026100001 Content-Type: text/plain; charset="utf-8" When booting or resuming the system MTRR state is saved on the boot processor and then this state is loaded into MTRRs of all other cpus. During update of the MTRRs the MTRR mechanism needs to be disabled by writing the related MSR. The old contents of this MSR are saved in a set of static variables and later those static variables are used to restore the MSR. In case the MSR contents need to be modified on a cpu due to the MSR not having been initialized properly by the BIOS, the related update function is modifying the static variables accordingly. Unfortunately the MTRR state update is usually running on all cpus at the same time, so using just one set of static variables for all cpus is racy in case the MSR contents differ across cpus. Fix that by using percpu variables for saving the MSR contents. Cc: stable@vger.kernel.org Signed-off-by: Juergen Gross --- I thought adding a "Fixes:" tag for the kernel's initial git commit would maybe be entertaining, but without being really helpful. The percpu variables were preferred over on-stack ones in order to avoid more code churn in followup patches decoupling PAT from MTRR support. V2: - new patch --- arch/x86/kernel/cpu/mtrr/generic.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/= generic.c index 558108296f3c..3d185fcf08ca 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -679,7 +679,8 @@ static bool set_mtrr_var_ranges(unsigned int index, str= uct mtrr_var_range *vr) return changed; } =20 -static u32 deftype_lo, deftype_hi; +static DEFINE_PER_CPU(u32, deftype_lo); +static DEFINE_PER_CPU(u32, deftype_hi); =20 /** * set_mtrr_state - Set the MTRR state for this CPU. @@ -691,6 +692,7 @@ static unsigned long set_mtrr_state(void) { unsigned long change_mask =3D 0; unsigned int i; + u32 *lo =3D this_cpu_ptr(&deftype_lo); =20 for (i =3D 0; i < num_var_ranges; i++) { if (set_mtrr_var_ranges(i, &mtrr_state.var_ranges[i])) @@ -704,10 +706,10 @@ static unsigned long set_mtrr_state(void) * Set_mtrr_restore restores the old value of MTRRdefType, * so to set it we fiddle with the saved value: */ - if ((deftype_lo & 0xff) !=3D mtrr_state.def_type - || ((deftype_lo & 0xc00) >> 10) !=3D mtrr_state.enabled) { + if ((*lo & 0xff) !=3D mtrr_state.def_type + || ((*lo & 0xc00) >> 10) !=3D mtrr_state.enabled) { =20 - deftype_lo =3D (deftype_lo & ~0xcff) | mtrr_state.def_type | + *lo =3D (*lo & ~0xcff) | mtrr_state.def_type | (mtrr_state.enabled << 10); change_mask |=3D MTRR_CHANGE_MASK_DEFTYPE; } @@ -729,6 +731,8 @@ static DEFINE_RAW_SPINLOCK(set_atomicity_lock); static void prepare_set(void) __acquires(set_atomicity_lock) { unsigned long cr0; + u32 *lo =3D this_cpu_ptr(&deftype_lo); + u32 *hi =3D this_cpu_ptr(&deftype_hi); =20 /* * Note that this is not ideal @@ -763,10 +767,10 @@ static void prepare_set(void) __acquires(set_atomicit= y_lock) flush_tlb_local(); =20 /* Save MTRR state */ - rdmsr(MSR_MTRRdefType, deftype_lo, deftype_hi); + rdmsr(MSR_MTRRdefType, *lo, *hi); =20 /* Disable MTRRs, and set the default type to uncached */ - mtrr_wrmsr(MSR_MTRRdefType, deftype_lo & ~0xcff, deftype_hi); + mtrr_wrmsr(MSR_MTRRdefType, *lo & ~0xcff, *hi); =20 /* Again, only flush caches if we have to. */ if (!static_cpu_has(X86_FEATURE_SELFSNOOP)) @@ -775,12 +779,15 @@ static void prepare_set(void) __acquires(set_atomicit= y_lock) =20 static void post_set(void) __releases(set_atomicity_lock) { + u32 *lo =3D this_cpu_ptr(&deftype_lo); + u32 *hi =3D this_cpu_ptr(&deftype_hi); + /* Flush TLBs (no need to flush caches - they are disabled) */ count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL); flush_tlb_local(); =20 /* Intel (P6) standard MTRRs */ - mtrr_wrmsr(MSR_MTRRdefType, deftype_lo, deftype_hi); + mtrr_wrmsr(MSR_MTRRdefType, *lo, *hi); =20 /* Enable caches */ write_cr0(read_cr0() & ~X86_CR0_CD); --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987571; cv=none; d=zohomail.com; s=zohoarc; b=MOYZKuj/iAyac/yskXwHRyixkD+hPtR9Jm4i4xH57QOprH4dHp10+GUaOlvgabpu0OibD/S3hYRCRkIPJQamicyiEnq1Ys/RLa9i6znzLgycrJ+f+ET0a3c10T58kW4JOyXKlha5kjQ4gZfb4FQ0ui6JxaASquulk2zxPnQ2ntc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987571; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0gjmfenIvywBVf50iHOvJGRTYSPveQrXTU08CzyNoCY=; b=M57mSQo3qj8l6JKNEgu6/BxTPB1zEvmlawU8zAFZxy4KthM6Yh7TmVGcCOBoG/v2C8CWrho85Taofz9rtvR7IiiMR18uqnbTzym1FQzSDrSs+WDEJvfgRtjjgGu7MjqDShUzkr7NEJmkLuu/PgsbvbtL3JZBdcx3SqWZEkbc2m8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987571298418.8935699335814; Sat, 20 Aug 2022 02:26:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390680.628195 (Exim 4.92) (envelope-from ) id 1oPKjd-0006hn-9D; Sat, 20 Aug 2022 09:25:41 +0000 Received: by outflank-mailman (output) from mailman id 390680.628195; Sat, 20 Aug 2022 09:25:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjd-0006fB-3P; Sat, 20 Aug 2022 09:25:41 +0000 Received: by outflank-mailman (input) for mailman id 390680; Sat, 20 Aug 2022 09:25:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjb-0006NK-SS for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:39 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0d049294-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:37 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 74AEB1FAB0; Sat, 20 Aug 2022 09:25:37 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2EA7513440; Sat, 20 Aug 2022 09:25:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id oKf+CZGoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0d049294-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987537; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0gjmfenIvywBVf50iHOvJGRTYSPveQrXTU08CzyNoCY=; b=BqdX/14cAakxRQyKAhO65J4h9xsR0T7dbg6/jEYZjEbmtxjh0iAKszwYrhWIOXn9Z6MxvT xoYP+siT0ssDMrA2jTHAKGCC3fTwN9qvGW+BeiP/B/kIgLR+Mh0DfmcuIh/VU6Xe79Amxv ERgSGeKIrH4w+QpK9axgoFZsg+7FNwo= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v2 02/10] x86/mtrr: remove unused cyrix_set_all() function Date: Sat, 20 Aug 2022 11:25:25 +0200 Message-Id: <20220820092533.29420-3-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987571645100001 Content-Type: text/plain; charset="utf-8" The Cyrix cpu specific MTRR function cyrix_set_all() will never be called, as the struct mtrr_ops set_all() callback will only be called in the use_intel() case, which would require the use_intel_if member of struct mtrr_ops to be set, which isn't the case for Cyrix. Signed-off-by: Juergen Gross --- V2: - new patch --- arch/x86/kernel/cpu/mtrr/cyrix.c | 34 -------------------------------- 1 file changed, 34 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/cyrix.c b/arch/x86/kernel/cpu/mtrr/cy= rix.c index ca670919b561..c77d3b0a5bf2 100644 --- a/arch/x86/kernel/cpu/mtrr/cyrix.c +++ b/arch/x86/kernel/cpu/mtrr/cyrix.c @@ -234,42 +234,8 @@ static void cyrix_set_arr(unsigned int reg, unsigned l= ong base, post_set(); } =20 -typedef struct { - unsigned long base; - unsigned long size; - mtrr_type type; -} arr_state_t; - -static arr_state_t arr_state[8] =3D { - {0UL, 0UL, 0UL}, {0UL, 0UL, 0UL}, {0UL, 0UL, 0UL}, {0UL, 0UL, 0UL}, - {0UL, 0UL, 0UL}, {0UL, 0UL, 0UL}, {0UL, 0UL, 0UL}, {0UL, 0UL, 0UL} -}; - -static unsigned char ccr_state[7] =3D { 0, 0, 0, 0, 0, 0, 0 }; - -static void cyrix_set_all(void) -{ - int i; - - prepare_set(); - - /* the CCRs are not contiguous */ - for (i =3D 0; i < 4; i++) - setCx86(CX86_CCR0 + i, ccr_state[i]); - for (; i < 7; i++) - setCx86(CX86_CCR4 + i, ccr_state[i]); - - for (i =3D 0; i < 8; i++) { - cyrix_set_arr(i, arr_state[i].base, - arr_state[i].size, arr_state[i].type); - } - - post_set(); -} - static const struct mtrr_ops cyrix_mtrr_ops =3D { .vendor =3D X86_VENDOR_CYRIX, - .set_all =3D cyrix_set_all, .set =3D cyrix_set_arr, .get =3D cyrix_get_arr, .get_free_region =3D cyrix_get_free_region, --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987574; cv=none; d=zohomail.com; s=zohoarc; b=VqKOa/nQbTj7Frt+P6kTbRUMaPMBlCsecs5w63U/Egeyi6uNtaio6il3iKj1BCeIaUsV51JvMcxvKOyRK904J6F7zD1CTnuV2Dd4WT2PrNEqvWmnwVR6k8a6zex6m8DTZ53sfKy51SF0DKTvhOzI41nXtYJEKW/KZXJAsyec6j0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987574; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zH7OrEbGSHDEOBqN5ycZpM6fL4oud1epAjDEuL53dko=; b=Ucm2uRn41EiDJ5t2TUV66Zs25XLKJ3MIBDt6XdYeTroIooVwDBEHGQWW7jUxx3rUIrWSCyATb4k7NfibsH0+wM2EsbmBC2UE1fpeLY5+dGEvicBtbnga/HKfArmsVT/tpLj4TJAY0G16IhjwivjokNJCO2fFFzcTQaAWZeMm5Bs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987574271248.05700187577133; Sat, 20 Aug 2022 02:26:14 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390679.628189 (Exim 4.92) (envelope-from ) id 1oPKjc-0006dC-W9; Sat, 20 Aug 2022 09:25:40 +0000 Received: by outflank-mailman (output) from mailman id 390679.628189; Sat, 20 Aug 2022 09:25:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjc-0006d2-RL; Sat, 20 Aug 2022 09:25:40 +0000 Received: by outflank-mailman (input) for mailman id 390679; Sat, 20 Aug 2022 09:25:39 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjb-0006NL-MY for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:39 +0000 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 0d31c4db-206a-11ed-bd2e-47488cf2e6aa; Sat, 20 Aug 2022 11:25:38 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id BDE3234355; Sat, 20 Aug 2022 09:25:37 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7F1DF13440; Sat, 20 Aug 2022 09:25:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mMnGHZGoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0d31c4db-206a-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987537; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zH7OrEbGSHDEOBqN5ycZpM6fL4oud1epAjDEuL53dko=; b=uGhdI1IZGR+h8vzojT8JBtc7Y8Sk+yQ8KmTZ+ftXj6xsBac4HIS8dsBha3w9fmzifD4ZjE rcw5kNiNK9kq8REjgLzYRO/hCeVKHLcPlohrxNJHV4ikrUROnhEW/I6B5KGObt+tTnnyFX VfA79XO2ZJQ+P71Og/D4kNvZgwJ+0yk= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v2 03/10] x86/mtrr: replace use_intel() with a local flag Date: Sat, 20 Aug 2022 11:25:26 +0200 Message-Id: <20220820092533.29420-4-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987575815100017 Content-Type: text/plain; charset="utf-8" In MTRR code use_intel() is only used in one source file, and the relevant use_intel_if member of struct mtrr_ops is set only in generic_mtrr_ops. Replace use_intel() with a single flag in cacheinfo.c, which can be set when assigning generic_mtrr_ops to mtrr_if. This allows to drop use_intel_if from mtrr_ops, while preparing to support PAT without MTRR. As another preparation for the PAT/MTRR decoupling use a bit for MTRR control and one for PAT control. For now set both bits together, this can be changed later. As the new flag will be set only if mtrr_enabled is set, the test for mtrr_enabled can be dropped at some places. At the same time drop the local mtrr_enabled() function and rename the __mtrr_enabled flag to mtrr_enabled. Signed-off-by: Juergen Gross --- V2: - new patch --- arch/x86/include/asm/cacheinfo.h | 5 +++ arch/x86/kernel/cpu/cacheinfo.c | 3 ++ arch/x86/kernel/cpu/mtrr/generic.c | 1 - arch/x86/kernel/cpu/mtrr/mtrr.c | 58 ++++++++++++++---------------- arch/x86/kernel/cpu/mtrr/mtrr.h | 2 -- 5 files changed, 35 insertions(+), 34 deletions(-) diff --git a/arch/x86/include/asm/cacheinfo.h b/arch/x86/include/asm/cachei= nfo.h index 86b2e0dcc4bf..1aeafa9888f7 100644 --- a/arch/x86/include/asm/cacheinfo.h +++ b/arch/x86/include/asm/cacheinfo.h @@ -2,6 +2,11 @@ #ifndef _ASM_X86_CACHEINFO_H #define _ASM_X86_CACHEINFO_H =20 +/* Kernel controls MTRR and/or PAT MSRs. */ +extern unsigned int cache_generic; +#define CACHE_GENERIC_MTRR 0x01 +#define CACHE_GENERIC_PAT 0x02 + void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, int cpu); void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, int cpu); =20 diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index 66556833d7af..3b05d3ade7a6 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -35,6 +35,9 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_= map); /* Shared L2 cache maps */ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map); =20 +/* Kernel controls MTRR and/or PAT MSRs. */ +unsigned int cache_generic; + struct _cache_table { unsigned char descriptor; char cache_type; diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/= generic.c index 3d185fcf08ca..3c3c7919de5b 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -921,7 +921,6 @@ int positive_have_wrcomb(void) * Generic structure... */ const struct mtrr_ops generic_mtrr_ops =3D { - .use_intel_if =3D 1, .set_all =3D generic_set_all, .get =3D generic_get_mtrr, .get_free_region =3D generic_get_free_region, diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtr= r.c index 2746cac9d8a9..4cd9c8d1473e 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -46,6 +46,7 @@ #include #include =20 +#include #include #include #include @@ -58,12 +59,7 @@ #define MTRR_TO_PHYS_WC_OFFSET 1000 =20 u32 num_var_ranges; -static bool __mtrr_enabled; - -static bool mtrr_enabled(void) -{ - return __mtrr_enabled; -} +static bool mtrr_enabled; =20 unsigned int mtrr_usage_table[MTRR_MAX_VAR_RANGES]; static DEFINE_MUTEX(mtrr_mutex); @@ -119,11 +115,11 @@ static int have_wrcomb(void) } =20 /* This function returns the number of variable MTRRs */ -static void __init set_num_var_ranges(void) +static void __init set_num_var_ranges(bool use_generic) { unsigned long config =3D 0, dummy; =20 - if (use_intel()) + if (use_generic) rdmsr(MSR_MTRRcap, config, dummy); else if (is_cpu(AMD) || is_cpu(HYGON)) config =3D 2; @@ -303,7 +299,7 @@ int mtrr_add_page(unsigned long base, unsigned long siz= e, int i, replace, error; mtrr_type ltype; =20 - if (!mtrr_enabled()) + if (!mtrr_enabled) return -ENXIO; =20 error =3D mtrr_if->validate_add_page(base, size, type); @@ -451,7 +447,7 @@ static int mtrr_check(unsigned long base, unsigned long= size) int mtrr_add(unsigned long base, unsigned long size, unsigned int type, bool increment) { - if (!mtrr_enabled()) + if (!mtrr_enabled) return -ENODEV; if (mtrr_check(base, size)) return -EINVAL; @@ -480,7 +476,7 @@ int mtrr_del_page(int reg, unsigned long base, unsigned= long size) unsigned long lbase, lsize; int error =3D -EINVAL; =20 - if (!mtrr_enabled()) + if (!mtrr_enabled) return -ENODEV; =20 max =3D num_var_ranges; @@ -540,7 +536,7 @@ int mtrr_del_page(int reg, unsigned long base, unsigned= long size) */ int mtrr_del(int reg, unsigned long base, unsigned long size) { - if (!mtrr_enabled()) + if (!mtrr_enabled) return -ENODEV; if (mtrr_check(base, size)) return -EINVAL; @@ -566,7 +562,7 @@ int arch_phys_wc_add(unsigned long base, unsigned long = size) { int ret; =20 - if (pat_enabled() || !mtrr_enabled()) + if (pat_enabled() || !mtrr_enabled) return 0; /* Success! (We don't need to do anything.) */ =20 ret =3D mtrr_add(base, size, MTRR_TYPE_WRCOMB, true); @@ -687,6 +683,7 @@ int __initdata changed_by_mtrr_cleanup; void __init mtrr_bp_init(void) { u32 phys_addr; + bool use_generic =3D false; =20 init_ifs(); =20 @@ -694,6 +691,7 @@ void __init mtrr_bp_init(void) =20 if (boot_cpu_has(X86_FEATURE_MTRR)) { mtrr_if =3D &generic_mtrr_ops; + use_generic =3D true; size_or_mask =3D SIZE_OR_MASK_BITS(36); size_and_mask =3D 0x00f00000; phys_addr =3D 36; @@ -755,15 +753,18 @@ void __init mtrr_bp_init(void) } =20 if (mtrr_if) { - __mtrr_enabled =3D true; - set_num_var_ranges(); + mtrr_enabled =3D true; + set_num_var_ranges(use_generic); init_table(); - if (use_intel()) { + if (use_generic) { /* BIOS may override */ - __mtrr_enabled =3D get_mtrr_state(); + mtrr_enabled =3D get_mtrr_state(); =20 - if (mtrr_enabled()) + if (mtrr_enabled) { mtrr_bp_pat_init(); + cache_generic |=3D CACHE_GENERIC_MTRR | + CACHE_GENERIC_PAT; + } =20 if (mtrr_cleanup(phys_addr)) { changed_by_mtrr_cleanup =3D 1; @@ -772,7 +773,7 @@ void __init mtrr_bp_init(void) } } =20 - if (!mtrr_enabled()) { + if (!mtrr_enabled) { pr_info("Disabled\n"); =20 /* @@ -786,10 +787,7 @@ void __init mtrr_bp_init(void) =20 void mtrr_ap_init(void) { - if (!mtrr_enabled()) - return; - - if (!use_intel() || mtrr_aps_delayed_init) + if (!cache_generic || mtrr_aps_delayed_init) return; =20 /* @@ -816,7 +814,7 @@ void mtrr_save_state(void) { int first_cpu; =20 - if (!mtrr_enabled()) + if (!mtrr_enabled) return; =20 first_cpu =3D cpumask_first(cpu_online_mask); @@ -825,9 +823,7 @@ void mtrr_save_state(void) =20 void set_mtrr_aps_delayed_init(void) { - if (!mtrr_enabled()) - return; - if (!use_intel()) + if (!cache_generic) return; =20 mtrr_aps_delayed_init =3D true; @@ -838,7 +834,7 @@ void set_mtrr_aps_delayed_init(void) */ void mtrr_aps_init(void) { - if (!use_intel() || !mtrr_enabled()) + if (!cache_generic) return; =20 /* @@ -855,7 +851,7 @@ void mtrr_aps_init(void) =20 void mtrr_bp_restore(void) { - if (!use_intel() || !mtrr_enabled()) + if (!cache_generic) return; =20 mtrr_if->set_all(); @@ -863,10 +859,10 @@ void mtrr_bp_restore(void) =20 static int __init mtrr_init_finialize(void) { - if (!mtrr_enabled()) + if (!mtrr_enabled) return 0; =20 - if (use_intel()) { + if (cache_generic & CACHE_GENERIC_MTRR) { if (!changed_by_mtrr_cleanup) mtrr_state_warn(); return 0; diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtr= r.h index 2ac99e561181..88b1c4b6174a 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.h +++ b/arch/x86/kernel/cpu/mtrr/mtrr.h @@ -14,7 +14,6 @@ extern unsigned int mtrr_usage_table[MTRR_MAX_VAR_RANGES]; =20 struct mtrr_ops { u32 vendor; - u32 use_intel_if; void (*set)(unsigned int reg, unsigned long base, unsigned long size, mtrr_type type); void (*set_all)(void); @@ -61,7 +60,6 @@ extern u64 size_or_mask, size_and_mask; extern const struct mtrr_ops *mtrr_if; =20 #define is_cpu(vnd) (mtrr_if && mtrr_if->vendor =3D=3D X86_VENDOR_##vnd) -#define use_intel() (mtrr_if && mtrr_if->use_intel_if =3D=3D 1) =20 extern unsigned int num_var_ranges; extern u64 mtrr_tom2; --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987571; cv=none; d=zohomail.com; s=zohoarc; b=EAV5bj7JGORA6uD5xMiUrbElpsGDfUO5CeFnbhhlsJpHd1PcxRiVqe9tYvLnDdveCOZEhOpUUUc33ROp9JxeCIAzji3CJCe1qlvz5EjbtIpZlHvuUrAgUJiGKwgLx1vW1bfGpdCOV2Lt/qQd23uC+mc9Grhfq8WbEAYuoqbD6RE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987571; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JDumrHgKrBNPtMTfhWyZ4csCcjK0/8ieooTma4S3Dm8=; b=kRlTLgvD8+mpS6QVv8JEyNDAspe3xf0YCTSZhkpS3dXjxgZncLxhaRmbA8drUZGnr85LExQZrh3N/nQVU6m2mF5dyAskv7iCn1DAe3VV3nkEhFadE+k0FBSgV+jTlinHltPrOdOINxSQWopVyuK50iabamWkwInSVkk+rsqmy7U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987571918856.221711766933; Sat, 20 Aug 2022 02:26:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390683.628228 (Exim 4.92) (envelope-from ) id 1oPKjf-0007R2-C9; Sat, 20 Aug 2022 09:25:43 +0000 Received: by outflank-mailman (output) from mailman id 390683.628228; Sat, 20 Aug 2022 09:25:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjf-0007Q2-4W; Sat, 20 Aug 2022 09:25:43 +0000 Received: by outflank-mailman (input) for mailman id 390683; Sat, 20 Aug 2022 09:25:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjc-0006NK-Pi for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:40 +0000 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0d6a01b0-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:38 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 206943435A; Sat, 20 Aug 2022 09:25:38 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C94FE13440; Sat, 20 Aug 2022 09:25:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 8G7BL5GoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0d6a01b0-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987538; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JDumrHgKrBNPtMTfhWyZ4csCcjK0/8ieooTma4S3Dm8=; b=svZxP+pm6baKT31kYNqJaURbLddSo6zE8EiJggJc9DT217A0hg390iq3f4fncdBUcVjmeD jA9kPvyw+AeFUXDl4ip2lT+++L70em15XET/aD6NthQMWxb6VQGxPoH0qTsjd1wix0sK+s Yx+szNaUm2d2QxYpP2p6nI5l9fCYuh0= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v2 04/10] x86: move some code out of arch/x86/kernel/cpu/mtrr Date: Sat, 20 Aug 2022 11:25:27 +0200 Message-Id: <20220820092533.29420-5-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987573717100007 Content-Type: text/plain; charset="utf-8" Prepare making PAT and MTRR support independent from each other by moving some code needed by both out of the MTRR specific sources. Signed-off-by: Juergen Gross --- V2: - move code from cpu/common.c to cpu/cacheinfo.c (Boris Petkov) --- arch/x86/include/asm/cacheinfo.h | 3 ++ arch/x86/include/asm/mtrr.h | 4 ++ arch/x86/kernel/cpu/cacheinfo.c | 77 ++++++++++++++++++++++++++++ arch/x86/kernel/cpu/mtrr/generic.c | 80 ++++-------------------------- 4 files changed, 93 insertions(+), 71 deletions(-) diff --git a/arch/x86/include/asm/cacheinfo.h b/arch/x86/include/asm/cachei= nfo.h index 1aeafa9888f7..313a6920d0f9 100644 --- a/arch/x86/include/asm/cacheinfo.h +++ b/arch/x86/include/asm/cacheinfo.h @@ -10,4 +10,7 @@ extern unsigned int cache_generic; void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, int cpu); void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, int cpu); =20 +void cache_disable(void); +void cache_enable(void); + #endif /* _ASM_X86_CACHEINFO_H */ diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index 76d726074c16..12a16caed395 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -48,6 +48,8 @@ extern void mtrr_aps_init(void); extern void mtrr_bp_restore(void); extern int mtrr_trim_uncached_memory(unsigned long end_pfn); extern int amd_special_default_mtrr(void); +void mtrr_disable(void); +void mtrr_enable(void); # else static inline u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform) { @@ -87,6 +89,8 @@ static inline void mtrr_centaur_report_mcr(int mcr, u32 l= o, u32 hi) #define set_mtrr_aps_delayed_init() do {} while (0) #define mtrr_aps_init() do {} while (0) #define mtrr_bp_restore() do {} while (0) +#define mtrr_disable() do {} while (0) +#define mtrr_enable() do {} while (0) # endif =20 #ifdef CONFIG_COMPAT diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index 3b05d3ade7a6..47e2c72fa8a4 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -20,6 +20,8 @@ #include #include #include +#include +#include =20 #include "cpu.h" =20 @@ -1043,3 +1045,78 @@ int populate_cache_leaves(unsigned int cpu) =20 return 0; } + +/* + * Disable and enable caches. Needed for changing MTRRs and the PAT MSR. + * + * Since we are disabling the cache don't allow any interrupts, + * they would run extremely slow and would only increase the pain. + * + * The caller must ensure that local interrupts are disabled and + * are reenabled after cache_enable() has been called. + */ +static unsigned long saved_cr4; +static DEFINE_RAW_SPINLOCK(cache_disable_lock); + +void cache_disable(void) __acquires(cache_disable_lock) +{ + unsigned long cr0; + + /* + * Note that this is not ideal + * since the cache is only flushed/disabled for this CPU while the + * MTRRs are changed, but changing this requires more invasive + * changes to the way the kernel boots + */ + + raw_spin_lock(&cache_disable_lock); + + /* Enter the no-fill (CD=3D1, NW=3D0) cache mode and flush caches. */ + cr0 =3D read_cr0() | X86_CR0_CD; + write_cr0(cr0); + + /* + * Cache flushing is the most time-consuming step when programming + * the MTRRs. Fortunately, as per the Intel Software Development + * Manual, we can skip it if the processor supports cache self- + * snooping. + */ + if (!static_cpu_has(X86_FEATURE_SELFSNOOP)) + wbinvd(); + + /* Save value of CR4 and clear Page Global Enable (bit 7) */ + if (boot_cpu_has(X86_FEATURE_PGE)) { + saved_cr4 =3D __read_cr4(); + __write_cr4(saved_cr4 & ~X86_CR4_PGE); + } + + /* Flush all TLBs via a mov %cr3, %reg; mov %reg, %cr3 */ + count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL); + flush_tlb_local(); + + if (boot_cpu_has(X86_FEATURE_MTRR)) + mtrr_disable(); + + /* Again, only flush caches if we have to. */ + if (!static_cpu_has(X86_FEATURE_SELFSNOOP)) + wbinvd(); +} + +void cache_enable(void) __releases(cache_disable_lock) +{ + /* Flush TLBs (no need to flush caches - they are disabled) */ + count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL); + flush_tlb_local(); + + if (boot_cpu_has(X86_FEATURE_MTRR)) + mtrr_enable(); + + /* Enable caches */ + write_cr0(read_cr0() & ~X86_CR0_CD); + + /* Restore value of CR4 */ + if (boot_cpu_has(X86_FEATURE_PGE)) + __write_cr4(saved_cr4); + + raw_spin_unlock(&cache_disable_lock); +} diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/= generic.c index 3c3c7919de5b..a9ac159b1566 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -10,6 +10,7 @@ #include =20 #include +#include #include #include #include @@ -396,9 +397,6 @@ print_fixed(unsigned base, unsigned step, const mtrr_ty= pe *types) } } =20 -static void prepare_set(void); -static void post_set(void); - static void __init print_mtrr_state(void) { unsigned int i; @@ -450,11 +448,11 @@ void __init mtrr_bp_pat_init(void) unsigned long flags; =20 local_irq_save(flags); - prepare_set(); + cache_disable(); =20 pat_init(); =20 - post_set(); + cache_enable(); local_irq_restore(flags); } =20 @@ -717,85 +715,25 @@ static unsigned long set_mtrr_state(void) return change_mask; } =20 - -static unsigned long cr4; -static DEFINE_RAW_SPINLOCK(set_atomicity_lock); - -/* - * Since we are disabling the cache don't allow any interrupts, - * they would run extremely slow and would only increase the pain. - * - * The caller must ensure that local interrupts are disabled and - * are reenabled after post_set() has been called. - */ -static void prepare_set(void) __acquires(set_atomicity_lock) +void mtrr_disable(void) { - unsigned long cr0; u32 *lo =3D this_cpu_ptr(&deftype_lo); u32 *hi =3D this_cpu_ptr(&deftype_hi); =20 - /* - * Note that this is not ideal - * since the cache is only flushed/disabled for this CPU while the - * MTRRs are changed, but changing this requires more invasive - * changes to the way the kernel boots - */ - - raw_spin_lock(&set_atomicity_lock); - - /* Enter the no-fill (CD=3D1, NW=3D0) cache mode and flush caches. */ - cr0 =3D read_cr0() | X86_CR0_CD; - write_cr0(cr0); - - /* - * Cache flushing is the most time-consuming step when programming - * the MTRRs. Fortunately, as per the Intel Software Development - * Manual, we can skip it if the processor supports cache self- - * snooping. - */ - if (!static_cpu_has(X86_FEATURE_SELFSNOOP)) - wbinvd(); - - /* Save value of CR4 and clear Page Global Enable (bit 7) */ - if (boot_cpu_has(X86_FEATURE_PGE)) { - cr4 =3D __read_cr4(); - __write_cr4(cr4 & ~X86_CR4_PGE); - } - - /* Flush all TLBs via a mov %cr3, %reg; mov %reg, %cr3 */ - count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL); - flush_tlb_local(); - /* Save MTRR state */ rdmsr(MSR_MTRRdefType, *lo, *hi); =20 /* Disable MTRRs, and set the default type to uncached */ mtrr_wrmsr(MSR_MTRRdefType, *lo & ~0xcff, *hi); - - /* Again, only flush caches if we have to. */ - if (!static_cpu_has(X86_FEATURE_SELFSNOOP)) - wbinvd(); } =20 -static void post_set(void) __releases(set_atomicity_lock) +void mtrr_enable(void) { u32 *lo =3D this_cpu_ptr(&deftype_lo); u32 *hi =3D this_cpu_ptr(&deftype_hi); =20 - /* Flush TLBs (no need to flush caches - they are disabled) */ - count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL); - flush_tlb_local(); - /* Intel (P6) standard MTRRs */ mtrr_wrmsr(MSR_MTRRdefType, *lo, *hi); - - /* Enable caches */ - write_cr0(read_cr0() & ~X86_CR0_CD); - - /* Restore value of CR4 */ - if (boot_cpu_has(X86_FEATURE_PGE)) - __write_cr4(cr4); - raw_spin_unlock(&set_atomicity_lock); } =20 static void generic_set_all(void) @@ -804,7 +742,7 @@ static void generic_set_all(void) unsigned long flags; =20 local_irq_save(flags); - prepare_set(); + cache_disable(); =20 /* Actually set the state */ mask =3D set_mtrr_state(); @@ -812,7 +750,7 @@ static void generic_set_all(void) /* also set PAT */ pat_init(); =20 - post_set(); + cache_enable(); local_irq_restore(flags); =20 /* Use the atomic bitops to update the global mask */ @@ -843,7 +781,7 @@ static void generic_set_mtrr(unsigned int reg, unsigned= long base, vr =3D &mtrr_state.var_ranges[reg]; =20 local_irq_save(flags); - prepare_set(); + cache_disable(); =20 if (size =3D=3D 0) { /* @@ -862,7 +800,7 @@ static void generic_set_mtrr(unsigned int reg, unsigned= long base, mtrr_wrmsr(MTRRphysMask_MSR(reg), vr->mask_lo, vr->mask_hi); } =20 - post_set(); + cache_enable(); local_irq_restore(flags); } =20 --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987573; cv=none; d=zohomail.com; s=zohoarc; b=dlCkn2wzTzp/Qy6Q8ZMrcNRm5dHxZhNfSkJ343sQBJb5raYSH0k3LlxB2wkR5QJUVyZ4lovVaTZiv4sysXjqulW7zD4D/T8PuxnEQ9YeaJhjwJHfEugW+Z9SuyKD2ue0E2ezWvGNAkxWUzwxpStK31msGRTltATIyZgR2pz7b90= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987573; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=I9u1SaUR5yI9p8R8N8ofvxILRs1M2hzJI4h2vyFfrA8=; b=Qz//lLbEq932bSMWR11rEs0I2V7BuKO9KCc8x6AF0V891tV7ljc3tX/mR8n29QukXiF1b6sMcOji6kWch+YV+/NWNAFbvjibGEBmj5uf46XJB571h5vQJHgORwaAetdu6BLtAHzkuFzp9b2qWUoaQ6UPrR7WMJjxzCegacAMlw0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987573056827.1658311415147; Sat, 20 Aug 2022 02:26:13 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390684.628236 (Exim 4.92) (envelope-from ) id 1oPKjg-0007da-3T; Sat, 20 Aug 2022 09:25:44 +0000 Received: by outflank-mailman (output) from mailman id 390684.628236; Sat, 20 Aug 2022 09:25:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjf-0007az-MA; Sat, 20 Aug 2022 09:25:43 +0000 Received: by outflank-mailman (input) for mailman id 390684; Sat, 20 Aug 2022 09:25:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjd-0006NK-Ji for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:41 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [2001:67c:2178:6::1d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0d9746f2-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:38 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 69FA11FDBD; Sat, 20 Aug 2022 09:25:38 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2816A13440; Sat, 20 Aug 2022 09:25:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id MAGKCJKoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0d9746f2-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987538; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I9u1SaUR5yI9p8R8N8ofvxILRs1M2hzJI4h2vyFfrA8=; b=gGTAo9BwxgqV8mrviLJWJvW+8Y9yx32TtlD70JNrIT2jvHqRZDQDlM+aSwXvJbTerU0s+A LRBiVpAJbg3pfdvDw8VDueKkXg1R/+nOtwtHgx4IQL1C1EqOfYPPw601t5zpbD+YWObL9i +Jkc4nRJLm6Vrokm/eX/eVMvL59nknw= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v2 05/10] x86/mtrr: split generic_set_all() Date: Sat, 20 Aug 2022 11:25:28 +0200 Message-Id: <20220820092533.29420-6-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987573682100005 Content-Type: text/plain; charset="utf-8" Split generic_set_all() into multiple parts, while moving the main function body into cacheinfo.c. This prepares the support of PAT without needing MTRR support by moving the main function body of generic_set_all() into cacheinfo.c while renaming it to cache_cpu_init(). The MTRR specific parts are moved into a dedicated small function called by cache_cpu_init(). The PAT and MTRR specific functions are called conditionally based on the cache_generic bit settings. The setting of smp_changes_mask is merged into the (new) function mtrr_generic_set_state() used to call set_mtrr_state(). It was probably split in ancient times, as atomic operations while running uncached might be quite expensive, but OTOH only systems with a broken BIOS should ever require to set any bit in smp_changes_mask, so just hurting those devices with a penalty of a few microseconds during boot shouldn't be a real issue. Signed-off-by: Juergen Gross --- V2: - new patch --- arch/x86/include/asm/cacheinfo.h | 1 + arch/x86/include/asm/mtrr.h | 2 ++ arch/x86/kernel/cpu/cacheinfo.c | 19 +++++++++++++++++++ arch/x86/kernel/cpu/mtrr/generic.c | 15 ++------------- 4 files changed, 24 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/cacheinfo.h b/arch/x86/include/asm/cachei= nfo.h index 313a6920d0f9..563d9cb5fcf5 100644 --- a/arch/x86/include/asm/cacheinfo.h +++ b/arch/x86/include/asm/cacheinfo.h @@ -12,5 +12,6 @@ void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, i= nt cpu); =20 void cache_disable(void); void cache_enable(void); +void cache_cpu_init(void); =20 #endif /* _ASM_X86_CACHEINFO_H */ diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index 12a16caed395..986249a2b9b6 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -50,6 +50,7 @@ extern int mtrr_trim_uncached_memory(unsigned long end_pf= n); extern int amd_special_default_mtrr(void); void mtrr_disable(void); void mtrr_enable(void); +void mtrr_generic_set_state(void); # else static inline u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform) { @@ -91,6 +92,7 @@ static inline void mtrr_centaur_report_mcr(int mcr, u32 l= o, u32 hi) #define mtrr_bp_restore() do {} while (0) #define mtrr_disable() do {} while (0) #define mtrr_enable() do {} while (0) +#define mtrr_generic_set_state() do {} while (0) # endif =20 #ifdef CONFIG_COMPAT diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index 47e2c72fa8a4..36378604ec61 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -1120,3 +1120,22 @@ void cache_enable(void) __releases(cache_disable_loc= k) =20 raw_spin_unlock(&cache_disable_lock); } + +void cache_cpu_init(void) +{ + unsigned long flags; + + local_irq_save(flags); + cache_disable(); + + /* Set MTRR state. */ + if (cache_generic & CACHE_GENERIC_MTRR) + mtrr_generic_set_state(); + + /* Set PAT. */ + if (cache_generic & CACHE_GENERIC_PAT) + pat_init(); + + cache_enable(); + local_irq_restore(flags); +} diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/= generic.c index a9ac159b1566..0be6d43ec1fe 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -736,30 +736,19 @@ void mtrr_enable(void) mtrr_wrmsr(MSR_MTRRdefType, *lo, *hi); } =20 -static void generic_set_all(void) +void mtrr_generic_set_state(void) { unsigned long mask, count; - unsigned long flags; - - local_irq_save(flags); - cache_disable(); =20 /* Actually set the state */ mask =3D set_mtrr_state(); =20 - /* also set PAT */ - pat_init(); - - cache_enable(); - local_irq_restore(flags); - /* Use the atomic bitops to update the global mask */ for (count =3D 0; count < sizeof(mask) * 8; ++count) { if (mask & 0x01) set_bit(count, &smp_changes_mask); mask >>=3D 1; } - } =20 /** @@ -859,7 +848,7 @@ int positive_have_wrcomb(void) * Generic structure... */ const struct mtrr_ops generic_mtrr_ops =3D { - .set_all =3D generic_set_all, + .set_all =3D cache_cpu_init, .get =3D generic_get_mtrr, .get_free_region =3D generic_get_free_region, .set =3D generic_set_mtrr, --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987574; cv=none; d=zohomail.com; s=zohoarc; b=AMsV6DnrX6KfjE82WBaDLEniSUcjWWWxzRURUYuELwVbSIsrm+8gDBWzeNTYL5zZ8IF0x3FD12Zh9cZbDPlVSPZQAWO4QppA1IArJThYNPmjFCnqF0wVO3MxaW4bX7RE2dIrRQw28+rAsnKFSvO7jg0JfqAQmIr03JPIbYMWhtQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987574; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UlbRhwJUB+EdBZOc37IvbCp3QAOY+aESAq1E4i9TKiY=; b=h8t7WbBIZwbpOhre4CmFvhfxUDzHN8bE5fNmYjnuCr4WwR0N6onCTDLy8445CR8pYU19XC4/ET5CQu0ZJFfjrtMUDhfyoJ0uYzL6sh1jwNixFNBwXIsnI9ukktV2dPQwB93QzglFRsLRwmDuIeRSD7cGM75tyrgPJLjCabIQ8ls= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987574197480.1371265675285; Sat, 20 Aug 2022 02:26:14 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390685.628241 (Exim 4.92) (envelope-from ) id 1oPKjg-0007kM-F3; Sat, 20 Aug 2022 09:25:44 +0000 Received: by outflank-mailman (output) from mailman id 390685.628241; Sat, 20 Aug 2022 09:25:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjg-0007hw-4Q; Sat, 20 Aug 2022 09:25:44 +0000 Received: by outflank-mailman (input) for mailman id 390685; Sat, 20 Aug 2022 09:25:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKje-0006NK-Jp for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:42 +0000 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0dc3056f-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:38 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B0B3034367; Sat, 20 Aug 2022 09:25:38 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 728BA13440; Sat, 20 Aug 2022 09:25:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mIm5GpKoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0dc3056f-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987538; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UlbRhwJUB+EdBZOc37IvbCp3QAOY+aESAq1E4i9TKiY=; b=JshuIRAceP6IJvA0riFQZSd2Htq4cpLIftqmAiiPsVriEG1H1DgpOwVRp2+8gYx4bUYnBb sQbmVHPAY3GUOWlsvTHo51jdOnlUOMv3p5HaCzn7K4N9tsevg+T3Ozz1KMjE1SvLp4mQu8 n4aCd4tPKKFeEe1Nguuec2jpFNoUCig= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v2 06/10] x86/mtrr: remove set_all callback from struct mtrr_ops Date: Sat, 20 Aug 2022 11:25:29 +0200 Message-Id: <20220820092533.29420-7-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987575783100015 Content-Type: text/plain; charset="utf-8" Instead of using an indirect call to mtrr_if->set_all just call the only possible target cache_cpu_init() directly. This enables to remove the set_all callback from struct mtrr_ops. Signed-off-by: Juergen Gross --- arch/x86/kernel/cpu/mtrr/generic.c | 1 - arch/x86/kernel/cpu/mtrr/mtrr.c | 10 +++++----- arch/x86/kernel/cpu/mtrr/mtrr.h | 2 -- 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/= generic.c index 0be6d43ec1fe..1c6260b94cb9 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -848,7 +848,6 @@ int positive_have_wrcomb(void) * Generic structure... */ const struct mtrr_ops generic_mtrr_ops =3D { - .set_all =3D cache_cpu_init, .get =3D generic_get_mtrr, .get_free_region =3D generic_get_free_region, .set =3D generic_set_mtrr, diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtr= r.c index 4cd9c8d1473e..19662ab057be 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -165,15 +165,15 @@ static int mtrr_rendezvous_handler(void *info) * saved, and we want to replicate that across all the cpus that come * online (either at the end of boot or resume or during a runtime cpu * online). If we're doing that, @reg is set to something special and on - * all the cpu's we do mtrr_if->set_all() (On the logical cpu that + * all the cpu's we do cache_cpu_init() (On the logical cpu that * started the boot/resume sequence, this might be a duplicate - * set_all()). + * cache_cpu_init()). */ if (data->smp_reg !=3D ~0U) { mtrr_if->set(data->smp_reg, data->smp_base, data->smp_size, data->smp_type); } else if (mtrr_aps_delayed_init || !cpu_online(smp_processor_id())) { - mtrr_if->set_all(); + cache_cpu_init(); } return 0; } @@ -768,7 +768,7 @@ void __init mtrr_bp_init(void) =20 if (mtrr_cleanup(phys_addr)) { changed_by_mtrr_cleanup =3D 1; - mtrr_if->set_all(); + cache_cpu_init(); } } } @@ -854,7 +854,7 @@ void mtrr_bp_restore(void) if (!cache_generic) return; =20 - mtrr_if->set_all(); + cache_cpu_init(); } =20 static int __init mtrr_init_finialize(void) diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtr= r.h index 88b1c4b6174a..3b1883185185 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.h +++ b/arch/x86/kernel/cpu/mtrr/mtrr.h @@ -16,8 +16,6 @@ struct mtrr_ops { u32 vendor; void (*set)(unsigned int reg, unsigned long base, unsigned long size, mtrr_type type); - void (*set_all)(void); - void (*get)(unsigned int reg, unsigned long *base, unsigned long *size, mtrr_type *type); int (*get_free_region)(unsigned long base, unsigned long size, --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987573; cv=none; d=zohomail.com; s=zohoarc; b=Qk86Zb6xD1AK/Js5hGI/OmQQUtPc7ZvXnB2YxIz+kk58ehY8AnOW8NgJDrkNrEaLnI24yUPjCkC7Kh1IhEJuBjCwwTz6xkUBhLmDHMVFUIBpijiJAfbfDghcAOCfmoWwqJ943BuavbSNWRnCt8J/GFZjZZAWi9NOMMwMGla4QHw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987573; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ApIUNLFVl6HZvnRkU+2Ix/2f56BO+zXK/TglyOeKv+E=; b=fBlJpcg4fwlrI1AHMrjMzoUGJXNkzBW4rm21q+uT8NH6/DysIcNcjEgDwSwwgjJDzEnXDUK62GhexxjgkmudZyCFOsXU7vmbXL8XAAUvr80Li9HEmtD7s4RbQX02mPOj7IXWWM6BPkLfkmf1nmyoksEoOlJ3f/XRYrZu7lR/JRc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987573057893.3375671011696; Sat, 20 Aug 2022 02:26:13 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390682.628221 (Exim 4.92) (envelope-from ) id 1oPKje-0007KS-Un; Sat, 20 Aug 2022 09:25:42 +0000 Received: by outflank-mailman (output) from mailman id 390682.628221; Sat, 20 Aug 2022 09:25:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKje-0007Jq-P9; Sat, 20 Aug 2022 09:25:42 +0000 Received: by outflank-mailman (input) for mailman id 390682; Sat, 20 Aug 2022 09:25:40 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjc-0006NL-KS for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:40 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [2001:67c:2178:6::1d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 0df6ce76-206a-11ed-bd2e-47488cf2e6aa; Sat, 20 Aug 2022 11:25:39 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 023FA1FEFD; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B9C3D13440; Sat, 20 Aug 2022 09:25:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id ADMoLJKoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0df6ce76-206a-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987539; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ApIUNLFVl6HZvnRkU+2Ix/2f56BO+zXK/TglyOeKv+E=; b=OVoRIvF8goFpMJZ0uw8ySeIl4Um60lYLwlPgpnYvUUYtNsnSHo8XctJ0ZyOjXo9aoCYVY/ HbZEEu/C4KJYA4TEC5WeYmVy5Mu7qV0eKj4IAfguw+7mzASrNc6L9fylPZZ+eikjdzX2Qv R9IyrLHZQ9dN6daxeRRiHELCANSgRLw= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v2 07/10] x86/mtrr: simplify mtrr_bp_init() Date: Sat, 20 Aug 2022 11:25:30 +0200 Message-Id: <20220820092533.29420-8-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987573972100012 Content-Type: text/plain; charset="utf-8" In case of the generic cache interface being used (Intel cpus or a 64-bit system), the initialization sequence of the boot cpu is more complicated as necessary: - check if MTRR enabled, if yes, call mtrr_bp_pat_init() which will disable caching, set the PAT MSR, and reenable caching - call mtrr_cleanup(), in case that changed anything, call cache_cpu_init() doing the same caching disable/enable dance as above, but this time with setting the (modified) MTRR state (even if MTRR was disabled) AND setting the PAT MSR (again even with disabled MTRR) The sequence can be simplified a lot while removing potential inconsistencies: - check if MTRR enabled, if yes, call mtrr_cleanup() and then cache_cpu_init() This ensures to: - no longer disable/enable caching more than once - avoid to set MTRRs and/or the PAT MSR on the boot processor in case of MTRR cleanups even if MTRRs meant to be disabled With that mtrr_bp_pat_init() can be removed. Signed-off-by: Juergen Gross --- V2: - new patch --- arch/x86/kernel/cpu/mtrr/generic.c | 14 -------------- arch/x86/kernel/cpu/mtrr/mtrr.c | 7 ++----- arch/x86/kernel/cpu/mtrr/mtrr.h | 1 - 3 files changed, 2 insertions(+), 20 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/= generic.c index 1c6260b94cb9..b6be476c77af 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -442,20 +442,6 @@ static void __init print_mtrr_state(void) pr_debug("TOM2: %016llx aka %lldM\n", mtrr_tom2, mtrr_tom2>>20); } =20 -/* PAT setup for BP. We need to go through sync steps here */ -void __init mtrr_bp_pat_init(void) -{ - unsigned long flags; - - local_irq_save(flags); - cache_disable(); - - pat_init(); - - cache_enable(); - local_irq_restore(flags); -} - /* Grab all of the MTRR state for this CPU into *state */ bool __init get_mtrr_state(void) { diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtr= r.c index 19662ab057be..ca348dcc5bf8 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -761,13 +761,10 @@ void __init mtrr_bp_init(void) mtrr_enabled =3D get_mtrr_state(); =20 if (mtrr_enabled) { - mtrr_bp_pat_init(); cache_generic |=3D CACHE_GENERIC_MTRR | CACHE_GENERIC_PAT; - } - - if (mtrr_cleanup(phys_addr)) { - changed_by_mtrr_cleanup =3D 1; + changed_by_mtrr_cleanup =3D + mtrr_cleanup(phys_addr); cache_cpu_init(); } } diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtr= r.h index 3b1883185185..c98928ceee6a 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.h +++ b/arch/x86/kernel/cpu/mtrr/mtrr.h @@ -50,7 +50,6 @@ void set_mtrr_prepare_save(struct set_mtrr_context *ctxt); void fill_mtrr_var_range(unsigned int index, u32 base_lo, u32 base_hi, u32 mask_lo, u32 mask_hi); bool get_mtrr_state(void); -void mtrr_bp_pat_init(void); =20 extern void __init set_mtrr_ops(const struct mtrr_ops *ops); =20 --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987571; cv=none; d=zohomail.com; s=zohoarc; b=OYLkCJR5CVfS1VoRfJZ4nEqo2Su7jBvHRnXchMd91Y25DRJt9h1sShskNHqblZZSg3/JI21f/+ctIIK+dTeJZLmA+kKEnbYLbvVm9XHDwkUM+c2bvjXWhizk2b0YX7D5whQz4njgzVtilrk90dnrryhsooL0MH/g333iyTu4vPw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987571; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1+9qSe12eN9aldkpJUagir7LS9P8+Ie9bYA0m7OCmgU=; b=n1CNqrVapt6x5e2uKcgo8vfC2sknlh0Coir2itSvAMWOkROQ0JNvMZR+mNaOLAaUh6NjgDZ+G87RQDGcB4K2b4hOvpZZKnA6WlHV0+5UbvEJNY6xjBFbMqNsxpPfccGKmh+4rWB4z0cOdcXf4BA7WY6W5xaXcQUfp4gIxRDHTXI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 166098757122135.26071286506999; Sat, 20 Aug 2022 02:26:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390686.628258 (Exim 4.92) (envelope-from ) id 1oPKjh-0008Cu-Ul; Sat, 20 Aug 2022 09:25:45 +0000 Received: by outflank-mailman (output) from mailman id 390686.628258; Sat, 20 Aug 2022 09:25:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjh-0008BU-MS; Sat, 20 Aug 2022 09:25:45 +0000 Received: by outflank-mailman (input) for mailman id 390686; Sat, 20 Aug 2022 09:25:43 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjf-0006NK-K0 for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:43 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0e2af96d-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:39 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4EEFC1FF2C; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0B50113440; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id mJiRAZOoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0e2af96d-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987539; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1+9qSe12eN9aldkpJUagir7LS9P8+Ie9bYA0m7OCmgU=; b=jwQV9DUHU1m/w4KwDZAHaiEgHp1+vBu8JBDI1SuKu/yWHL8/ABmTvpPvgpWrYOuArNxHcA 6u/QFmufL8ScBAHFisN+w6l5yU12CyLZAMh/A+rxn6+kt31lLZHX1UQhA3ImTT+cIu95tf VJgKfXxWw3RWqW6W5SSwEXsd1mu8Ztg= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v2 08/10] x86/mtrr: let cache_aps_delayed_init replace mtrr_aps_delayed_init Date: Sat, 20 Aug 2022 11:25:31 +0200 Message-Id: <20220820092533.29420-9-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987571654100002 Content-Type: text/plain; charset="utf-8" In order to prepare decoupling MTRR and PAT replace the MTRR specific mtrr_aps_delayed_init flag with a more generic cache_aps_delayed_init one. Signed-off-by: Juergen Gross --- V2: - new patch --- arch/x86/include/asm/cacheinfo.h | 2 ++ arch/x86/include/asm/mtrr.h | 2 -- arch/x86/kernel/cpu/cacheinfo.c | 2 ++ arch/x86/kernel/cpu/mtrr/mtrr.c | 17 ++++------------- arch/x86/kernel/smpboot.c | 5 +++-- 5 files changed, 11 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/cacheinfo.h b/arch/x86/include/asm/cachei= nfo.h index 563d9cb5fcf5..e80ed3c523c8 100644 --- a/arch/x86/include/asm/cacheinfo.h +++ b/arch/x86/include/asm/cacheinfo.h @@ -7,6 +7,8 @@ extern unsigned int cache_generic; #define CACHE_GENERIC_MTRR 0x01 #define CACHE_GENERIC_PAT 0x02 =20 +extern bool cache_aps_delayed_init; + void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, int cpu); void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, int cpu); =20 diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index 986249a2b9b6..5d31219c8529 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -43,7 +43,6 @@ extern int mtrr_del(int reg, unsigned long base, unsigned= long size); extern int mtrr_del_page(int reg, unsigned long base, unsigned long size); extern void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi); extern void mtrr_ap_init(void); -extern void set_mtrr_aps_delayed_init(void); extern void mtrr_aps_init(void); extern void mtrr_bp_restore(void); extern int mtrr_trim_uncached_memory(unsigned long end_pfn); @@ -87,7 +86,6 @@ static inline void mtrr_centaur_report_mcr(int mcr, u32 l= o, u32 hi) { } #define mtrr_ap_init() do {} while (0) -#define set_mtrr_aps_delayed_init() do {} while (0) #define mtrr_aps_init() do {} while (0) #define mtrr_bp_restore() do {} while (0) #define mtrr_disable() do {} while (0) diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index 36378604ec61..c6e7c93e45e8 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -1139,3 +1139,5 @@ void cache_cpu_init(void) cache_enable(); local_irq_restore(flags); } + +bool cache_aps_delayed_init; diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtr= r.c index ca348dcc5bf8..b68ec43e33a2 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -65,7 +65,6 @@ unsigned int mtrr_usage_table[MTRR_MAX_VAR_RANGES]; static DEFINE_MUTEX(mtrr_mutex); =20 u64 size_or_mask, size_and_mask; -static bool mtrr_aps_delayed_init; =20 static const struct mtrr_ops *mtrr_ops[X86_VENDOR_NUM] __ro_after_init; =20 @@ -172,7 +171,7 @@ static int mtrr_rendezvous_handler(void *info) if (data->smp_reg !=3D ~0U) { mtrr_if->set(data->smp_reg, data->smp_base, data->smp_size, data->smp_type); - } else if (mtrr_aps_delayed_init || !cpu_online(smp_processor_id())) { + } else if (cache_aps_delayed_init || !cpu_online(smp_processor_id())) { cache_cpu_init(); } return 0; @@ -784,7 +783,7 @@ void __init mtrr_bp_init(void) =20 void mtrr_ap_init(void) { - if (!cache_generic || mtrr_aps_delayed_init) + if (!cache_generic || cache_aps_delayed_init) return; =20 /* @@ -818,14 +817,6 @@ void mtrr_save_state(void) smp_call_function_single(first_cpu, mtrr_save_fixed_ranges, NULL, 1); } =20 -void set_mtrr_aps_delayed_init(void) -{ - if (!cache_generic) - return; - - mtrr_aps_delayed_init =3D true; -} - /* * Delayed MTRR initialization for all AP's */ @@ -839,11 +830,11 @@ void mtrr_aps_init(void) * by doing set_mtrr_aps_delayed_init(), prior to this point. If not, * then we are done. */ - if (!mtrr_aps_delayed_init) + if (!cache_aps_delayed_init) return; =20 set_mtrr(~0U, 0, 0, 0); - mtrr_aps_delayed_init =3D false; + cache_aps_delayed_init =3D false; } =20 void mtrr_bp_restore(void) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index f24227bc3220..ef7bce21cbe8 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -58,6 +58,7 @@ #include =20 #include +#include #include #include #include @@ -1428,7 +1429,7 @@ void __init native_smp_prepare_cpus(unsigned int max_= cpus) =20 uv_system_init(); =20 - set_mtrr_aps_delayed_init(); + cache_aps_delayed_init =3D true; =20 smp_quirk_init_udelay(); =20 @@ -1439,7 +1440,7 @@ void __init native_smp_prepare_cpus(unsigned int max_= cpus) =20 void arch_thaw_secondary_cpus_begin(void) { - set_mtrr_aps_delayed_init(); + cache_aps_delayed_init =3D true; } =20 void arch_thaw_secondary_cpus_end(void) --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987573; cv=none; d=zohomail.com; s=zohoarc; b=XVEP4IgB9MR7hpmWvIng3SEHQE0AErMoZuB+lC4mOYZWRvQckNXWJOJukgr0Av4Ra5zjtRRstwZgiPVDAfooQk3ATz756CAhy4f0u0kyn5LjCtmsg63G9zpY8vK0DFqDJq8f7XsNpFREi0VKG6NyrGg2k3SZNsNUMNXkAz2iJgk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987573; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JUJm8AzApP3BXUfz/8kYW4X5+H9Ehhz9tOTxTyy5q78=; b=EfFEJtruwhJWByjf9vZNz1BbOvsts2FHTcexIhwAKXlw9sKhqVhUn+UGQ3JxxpfdyuvyVVnLjFmWrqwI2EUZENN+By+6M79cFG8l12y1Sw4/MU8Icxe5Qs8TF61p8NWMfoVxa1YbvyGHA7+pASFILJ5xST0UVqcBIh9lVaddzdw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987573111400.6117840870629; Sat, 20 Aug 2022 02:26:13 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390687.628273 (Exim 4.92) (envelope-from ) id 1oPKjj-0000FF-Nw; Sat, 20 Aug 2022 09:25:47 +0000 Received: by outflank-mailman (output) from mailman id 390687.628273; Sat, 20 Aug 2022 09:25:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjj-0000C7-ED; Sat, 20 Aug 2022 09:25:47 +0000 Received: by outflank-mailman (input) for mailman id 390687; Sat, 20 Aug 2022 09:25:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjg-0006NK-KD for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:44 +0000 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0e5d84f3-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:39 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AF1872005B; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5933213440; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id KAScFJOoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0e5d84f3-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987539; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JUJm8AzApP3BXUfz/8kYW4X5+H9Ehhz9tOTxTyy5q78=; b=pLwDNZdYw2eOdoTT+7tU43E4Jb9JnNXc6u/4Ne/QAtm6XHwsyV2Fd9Ul8cM8HsqGgbKKbP bJTrtLPgUbNpMM20fcnP4tOowZ4b2qR7nc8zd8Q9h8+omZJH7S28LBFKap+aBkpIK0QCoy G2ypbxWHcgr96ClzCmZzKbr93DnKD24= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , "Rafael J. Wysocki" , Pavel Machek Subject: [PATCH v2 09/10] x86/mtrr: add a stop_machine() handler calling only cache_cpu_init() Date: Sat, 20 Aug 2022 11:25:32 +0200 Message-Id: <20220820092533.29420-10-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987573709100006 Content-Type: text/plain; charset="utf-8" Instead of having a stop_machine() handler for either a specific MTRR register or all state at once, add a handler just for calling cache_cpu_init() if appropriate. Add functions for calling stop_machine() with this handler as well. Add a generic replacements for mtrr_bp_restore() and a wrapper for mtrr_bp_init(). Signed-off-by: Juergen Gross --- V2: - completely new replacement of former patch 2 --- arch/x86/include/asm/cacheinfo.h | 5 +- arch/x86/include/asm/mtrr.h | 4 -- arch/x86/kernel/cpu/cacheinfo.c | 59 +++++++++++++++++++++- arch/x86/kernel/cpu/common.c | 3 +- arch/x86/kernel/cpu/mtrr/mtrr.c | 87 +------------------------------- arch/x86/kernel/setup.c | 3 +- arch/x86/kernel/smpboot.c | 4 +- arch/x86/power/cpu.c | 3 +- 8 files changed, 72 insertions(+), 96 deletions(-) diff --git a/arch/x86/include/asm/cacheinfo.h b/arch/x86/include/asm/cachei= nfo.h index e80ed3c523c8..a122a1aad936 100644 --- a/arch/x86/include/asm/cacheinfo.h +++ b/arch/x86/include/asm/cacheinfo.h @@ -14,6 +14,9 @@ void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, i= nt cpu); =20 void cache_disable(void); void cache_enable(void); -void cache_cpu_init(void); +void cache_bp_init(void); +void cache_bp_restore(void); +void cache_ap_init(void); +void cache_aps_init(void); =20 #endif /* _ASM_X86_CACHEINFO_H */ diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index 5d31219c8529..ec73d1e5bafb 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -42,8 +42,6 @@ extern int mtrr_add_page(unsigned long base, unsigned lon= g size, extern int mtrr_del(int reg, unsigned long base, unsigned long size); extern int mtrr_del_page(int reg, unsigned long base, unsigned long size); extern void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi); -extern void mtrr_ap_init(void); -extern void mtrr_aps_init(void); extern void mtrr_bp_restore(void); extern int mtrr_trim_uncached_memory(unsigned long end_pfn); extern int amd_special_default_mtrr(void); @@ -85,8 +83,6 @@ static inline int mtrr_trim_uncached_memory(unsigned long= end_pfn) static inline void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi) { } -#define mtrr_ap_init() do {} while (0) -#define mtrr_aps_init() do {} while (0) #define mtrr_bp_restore() do {} while (0) #define mtrr_disable() do {} while (0) #define mtrr_enable() do {} while (0) diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index c6e7c93e45e8..4946f93eb16f 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -15,6 +15,7 @@ #include #include #include +#include =20 #include #include @@ -1121,7 +1122,7 @@ void cache_enable(void) __releases(cache_disable_lock) raw_spin_unlock(&cache_disable_lock); } =20 -void cache_cpu_init(void) +static void cache_cpu_init(void) { unsigned long flags; =20 @@ -1141,3 +1142,59 @@ void cache_cpu_init(void) } =20 bool cache_aps_delayed_init; + +static int cache_rendezvous_handler(void *unused) +{ + if (cache_aps_delayed_init || !cpu_online(smp_processor_id())) + cache_cpu_init(); + + return 0; +} + +void __init cache_bp_init(void) +{ + mtrr_bp_init(); + + if (cache_generic) + cache_cpu_init(); +} + +void cache_bp_restore(void) +{ + if (cache_generic) + cache_cpu_init(); +} + +void cache_ap_init(void) +{ + if (!cache_generic || cache_aps_delayed_init) + return; + + /* + * Ideally we should hold mtrr_mutex here to avoid mtrr entries + * changed, but this routine will be called in cpu boot time, + * holding the lock breaks it. + * + * This routine is called in two cases: + * + * 1. very early time of software resume, when there absolutely + * isn't mtrr entry changes; + * + * 2. cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug + * lock to prevent mtrr entry changes + */ + stop_machine_from_inactive_cpu(cache_rendezvous_handler, NULL, + cpu_callout_mask); +} + +/* + * Delayed cache initialization for all AP's + */ +void cache_aps_init(void) +{ + if (!cache_generic || !cache_aps_delayed_init) + return; + + stop_machine(cache_rendezvous_handler, NULL, cpu_online_mask); + cache_aps_delayed_init =3D false; +} diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 64a73f415f03..5c122dd17ec2 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -1936,7 +1937,7 @@ void identify_secondary_cpu(struct cpuinfo_x86 *c) #ifdef CONFIG_X86_32 enable_sep_cpu(); #endif - mtrr_ap_init(); + cache_ap_init(); validate_apic_and_package_id(c); x86_spec_ctrl_setup_ap(); update_srbds_msr(); diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtr= r.c index b68ec43e33a2..38531e021581 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -70,9 +70,6 @@ static const struct mtrr_ops *mtrr_ops[X86_VENDOR_NUM] __= ro_after_init; =20 const struct mtrr_ops *mtrr_if; =20 -static void set_mtrr(unsigned int reg, unsigned long base, - unsigned long size, mtrr_type type); - void __init set_mtrr_ops(const struct mtrr_ops *ops) { if (ops->vendor && ops->vendor < X86_VENDOR_NUM) @@ -155,25 +152,8 @@ static int mtrr_rendezvous_handler(void *info) { struct set_mtrr_data *data =3D info; =20 - /* - * We use this same function to initialize the mtrrs during boot, - * resume, runtime cpu online and on an explicit request to set a - * specific MTRR. - * - * During boot or suspend, the state of the boot cpu's mtrrs has been - * saved, and we want to replicate that across all the cpus that come - * online (either at the end of boot or resume or during a runtime cpu - * online). If we're doing that, @reg is set to something special and on - * all the cpu's we do cache_cpu_init() (On the logical cpu that - * started the boot/resume sequence, this might be a duplicate - * cache_cpu_init()). - */ - if (data->smp_reg !=3D ~0U) { - mtrr_if->set(data->smp_reg, data->smp_base, - data->smp_size, data->smp_type); - } else if (cache_aps_delayed_init || !cpu_online(smp_processor_id())) { - cache_cpu_init(); - } + mtrr_if->set(data->smp_reg, data->smp_base, + data->smp_size, data->smp_type); return 0; } =20 @@ -243,19 +223,6 @@ static void set_mtrr_cpuslocked(unsigned int reg, unsi= gned long base, stop_machine_cpuslocked(mtrr_rendezvous_handler, &data, cpu_online_mask); } =20 -static void set_mtrr_from_inactive_cpu(unsigned int reg, unsigned long bas= e, - unsigned long size, mtrr_type type) -{ - struct set_mtrr_data data =3D { .smp_reg =3D reg, - .smp_base =3D base, - .smp_size =3D size, - .smp_type =3D type - }; - - stop_machine_from_inactive_cpu(mtrr_rendezvous_handler, &data, - cpu_callout_mask); -} - /** * mtrr_add_page - Add a memory type region * @base: Physical base address of region in pages (in units of 4 kB!) @@ -764,7 +731,6 @@ void __init mtrr_bp_init(void) CACHE_GENERIC_PAT; changed_by_mtrr_cleanup =3D mtrr_cleanup(phys_addr); - cache_cpu_init(); } } } @@ -781,27 +747,6 @@ void __init mtrr_bp_init(void) } } =20 -void mtrr_ap_init(void) -{ - if (!cache_generic || cache_aps_delayed_init) - return; - - /* - * Ideally we should hold mtrr_mutex here to avoid mtrr entries - * changed, but this routine will be called in cpu boot time, - * holding the lock breaks it. - * - * This routine is called in two cases: - * - * 1. very early time of software resume, when there absolutely - * isn't mtrr entry changes; - * - * 2. cpu hotadd time. We let mtrr_add/del_page hold cpuhotplug - * lock to prevent mtrr entry changes - */ - set_mtrr_from_inactive_cpu(~0U, 0, 0, 0); -} - /** * mtrr_save_state - Save current fixed-range MTRR state of the first * cpu in cpu_online_mask. @@ -817,34 +762,6 @@ void mtrr_save_state(void) smp_call_function_single(first_cpu, mtrr_save_fixed_ranges, NULL, 1); } =20 -/* - * Delayed MTRR initialization for all AP's - */ -void mtrr_aps_init(void) -{ - if (!cache_generic) - return; - - /* - * Check if someone has requested the delay of AP MTRR initialization, - * by doing set_mtrr_aps_delayed_init(), prior to this point. If not, - * then we are done. - */ - if (!cache_aps_delayed_init) - return; - - set_mtrr(~0U, 0, 0, 0); - cache_aps_delayed_init =3D false; -} - -void mtrr_bp_restore(void) -{ - if (!cache_generic) - return; - - cache_cpu_init(); -} - static int __init mtrr_init_finialize(void) { if (!mtrr_enabled) diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 216fee7144ee..e0e185ee0229 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -1075,7 +1076,7 @@ void __init setup_arch(char **cmdline_p) =20 /* update e820 for memory not covered by WB MTRRs */ if (IS_ENABLED(CONFIG_MTRR)) - mtrr_bp_init(); + cache_bp_init(); else pat_disable("PAT support disabled because CONFIG_MTRR is disabled in the= kernel."); =20 diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index ef7bce21cbe8..ff793f436904 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1445,7 +1445,7 @@ void arch_thaw_secondary_cpus_begin(void) =20 void arch_thaw_secondary_cpus_end(void) { - mtrr_aps_init(); + cache_aps_init(); } =20 /* @@ -1488,7 +1488,7 @@ void __init native_smp_cpus_done(unsigned int max_cpu= s) =20 nmi_selftest(); impress_friends(); - mtrr_aps_init(); + cache_aps_init(); } =20 static int __initdata setup_possible_cpus =3D -1; diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c index bb176c72891c..754221c9a1c3 100644 --- a/arch/x86/power/cpu.c +++ b/arch/x86/power/cpu.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -261,7 +262,7 @@ static void notrace __restore_processor_state(struct sa= ved_context *ctxt) do_fpu_end(); tsc_verify_tsc_adjust(true); x86_platform.restore_sched_clock_state(); - mtrr_bp_restore(); + cache_bp_restore(); perf_restore_debug_store(); =20 c =3D &cpu_data(smp_processor_id()); --=20 2.35.3 From nobody Sun May 19 00:42:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1660987574; cv=none; d=zohomail.com; s=zohoarc; b=PNjMJHf1mUjWF5szniYyKHyJrxyglEiuXyk8Idw5DDhh09H2flzCSluFuOMICWuALs0qYlrvG7bEaesldAnSuF5Q83NPtyMnttpxqYCU4u5rrb7QKh6GET/vSOS/I7prUY8dGwN7S6+Zd4i4ApdrRrN3SNz2QCo6jT3gxdJIJhk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1660987574; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=g4lisHKlxJMHmN7MLyKucg4rSOW6FXqUJI1lqd+zyHc=; b=n2ll94uwMbPqSTQhJiPi+7ubRrh6MRryByuZxLv6wmC/YB09C+Bb78yshS5+LvzD5xhJ9JBBPNFtd06qbfb1kgklxPGIZ4mo5Pttm93gpTVH+6bfGNc03HZSdhXg7eIrfXz4nh2efAO14c34+fSDqyMKVHaTBGygjIgndmDycsE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1660987574281984.9644346625905; Sat, 20 Aug 2022 02:26:14 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.390688.628281 (Exim 4.92) (envelope-from ) id 1oPKjk-0000RX-Gt; Sat, 20 Aug 2022 09:25:48 +0000 Received: by outflank-mailman (output) from mailman id 390688.628281; Sat, 20 Aug 2022 09:25:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjk-0000Nn-76; Sat, 20 Aug 2022 09:25:48 +0000 Received: by outflank-mailman (input) for mailman id 390688; Sat, 20 Aug 2022 09:25:46 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oPKjh-0006NK-KR for xen-devel@lists.xenproject.org; Sat, 20 Aug 2022 09:25:45 +0000 Received: from smtp-out1.suse.de (smtp-out1.suse.de [2001:67c:2178:6::1c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0e930528-206a-11ed-9250-1f966e50362f; Sat, 20 Aug 2022 11:25:40 +0200 (CEST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 134593436B; Sat, 20 Aug 2022 09:25:40 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B69C913B04; Sat, 20 Aug 2022 09:25:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id kMxPK5OoAGPJMAAAMHmgww (envelope-from ); Sat, 20 Aug 2022 09:25:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0e930528-206a-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1660987540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g4lisHKlxJMHmN7MLyKucg4rSOW6FXqUJI1lqd+zyHc=; b=kj/w/8TFOzdZqNCcGJL3EtijbDuNmVRNPtYFAVa+o2+bHMrLc2NUQO5X7c9Uz0/bo4sulB 6HUH5C1stD/XQS2cK7zTpGrumqWbYpRgUWWyTNAmA8ZhQKZxFHXoo4Ld4crAlpgQQWI1VH 6t8torGRK2oftr9UT8AMdU/X+ZFoJGU= From: Juergen Gross To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra Subject: [PATCH v2 10/10] x86: decouple pat and mtrr handling Date: Sat, 20 Aug 2022 11:25:33 +0200 Message-Id: <20220820092533.29420-11-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220820092533.29420-1-jgross@suse.com> References: <20220820092533.29420-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1660987575808100016 Content-Type: text/plain; charset="utf-8" Today PAT is usable only with MTRR being active, with some nasty tweaks to make PAT usable when running as Xen PV guest, which doesn't support MTRR. The reason for this coupling is, that both, PAT MSR changes and MTRR changes, require a similar sequence and so full PAT support was added using the already available MTRR handling. Xen PV PAT handling can work without MTRR, as it just needs to consume the PAT MSR setting done by the hypervisor without the ability and need to change it. This in turn has resulted in a convoluted initialization sequence and wrong decisions regarding cache mode availability due to misguiding PAT availability flags. Fix all of that by allowing to use PAT without MTRR and by reworking the current PAT initialization sequence to match better with the newly introduced generic cache initialization. This removes the need of the recently added pat_force_disabled flag, so remove the remnants of the patch adding it. Signed-off-by: Juergen Gross --- V2: - former patch 3 completely reworked --- arch/x86/include/asm/memtype.h | 5 +- arch/x86/kernel/cpu/cacheinfo.c | 3 +- arch/x86/kernel/cpu/mtrr/mtrr.c | 13 +--- arch/x86/kernel/setup.c | 13 +--- arch/x86/mm/pat/memtype.c | 127 ++++++++++---------------------- 5 files changed, 45 insertions(+), 116 deletions(-) diff --git a/arch/x86/include/asm/memtype.h b/arch/x86/include/asm/memtype.h index 9ca760e430b9..113b2fa51849 100644 --- a/arch/x86/include/asm/memtype.h +++ b/arch/x86/include/asm/memtype.h @@ -6,9 +6,8 @@ #include =20 extern bool pat_enabled(void); -extern void pat_disable(const char *reason); -extern void pat_init(void); -extern void init_cache_modes(void); +extern void pat_bp_init(void); +extern void pat_cpu_init(void); =20 extern int memtype_reserve(u64 start, u64 end, enum page_cache_mode req_pcm, enum page_cache_mode *ret_pcm); diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index 4946f93eb16f..08130919d55d 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -1135,7 +1135,7 @@ static void cache_cpu_init(void) =20 /* Set PAT. */ if (cache_generic & CACHE_GENERIC_PAT) - pat_init(); + pat_cpu_init(); =20 cache_enable(); local_irq_restore(flags); @@ -1154,6 +1154,7 @@ static int cache_rendezvous_handler(void *unused) void __init cache_bp_init(void) { mtrr_bp_init(); + pat_bp_init(); =20 if (cache_generic) cache_cpu_init(); diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtr= r.c index 38531e021581..90aa15610692 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -727,24 +727,15 @@ void __init mtrr_bp_init(void) mtrr_enabled =3D get_mtrr_state(); =20 if (mtrr_enabled) { - cache_generic |=3D CACHE_GENERIC_MTRR | - CACHE_GENERIC_PAT; + cache_generic |=3D CACHE_GENERIC_MTRR; changed_by_mtrr_cleanup =3D mtrr_cleanup(phys_addr); } } } =20 - if (!mtrr_enabled) { + if (!mtrr_enabled) pr_info("Disabled\n"); - - /* - * PAT initialization relies on MTRR's rendezvous handler. - * Skip PAT init until the handler can initialize both - * features independently. - */ - pat_disable("MTRRs disabled, skipping PAT initialization too."); - } } =20 /** diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index e0e185ee0229..aacaa96f0195 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1075,23 +1075,12 @@ void __init setup_arch(char **cmdline_p) max_pfn =3D e820__end_of_ram_pfn(); =20 /* update e820 for memory not covered by WB MTRRs */ - if (IS_ENABLED(CONFIG_MTRR)) - cache_bp_init(); - else - pat_disable("PAT support disabled because CONFIG_MTRR is disabled in the= kernel."); - + cache_bp_init(); if (mtrr_trim_uncached_memory(max_pfn)) max_pfn =3D e820__end_of_ram_pfn(); =20 max_possible_pfn =3D max_pfn; =20 - /* - * This call is required when the CPU does not support PAT. If - * mtrr_bp_init() invoked it already via pat_init() the call has no - * effect. - */ - init_cache_modes(); - /* * Define random base addresses for memory sections after max_pfn is * defined and before each memory section base is used. diff --git a/arch/x86/mm/pat/memtype.c b/arch/x86/mm/pat/memtype.c index 66a209f7eb86..891f7c5f0f09 100644 --- a/arch/x86/mm/pat/memtype.c +++ b/arch/x86/mm/pat/memtype.c @@ -43,6 +43,7 @@ #include =20 #include +#include #include #include #include @@ -60,41 +61,34 @@ #undef pr_fmt #define pr_fmt(fmt) "" fmt =20 -static bool __read_mostly pat_bp_initialized; static bool __read_mostly pat_disabled =3D !IS_ENABLED(CONFIG_X86_PAT); -static bool __initdata pat_force_disabled =3D !IS_ENABLED(CONFIG_X86_PAT); -static bool __read_mostly pat_bp_enabled; -static bool __read_mostly pat_cm_initialized; +static u64 __read_mostly pat_msr_val; =20 /* * PAT support is enabled by default, but can be disabled for * various user-requested or hardware-forced reasons: */ -void pat_disable(const char *msg_reason) +static void __init pat_disable(const char *msg_reason) { if (pat_disabled) return; =20 - if (pat_bp_initialized) { - WARN_ONCE(1, "x86/PAT: PAT cannot be disabled after initialization\n"); - return; - } - pat_disabled =3D true; pr_info("x86/PAT: %s\n", msg_reason); + + cache_generic &=3D ~CACHE_GENERIC_PAT; } =20 static int __init nopat(char *str) { pat_disable("PAT support disabled via boot option."); - pat_force_disabled =3D true; return 0; } early_param("nopat", nopat); =20 bool pat_enabled(void) { - return pat_bp_enabled; + return !pat_disabled; } EXPORT_SYMBOL_GPL(pat_enabled); =20 @@ -192,7 +186,8 @@ enum { =20 #define CM(c) (_PAGE_CACHE_MODE_ ## c) =20 -static enum page_cache_mode pat_get_cache_mode(unsigned pat_val, char *msg) +static enum page_cache_mode __init pat_get_cache_mode(unsigned int pat_val, + char *msg) { enum page_cache_mode cache; char *cache_mode; @@ -219,14 +214,12 @@ static enum page_cache_mode pat_get_cache_mode(unsign= ed pat_val, char *msg) * configuration. * Using lower indices is preferred, so we start with highest index. */ -static void __init_cache_modes(u64 pat) +static void __init init_cache_modes(u64 pat) { enum page_cache_mode cache; char pat_msg[33]; int i; =20 - WARN_ON_ONCE(pat_cm_initialized); - pat_msg[32] =3D 0; for (i =3D 7; i >=3D 0; i--) { cache =3D pat_get_cache_mode((pat >> (i * 8)) & 7, @@ -234,34 +227,11 @@ static void __init_cache_modes(u64 pat) update_cache_mode_entry(i, cache); } pr_info("x86/PAT: Configuration [0-7]: %s\n", pat_msg); - - pat_cm_initialized =3D true; } =20 #define PAT(x, y) ((u64)PAT_ ## y << ((x)*8)) =20 -static void pat_bp_init(u64 pat) -{ - u64 tmp_pat; - - if (!boot_cpu_has(X86_FEATURE_PAT)) { - pat_disable("PAT not supported by the CPU."); - return; - } - - rdmsrl(MSR_IA32_CR_PAT, tmp_pat); - if (!tmp_pat) { - pat_disable("PAT support disabled by the firmware."); - return; - } - - wrmsrl(MSR_IA32_CR_PAT, pat); - pat_bp_enabled =3D true; - - __init_cache_modes(pat); -} - -static void pat_ap_init(u64 pat) +void pat_cpu_init(void) { if (!boot_cpu_has(X86_FEATURE_PAT)) { /* @@ -271,30 +241,35 @@ static void pat_ap_init(u64 pat) panic("x86/PAT: PAT enabled, but not supported by secondary CPU\n"); } =20 - wrmsrl(MSR_IA32_CR_PAT, pat); + wrmsrl(MSR_IA32_CR_PAT, pat_msr_val); } =20 -void __init init_cache_modes(void) +/** + * pat_bp_init - Initialize the PAT MSR value and PAT table + * + * This function initializes PAT MSR value and PAT table with an OS-defined + * value to enable additional cache attributes, WC, WT and WP. + * + * This function prepares the calls of pat_cpu_init() via cache_cpu_init() + * on all cpus. + */ +void __init pat_bp_init(void) { u64 pat =3D 0; + struct cpuinfo_x86 *c =3D &boot_cpu_data; =20 - if (pat_cm_initialized) - return; +#ifndef CONFIG_X86_PAT + pr_info_once("x86/PAT: PAT support disabled because CONFIG_X86_PAT is dis= abled in the kernel.\n"); +#endif =20 - if (boot_cpu_has(X86_FEATURE_PAT)) { - /* - * CPU supports PAT. Set PAT table to be consistent with - * PAT MSR. This case supports "nopat" boot option, and - * virtual machine environments which support PAT without - * MTRRs. In specific, Xen has unique setup to PAT MSR. - * - * If PAT MSR returns 0, it is considered invalid and emulates - * as No PAT. - */ + if (!boot_cpu_has(X86_FEATURE_PAT)) + pat_disable("PAT not supported by the CPU."); + else rdmsrl(MSR_IA32_CR_PAT, pat); - } =20 if (!pat) { + pat_disable("PAT support disabled by the firmware."); + /* * No PAT. Emulate the PAT table that corresponds to the two * cache bits, PWT (Write Through) and PCD (Cache Disable). @@ -315,38 +290,14 @@ void __init init_cache_modes(void) */ pat =3D PAT(0, WB) | PAT(1, WT) | PAT(2, UC_MINUS) | PAT(3, UC) | PAT(4, WB) | PAT(5, WT) | PAT(6, UC_MINUS) | PAT(7, UC); - } else if (!pat_force_disabled && cpu_feature_enabled(X86_FEATURE_HYPERVI= SOR)) { - /* - * Clearly PAT is enabled underneath. Allow pat_enabled() to - * reflect this. - */ - pat_bp_enabled =3D true; } =20 - __init_cache_modes(pat); -} - -/** - * pat_init - Initialize the PAT MSR and PAT table on the current CPU - * - * This function initializes PAT MSR and PAT table with an OS-defined value - * to enable additional cache attributes, WC, WT and WP. - * - * This function must be called on all CPUs using the specific sequence of - * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this - * procedure for PAT. - */ -void pat_init(void) -{ - u64 pat; - struct cpuinfo_x86 *c =3D &boot_cpu_data; - -#ifndef CONFIG_X86_PAT - pr_info_once("x86/PAT: PAT support disabled because CONFIG_X86_PAT is dis= abled in the kernel.\n"); -#endif + /* Xen PV doesn't allow to set PAT MSR, but all cache modes are fine. */ + if (pat_disabled || cpu_feature_enabled(X86_FEATURE_XENPV)) { + init_cache_modes(pat); =20 - if (pat_disabled) return; + } =20 if ((c->x86_vendor =3D=3D X86_VENDOR_INTEL) && (((c->x86 =3D=3D 0x6) && (c->x86_model <=3D 0xd)) || @@ -404,12 +355,10 @@ void pat_init(void) PAT(4, WB) | PAT(5, WP) | PAT(6, UC_MINUS) | PAT(7, WT); } =20 - if (!pat_bp_initialized) { - pat_bp_init(pat); - pat_bp_initialized =3D true; - } else { - pat_ap_init(pat); - } + pat_msr_val =3D pat; + cache_generic |=3D CACHE_GENERIC_PAT; + + init_cache_modes(pat); } =20 #undef PAT --=20 2.35.3