From nobody Tue Dec 9 03:03:07 2025 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8B2E28248C; Thu, 13 Nov 2025 05:21:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=78.133.224.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763011275; cv=none; b=LDDuB0zcoCxeA1GfPc6Y55xGM2fSv9g1W+XVhJ7tzKs21L5zVrXxuUO3WP6ZPVFBAq8S5XBXQ9bNANJAHPipoILijUmTaRAVVIRLRq7xB8XnuGQNX+YcoNSZp7uDD3gURbmKP7QeqCC4RhqleCQihjEh6amjIsGW7CAEcSUXn2E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763011275; c=relaxed/simple; bh=jO8jB74KjHuUDAYmA+SP2Z73Mt+40xwFkVV1azxk90w=; h=Date:From:To:cc:Subject:Message-ID:MIME-Version:Content-Type; b=rJn8sxFN8YciMZ3a9pPvUlfxyzjONeCN6v4wUgefutueMFLt33TDHY1DpJGlf9v1kXUQWLrdeZGxyYOsaHAOSD8/XcNmRUMTsyStWqLFszp0mCAMn7mOVEmKLFF0WU3Pxm57LsDmxz9w6tqe8+nVUBFq3YXxvGbJ5gBIh+oVzuM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=orcam.me.uk; spf=none smtp.mailfrom=orcam.me.uk; arc=none smtp.client-ip=78.133.224.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=orcam.me.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=orcam.me.uk Received: by angie.orcam.me.uk (Postfix, from userid 500) id 8975B92009C; Thu, 13 Nov 2025 06:21:10 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 7944C92009B; Thu, 13 Nov 2025 05:21:10 +0000 (GMT) Date: Thu, 13 Nov 2025 05:21:10 +0000 (GMT) From: "Maciej W. Rozycki" To: Nick Bowler , Thomas Bogendoerfer , Jiaxun Yang cc: linux-mips@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3] MIPS: mm: Prevent a TLB shutdown on initial uniquification Message-ID: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Depending on the particular CPU implementation a TLB shutdown may occur=20 if multiple matching entries are detected upon the execution of a TLBP=20 or the TLBWI/TLBWR instructions. Given that we don't know what entries=20 we have been handed we need to be very careful with the initial TLB=20 setup and avoid all these instructions. Therefore read all the TLB entries one by one with the TLBR instruction,=20 bypassing the content addressing logic, and truncate any large pages in=20 place so as to avoid a case in the second step where an incoming entry=20 for a large page at a lower address overlaps with a replacement entry=20 chosen at another index. Then preinitialize the TLB using addresses=20 outside our usual unique range and avoiding clashes with any entries=20 received, before making the usual call to local_flush_tlb_all(). This fixes (at least) R4x00 cores if TLBP hits multiple matching TLB=20 entries (SGI IP22 PROM for examples sets up all TLBs to the same virtual=20 address). Signed-off-by: Maciej W. Rozycki Fixes: 35ad7e181541 ("MIPS: mm: tlb-r4k: Uniquify TLB entries on init") Cc: stable@vger.kernel.org # v6.17+ Reviewed-by: Jiaxun Yang Tested-by: Jiaxun Yang # Boston I6400, M5150 sim --- Hi, On second thoughts I decided against including wired entries in our VPN=20 matching discovery for clash avoidance. The reason is as I wrote before=20 it makes no sense to have wired entries for KSEG0 addresses, so it should=20 be safe to assume we won't ever make one, and then if a wired entry maps a=20 large page, which is quite likely, then our clash avoidance logic won't=20 handle an overlap where the two VPN values of a clashing pair don't match,=20 so it makes no sense to pretend we can handle wired entries with the code=20 as proposed. Pasting v2 discussion below verbatim as it still applies. Verified the same way as before, also with some diagnostics added so as=20 to make sure things get set up correctly, with my Malta/74Kf system for a=20 32-bit configuration and with my SWARM/BCM1250 system for a 64-bit one. In addition to the Wired register setup discussed with v1 I have realised=20 the incoming entries may include large pages, possibly exceeding the size=20 of KSEG0 even. Such entries may overlap with our temporary entries added=20 in the second step, so truncate any large pages in place as this ensures=20 no clash happens with the received contents of the TLB. NB this doesn't handle incoming PageGrain.ESP having been set, but it's=20 an unrelated preexisting issue that would have to be handled elsewhere. =20 Possibly it doesn't matter in reality. Additionally PageMask is left set at what has been retrieved from the=20 last incoming TLB entry in the first step and has to be reset to our page=20 size before proceeding with the second step. And last but not least the comparator function returned 0 incorrectly=20 when the difference between 64-bit elements was positive but with none of=20 the high-order 32 bits set. Fixed with a branchless sequence of 3 machine=20 instructions, which I think is the minimum here (only the sign and zero=20 matter here, but this sequence actually produces -1/0/1, because why not). No change for the 32-bit case, the difference is returned as is. Maciej Changes from v2 (at=20 ): - Revert the v2 update to include wired entries while reading original=20 contents of TLB. Changes from v1 (at=20 ): - Also include wired entries while reading original contents of TLB. - Truncate any large pages in place while reading original TLB entries. - Reset PageMask to PM_DEFAULT_MASK after reading in TLB entries. - Fix the 64-bit case for the sort comparator. --- arch/mips/mm/tlb-r4k.c | 100 ++++++++++++++++++++++++++++++--------------= ----- 1 file changed, 63 insertions(+), 37 deletions(-) linux-mips-tlb-r4k-uniquify-fix.diff Index: linux-swarm64/arch/mips/mm/tlb-r4k.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-swarm64.orig/arch/mips/mm/tlb-r4k.c +++ linux-swarm64/arch/mips/mm/tlb-r4k.c @@ -15,6 +15,7 @@ #include #include #include +#include =20 #include #include @@ -508,54 +509,78 @@ static int __init set_ntlb(char *str) =20 __setup("ntlb=3D", set_ntlb); =20 -/* Initialise all TLB entries with unique values */ + +/* Comparison function for EntryHi VPN fields. */ +static int r4k_vpn_cmp(const void *a, const void *b) +{ + long v =3D *(unsigned long *)a - *(unsigned long *)b; + int s =3D sizeof(long) > sizeof(int) ? sizeof(long) * 8 - 1: 0; + return s ? (v !=3D 0) | v >> s : v; +} + +/* + * Initialise all TLB entries with unique values that do not clash with + * what we have been handed over and what we'll be using ourselves. + */ static void r4k_tlb_uniquify(void) { - int entry =3D num_wired_entries(); + unsigned long tlb_vpns[1 << MIPS_CONF1_TLBS_SIZE]; + int tlbsize =3D current_cpu_data.tlbsize; + int start =3D num_wired_entries(); + unsigned long vpn_mask; + int cnt, ent, idx, i; + + vpn_mask =3D GENMASK(cpu_vmbits - 1, 13); + vpn_mask |=3D IS_ENABLED(CONFIG_64BIT) ? 3ULL << 62 : 1 << 31; =20 htw_stop(); + + for (i =3D start, cnt =3D 0; i < tlbsize; i++, cnt++) { + unsigned long vpn; + + write_c0_index(i); + mtc0_tlbr_hazard(); + tlb_read(); + tlb_read_hazard(); + vpn =3D read_c0_entryhi(); + vpn &=3D vpn_mask & PAGE_MASK; + tlb_vpns[cnt] =3D vpn; + + /* Prevent any large pages from overlapping regular ones. */ + write_c0_pagemask(read_c0_pagemask() & PM_DEFAULT_MASK); + mtc0_tlbw_hazard(); + tlb_write_indexed(); + tlbw_use_hazard(); + } + + sort(tlb_vpns, cnt, sizeof(tlb_vpns[0]), r4k_vpn_cmp, NULL); + + write_c0_pagemask(PM_DEFAULT_MASK); write_c0_entrylo0(0); write_c0_entrylo1(0); =20 - while (entry < current_cpu_data.tlbsize) { - unsigned long asid_mask =3D cpu_asid_mask(¤t_cpu_data); - unsigned long asid =3D 0; - int idx; + idx =3D 0; + ent =3D tlbsize; + for (i =3D start; i < tlbsize; i++) + while (1) { + unsigned long entryhi, vpn; =20 - /* Skip wired MMID to make ginvt_mmid work */ - if (cpu_has_mmid) - asid =3D MMID_KERNEL_WIRED + 1; + entryhi =3D UNIQUE_ENTRYHI(ent); + vpn =3D entryhi & vpn_mask & PAGE_MASK; =20 - /* Check for match before using UNIQUE_ENTRYHI */ - do { - if (cpu_has_mmid) { - write_c0_memorymapid(asid); - write_c0_entryhi(UNIQUE_ENTRYHI(entry)); + if (idx >=3D cnt || vpn < tlb_vpns[idx]) { + write_c0_entryhi(entryhi); + write_c0_index(i); + mtc0_tlbw_hazard(); + tlb_write_indexed(); + ent++; + break; + } else if (vpn =3D=3D tlb_vpns[idx]) { + ent++; } else { - write_c0_entryhi(UNIQUE_ENTRYHI(entry) | asid); + idx++; } - mtc0_tlbw_hazard(); - tlb_probe(); - tlb_probe_hazard(); - idx =3D read_c0_index(); - /* No match or match is on current entry */ - if (idx < 0 || idx =3D=3D entry) - break; - /* - * If we hit a match, we need to try again with - * a different ASID. - */ - asid++; - } while (asid < asid_mask); - - if (idx >=3D 0 && idx !=3D entry) - panic("Unable to uniquify TLB entry %d", idx); - - write_c0_index(entry); - mtc0_tlbw_hazard(); - tlb_write_indexed(); - entry++; - } + } =20 tlbw_use_hazard(); htw_start(); @@ -602,6 +627,7 @@ static void r4k_tlb_configure(void) =20 /* From this point on the ARC firmware is dead. */ r4k_tlb_uniquify(); + local_flush_tlb_all(); =20 /* Did I tell you that ARC SUCKS? */ }