From nobody Mon Dec 1 22:36:54 2025 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E25E332EB2 for ; Thu, 27 Nov 2025 14:11:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764252712; cv=none; b=YXEQ14m/hCnHIaleUJ++9Zgg0DrzyWymHN9iByO1WOIFy9sPDGTH7UgQKH2hTTWEoENT9wq04baZA1VGDDdkWYLQsvV2Kcynu5v0JIBAhNX/HjPGHC23lHAlRHuXUGcQRPKsdlWjsUhcBvPWStpGjhRWVjLZmPdxCsT0oRFQIoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764252712; c=relaxed/simple; bh=6WgF+HUgdtvQQTctNmbv7ojAGa2BGON60tch8qSWMbg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ntyjz0xzndrJ4I+oLg3xeo5ze+o2IITwz3Uv895VRW283fjwruZI1MxLU5BdxDv9U8x+kiPhADCL/kPepcfL7iC+5lkzogUqfUn1Naurcxw15/WfYUsyVWNYHaFaOk+o8XbR/3k28vea3IwU3l7ln4amv03quYaMc1OAV6yazKE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=Df/MJMnQ; arc=none smtp.client-ip=209.85.216.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="Df/MJMnQ" Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-3434700be69so1217415a91.1 for ; Thu, 27 Nov 2025 06:11:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1764252710; x=1764857510; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OGHgnOknc2HR28gygrywMGZEYzdqi18gD0LqvxY1V9E=; b=Df/MJMnQBdMFBRz3kBeHlN4njecNvUYRp9XTAq4qul2GOSyjIajxkBcBc47cAzg/Z6 xEvrlCcff8SaI0kHns/oqVYSr0a4vMt+peIHpbgkTFEm3/41QpVxHnAEwYK4gLRMD7UK vLHNSoLxC0iVoPHD2K8tu2F7TD8zPYZLLr9CoNkZ2uueS6GgGBblPQPd/R0P+yBt4UNu 03r6ick5x2dnK2dZqGY51S8usxQte/pzbw3OaR/vgLoYsYBvxKjmnD+fUuddS0B7APMy DYYwUxStCC2IWRO+xlmV1u7Rnh529p8T0nk6X66g9xWN4PKtDyVUutXH+bG8Vxdt5Tdd 4y5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764252710; x=1764857510; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=OGHgnOknc2HR28gygrywMGZEYzdqi18gD0LqvxY1V9E=; b=O8f+bfC0BZdwFXuNV8frvC1D0uLVamBPr4jlhQEc+xR/jVhsHtakG7NilwbFF06tZM r/YFJ3/z79ROvvQa4kEbJLqIMj5J5tOWoi9YfB95T1GHPYYk6orcdCnS2o84eoMZ1nr4 /py5rYxP/87dvLdwAuj6pWon/R+AhWA6aHOgI/h8I6XT8YnkVGsD9KJ8RQJsY0rlbHcE cIXPW9JlW0u4l6lP6Odap2xiG3EXvBoR6M0M2o13ygB602CQt8i+c52U/FpWJArOXZ5Z 2GrQYEKZ2tPhTDhRmfbXwSeD9orVIjdHszrwK0wBi46AcT+1DE9Lu4IpfHTDytjjxj5x HFwQ== X-Forwarded-Encrypted: i=1; AJvYcCU2btPMRkYnVAel3aaXO5tSf2O5HvQ7MykEQVCSf/lA/2a6Iz+gg7hXyOJaALHfGu9jqk+NllzsAEHAKY4=@vger.kernel.org X-Gm-Message-State: AOJu0YzmhLRcbthRSwwhzcqikUyhwgdkin7UGrO4Kc82reXK4g7cSNrN sZ1Vpfm9Jw7g2LDcOJ8amxTgKWLCO2S6drkM/nNlVu0oTDNPWljzeLpPnAdcfL+GAns= X-Gm-Gg: ASbGncvMGqBwgV0JiSvG51TJ0u0lE8xI5AiEp9z12aWBHspBGasSDZ2OBARqoUX1Aqs 7KHLGv4EHu62h7HD7qMT5OYGntOwWPrUttoacDxvEmSjR1U6nRMN587HWJLBmb4Io44XPCUSbD2 GwuxVZPAufTFZaxpcEL+OyO7eKeMdGyLbvOBJUAqt/hicBPXv8NKViZf9dTXpBTNq+MTOyjYqY3 kCe/kWFd2oaSV4phctM858M/vNtoDiHk/AIwSem2nR+Nhv5sx7gWZDWi+dvqLnHxsqGgwHuh6rC j5SJFCEr4J5YEWQnfeGFIFDLQ5Z2rDV6F30EFQM9z+9LViX5+mPcrS53/XXS2iPIhj28+eDw5nP t648G22N/u7+MBKAEXc3oCjqIu3HxCRMZ5MWKVCPH2i9Y3Nk37rNK5HuWNsBMmevRnaNQjunhzh 8vhHbLiMm0v/iLaU9lFa9h0bBSVBwxjD9CmClpadGMZh17+kOzDJSXSFoUHyuPgisTIOMqZ99ke w== X-Google-Smtp-Source: AGHT+IH9shCFmiY257H7nfVJqHK2TfMQ1cqSxBQ+VJIOMrKykl/Xamof7TK0iwdrGZLERz9FgKIyJw== X-Received: by 2002:a17:90b:4c8f:b0:341:133:e13d with SMTP id 98e67ed59e1d1-3475ebd2d92mr11210465a91.5.1764252710205; Thu, 27 Nov 2025 06:11:50 -0800 (PST) Received: from J9GPGXL7NT.bytedance.net ([61.213.176.58]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3477b7341d2sm2030249a91.11.2025.11.27.06.11.45 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 27 Nov 2025 06:11:49 -0800 (PST) From: Xu Lu To: pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, kees@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, akpm@linux-foundation.org, david@redhat.com, apatel@ventanamicro.com, guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Xu Lu Subject: [RFC PATCH v2 2/9] riscv: mm: Apply a threshold to the number of active ASIDs on each CPU Date: Thu, 27 Nov 2025 22:11:10 +0800 Message-ID: <20251127141117.87420-3-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251127141117.87420-1-luxu.kernel@bytedance.com> References: <20251127141117.87420-1-luxu.kernel@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since each CPU has limited TLB entries, there exist limited active ASIDs in each CPU's TLB at the same time. Thus we apply a threshold here. When a mm_struct is loaded, we mark its ASID as active. If the number of active ASIDs exceeds the threshold, we evict the mm_struct that has not been used for the longest time, flush its TLB entries, mark its ASID inactive, and clear current CPU in its mm_cpumask. Signed-off-by: Xu Lu --- arch/riscv/include/asm/tlbflush.h | 27 +++++++++++++ arch/riscv/mm/context.c | 1 + arch/riscv/mm/tlbflush.c | 66 +++++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlb= flush.h index eed0abc405143..3f83fd5ef36db 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -66,6 +66,33 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unma= p_batch *batch, void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); =20 extern unsigned long tlb_flush_all_threshold; + +#ifdef CONFIG_RISCV_LAZY_TLB_FLUSH + +#define MAX_LOADED_MM 6 + +struct tlb_context { + struct mm_struct *mm; + unsigned int gen; +}; + +struct tlb_info { + rwlock_t rwlock; + struct mm_struct *active_mm; + unsigned int next_gen; + struct tlb_context contexts[MAX_LOADED_MM]; +}; + +DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_info, tlbinfo); + +void local_load_tlb_mm(struct mm_struct *mm); + +#else /* CONFIG_RISCV_LAZY_TLB_FLUSH */ + +static inline void local_load_tlb_mm(struct mm_struct *mm) {} + +#endif /* CONFIG_RISCV_LAZY_TLB_FLUSH */ + #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 55c20ad1f7444..a7cf36ad34678 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -217,6 +217,7 @@ static inline void set_mm(struct mm_struct *prev, */ cpumask_set_cpu(cpu, mm_cpumask(next)); if (static_branch_unlikely(&use_asid_allocator)) { + local_load_tlb_mm(next); set_mm_asid(next, cpu); } else { cpumask_clear_cpu(cpu, mm_cpumask(prev)); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 8404530ec00f9..0b1c21c7aafb8 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -103,6 +103,15 @@ struct flush_tlb_range_data { unsigned long stride; }; =20 +#ifdef CONFIG_RISCV_LAZY_TLB_FLUSH +DEFINE_PER_CPU_SHARED_ALIGNED(struct tlb_info, tlbinfo) =3D { + .rwlock =3D __RW_LOCK_UNLOCKED(tlbinfo.rwlock), + .active_mm =3D NULL, + .next_gen =3D 1, + .contexts =3D { { NULL, 0, }, }, +}; +#endif /* CONFIG_RISCV_LAZY_TLB_FLUSH */ + static void __ipi_flush_tlb_range_asid(void *info) { struct flush_tlb_range_data *d =3D info; @@ -240,3 +249,60 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_ba= tch *batch) 0, FLUSH_TLB_MAX_SIZE, PAGE_SIZE); cpumask_clear(&batch->cpumask); } + +#ifdef CONFIG_RISCV_LAZY_TLB_FLUSH + +static inline unsigned int new_tlb_gen(struct tlb_info *info) +{ + unsigned int gen =3D info->next_gen++; + unsigned int i; + + if (unlikely(!info->next_gen)) { + for (i =3D 0; i < MAX_LOADED_MM; i++) { + if (info->contexts[i].gen) + info->contexts[i].gen =3D 1; + } + info->next_gen =3D 1; + gen =3D info->next_gen++; + } + + return gen; +} + +void local_load_tlb_mm(struct mm_struct *mm) +{ + struct tlb_info *info =3D this_cpu_ptr(&tlbinfo); + struct tlb_context *contexts =3D info->contexts; + struct mm_struct *victim =3D NULL; + unsigned int i, pos =3D 0, min =3D UINT_MAX; + + for (i =3D 0; i < MAX_LOADED_MM; i++) { + if (contexts[i].mm =3D=3D mm) { + pos =3D i; + break; + } + if (min > contexts[i].gen) { + min =3D contexts[i].gen; + pos =3D i; + } + } + + write_lock(&info->rwlock); + + info->active_mm =3D mm; + + if (contexts[pos].mm !=3D mm) { + victim =3D contexts[pos].mm; + contexts[pos].mm =3D mm; + } + contexts[pos].gen =3D new_tlb_gen(info); + + write_unlock(&info->rwlock); + + if (victim) { + cpumask_clear_cpu(raw_smp_processor_id(), mm_cpumask(victim)); + local_flush_tlb_all_asid(get_mm_asid(victim)); + } +} + +#endif /* CONFIG_RISCV_LAZY_TLB_FLUSH */ --=20 2.20.1