From nobody Sat Feb 7 17:04:24 2026 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7FEE242D78 for ; Thu, 30 Oct 2025 13:57:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832629; cv=none; b=h4Il8E+ZeW84ILTAlYfs3eOLTWCJNANE/qCPKtidLdw5p2zxpeeRoGGlRd7Bm8/TBz7JFNeqOQOcEuVwwsqGwiiXRW/IgAqOUUGhCTWkE+oo3ICbZmYyz25ZFSqqM/TYxvO6PUzCaood1/idZ5KUpckmGdiEuKpiF0AomJWoWGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832629; c=relaxed/simple; bh=s5iozbZgWNVtTbraAacnZCEwXQ1F+RRaD5pBFzrjkIg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LV/Dlz+ut4tvgnjAc9pI4y5saZGpu1dXBSxMlhpPMCeC8WbpG2P1ddVLORGBq4ecr5DC5wHvh29ywDmvXeBm6ZDJqBVBWsIwYxC3DxEUElWipiWeuM8LyDGKt2bqnyDlMIonl9OB5SSxN4+lvk3zUrR4vtX6tx3X8KyN33O9WAk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=QFP0nzRi; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="QFP0nzRi" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-27c369f8986so10225695ad.3 for ; Thu, 30 Oct 2025 06:57:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1761832627; x=1762437427; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c8VyXh52m2HmF6WkbMPxGheT256pmfI79AWNunisbZ0=; b=QFP0nzRiwmJbw11lW6uJCrT99+qUXIQw9kg3z6zjZOoAuQk6Px6T3Gx/l70aRmxADx hYeXBqZIe1sOtV192vI9mXxnpNTuWXUXenHP7f2TeTKhGNoZ6Maz9iOasAVekH2AZ1wY MBsA4CbQUnEU7nJobfjGeXC1/MuSDW4OiHyUxaGEpWrYbE9AReveyTXB9s9ibAQ1UbF/ HEx7Hd4hNDmeIQPEeI/fO1E+2mBkrYJVuJtDUvDsN+X+UzuByzkV0A+1afwi2FNogY1r IOl2oIhINeOUnSdgls1W/NPwh0BYFUH5atw0+P4GavoDC6xmtFQsPZJAk+PyWBee+E87 68ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761832627; x=1762437427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c8VyXh52m2HmF6WkbMPxGheT256pmfI79AWNunisbZ0=; b=Rv+52st7BvmmnFqrVSbznbeEJlAI4Q0KqnM9aqAPDjxjCnMlEqy3tBxK6apiJEPFdX h4P6RchrcPqVBby0flB5ppbawPVfRpjSQ1gnX66qOUr+B5rPWr6raFHSTIWSQy3FuQvz ZMHd4fOC95iV6rOkYPctiy7PXqOxc8mjLK1burufkdE6NZFTVP0ux9XglykraBBUaq7O Cf/QyuLiPjACVT6MkUS6u8FyjexI+YTyyR1SjBmB1XK2n0wyzdp8DkhRC/qxzzyAon7H 0TjJSCB2Bvrxjj+xaheHeL3GMeDndc++FWcZswrTWGtVgid//CwGJX040PFO0dC/txt1 jpsw== X-Forwarded-Encrypted: i=1; AJvYcCXhtvrhdivxVThLierxakDmn00QIGbC1PV+WPpNwVmKdtz9ZWxioqdhbJHJ9kZTSKmRc7hYOsw8jMchEjs=@vger.kernel.org X-Gm-Message-State: AOJu0YxP2T9ghxUXlwcpUre5dAatF/+rFGzaEz4cFvmPU1w8lhyFKGmL vKcNKG3NAWiISr5+pT31JfAcr9YFhFnc9aQE7bB6iw9ahIdq7u+2v2Uo2mtbAWHxtqI= X-Gm-Gg: ASbGncvdjvviYcUPyMd0P6po83tLViUvqzrJmKwkZwxcGcWNhE8g/7tb/nEycIAm7VH ZOXHDvKSSwJHcg2L/XgKlyTTu8fmfV4b4fyZO7m8oJRnKslx7YOcHOizEmndjiSHa2ocZqREVnC h4lGHhinOYqOvHiztlKBfykfTt1TnHnlmUgk9PCCozqNCtXlrhF+YbumGlmkzhDiE8nxKk77GKK uP1/qQEmL3M4/RaAAr7ob3dkLAzOtWp+v0aCUVe+wN5lifphIWMju1BZsvQoPgF7qJs2G/22pvW OmWEdlSnh097VlZD681jhJngSDsKaIbM1JXt/GZxqOH7WOb5FgljBcW0eRrcFHOQfxKBNPN6sbm CJ8CqcpYpL6OPREQcNuc4JInkbcnVZUmO/qupdpbpAJPmmd9g5ZSmJZ9aolu5+OcmTE2EFS4zES jcgHENdXTGEyvMV17wNAhlE47hU53gpUMD/xOPQ594G0qEZggt9JSga0oTA17SmAY= X-Google-Smtp-Source: AGHT+IGCoAuBzTrQiSWc/Ld2jXG5kw739zrV1CLzqkJDo2K/6TZfdSsStOQfI7qh++gzFJ3h/uvC5g== X-Received: by 2002:a17:902:f550:b0:277:3488:787e with SMTP id d9443c01a7336-294dee0cebemr84296405ad.12.1761832626724; Thu, 30 Oct 2025 06:57:06 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([61.213.176.55]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29498e495e8sm187071905ad.110.2025.10.30.06.57.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 30 Oct 2025 06:57:06 -0700 (PDT) From: Xu Lu To: pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, apatel@ventanamicro.com, guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Xu Lu Subject: [RFC PATCH v1 1/4] riscv: mm: Introduce percpu loaded_asid Date: Thu, 30 Oct 2025 21:56:49 +0800 Message-ID: <20251030135652.63837-2-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251030135652.63837-1-luxu.kernel@bytedance.com> References: <20251030135652.63837-1-luxu.kernel@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The percpu loaded_asid records the asid currently used by each CPU. Signed-off-by: Xu Lu --- arch/riscv/include/asm/mmu_context.h | 1 + arch/riscv/mm/context.c | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/arch/riscv/include/asm/mmu_context.h b/arch/riscv/include/asm/= mmu_context.h index 8c4bc49a3a0f5..fd532f8e8d057 100644 --- a/arch/riscv/include/asm/mmu_context.h +++ b/arch/riscv/include/asm/mmu_context.h @@ -39,6 +39,7 @@ static inline int init_new_context(struct task_struct *ts= k, } =20 DECLARE_STATIC_KEY_FALSE(use_asid_allocator); +DECLARE_PER_CPU(unsigned long, loaded_asid); =20 #ifdef CONFIG_RISCV_ISA_SUPM #define mm_untag_mask mm_untag_mask diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 55c20ad1f7444..4d5792c3a8c19 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -32,6 +32,8 @@ static unsigned long *context_asid_map; static DEFINE_PER_CPU(atomic_long_t, active_context); static DEFINE_PER_CPU(unsigned long, reserved_context); =20 +DEFINE_PER_CPU(unsigned long, loaded_asid) =3D 0; + static bool check_update_reserved_context(unsigned long cntx, unsigned long newcntx) { @@ -193,6 +195,8 @@ static void set_mm_asid(struct mm_struct *mm, unsigned = int cpu) (cntx2asid(cntx) << SATP_ASID_SHIFT) | satp_mode); =20 + this_cpu_write(loaded_asid, cntx2asid(cntx)); + if (need_flush_tlb) local_flush_tlb_all(); } --=20 2.20.1 From nobody Sat Feb 7 17:04:24 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42AB1257825 for ; Thu, 30 Oct 2025 13:57:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832633; cv=none; b=UbnWWrnJFyHF7GDchOnv/QWeRjjZsYC4m6FVtqWJN3nBCkZd0exHEg+dNcd40PVbKBr5vUFOA6d7nLsnkk5jnM7atxYdgYZtprFGR9SVuxVDh4n5yQVECcJapjD7gu7MLirI+ZSfJJlJY+feult3UNxHeGKss7GET3GtdVCDInU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832633; c=relaxed/simple; bh=sklcjzEcUTVDi/Q5wjy6gWoZ9JYYCnW2Ie2EGD2t+f4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kWL/ZG8jvMSKERI/9EIcj2Bvq4E9EpWfGgq6XfUjl1i5mYQIloblzLiA4d9P4Iaox+pJtT1o1EghVb42OdHgqoGTR7s7M/SikcpDqAW/3p21u5fMEnYpCzXi1sGbhkoaK7grv8EPnGWzVXbfKoNOc/NHxbMe2/zoR2/UjaXyo7k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=CL20xqFb; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="CL20xqFb" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-78af3fe5b17so1099941b3a.2 for ; Thu, 30 Oct 2025 06:57:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1761832631; x=1762437431; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=idtv0RG1n1Xu/2siY10CA00hyxq2ZPaiF1nMz1aq8l0=; b=CL20xqFbkYfYrwMgWT//8vyg6J71GVkJqhtlN9e44hQSeuPNH2Ag8EnFi5nGHow1yR omDqYyZBvh4/FB4rsUDfobip57jupY/Q9mOvGGzzYmXhxA3N5rnKIjpkL1Y+hq+N96gY 3gGpLoJUxYtnGnispysx2CrPsCspPLLe5HW9fy+Rf+8UqmY3rlY1jSsdt/EO5ulNoHvN d2SflKSb56b/FKRxGwCk/3zT1I7MvwBey1gfYx7gmFsusWyX5TN1u3GVZYj5Elh/f1oc RSZiXyM4jj/XFN66yTWrTtv0IpLKxZKTs+fCyQGV2qi4ATGzxVfce5D7QY0fSsa0oK2F CJ3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761832631; x=1762437431; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=idtv0RG1n1Xu/2siY10CA00hyxq2ZPaiF1nMz1aq8l0=; b=LiKEmPLaNKUT3Sbe8rD1Nhc+cdgTec9fgaJsoZ1Rx0WWRQQoDBBEmwyI2XoQlrhCWO o+6qP84dYfWjLEVjdRXOHCYhB/HfWQL/DbW4PumcCjI099s+MtMMNkHGwvTQpZVul5eW UCvhyTlkjp0EO5dM/aHA1axJUxrRBCYooc/Vf82bnJbh7FTr0bB5cAicjPgNaBPdbPa/ LIxaniVBYQp44qWu5XV3x1nbfxh9pfbPx8G3l/RxwRwg45L5sDQT7G768h+wK0bZcIvY IZL+t9CqFZXDBPeHnemeVjEHC24BHByPS6e8KGLHRByJqxZsrL9dvzlorR1fN1DkWtCS 6FgA== X-Forwarded-Encrypted: i=1; AJvYcCVju+FeHjoE72jjWPjSEJhNZHppQilmctg+DymIC8DJabM9X6bKg+zJrRjRkcoXfaax7/YOjKcZxrA7He4=@vger.kernel.org X-Gm-Message-State: AOJu0Yys8L+XMlm+M1vALj+nmrBkH4Mz7Ye5CYqZFkTk25ky/BQ/n3bI dcV3Jz9e0bqX8ct4cFEXP1WJ1MleuF7I6GmDMIylrgOtS6fPF88ncIFj4oHpbTGHqw4= X-Gm-Gg: ASbGncsIbQ5by46XqL22KklIjoCHLODu6sAqAOMSKExkXU6RApmtPbKCOVKxkbDJq0z oB+6QPAHP7EsjO5HexD0eX9DMAo2bl0CLmFDqVRAc7PLUFw8yW9Nt5fDQ0dTa3gJ8OASRlCs5NC pr8pzCzGZR/koiQd2rF4RivDy6Ch7kjm33vAmwApfYVljMQWInQFw3kSaQmTPmMda3AYPHXGwx1 0DXTolByWEpFZiH/zVVb3/KDIumeoIrQah/b4e0YPyF9p1imNVQrrU/uuD8tkLvHrh0hX+OLToN 6Juui+BfxbckPtfJuU92gWve+sSmKoY4uGeq08qBj1TjQhYyH6JLiNagQts/KYZo8MHkN9NzGws IsxtO78Qn+qomR4I3MQfMicV2oFlu5lW3SrpKmc5kz3K/yxaRa7AhGk/S3h60elCW39BigpHkxf lbon5MIXAPAsNb8JbWi6byJKchtVQFazRQrFVOkIxcADvVo744A0c//n/rctwCza0= X-Google-Smtp-Source: AGHT+IGbOUAtM2+Lu3qbpwcnntpyy8m+vA1T9owmvZzuSyODN0ovPcBr/za3iDV2dG9ZYLAg2yY1uw== X-Received: by 2002:a17:902:f682:b0:294:ec58:1d25 with SMTP id d9443c01a7336-294ee0a0fb8mr33408495ad.50.1761832631190; Thu, 30 Oct 2025 06:57:11 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([61.213.176.55]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29498e495e8sm187071905ad.110.2025.10.30.06.57.07 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 30 Oct 2025 06:57:10 -0700 (PDT) From: Xu Lu To: pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, apatel@ventanamicro.com, guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Xu Lu Subject: [RFC PATCH v1 2/4] riscv: mm: Introduce percpu tlb flush queue Date: Thu, 30 Oct 2025 21:56:50 +0800 Message-ID: <20251030135652.63837-3-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251030135652.63837-1-luxu.kernel@bytedance.com> References: <20251030135652.63837-1-luxu.kernel@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The percpu tlb flush queue is used to buffer the tlb flush tasks that each cpu needs to process. Signed-off-by: Xu Lu --- arch/riscv/mm/tlbflush.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 8404530ec00f9..aa8f1304ae5c4 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -103,6 +103,18 @@ struct flush_tlb_range_data { unsigned long stride; }; =20 +#define TLB_FLUSH_QUEUE_SIZE 16 +struct tlb_flush_queue { + struct flush_tlb_range_data tasks[TLB_FLUSH_QUEUE_SIZE]; + raw_spinlock_t lock; + unsigned int len; +}; + +DEFINE_PER_CPU(struct tlb_flush_queue, tlb_flush_queue) =3D { + .lock =3D __RAW_SPIN_LOCK_UNLOCKED(tlb_flush_queue.lock), + .len =3D 0, +}; + static void __ipi_flush_tlb_range_asid(void *info) { struct flush_tlb_range_data *d =3D info; --=20 2.20.1 From nobody Sat Feb 7 17:04:24 2026 Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA20524337B for ; Thu, 30 Oct 2025 13:57:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832638; cv=none; b=ECMAVrb41e7oWYjRPMUoBdfSx1UmFvy+AG1lT+xhV55T2wh9hvD5/ZUNFz6G5m9+lCPfE96yg63shGz+lyVLcAX+OIAf+73etgptF+XYGa0snxjseLVbtzlolMhy4yow2sX56njmE2xrGPQsPNY0km8eFMtxsdgKEnGZ9lXZZ/A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832638; c=relaxed/simple; bh=GZ13kGXVZV3MWjVjkglZlasmL+TOxO9NJy2cnrIsr0Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h0eMzuLmlYQdNmnivKXTIu8qASzSuUefsuhN5vYosnc5alnNjHUOYA8NTsy91LmklkdNDf/gY1juixYB7h2qKWj/0AeDw6FQi/TAE3J9/RHQOyZO2bK6XylNbJaLQLhdmquj8nAN5duabQrKi2/y63B45OpjDoPy2/2lq58pKLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=igSpNwBS; arc=none smtp.client-ip=209.85.215.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="igSpNwBS" Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-b6cf1a95273so707787a12.1 for ; Thu, 30 Oct 2025 06:57:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1761832636; x=1762437436; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JicIE58FwjHecYxgU5U3fS7oy4Wdsaoyxg9pwk2jRzo=; b=igSpNwBSUXipUq2scagJG7G94aFT5zQ6DedpRagacRC0PjCy9Ax+IKZUq+FCOrHD7i NV8iY2iGSDp9wecDQoQNLNXKRzADlFN3/rsi+LWZ6xkNdgiC06p7Y4529myY8Vkg3DGo P8MdTZzGk7HIFMPBTQ61kIJS8a6M3C2wvpYCY0yMAzB9csh3DJTFbYKdYGS8vXgnJqJr o45tb4l6v7AUFbcM+Jh8Ee9JDcVwlTOCX+S9H3lRZQCzky1jb8ZbBXiPMzvGwGJWoXq1 txecwcaK56V7VoRRFxmRmXKI1e0XKZLFVd2WEwGjcsNIhWjE511wDSoCCsSUh7jCiyB7 mEew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761832636; x=1762437436; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JicIE58FwjHecYxgU5U3fS7oy4Wdsaoyxg9pwk2jRzo=; b=nCxme8gtAwhsrljT83G9C7TXFmbh7bnbnSE9/DtSez9S375Rk8NQVgdufYhn1npGlp mPEgYynLMp1UteVhLDHfOoGInm9xihWDrcMHx8DUNx9ZfFun1OrtVkJN7QFbUZzMsmWL Zvjy+PX+PGWPn8h8Y/OI8vxCbWg0+J/h7FFSlz+UxPk+zKTBeq48yS/K143Fb5c0nG4t E4uQ+BYm2ynBd4WcDUI+/Xou5Y1LpmqO8GCCaERw2umqQyBddCTk4DZrJHfZ1w0cJe3D 05XQx7b2MHbwVGqvadu5p5Dd/+mhA35NOQrAkPqtqDlxnZTYOOgvRKOq2tQ8oL+U1iNH 306A== X-Forwarded-Encrypted: i=1; AJvYcCXy9rWlTDflUmStqVtGxsdxOisQKsyi4u2aO6pWKg27VOl1jwFtudBSj3o8oSgCm2yrb+JZWuP/ZMFn79E=@vger.kernel.org X-Gm-Message-State: AOJu0YxttenLXxmj2LVdzMM4uomq3bi2POEpbkF7swRd7LPoTTOuzCuo 0aoIGUm0VtsSQ1qAw/gtERcfXTDWXjoSdtzPgsoCZD5y55mWskU6k2OFK4D7ZLMoA7M= X-Gm-Gg: ASbGnct4ulmqQhhFIYPXQBtDdOKiZ3CdCe7Hw2JR+Sru4yvi9ueUXYJyBgbs9vZAdad MzdumQTXPOeR7IvLWXpi8SbrMU27SusNQOKhezobddadhImeA44phO3n8/jJvMfJfpWIvugd2W+ kkcNhgYThz0g12Bjxp9qqs7+TgEcigCa1C08B1Is/w0xQby0N7w/xMvPaIWxV10565MXBkhJCpj NbTqqkpdODxZWZeQqPk+liWIDSYuZLyyXrsCvMne2/1DQTW6kYB+NCxLUHS7VtuoLYgvIdNbM8Q CxUkMw04YwPSG+kxFk6KIiQHxH58tkvsj5fSXOtIfe65Us6495mrul/Cemd881B01jO3N1iRVQl X4IGdnatMjNRO+/ojyaT6XEUw2vDc5dxYYtyTkpbSYD/n4/MwwPldlgtTgVdgE8C5n7DnjZmQ3/ jMRrFzNKFoSLvkUw8W8f0S1EuWhMDyO65W7pHQlf2nRjd0VxQgyULpQ30irLeuidc= X-Google-Smtp-Source: AGHT+IHEJ20/piBU7YOuFAuDIHAXabWarffYfeSdFbAZ/TJ7BVINPJu9h3FE6SZpYUveWoSWx2Tx3Q== X-Received: by 2002:a17:902:db11:b0:294:f1fa:9097 with SMTP id d9443c01a7336-294f1fa9228mr37402885ad.34.1761832635722; Thu, 30 Oct 2025 06:57:15 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([61.213.176.55]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29498e495e8sm187071905ad.110.2025.10.30.06.57.11 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 30 Oct 2025 06:57:15 -0700 (PDT) From: Xu Lu To: pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, apatel@ventanamicro.com, guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Xu Lu Subject: [RFC PATCH v1 3/4] riscv: mm: Enqueue tlbflush info if task is not running on target cpu Date: Thu, 30 Oct 2025 21:56:51 +0800 Message-ID: <20251030135652.63837-4-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251030135652.63837-1-luxu.kernel@bytedance.com> References: <20251030135652.63837-1-luxu.kernel@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When need to flush tlb of a remote cpu, we only send ipi to the target cpu if the task is currently running on it. Otherwise, we only enqueue the tlb flush info in target cpu's tlb flush queue. Signed-off-by: Xu Lu --- arch/riscv/mm/tlbflush.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index aa8f1304ae5c4..f4333c3a6d251 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -115,6 +115,32 @@ DEFINE_PER_CPU(struct tlb_flush_queue, tlb_flush_queue= ) =3D { .len =3D 0, }; =20 +static bool should_ipi_flush(int cpu, void *info) +{ + struct tlb_flush_queue *queue =3D per_cpu_ptr(&tlb_flush_queue, cpu); + struct flush_tlb_range_data *d =3D info; + unsigned long flags; + + if (per_cpu(loaded_asid, cpu) =3D=3D d->asid) + return true; + + raw_spin_lock_irqsave(&queue->lock, flags); + if (queue->len < TLB_FLUSH_QUEUE_SIZE) { + queue->tasks[queue->len] =3D *d; + queue->len++; + } else { + raw_spin_unlock_irqrestore(&queue->lock, flags); + return true; + } + raw_spin_unlock_irqrestore(&queue->lock, flags); + + /* Recheck whether loaded_asid changed during enqueueing task */ + if (per_cpu(loaded_asid, cpu) =3D=3D d->asid) + return true; + + return false; +} + static void __ipi_flush_tlb_range_asid(void *info) { struct flush_tlb_range_data *d =3D info; @@ -152,7 +178,9 @@ static void __flush_tlb_range(struct mm_struct *mm, ftd.start =3D start; ftd.size =3D size; ftd.stride =3D stride; - on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); + on_each_cpu_cond_mask(should_ipi_flush, + __ipi_flush_tlb_range_asid, + &ftd, 1, cmask); } =20 put_cpu(); --=20 2.20.1 From nobody Sat Feb 7 17:04:24 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3056225C804 for ; Thu, 30 Oct 2025 13:57:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832642; cv=none; b=NV2zHi+769AuWC+7fOKj1r1xmgmbSDlLNLhNkGKCd0eMFb49ZAt+sJw4kT15Kvaiv95NWuIehDr0gAJdJ0C5Yp6QUJqCpsnO7+0KWZl6K+6t8Pcc8xiLLTuu8S0mDiWxqjvNn2N7TrqeHjNuwkQYVz35Gomja72A5PewrsAWrvA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761832642; c=relaxed/simple; bh=ouiSQ/bYXMLTEPf/wq+/mgBpKqBW2VXDy5s0N6Y96sM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lMNaN4iW0W6CDyAxlYKl69b5doNAyguG2uXbt9eMr6nzVb1WLRW+bVQglZMmqeL57mBK9AHVc3e0ISVq4W3ULxNJof7/O363wj3HqCFLRJM8LwfzwZYm/GhxGSUhQfEli9I1hSWguLAtc8/X9P6yGh8yFt85Q2ZMmeUj6XKorKY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=DGOCRQIC; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="DGOCRQIC" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-290ac2ef203so10568075ad.1 for ; Thu, 30 Oct 2025 06:57:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1761832640; x=1762437440; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bBFEkcQ8xUvhwX9gDA9IWLN2BELZaEr5a/ByZItGkVU=; b=DGOCRQICoMScUp8gvg2m/OxCOWchI6aWx0Ui/q1Qj5IeWzP/ZvaQD/2GJg6VVeqdDO 3Zp4mNk49HxK4jgGZh3zIWNFk32oLnIuHukL0VwhbzhoVcpHO01YJaafI7RRGCz+wUOA KtlNfd0LXloVdeKaBxUV00p6i/i3PW4of+t9B4Ekji+WPFUmoQNQpXgZIOuRwfGIN1hI dzbhwvEtEdzoxxJ6rKQWkuKh0e2gJMdkAJZNiO0I2xk7wmSFo0GNyPi8494x8LsoVZCQ l0Qk8VpQAdItGlebgccftyy5yy179fMckZYPL8W5IyGPI/v7RbXUCB7SCxkrdFiMXjfq dmdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761832640; x=1762437440; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bBFEkcQ8xUvhwX9gDA9IWLN2BELZaEr5a/ByZItGkVU=; b=WN81BhQ556eIHk647TTuHgL2sP/xMcIm9yDid8QbQGsZCLbV++ogDPaOHI50oF4oHw 0UbXOQ4ylsPFh0dP4dMZXaVBB5+zN0r9FjrsYfdU62iwMPADoZKigST5v9EAe74dXVFT /qFKWoDi6/FGqR5xGZ5jnLpo9WrACx5H8kFdD1rWw+T8dxCMB/vsMehgTL05e5qKk4Jp TN2TqEIf/1q0kE85+7rnfd6XazaSwtjJC1LDmKNtiFchsWsXPNphbwoZvjEO8frKo+Ik FiKP9wlePJkdej6T+sIgmp0rw+ImXG6VDW3OUHQlTdZZIpDcZJcfAqbBdCTG4FDTF5pO 1Ipg== X-Forwarded-Encrypted: i=1; AJvYcCXBfirexukN6Iw+8w5r0zAal9BNqJ7pPGroVJfyAbjjBwxSKTidmVwCsrpAjZo4HDG/n3p82Q72KSHBRHo=@vger.kernel.org X-Gm-Message-State: AOJu0YwSbNfol0/i82MSUjZUirgHIgnJsmXsY/TJQitOLzcEnJo6/Jwj KeVFz9zQghjwELX8biXZQQ+UsKu8vKHrDibvm9Gaeo8SjvSYF2yTUpCdw0/FQEaJMRI= X-Gm-Gg: ASbGncve3RlR2Xl9ZV7RGxp0Sj/GztMZCVv2J8AQtiicEz8a4/E7Z/QdptMYXWU+siT yIqIj7ebVJSeREYjJfOm3eBuj196ScAXBGuYTGak7Q7TW49LeaEiL2OFVso0LSXOgK2XMDHTVkG XUg6aJ5mQeHeE2519j4Q9V3hRWkxK1xElwYvwzmDMIhndGbSRgINxVCkOaKX4soR/WJ8rzFS4D1 7w2vk66wwkj6LMVVJd6tZjiJnX9IgxvBNcz54XUWy+BT/Gq0F39bB8lL+h9UIRJEf31YFuSaRmd er86xYHsDBZBsrXN032NDpeZlYa+Ou1RyaBEoD84JFn4SNrGF0B7CTZbtJqg67T7AqbzwNccu/F jVJs3IUqieBX6QV5fKt6i5WAgggO1qKJgP/jHlT9p1js4UIqLB2nYDRe5S7RJoect+VFyIe9Jzt FnLCanz0y0fyS9mC6Swvy9WpEKCcJgq5hV09uvcocxF7KCBXzq40MUzUWK8f+kF70= X-Google-Smtp-Source: AGHT+IEBVtXylNQASrr2YyG7D2Tj2EPDKv83suTEKGHYLNZD/7uPvsZga5WS/d7dp2LV6KOpbcdbCg== X-Received: by 2002:a17:903:11d1:b0:294:e095:3d3a with SMTP id d9443c01a7336-294e0953f16mr72086945ad.24.1761832640241; Thu, 30 Oct 2025 06:57:20 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([61.213.176.55]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29498e495e8sm187071905ad.110.2025.10.30.06.57.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 30 Oct 2025 06:57:19 -0700 (PDT) From: Xu Lu To: pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, apatel@ventanamicro.com, guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Xu Lu Subject: [RFC PATCH v1 4/4] riscv: mm: Perform tlb flush during context_switch Date: Thu, 30 Oct 2025 21:56:52 +0800 Message-ID: <20251030135652.63837-5-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251030135652.63837-1-luxu.kernel@bytedance.com> References: <20251030135652.63837-1-luxu.kernel@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" During context_switch, check the percpu tlb flush queue and lazily perform tlb flush. Signed-off-by: Xu Lu --- arch/riscv/include/asm/tlbflush.h | 4 ++++ arch/riscv/mm/context.c | 6 ++++++ arch/riscv/mm/tlbflush.c | 34 +++++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlb= flush.h index eed0abc405143..7735c36f13d9f 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -66,6 +66,10 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unma= p_batch *batch, void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); =20 extern unsigned long tlb_flush_all_threshold; + +DECLARE_PER_CPU(bool, need_tlb_flush); +void local_tlb_flush_queue_drain(void); + #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 4d5792c3a8c19..82b743bc81e4c 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -199,6 +199,12 @@ static void set_mm_asid(struct mm_struct *mm, unsigned= int cpu) =20 if (need_flush_tlb) local_flush_tlb_all(); + + /* Paired with RISCV_FENCE in should_ipi_flush() */ + RISCV_FENCE(w, r); + + if (this_cpu_read(need_tlb_flush)) + local_tlb_flush_queue_drain(); } =20 static void set_mm_noasid(struct mm_struct *mm) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index f4333c3a6d251..6592f72354df9 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -115,6 +115,8 @@ DEFINE_PER_CPU(struct tlb_flush_queue, tlb_flush_queue)= =3D { .len =3D 0, }; =20 +DEFINE_PER_CPU(bool, need_tlb_flush) =3D false; + static bool should_ipi_flush(int cpu, void *info) { struct tlb_flush_queue *queue =3D per_cpu_ptr(&tlb_flush_queue, cpu); @@ -134,6 +136,14 @@ static bool should_ipi_flush(int cpu, void *info) } raw_spin_unlock_irqrestore(&queue->lock, flags); =20 + /* Ensure tlb flush info is queued before setting need_tlb_flush flag */ + smp_wmb(); + + per_cpu(need_tlb_flush, cpu) =3D true; + + /* Paired with RISCV_FENCE in set_mm_asid() */ + RISCV_FENCE(w, r); + /* Recheck whether loaded_asid changed during enqueueing task */ if (per_cpu(loaded_asid, cpu) =3D=3D d->asid) return true; @@ -146,6 +156,9 @@ static void __ipi_flush_tlb_range_asid(void *info) struct flush_tlb_range_data *d =3D info; =20 local_flush_tlb_range_asid(d->start, d->size, d->stride, d->asid); + + if (this_cpu_read(need_tlb_flush)) + local_tlb_flush_queue_drain(); } =20 static inline unsigned long get_mm_asid(struct mm_struct *mm) @@ -280,3 +293,24 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_ba= tch *batch) 0, FLUSH_TLB_MAX_SIZE, PAGE_SIZE); cpumask_clear(&batch->cpumask); } + +void local_tlb_flush_queue_drain(void) +{ + struct tlb_flush_queue *queue =3D this_cpu_ptr(&tlb_flush_queue); + struct flush_tlb_range_data *d; + unsigned int i; + + this_cpu_write(need_tlb_flush, false); + + /* Ensure clearing the need_tlb_flush flags before real tlb flush */ + smp_wmb(); + + raw_spin_lock(&queue->lock); + for (i =3D 0; i < queue->len; i++) { + d =3D &queue->tasks[i]; + local_flush_tlb_range_asid(d->start, d->size, d->stride, + d->asid); + } + queue->len =3D 0; + raw_spin_unlock(&queue->lock); +} --=20 2.20.1