From nobody Fri Sep 25 08:47:20 2026 Received: from mail-oo2-f42.google.com (mail-oo2-f42.google.com [74.125.231.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C250390CAE for ; Tue, 15 Sep 2026 02:03:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.231.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789437837; cv=none; b=Vdf752NmQ7Ye/5deVsPhSD3Cl/kIqkLSEevnTyijMXCVWlZtYRBXxwiIvZpic0TTHjsIUR92659JJiKmRcPWvEieK+v6mAy+dOS5ZVodK+36i5j0nwhoPVJq/sWQiPq893Y2PK/oVliEPV+Jmz/1Lq482Wl9xNya++gG9pOUK5o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789437837; c=relaxed/simple; bh=4KlU6IioCAwNk3jDxqQyOgDpHDc+AmeDeXmdSyc+iYU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GIoa7Uj/x25XR63cKbnHfwTSvP2JSyVlZwn4JWj8A/9A1qAvBB9crg61T0F3STNZ/bXwhrIErMVTh1AoWxdeG3uKFpedsvPQ1gvhM2LIqhXSBE+IJhEZt/NJjAJxjetfnQUmNu3SRnMNo+p4kTlLCWJbljGtflTVzJHzwhlIcq8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b=AXozjsoW; arc=none smtp.client-ip=74.125.231.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="AXozjsoW" Received: by mail-oo2-f42.google.com with SMTP id 46e09a7af769-7f4f0cfb29cso1288610a34.1 for ; Mon, 14 Sep 2026 19:03:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1789437834; x=1790042634; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/CaPcfjF5HqH3+aS6ruNpmYZeARPkCepH2Ne0CkwEFA=; b=AXozjsoWqy9neo+jCEGvb3bill5+DF9FUr2yVLJGa7+0kr+KvDCZFyiSeZ5Rb9Z8JB 9IIM27xav14EzxUsYTwSLG3BYZ8+31FG+zkYwgradPE/8RSTZoD3noVSYnx5WoUjbLPH HOAfpNrP+Inn2bnBb84ckHlS5uRYMTpoMKdHHd/DIxVU4gMe5YXFYmbxA+PVZEztFHkt H8ef5cHIrEVo83Oe2Jeql/FBV/qLXjwsJLXgwxdE+L4I03f4TkPQPkoRJwX6BYQ8v0bh LmJTdJj6Vt/pPj/SAodfBro+7QrNkLDaIBq753jqcla2IQhiuulBx1Si4pj2huNmKgNH QiXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789437834; x=1790042634; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=/CaPcfjF5HqH3+aS6ruNpmYZeARPkCepH2Ne0CkwEFA=; b=Pm1tduitINxRrue/BdVtR85cP+iPf9KC21LR7m44JrGGgFdAak2fIOTyV5n2MbuUXW 3I2mkW5soulq5YAtXl2ZltssEwYjDABz0CdzEZd2t9WNnVlB4NtIatr/Z7qYMbHnIrR5 n78kCnboYnb4SyueWstbWNVdFhMYDhxpHNntQSiwQe5K1XjyqPew+ZJi05L25cVmnqQG MFdtq8PQMK8KNLTN3hH5hxZW8L1pqI8olDHGJtEyG1Ctgj2/11eOP7k9cflsx+89Hsca r3Tk8GDGN+fGPQciitS8qXrRfqS9ZAo5i+9wy+x6Lh7WJHgXuo+XdcgYiempq0R7mCzE W2eg== X-Forwarded-Encrypted: i=1; AKwUvByQIpSUWmDSbpFVTfR4Cu2MOuX7fDPacRpguH9La9Dup0smGYhEkBO4qq2eqCtSr/+e5VeqLu4AyNdm048=@vger.kernel.org X-Gm-Message-State: AFuF++l1IcyUQCD9D9xqRTdgkSqcfdFCnCaAaN9jTfxYZIJYLLiWcovt jtCcwrhoTNFlAGUnFMhutk+/h574xiLDyw1JxRoePDAAwNHHhgmZFJhkxJr7f6yS5JQ= X-Gm-Gg: AYBFou0YUiMIjr3DLjk2sv4EsI4KiIcrVeyP7oCwQmcMsbyc4eiwmQr9i7fhwSpkKv/ XHmdgRmfXhOW3O7lPji9XQ4A5cK3mJ0tirM0RiX3sebtfB1jWM8jbd6j3UZGHxQDUdDB7k4JU+o 0YxM2G5Ea0y0LVbsF3/OcOOx//TV/tQbp0fVM8RXkEw1dihvVNs/H6txbL31zF4Y3RwJvs0zm65 hWT68nPVJKMRcfFtgitlYqQzdU94Kcc7yCuTzTyqUzJr/IsSXrV7qL8A72TBmHrHD4QcGzDXJqg iy9+56kqLoTQ/QBt1cwDdLhU0+RU/z0Wk5A7RlWZeZwAONkTPod32kYgI4SZpHirCto8MVyRqMq 4SAcuSRDBMI3j6dHP3o1kTnemr5NxFUHPvmYo5ZQ3F+qAwETwJSvJvzBRqkrWZyJhXC6X9fZKyV UUl7Pbph3UklYjGpfN/9YiNNxtJRCsk4CHuEPDDrD8fHmAqDlR5oAI8WqXvQycOY+HxuAo4U0= X-Received: by 2002:a05:6830:8285:b0:806:1728:aed9 with SMTP id 46e09a7af769-8089759dffcmr5874003a34.5.1789437834177; Mon, 14 Sep 2026 19:03:54 -0700 (PDT) Received: from [127.0.1.1] ([2a09:bac6:bf21:2e46::49c:4c]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-804de582afcsm10795696a34.16.2026.09.14.19.03.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Sep 2026 19:03:52 -0700 (PDT) From: Chris J Arges Date: Mon, 14 Sep 2026 21:03:35 -0500 Subject: [PATCH net-next v2 1/3] ipv4: hash uncached routes by device Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260914-hash-bucket-route-lists-v2-1-29f6297d8a5a@cloudflare.com> References: <20260914-hash-bucket-route-lists-v2-0-29f6297d8a5a@cloudflare.com> In-Reply-To: <20260914-hash-bucket-route-lists-v2-0-29f6297d8a5a@cloudflare.com> To: David Ahern , Ido Schimmel , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, Chris J Arges X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1789437828; l=4312; i=carges@cloudflare.com; h=from:subject:message-id; bh=4KlU6IioCAwNk3jDxqQyOgDpHDc+AmeDeXmdSyc+iYU=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgaxY1IIT5oTohBZJmhnVgJo2HsM7Sv 9I0LdJCgpeGX6gAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QO7WoBzgXhIPhQvuy6eQZpcCD8KJdm3/hK/o2ENFnBRTFbGEk5OtQu7hB1DhZyQqI7qL7HYZVnS Zy/j/lz3suAo= X-Developer-Key: i=carges@cloudflare.com; a=openssh; fpr=SHA256:Cun99EBiH0EV7wvmfTBF9eDrld2NJx+aD4ScWZ45Q5M rt_flush_dev() currently walks every per-CPU uncached route list for each device being removed. This repeatedly examines unrelated routes and makes teardown increasingly expensive as the number of devices grows. Replace each per-CPU list with a hash table keyed by the route's netdevice. Keep the owning-list pointer in dst_entry so route removal remains unchanged, while device teardown only walks the matching bucket on each CPU. Hash collisions are filtered by the existing device comparison. The table has 2^CONFIG_IP_UNCACHED_ROUTE_HASH_BITS buckets and defaults to 64. Larger values shorten each bucket, but every additional bit doubles the per-CPU memory used by the table. The default costs approximately 1.5 KiB per possible CPU on x86-64. Signed-off-by: Chris J Arges --- net/ipv4/Kconfig | 13 +++++++++++++ net/ipv4/route.c | 36 +++++++++++++++++++++++++++++------- 2 files changed, 42 insertions(+), 7 deletions(-) diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig index 301b47660305..7d40ca22d2b2 100644 --- a/net/ipv4/Kconfig +++ b/net/ipv4/Kconfig @@ -103,6 +103,19 @@ config IP_ROUTE_VERBOSE config IP_ROUTE_CLASSID bool =20 +config IP_UNCACHED_ROUTE_HASH_BITS + int "IPv4 uncached route hash bits" + range 1 10 + default 6 + help + This option sets the number of buckets used in the IPv4 uncached + route hash table to 2^IP_UNCACHED_ROUTE_HASH_BITS buckets. The + allowed values select between 2 and 1024 buckets. Larger values + reduce collisions, but each additional bit doubles the per-CPU + memory used by the table. + + If unsure, use the default of 6 bits (64 buckets). + config IP_PNP bool "IP: kernel level autoconfiguration" help diff --git a/net/ipv4/route.c b/net/ipv4/route.c index d7da2f1acbb5..e28e2140cf62 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -74,6 +74,7 @@ #include #include #include +#include #include #include #include @@ -1552,11 +1553,21 @@ struct uncached_list { struct list_head head; }; =20 -static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt_uncached_list); +#define RT_UNCACHED_HASH_SIZE BIT(CONFIG_IP_UNCACHED_ROUTE_HASH_BITS) + +struct uncached_table { + struct uncached_list buckets[RT_UNCACHED_HASH_SIZE]; +}; + +static DEFINE_PER_CPU_ALIGNED(struct uncached_table, rt_uncached_table); =20 void rt_add_uncached_list(struct rtable *rt) { - struct uncached_list *ul =3D raw_cpu_ptr(&rt_uncached_list); + struct uncached_table *table =3D raw_cpu_ptr(&rt_uncached_table); + struct uncached_list *ul; + + ul =3D &table->buckets[hash_ptr(dst_dev(&rt->dst), + CONFIG_IP_UNCACHED_ROUTE_HASH_BITS)]; =20 rt->dst.rt_uncached_list =3D ul; =20 @@ -1588,14 +1599,19 @@ void rt_flush_dev(struct net_device *dev) int cpu; =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D &per_cpu(rt_uncached_list, cpu); + struct uncached_table *table; + struct uncached_list *ul; + + table =3D per_cpu_ptr(&rt_uncached_table, cpu); + ul =3D &table->buckets[hash_ptr(dev, + CONFIG_IP_UNCACHED_ROUTE_HASH_BITS)]; =20 if (list_empty(&ul->head)) continue; =20 spin_lock_bh(&ul->lock); list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { - if (rt->dst.dev !=3D dev) + if (dst_dev(&rt->dst) !=3D dev) continue; rcu_assign_pointer(rt->dst.dev_rcu, blackhole_netdev); netdev_ref_replace(dev, blackhole_netdev, @@ -3771,10 +3787,16 @@ int __init ip_rt_init(void) ip_tstamps =3D idents_hash + (ip_idents_mask + 1) * sizeof(*ip_idents); =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D &per_cpu(rt_uncached_list, cpu); + struct uncached_table *table; + int bucket; + + table =3D per_cpu_ptr(&rt_uncached_table, cpu); + for (bucket =3D 0; bucket < RT_UNCACHED_HASH_SIZE; bucket++) { + struct uncached_list *ul =3D &table->buckets[bucket]; =20 - INIT_LIST_HEAD(&ul->head); - spin_lock_init(&ul->lock); + INIT_LIST_HEAD(&ul->head); + spin_lock_init(&ul->lock); + } } #ifdef CONFIG_IP_ROUTE_CLASSID ip_rt_acct =3D __alloc_percpu(256 * sizeof(struct ip_rt_acct), __alignof_= _(struct ip_rt_acct)); --=20 2.43.0 From nobody Fri Sep 25 08:47:20 2026 Received: from mail-oo2-f42.google.com (mail-oo2-f42.google.com [74.125.231.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 482093D565A for ; Tue, 15 Sep 2026 02:03:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.231.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789437840; cv=none; b=qzg8R5pZsxbsDrG/6cHjqDPvBH91c3ax6vCgPBQJQdkMewGCWMIblEdkcWXtepOEaPna3mrL4FrNcH4wdD97fJob8jiARViG/wgFpoAvMOAipjKqAE+HpynW31CioUEWTmueIQsFdmKXUi1CVMcANHnXHrlKiM0e7dC67qu0xPU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789437840; c=relaxed/simple; bh=ujawG8SxKb6jFUdKWZQvD9jJs2Uc4neq7Xq9fXpMjTk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MtZT4YMwB9iZvgT5r+uI/jJCt+HyMoiEzpZOg+dKdVDoTf2ZTEdPrv9oaeHR46bGx3pRO1Zusq1MK6wtlR7GlUl0wrDH6viRJjYH4KY/ZsaPqvn+6WHlFVEd3V1mvv0rbfqRwWz23YQqz+30tAR2YIA2ABZ97CaFezFKIaSValo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b=BeipdoP2; arc=none smtp.client-ip=74.125.231.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="BeipdoP2" Received: by mail-oo2-f42.google.com with SMTP id 46e09a7af769-7f4f0c89e35so1817460a34.2 for ; Mon, 14 Sep 2026 19:03:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1789437837; x=1790042637; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=+gC8oQEQoLs54N5KYcANahGyWqaEmEB/cMaxCtlcefw=; b=BeipdoP23Oimdl5L4N6lT4grG9LA5YRqY2WYhO689wsx2ElUSYyFhCYo6wK7lc2aQI BkZMEfpvEHfly4MYoWtJnF7gbzxtZLWhuqIfV7RdXpMNMO7o5e4k/+WwVe9gOCIULIFG 2ZZP6lmUDrrahG/dcKfh2VuQR2ev+wy7oGc3c2/QCUeVx4yXO6PwdedTV1IEyDYpzUFD W/TlCDCGMeuxw8LRbIy6WL+tvrNtuPabu48cXMo7wPwFPw6q0+7UJH6PhZmklvpSp/uY 32y03OyX/xLLIgiyh3Qo8KaOYigYI3fWYcAvF/iWkGAUNTqWEVWT3Qf2nUTRpnHrBX/N kLqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789437837; x=1790042637; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=+gC8oQEQoLs54N5KYcANahGyWqaEmEB/cMaxCtlcefw=; b=yCLT0QzlaohIqti4xF6Er6MroJ2Osjb2MrvibnyOHo8A7bi3NKZFZqy3nCOj4VnyYT /YMkCceYPS/7ROmgzp4lDmUXJqtjPG2KLNevIWHYXlFhj6RVNr42WGKsj3c2k5ZKg8Ws n30Z601WNxx50jyNZQb0/+9oyE6vC1yzT2ortS8EY/sZw6+seEXThhuaPldaAhkER8Kv ///RuUXfqSBev+znr7Iuy7+nEysJ2lioFENkldCPLuReafaLY2/Iu3YDS47ujv9l1pfU Gk7+u4X5b9HMwaF1GhVSlJPZ9r59AiKPxpUl2P1KmgtcmmjTU5OE0DIY3SsDAkPw6z9c XFiw== X-Forwarded-Encrypted: i=1; AKwUvByN7uwVgeDvgYqs5kNhYPWmkGHEYAbfVVC7GyCABiHg9d4GBeJOPwzbUgewlBi0Nsgm6UpaxxSDXBORT7k=@vger.kernel.org X-Gm-Message-State: AFuF++mj439eptXRNJK8I/fw9nLsldREZdDdHzkNFC7cZSZxDqndOUIw iGzSl1HvDibrkG/N1AEFvQXOllEs/zfYAJCRcCQaMi0BkO4/sh3A2hRdP17pQyFj0NorwG+ZYH+ YeWJhvnvUcA== X-Gm-Gg: AYBFou2l0/H8wx9aPJinIw0nz9TOT2aEJ8FQyFbfq0ssqi/i76WhWrphRMxMUcRnB/e hwMFEoqnTT1y5yuQ7/GmTTLcBTW9BWqRL2sUd76/kON4aCLGqBiIvHD1dj4jceAJ7cVtZjEXSe7 wFtH0n4kfpzffW1Jq6S1gq6+muJoP84wHkE8V2CzgM/36GHs4YsLfECrEn8HHt5QYmrZ3lNIulZ MfgSZhS8rj2Uthat0FuYIFd0RM7Z10JPgEXcqy2ZE4xaP/ZTW9r7D+D0gv3p3/h5R4JScnxYFyb LWswoBmiiTop3lB3A83WTijMOc6htB8vFDIO26Md7+EPGofb3u++wT8H42dDl3pjuXHZ60X25Kg 3E9CS1el9Ow/7N3yd1ENksI+ahjNMFrR5ARz0bNxmG2/HRweQno5SS0Fy3fS6mHhBaSxm4Spsaw XP6dKYRTDhE1STkCeh6H/9mwzMX6b2z3MFdgoE97gckDBxdQS2kM1PNyYohomx9Q== X-Received: by 2002:a05:6830:4c07:b0:7f4:bae7:b00d with SMTP id 46e09a7af769-808945f12efmr3412972a34.0.1789437836931; Mon, 14 Sep 2026 19:03:56 -0700 (PDT) Received: from [127.0.1.1] ([2a09:bac6:bf21:2e46::49c:4c]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-804de582afcsm10795696a34.16.2026.09.14.19.03.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Sep 2026 19:03:55 -0700 (PDT) From: Chris J Arges Date: Mon, 14 Sep 2026 21:03:36 -0500 Subject: [PATCH net-next v2 2/3] ipv6: hash uncached routes by device Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260914-hash-bucket-route-lists-v2-2-29f6297d8a5a@cloudflare.com> References: <20260914-hash-bucket-route-lists-v2-0-29f6297d8a5a@cloudflare.com> In-Reply-To: <20260914-hash-bucket-route-lists-v2-0-29f6297d8a5a@cloudflare.com> To: David Ahern , Ido Schimmel , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, Chris J Arges X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1789437828; l=6072; i=carges@cloudflare.com; h=from:subject:message-id; bh=ujawG8SxKb6jFUdKWZQvD9jJs2Uc4neq7Xq9fXpMjTk=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgaxY1IIT5oTohBZJmhnVgJo2HsM7Sv 9I0LdJCgpeGX6gAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QBIPZmYi/sxYsGJ+GBEi9RpcnUOfNENWfcAEBRI0yRIK/m6M0T0P9ATIIgs0dNBlC15gOaDi206 nXVi/+P749Q8= X-Developer-Key: i=carges@cloudflare.com; a=openssh; fpr=SHA256:Cun99EBiH0EV7wvmfTBF9eDrld2NJx+aD4ScWZ45Q5M rt6_uncached_list_flush_dev() currently walks every per-CPU uncached route list for each device being removed. Hash uncached routes by their inet6 device so ordinary device teardown only visits the matching bucket on each CPU. ip6_rt_get_dev_rcu() can return loopback or an L3 master while rt6i_idev still refers to the original interface, so such a route must be reachable from either device. Place those routes on a separate per-CPU list that is always visited in addition to the keyed bucket. This avoids growing struct rt6_info while filtering most unrelated routes from ordinary device teardown. The table has 2^CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS buckets and defaults to 64. Larger values shorten each bucket, but every additional bit doubles the per-CPU memory used by the table. The default costs approximately 1.5 KiB per possible CPU on x86-64. Signed-off-by: Chris J Arges --- net/ipv6/Kconfig | 13 +++++++ net/ipv6/route.c | 102 +++++++++++++++++++++++++++++++++++++--------------= ---- 2 files changed, 82 insertions(+), 33 deletions(-) diff --git a/net/ipv6/Kconfig b/net/ipv6/Kconfig index c3806c6ac96f..0253178668bd 100644 --- a/net/ipv6/Kconfig +++ b/net/ipv6/Kconfig @@ -18,6 +18,19 @@ menuconfig IPV6 =20 if IPV6 =20 +config IPV6_UNCACHED_ROUTE_HASH_BITS + int "IPv6 uncached route hash bits" + range 1 10 + default 6 + help + This option sets the number of buckets used in the IPv6 uncached + route hash table to 2^IPV6_UNCACHED_ROUTE_HASH_BITS buckets. The + allowed values select between 2 and 1024 buckets. Larger values + reduce collisions, but each additional bit doubles the per-CPU + memory used by the table. + + If unsure, use the default of 6 bits (64 buckets). + config IPV6_ROUTER_PREF bool "IPv6: Router Preference (RFC 4191) support" help diff --git a/net/ipv6/route.c b/net/ipv6/route.c index 7535b09068a0..080dce329168 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -133,11 +134,27 @@ struct uncached_list { struct list_head head; }; =20 -static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt6_uncached_list); +#define RT6_UNCACHED_HASH_SIZE BIT(CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS) + +struct rt6_uncached_table { + struct uncached_list buckets[RT6_UNCACHED_HASH_SIZE]; + /* Routes that must be discoverable through two different devices. */ + struct uncached_list mismatch; +}; + +static DEFINE_PER_CPU_ALIGNED(struct rt6_uncached_table, rt6_uncached_tabl= e); =20 void rt6_uncached_list_add(struct rt6_info *rt) { - struct uncached_list *ul =3D raw_cpu_ptr(&rt6_uncached_list); + struct rt6_uncached_table *table =3D raw_cpu_ptr(&rt6_uncached_table); + struct net_device *rt_dev =3D dst_dev(&rt->dst); + struct uncached_list *ul; + + if (rt->rt6i_idev && rt->rt6i_idev->dev !=3D rt_dev) + ul =3D &table->mismatch; + else + ul =3D &table->buckets[hash_ptr(rt_dev, + CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS)]; =20 rt->dst.rt_uncached_list =3D ul; =20 @@ -157,40 +174,51 @@ void rt6_uncached_list_del(struct rt6_info *rt) } } =20 +static void rt6_uncached_list_flush(struct uncached_list *ul, + struct net_device *dev) +{ + struct rt6_info *rt, *safe; + + if (list_empty(&ul->head)) + return; + + spin_lock_bh(&ul->lock); + list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { + struct inet6_dev *rt_idev =3D rt->rt6i_idev; + struct net_device *rt_dev =3D dst_dev(&rt->dst); + bool handled =3D false; + + if (rt_idev && rt_idev->dev =3D=3D dev) { + rt->rt6i_idev =3D in6_dev_get(blackhole_netdev); + in6_dev_put(rt_idev); + handled =3D true; + } + + if (rt_dev =3D=3D dev) { + rt->dst.dev =3D blackhole_netdev; + netdev_ref_replace(rt_dev, blackhole_netdev, + &rt->dst.dev_tracker, GFP_ATOMIC); + handled =3D true; + } + if (handled) + list_del_init(&rt->dst.rt_uncached); + } + spin_unlock_bh(&ul->lock); +} + static void rt6_uncached_list_flush_dev(struct net_device *dev) { int cpu; =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D per_cpu_ptr(&rt6_uncached_list, cpu); - struct rt6_info *rt, *safe; + struct rt6_uncached_table *table; + struct uncached_list *ul; =20 - if (list_empty(&ul->head)) - continue; - - spin_lock_bh(&ul->lock); - list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { - struct inet6_dev *rt_idev =3D rt->rt6i_idev; - struct net_device *rt_dev =3D rt->dst.dev; - bool handled =3D false; - - if (rt_idev && rt_idev->dev =3D=3D dev) { - rt->rt6i_idev =3D in6_dev_get(blackhole_netdev); - in6_dev_put(rt_idev); - handled =3D true; - } - - if (rt_dev =3D=3D dev) { - rt->dst.dev =3D blackhole_netdev; - netdev_ref_replace(rt_dev, blackhole_netdev, - &rt->dst.dev_tracker, - GFP_ATOMIC); - handled =3D true; - } - if (handled) - list_del_init(&rt->dst.rt_uncached); - } - spin_unlock_bh(&ul->lock); + table =3D per_cpu_ptr(&rt6_uncached_table, cpu); + ul =3D &table->buckets[hash_ptr(dev, + CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS)]; + rt6_uncached_list_flush(ul, dev); + rt6_uncached_list_flush(&table->mismatch, dev); } } =20 @@ -6987,10 +7015,18 @@ int __init ip6_route_init(void) #endif =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D per_cpu_ptr(&rt6_uncached_list, cpu); + struct rt6_uncached_table *table; + int bucket; + + table =3D per_cpu_ptr(&rt6_uncached_table, cpu); + for (bucket =3D 0; bucket < RT6_UNCACHED_HASH_SIZE; bucket++) { + struct uncached_list *ul =3D &table->buckets[bucket]; =20 - INIT_LIST_HEAD(&ul->head); - spin_lock_init(&ul->lock); + INIT_LIST_HEAD(&ul->head); + spin_lock_init(&ul->lock); + } + INIT_LIST_HEAD(&table->mismatch.head); + spin_lock_init(&table->mismatch.lock); } =20 out: --=20 2.43.0 From nobody Fri Sep 25 08:47:20 2026 Received: from mail-oo2-f41.google.com (mail-oo2-f41.google.com [74.125.231.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9F483D7D70 for ; Tue, 15 Sep 2026 02:04:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.231.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789437842; cv=none; b=D+WVK4f9s+fUt/dRRJRmfNK/JiLhdlEh73QUY7yWLSR/2TF/34AFmI5MPas0m591szli3Fg6QW4olfR1Z99NduugAiStiXSWbFPPVmocL9Ex4JW/+amBomRH8EYZVCZZiRoVaI/ZltOuYNnbM1kbNyZitdV5A0v9Pf5qyCynu1A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789437842; c=relaxed/simple; bh=8WdFP1v7/Oa+dKIJynboqHB29B7wDjj6AhOsc16jvtI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Vd1u+9QXIJXUCewJZqaf4O3IKDtffnJNp9AnNHM9LwTpcV6otHZGcbQijVLZqwugJwHs8dHEiz2PSz49k2WuMzjM31VGPyG3RkMpPS4o4Hop52u8aF7INw3EmUJqNCXjROyj9t+EpwHSKqqe1Y7HtoKjE8caq6y9KlR+aB702IA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b=J5+fb0gC; arc=none smtp.client-ip=74.125.231.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="J5+fb0gC" Received: by mail-oo2-f41.google.com with SMTP id 46e09a7af769-7f4f0d37f93so1433276a34.2 for ; Mon, 14 Sep 2026 19:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1789437840; x=1790042640; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=63GI428HLg6svgdiqmZ8ZkijlWs39+VHkT3kxwHKcMM=; b=J5+fb0gCkZb0yw0F/HqBYlxu/KDjBUXSTJOlo5Q7DqTcEfIsffH98AKU+KJJNBkcb2 xrDlOqZL1htnWA9dMZj+AGA9e/judpS9fXb5XCzs2NwADdDKuPhM1fv6w4mJGbwIYCmP Ue8jurfIsxXTRccbe8oG8ByQggVY2m6NdBlrgZhcVEDu3/mhpbVOS/yws7PGBUK6IhNJ FnqXfLrWBwvgeFIIRStjL0aAWSTZSMeuJKJzFiuz8plnrhqyyQWjkboj190yLoXREzDw qQO7pY5V1KOddeEbPTPeHhu0FnrdibWz/54CLCp/G06LgmY2cUDoFWOm4K+wHGndFiI7 Pxhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789437840; x=1790042640; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=63GI428HLg6svgdiqmZ8ZkijlWs39+VHkT3kxwHKcMM=; b=bcD4siXBhACA7WQqv/c+TG8viN+CFHCtzyQ87B04nWWlJ514sKf8lGUw4X9XqkGoCh Wvcyot3C/lRQQvE5PXSDsa4RCjtXmXvK4q+oXfC8M92GPSrDJys8TRjAFZD4d63KEqa/ hmx46exmTk/sLY5RB3k/6mVbor7sUX6oaAVFUe2Uaa3llYE2DU7qKPqc91hKZPCWntcV zdkURfDuA5eeYUroMGtjSuc2EeEwaJnx2puDnEF1WzKvbItkyLjrdOmhqtKdm3RPRwV8 r4vWWHQvvwGjLp9nomoJTriqDFDKrLEGsXdcF0DoNY46HMsj6RFv0GfepmZCrYpccirH 2V0g== X-Forwarded-Encrypted: i=1; AKwUvBzPD//Pg8fWpoJoUC/7dj6sjGat1tcaaNsc4VmT8FGQKlHVFHQWtFDpUKDyx3L25krKxiHswYTtdO52GV0=@vger.kernel.org X-Gm-Message-State: AFuF++kGyGLUs/m+qkcEB8BQVpC0LCEtdZR1JoaigwkjBzEu1uhFiO8r jFjj2sgpUtDR+nyiLpOwK5DpxawXnzxDKUWSzkrFGWRF70Nn57kzmwwgvUwqVkIYzRM= X-Gm-Gg: AYBFou164JH9qy/U+ErJlCFE/dT5SRboj11WE2HSxRbPbz7/54McMPDSw+EVOcxA1va r0eww/K5G7Tm3B+kg++X75G+dVZTk/pNhoy2aFb72gW1n99n1rp+6PMATpE/VBRuLKBVV+fNLAa KgphntOuWIL0fQGcu8xsLqiz2F6MwpACGK3jCDus2hCkA2N7JSSlXfjZGs/u6IsBuP29qWpQDUC h7bhwEzzj3isSbqwfPfeMSSYLG4LSV5xbYIJCOs6OKNEAWAtUKQ7mO98vJuSvib9JsQAT0RhOaa a76wuzHjhXUR1y+PEPPfJCSbTZmNNjcqKICjfC01A7OItuTGZnDDSPCfX436vtugpWks9wxojXw fUqj84SpEmtCc7v8E9C242SwwNgN2LEg+kE3CnnceXNPScnuey3/mUPpVsv9BhwaMtXQLriKX7S 1IwZvw+ni3EW6iu/nQN0K2t0Ir4fFPwOi4Q/TvWIJrgAz7DMA94+4cm5wgKvk0 X-Received: by 2002:a05:6830:2b25:b0:7f4:bd7e:1394 with SMTP id 46e09a7af769-8089709640emr3666718a34.1.1789437839686; Mon, 14 Sep 2026 19:03:59 -0700 (PDT) Received: from [127.0.1.1] ([2a09:bac6:bf21:2e46::49c:4c]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-804de582afcsm10795696a34.16.2026.09.14.19.03.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Sep 2026 19:03:58 -0700 (PDT) From: Chris J Arges Date: Mon, 14 Sep 2026 21:03:37 -0500 Subject: [PATCH net-next v2 3/3] selftests: net: cover IPv6 uncached route device mismatch Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260914-hash-bucket-route-lists-v2-3-29f6297d8a5a@cloudflare.com> References: <20260914-hash-bucket-route-lists-v2-0-29f6297d8a5a@cloudflare.com> In-Reply-To: <20260914-hash-bucket-route-lists-v2-0-29f6297d8a5a@cloudflare.com> To: David Ahern , Ido Schimmel , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, Chris J Arges X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1789437828; l=2338; i=carges@cloudflare.com; h=from:subject:message-id; bh=8WdFP1v7/Oa+dKIJynboqHB29B7wDjj6AhOsc16jvtI=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgaxY1IIT5oTohBZJmhnVgJo2HsM7Sv 9I0LdJCgpeGX6gAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QB1uSHqNTuHSEUu0oOnVIk7OUTk3RZzaswZwmFCmorRKp9gGkFSsUhv6PRzmEUm+JAD+57TJR82 YoQmqb3O9bwE= X-Developer-Key: i=carges@cloudflare.com; a=openssh; fpr=SHA256:Cun99EBiH0EV7wvmfTBF9eDrld2NJx+aD4ScWZ45Q5M Local IPv6 routes through a VRF can use the VRF as dst.dev while retaining the VRF member interface in rt6i_idev. Exercise device teardown while such uncached routes are retained by a delayed qdisc. Reuse the existing VRF topology and msg_zerocopy raw-header sender, and verify route creation, qdisc retention, and prompt interface deletion. Signed-off-by: Chris J Arges --- tools/testing/selftests/net/vrf-xfrm-tests.sh | 35 +++++++++++++++++++++++= ++++ 1 file changed, 35 insertions(+) diff --git a/tools/testing/selftests/net/vrf-xfrm-tests.sh b/tools/testing/= selftests/net/vrf-xfrm-tests.sh index b64dd891699d..4f409d135a99 100755 --- a/tools/testing/selftests/net/vrf-xfrm-tests.sh +++ b/tools/testing/selftests/net/vrf-xfrm-tests.sh @@ -385,6 +385,37 @@ run_tests() cleanup_xfrm_dev } =20 +test_ipv6_uncached_mismatch() +{ + local sender_pid + local backlog + local rc + + # A local route through a VRF uses the VRF as dst.dev while retaining + # the VRF member interface in rt6i_idev. Raw header sends create uncached + # routes, and netem keeps them referenced while the interface is deleted. + run_cmd_host1 tc qdisc replace dev ${VRF} root netem limit 1 delay 10s + ip -6 -netns "$host1" route add local ${HOST1_6}/128 dev eth0 + ip netns exec "$host1" ./msg_zerocopy -6 \ + -S ${HOST1_6} -D ${HOST1_6} -s 1200 -t 0 raw_hdrincl \ + >/dev/null 2>&1 & + sender_pid=3D$! + wait "$sender_pid" + rc=3D$? + log_test $rc 0 "Create uncached IPv6 routes with mismatched devices" + [ $rc -ne 0 ] && return + + backlog=3D$(ip netns exec "$host1" tc -s qdisc show dev ${VRF}) + if ! echo "$backlog" | grep -Eq 'backlog .* [1-9][0-9]*p'; then + log_test 1 0 "Retain uncached IPv6 routes in VRF qdisc" + return + fi + log_test 0 0 "Retain uncached IPv6 routes in VRF qdisc" + + run_cmd_host1 timeout 2 ip link del eth0 + log_test $? 0 "Flush uncached IPv6 routes with mismatched devices" +} + ##########################################################################= ###### # usage =20 @@ -425,6 +456,10 @@ echo echo "netem qdisc on VRF device" run_tests =20 +echo +echo "Uncached IPv6 route with mismatched devices" +test_ipv6_uncached_mismatch + printf "\nTests passed: %3d\n" ${nsuccess} printf "Tests failed: %3d\n" ${nfail} =20 --=20 2.43.0