From nobody Fri Sep 25 01:22:31 2026 Received: from mail-oi2-f13.google.com (mail-oi2-f13.google.com [74.125.231.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E35B03A453A for ; Thu, 17 Sep 2026 19:38:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.231.205 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789673915; cv=none; b=A61zovDOJagTzgWzl1sI8MYKjkNBK9HV8fxDaquYYc1p62BOaNtDwuwyEY4LGYum58mjTDWuLQNFTOTZlfKsxeO3wbAtYXP0R/E6eiFumtAK6DQmG2OlBR5z/4fLgR8X3CJRvKMi9qgOj3vGQKeaYJK/O1/lopJaL9G1yCK95HY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789673915; c=relaxed/simple; bh=VpMMQFtE0irAVVja4em612tK+k3J5NR1F8chlumvxas=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D6+v2qMfCFnGoz5SqR3LiVTiFofDeTMnZcNknhLVIaHNZfWUIuQIrXNhFD2lUw2sd6pTs+WTt7k2dL1K4XGI1zWdP7QyW+00cIOK34W8lmDbAH/sqew2oAceJ4HntTf3ATZtkjTr09yyfmE/M+srgzRkks+2kcSlZbmbODjsxGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b=UvA100iB; arc=none smtp.client-ip=74.125.231.205 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="UvA100iB" Received: by mail-oi2-f13.google.com with SMTP id 5614622812f47-4c4bf3dedf3so25837b6e.3 for ; Thu, 17 Sep 2026 12:38:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1789673912; x=1790278712; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=1zDN4PsZdWLOQV5bbiD9QzTK6IwoPKXQKegMYDTNvdY=; b=UvA100iBYWDObnB8518MeQusvO83Gr0JyUyb/yR81zyibi4zILk6e0ia2LDaN85TH3 iYpz/z0YtQZx45+1KMQD3+7hdGzNb1XrG193ZeHApFLJk4VP/ObpSFcW2D9bVtNFKn6N rfP+3FzZmZHldGg8GBu3MNUMwE7th328OVmfJtCRZMyhOQF7KXMOzLNbwoXhppF+xUnw fYUCt8o6yKY3BCFBujQ4FyOL3QYND2jWu4+wvJMBPN6rC0BPILQTuGCvPZVLAVo8TTYD NrvuIotrdUgSePNuKDplfACFeZ0IGZphPCf5Bw4S0E65E8/79/DlX2hMzKxiDfw/0AQf qCKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789673912; x=1790278712; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=1zDN4PsZdWLOQV5bbiD9QzTK6IwoPKXQKegMYDTNvdY=; b=zGvqwe4EdtVt2BKBEsthY2ugrLpEbtzCwSu4TCqid74iIB8+Y3/DfYZHumZT1JT3lB QwW3VllTV6nbBqElzcZAAF+h03nXHxTV0iPUd0iXLghuAwV/4fxJbc73BWasSxy9NRRQ mt8wRapRW/i9CRVxPNM+pzHWinuoGyStQrrY7e0XyFD7hT30H3u2rBg69ELZnHgYDofN lKccTH4kJu6xr1oVXhAX65lRDxAqHFPTH0HzyXB+IEARD6DlU0YEyf5ljHSTLOJNFjMw ToaVsPMXGypPCjBlKiwLWG3erDTAwdmgcHamPMKdInilbjih0QqAkW2uTdpkH5DCwqT6 0GhA== X-Forwarded-Encrypted: i=1; AKwUvBw5Unor606bTZr47mxmpK7vvGh+giFV3v20LT8pjOp2CsrVehDBjB6WiLYS+5j+Vc5BYVF/a/yjILho7rg=@vger.kernel.org X-Gm-Message-State: AFuF++nXCHBfdJHS3Xx/k/LYv0z9TawhyhPdmaWqg/8U/1hLVd7FBAgU EMcoCsYNpF5FzFDLr2fMb7bnzLzWUWLaTXpAe/vprK3kuOhpBu5/XyVgqM8T/EoZ5Z0= X-Gm-Gg: AYBFou22H5TrNjiBxFpjapqqp8BLIcn45y1afGsR6Ox6wsPrmWv16YU+QwVficADbj+ XlU6gzlt3/5wkdz+5NjIRYUHT5L6pMy9ZtLh5YulgpoLLCM7aJv7xqco71Yx9HgRZ/oI3jfUTwH F3L0mAAdqameceCcEyhHkNY8Sdu+yMgFlBZNANEqIiDDlIyX89KFYyGDv91aGFIn9vZmg8AG8Kk aTzKxY2z6z2aMioAv24Xcak5rxORxgllWDnHzJtuKDtKCg2MwuK+ceq3meazGthiMa7cXUUAiIN g4Vr6rK4XifWWjV5FKFBFsoBviRu92RTnK8AYreo+UcZY3kONfscCQOM1zH7kbyabwideLnsAY/ znKlkisg19/WS+CNwXe5p6knwyyaPEMFMfhd9pcbs/LZgxyXUs2sxDXDr4M6FRo+n3ec9zQDh67 /U0QbuShbPJvmOzfeiygBK+M0CvqNr/j6L7DdmGWk0EFr8BL6NkT5xA0Rr8OM= X-Received: by 2002:a05:6808:508e:b0:4c1:3047:9d23 with SMTP id 5614622812f47-4ccf8c496ccmr270320b6e.23.1789673912503; Thu, 17 Sep 2026 12:38:32 -0700 (PDT) Received: from [127.0.1.1] ([2a09:bac6:bf21:2632::3ce:23]) by smtp.gmail.com with ESMTPSA id 5614622812f47-4cb6d267495sm3576435b6e.12.2026.09.17.12.38.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2026 12:38:31 -0700 (PDT) From: Chris J Arges Date: Thu, 17 Sep 2026 14:38:22 -0500 Subject: [PATCH net-next v3 1/3] ipv4: hash uncached routes by device Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260917-hash-bucket-route-lists-v3-1-30493a37b6eb@cloudflare.com> References: <20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com> In-Reply-To: <20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com> To: David Ahern , Ido Schimmel , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, Chris J Arges X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1789673908; l=3315; i=carges@cloudflare.com; h=from:subject:message-id; bh=VpMMQFtE0irAVVja4em612tK+k3J5NR1F8chlumvxas=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgaxY1IIT5oTohBZJmhnVgJo2HsM7Sv 9I0LdJCgpeGX6gAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QLL4JZ1UCIcP9np2SFBYtXutgzYaWKx6FR3rXAgvEIAC5fRPVrK+nhDB49SIEqYqEkjGfR+A3rn h3c3NWCw8+Ag= X-Developer-Key: i=carges@cloudflare.com; a=openssh; fpr=SHA256:Cun99EBiH0EV7wvmfTBF9eDrld2NJx+aD4ScWZ45Q5M rt_flush_dev() currently walks every per-CPU uncached route list for each device being removed. This repeatedly examines unrelated routes and makes teardown increasingly expensive as the number of devices grows. Replace each per-CPU list with a hash table keyed by the route's netdevice. Keep the owning-list pointer in dst_entry so route removal remains unchanged, while device teardown only walks the matching bucket on each CPU. Hash collisions are filtered by the existing device comparison. The table has 64 buckets and costs approximately 1.5 KiB per possible CPU on x86-64. Signed-off-by: Chris J Arges --- net/ipv4/route.c | 36 +++++++++++++++++++++++++++++------- 1 file changed, 29 insertions(+), 7 deletions(-) diff --git a/net/ipv4/route.c b/net/ipv4/route.c index d7da2f1acbb5..e5539eda4307 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -74,6 +74,7 @@ #include #include #include +#include #include #include #include @@ -1552,11 +1553,22 @@ struct uncached_list { struct list_head head; }; =20 -static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt_uncached_list); +#define RT_UNCACHED_HASH_BITS 6 +#define RT_UNCACHED_HASH_SIZE BIT(RT_UNCACHED_HASH_BITS) + +struct uncached_table { + struct uncached_list buckets[RT_UNCACHED_HASH_SIZE]; +}; + +static DEFINE_PER_CPU_ALIGNED(struct uncached_table, rt_uncached_table); =20 void rt_add_uncached_list(struct rtable *rt) { - struct uncached_list *ul =3D raw_cpu_ptr(&rt_uncached_list); + struct uncached_table *table =3D raw_cpu_ptr(&rt_uncached_table); + struct uncached_list *ul; + + ul =3D &table->buckets[hash_ptr(dst_dev(&rt->dst), + RT_UNCACHED_HASH_BITS)]; =20 rt->dst.rt_uncached_list =3D ul; =20 @@ -1588,14 +1600,18 @@ void rt_flush_dev(struct net_device *dev) int cpu; =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D &per_cpu(rt_uncached_list, cpu); + struct uncached_table *table; + struct uncached_list *ul; + + table =3D per_cpu_ptr(&rt_uncached_table, cpu); + ul =3D &table->buckets[hash_ptr(dev, RT_UNCACHED_HASH_BITS)]; =20 if (list_empty(&ul->head)) continue; =20 spin_lock_bh(&ul->lock); list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { - if (rt->dst.dev !=3D dev) + if (dst_dev(&rt->dst) !=3D dev) continue; rcu_assign_pointer(rt->dst.dev_rcu, blackhole_netdev); netdev_ref_replace(dev, blackhole_netdev, @@ -3771,10 +3787,16 @@ int __init ip_rt_init(void) ip_tstamps =3D idents_hash + (ip_idents_mask + 1) * sizeof(*ip_idents); =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D &per_cpu(rt_uncached_list, cpu); + struct uncached_table *table; + int bucket; + + table =3D per_cpu_ptr(&rt_uncached_table, cpu); + for (bucket =3D 0; bucket < RT_UNCACHED_HASH_SIZE; bucket++) { + struct uncached_list *ul =3D &table->buckets[bucket]; =20 - INIT_LIST_HEAD(&ul->head); - spin_lock_init(&ul->lock); + INIT_LIST_HEAD(&ul->head); + spin_lock_init(&ul->lock); + } } #ifdef CONFIG_IP_ROUTE_CLASSID ip_rt_acct =3D __alloc_percpu(256 * sizeof(struct ip_rt_acct), __alignof_= _(struct ip_rt_acct)); --=20 2.43.0 From nobody Fri Sep 25 01:22:31 2026 Received: from mail-oi2-f13.google.com (mail-oi2-f13.google.com [74.125.231.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 504B84E36F4 for ; Thu, 17 Sep 2026 19:38:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.231.205 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789673919; cv=none; b=r+SPhoP52UcKNfs3nbmGFzGzsfWT09HWctb/HmvSB5WUA7PVGcmPyryKBm+suVFNC3wk2xh6tJ0HTXeH2043oeW+4lc9BbIutSG/+sd2K++LTELIepmhy/GZvXxzLb7zIt0SAB6FzjxhvzzlbaTSULPwcZ4A0UAmCtFNrzHJSrc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789673919; c=relaxed/simple; bh=LCVxHDVLkVrfWGjpRh2U9cCAwKrh9rkmPQLBh2nzJ1Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZYMKGB7ezqFYw/gcmhm9yaxLRk/G7i4wuCStAXR3q0u6AjG8pWdliZFlzkT7CHT3bv6gmPI96uzr6eLe++alJaQZamcs3+O63QVH5FAOC3YIk0MzxgtykWU+A6M2EpMbDkOZbdkra79yplAsnhwpx3efz+nM811QcrEhhIiDhO4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b=DNj5ETlk; arc=none smtp.client-ip=74.125.231.205 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="DNj5ETlk" Received: by mail-oi2-f13.google.com with SMTP id 5614622812f47-4c4bf3dedf3so25861b6e.3 for ; Thu, 17 Sep 2026 12:38:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1789673915; x=1790278715; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=DB7q2xGiAnPtsBjkJ+5oYrdPscazpDKtw7xgni/idcI=; b=DNj5ETlkQL7wJOaf3UtsRMlYxUl/CmmdZUfwmv3y50hOkmmA0rsUK11970E4SRGyMt dDz7Rpdb+v3sOt5Ep20BgEvxsja0bEUoFRwYLpxETJu6MU+lgRJg3bxEMteXgxWrSzF7 DlQydwsSPtEal06JyropgA99hGkchqCOp9t+b5cHajrN2OPItnOnmRVQFkrloRsrtIuN zoKi7BMva2FXn7U633HnlbFbWrlAfu+QyvXr8Og3VIMNqRWhQ6HqoDadnz8HdmuIWMMV iXWvWBIYAHuf+x5V/+7ylqJGjw0Q4wOJvprOUKLNlI5dD+vIPsd/ox4ydbH/q70xXjlN mpMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789673915; x=1790278715; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=DB7q2xGiAnPtsBjkJ+5oYrdPscazpDKtw7xgni/idcI=; b=WQMF7ICKAyvzy2HmSjA6+qH75ylINzfxuB1FJ+TxXLudsUeluHLT1MSRtCNDvTbE7T mb+l5IaWfGXDlBWs2pXWH8t4OmDNmfursm5INyvtU5laGF17o0X51FR669Ptq0K3WVnw J71CtmgSfOfPJR3emIbb80a0IrLOTGh7ZA7OL5PUjGi8WVtsn7LZpGSRW0cFAJlyaUDq bvM2fUJ8fCTtr1LJiDigXPBIRpPtjiAAz1ekotRTZIdnWDO1iMsy5Jq+IJBLPOJ0H0Mw u5ByZGzr3zhpF/agj8pEY7znFZqAKsG5ZU2CiSjb4b0jom7y7vjDhkpT8I+ygFJ9Zdm8 sAaw== X-Forwarded-Encrypted: i=1; AKwUvBzmk19icfaA/D9diLXZe2SEaeQ7oEloGp/VtQ9K1sU8oEnjM5BHn0WkBx2OSr1abAHWxe3w0bB3Q98h3dI=@vger.kernel.org X-Gm-Message-State: AFuF++livx6gzugT2jpCKi9di5XJhjOl7xTS+pW8YTk/r9pOZ5Js9mJm 96BPaRJgnMGPS+omqWaifgxl+sMAVY3PR2IfVLeJADpr2fBtWgspgakkJV7MzUV/Ic0= X-Gm-Gg: AYBFou0Zy/bKuZyxiSTOSb6bxZhK9N2lvcH5s8SlAFqlhTixB/J7PI/wTDD1zhLjsHP Bh6haMw2Eg92eZfvWMekxKKx44IqTMs1woV1adw4AEyZU8OiS9nWB0Hly3mATOjTX9HkSyDJSYf DBV7pGu6uMNJ377Vv+eBA/11hAft8OdlXvSLXZqPzLNc9drM0AQqGbOqeF/6QIng9cMCD3y+dBU ebtWzn5fqsUaL50LMcIp6/ki6jHWJV4qsroNGlonmS18oMMSNvMlreaLejFgWyKTRDIwhDuY2aT e98+AMqxHpQFvSwBU0f+VWT9IXFHZG4ghfd4+Kgor6561x+Gwfgn62fpz3hGpTiYDh8WipxjakS j8GVpSY9X91llmC9N4VmWYVduyu1qrZ6sbiXWj0hZfoVxqzFYF2gCQnSYL4ReJazDO3WG2E0tcK hWxlJOKRtdBLPZlh0HiczxsvWff6S44cSwJYKxbYzxMzh6yhPY2L8iLEiJDw4= X-Received: by 2002:a05:6808:c1e3:b0:4c4:1c0:ca38 with SMTP id 5614622812f47-4ccf9789f55mr257421b6e.34.1789673915107; Thu, 17 Sep 2026 12:38:35 -0700 (PDT) Received: from [127.0.1.1] ([2a09:bac6:bf21:2632::3ce:23]) by smtp.gmail.com with ESMTPSA id 5614622812f47-4cb6d267495sm3576435b6e.12.2026.09.17.12.38.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2026 12:38:33 -0700 (PDT) From: Chris J Arges Date: Thu, 17 Sep 2026 14:38:23 -0500 Subject: [PATCH net-next v3 2/3] ipv6: hash uncached routes by device Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260917-hash-bucket-route-lists-v3-2-30493a37b6eb@cloudflare.com> References: <20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com> In-Reply-To: <20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com> To: David Ahern , Ido Schimmel , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, Chris J Arges X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1789673908; l=5146; i=carges@cloudflare.com; h=from:subject:message-id; bh=LCVxHDVLkVrfWGjpRh2U9cCAwKrh9rkmPQLBh2nzJ1Q=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgaxY1IIT5oTohBZJmhnVgJo2HsM7Sv 9I0LdJCgpeGX6gAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QKoxtUisNdCVWi7GjJ/v8b0/ap8IUViZkPPi5uEDQDBw7ewZJbHArKtgiqL/FcMJi4JthhUIQS5 2tO/uXmihsQM= X-Developer-Key: i=carges@cloudflare.com; a=openssh; fpr=SHA256:Cun99EBiH0EV7wvmfTBF9eDrld2NJx+aD4ScWZ45Q5M rt6_uncached_list_flush_dev() currently walks every per-CPU uncached route list for each device being removed. Hash uncached routes by their inet6 device so ordinary device teardown only visits the matching bucket on each CPU. ip6_rt_get_dev_rcu() can return loopback or an L3 master while rt6i_idev still refers to the original interface. Key routes by rt6i_idev->dev when available and fall back to dst_dev(). Ordinary devices then require one bucket scan. Because loopback and L3 masters can instead be referenced by dst_dev(), scan all buckets when one of those devices is removed. This avoids growing struct rt6_info while filtering most unrelated routes from ordinary device teardown. The table has 64 buckets and costs approximately 1.5 KiB per possible CPU on x86-64. Signed-off-by: Chris J Arges --- net/ipv6/route.c | 101 +++++++++++++++++++++++++++++++++++++--------------= ---- 1 file changed, 69 insertions(+), 32 deletions(-) diff --git a/net/ipv6/route.c b/net/ipv6/route.c index 7535b09068a0..cda81e91be65 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -133,11 +134,23 @@ struct uncached_list { struct list_head head; }; =20 -static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt6_uncached_list); +#define RT6_UNCACHED_HASH_BITS 6 +#define RT6_UNCACHED_HASH_SIZE BIT(RT6_UNCACHED_HASH_BITS) + +struct rt6_uncached_table { + struct uncached_list buckets[RT6_UNCACHED_HASH_SIZE]; +}; + +static DEFINE_PER_CPU_ALIGNED(struct rt6_uncached_table, rt6_uncached_tabl= e); =20 void rt6_uncached_list_add(struct rt6_info *rt) { - struct uncached_list *ul =3D raw_cpu_ptr(&rt6_uncached_list); + struct rt6_uncached_table *table =3D raw_cpu_ptr(&rt6_uncached_table); + struct uncached_list *ul; + struct net_device *dev; + + dev =3D rt->rt6i_idev ? rt->rt6i_idev->dev : dst_dev(&rt->dst); + ul =3D &table->buckets[hash_ptr(dev, RT6_UNCACHED_HASH_BITS)]; =20 rt->dst.rt_uncached_list =3D ul; =20 @@ -157,40 +170,58 @@ void rt6_uncached_list_del(struct rt6_info *rt) } } =20 +static void rt6_uncached_list_flush(struct uncached_list *ul, + struct net_device *dev) +{ + struct rt6_info *rt, *safe; + + if (list_empty(&ul->head)) + return; + + spin_lock_bh(&ul->lock); + list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { + struct net_device *rt_dev =3D dst_dev(&rt->dst); + struct inet6_dev *rt_idev =3D rt->rt6i_idev; + bool handled =3D false; + + if (rt_idev && rt_idev->dev =3D=3D dev) { + rt->rt6i_idev =3D in6_dev_get(blackhole_netdev); + in6_dev_put(rt_idev); + handled =3D true; + } + + if (rt_dev =3D=3D dev) { + rcu_assign_pointer(rt->dst.dev_rcu, blackhole_netdev); + netdev_ref_replace(rt_dev, blackhole_netdev, + &rt->dst.dev_tracker, GFP_ATOMIC); + handled =3D true; + } + if (handled) + list_del_init(&rt->dst.rt_uncached); + } + spin_unlock_bh(&ul->lock); +} + static void rt6_uncached_list_flush_dev(struct net_device *dev) { + bool scan_all =3D dev->flags & IFF_LOOPBACK || netif_is_l3_master(dev); int cpu; =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D per_cpu_ptr(&rt6_uncached_list, cpu); - struct rt6_info *rt, *safe; - - if (list_empty(&ul->head)) + struct rt6_uncached_table *table; + struct uncached_list *ul; + int bucket; + + table =3D per_cpu_ptr(&rt6_uncached_table, cpu); + if (!scan_all) { + ul =3D &table->buckets[hash_ptr(dev, + RT6_UNCACHED_HASH_BITS)]; + rt6_uncached_list_flush(ul, dev); continue; - - spin_lock_bh(&ul->lock); - list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { - struct inet6_dev *rt_idev =3D rt->rt6i_idev; - struct net_device *rt_dev =3D rt->dst.dev; - bool handled =3D false; - - if (rt_idev && rt_idev->dev =3D=3D dev) { - rt->rt6i_idev =3D in6_dev_get(blackhole_netdev); - in6_dev_put(rt_idev); - handled =3D true; - } - - if (rt_dev =3D=3D dev) { - rt->dst.dev =3D blackhole_netdev; - netdev_ref_replace(rt_dev, blackhole_netdev, - &rt->dst.dev_tracker, - GFP_ATOMIC); - handled =3D true; - } - if (handled) - list_del_init(&rt->dst.rt_uncached); } - spin_unlock_bh(&ul->lock); + + for (bucket =3D 0; bucket < RT6_UNCACHED_HASH_SIZE; bucket++) + rt6_uncached_list_flush(&table->buckets[bucket], dev); } } =20 @@ -6987,10 +7018,16 @@ int __init ip6_route_init(void) #endif =20 for_each_possible_cpu(cpu) { - struct uncached_list *ul =3D per_cpu_ptr(&rt6_uncached_list, cpu); + struct rt6_uncached_table *table; + int bucket; + + table =3D per_cpu_ptr(&rt6_uncached_table, cpu); + for (bucket =3D 0; bucket < RT6_UNCACHED_HASH_SIZE; bucket++) { + struct uncached_list *ul =3D &table->buckets[bucket]; =20 - INIT_LIST_HEAD(&ul->head); - spin_lock_init(&ul->lock); + INIT_LIST_HEAD(&ul->head); + spin_lock_init(&ul->lock); + } } =20 out: --=20 2.43.0 From nobody Fri Sep 25 01:22:31 2026 Received: from mail-oi2-f12.google.com (mail-oi2-f12.google.com [74.125.231.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50F794E50A9 for ; Thu, 17 Sep 2026 19:38:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.231.204 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789673921; cv=none; b=ZGMNortZyoqF1X4tqmlG47FAa4Qxx4c3BBQ+/nwjR6sNW/lYuRtRBNZdeUgRNhO8IFe7eENbxPmRsKDqH3G1Dk1ht1gfRoUw7EAox6b0IaeTxYdTTMpRGRf4oNPZ/NF7dNUMaz1KQcJOiK9StM8IwGKmZSazoUSjFubxHNR5UDQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789673921; c=relaxed/simple; bh=8WdFP1v7/Oa+dKIJynboqHB29B7wDjj6AhOsc16jvtI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YP1yBPXfYaIkVOin7QARjQ++7giNMniA7vm2hDKgThtX6wAni4pP+AVZ20/yJWoDWp7JJM0EN4AybWfPyj9DzyFvBOaF3ZKNSK7lEcVEyVtL6JAB8OCekYT66Gitzskq8x22ulhhcwjo+BA0/PBdYIaHtpVEvX0e7PhU5xtoLLI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com; spf=pass smtp.mailfrom=cloudflare.com; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b=Qbvo7E6y; arc=none smtp.client-ip=74.125.231.204 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloudflare.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="Qbvo7E6y" Received: by mail-oi2-f12.google.com with SMTP id 5614622812f47-4b37a3c0d7aso7818b6e.1 for ; Thu, 17 Sep 2026 12:38:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google09082023; t=1789673918; x=1790278718; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=63GI428HLg6svgdiqmZ8ZkijlWs39+VHkT3kxwHKcMM=; b=Qbvo7E6yrNTNGp9RMmGVEJ7mvDowF6LmpBLTdg/Af0DUVt0dUM4pB+Zvi7oYIEKkkj 9ZJRd4hysWWzsHcaa7oGnOqX0li215pPndCf5WSmLvgnCfFrKt2heTnm0Gmt7y4fjJbK 68cFDBxe/LaRy9L72DNGc24bO5bM+aMsIsYz88BoSDaKk+v53Tsf3GJ8TuJv46W81GVo t2ok+lRuAzGILeMeEiN3soOORuA3ahERQ+NMwuaIqffl2EGScwugplwjk2YYWaWom469 K8rkGSqpNGt0rNwyb3gbTpTpC3WWbSIudCGb3M4WWZPT01hVDRBrSYwmcm09UMmfRTl8 cQVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789673918; x=1790278718; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=63GI428HLg6svgdiqmZ8ZkijlWs39+VHkT3kxwHKcMM=; b=Ca8A0h5bPNouNaOO9NXHBcHr1rUYt5HwWGSYxiteeP0+qtLEQRy5I/HwhVAlJFE2Fr TMGeSAunOi8nlKo74iKTBoWTvTQL0Yhx6ZEwSFHtSO+a8TSrPiiDupJwIp+SKlYQi/zL cOmZTNZAgU/ceBZVyzbCDogxyOi0L5Uubo64ioMxMgbX/MAf1K04k3xGLKAM+V0T2EJg XWOKIeWZfeev0mET0O50Fj6+UIY+f70HNe4yRO18m1JIOTNU4U1FCHkzUicjDBYP3RCK LqdRCZXDPvzF38B/LqTov2+W2W2Bwd+NlYX2Xeug2AIrtTpIRZaw8lEnzkLLNhQTZpxI h/PQ== X-Forwarded-Encrypted: i=1; AKwUvBxV/31Dj9YYh9qxDL9hkizBBLZSAAMz6XqzqnXo0H0ovw3MwGAZD5K6zk5oV0t7rWIg4kzIz2/fMhRS/40=@vger.kernel.org X-Gm-Message-State: AFuF++m8rjy8UfH1HvIL971kQof63GW6MrhvfLj60CDDRX/5sUizmwGp PC/s60KWTDDRulX3oQM9eQ2dSC01Qh2Ce7J/1aeemF3m/c6tVIflXxjRH0x9xA0X96s= X-Gm-Gg: AYBFou1Ld/ePVPz+jFBm+JgoWrbXG+6tSSc0U+utq1+5hi1NdeYci2yU0klq14r71Sm 9vxGleQ4T1W4xUTfq1RYYonM17eNZh6LRYv3aC940Nq+LvpQ+FOVA/FmZu8Qv7cUep8ZPnsAuLV eHupt7UYmW/CT5sqb+Woa506dbS41wXoBFiLDyXnIoBi7SJnohlH2fIQT3j1eQAFcvUI1ChIZNt uTwH3WUSz4TNKOr+TmX6aOV5lFZVdCi7Xb/RjQi7AYb7ubsm6+7zkJJTmqmees0Sm7nCyqfwjdq elCvHcCYdZR9e+gSaTA8p4tMLp2rze9Rl1YN+R47EsYcO3l8CLxKVYBnAmNIoEl+BYTp9TMuJJ2 5QJkpgsRTTNJCkXoasjm/V70hZJXbKH+7xXFNa+PfwoLt4Ogor06BUX1rpkMcEviV3HWwn1wx42 YJHz8HUEuZ54NL1J2OzvMQaZDJXjfJ7rcBnP6fwOtYgFvUo0c3nlYgVGaCbhIs X-Received: by 2002:a05:6808:1822:b0:4cb:26a3:a092 with SMTP id 5614622812f47-4cd00f722f7mr143159b6e.18.1789673917758; Thu, 17 Sep 2026 12:38:37 -0700 (PDT) Received: from [127.0.1.1] ([2a09:bac6:bf21:2632::3ce:23]) by smtp.gmail.com with ESMTPSA id 5614622812f47-4cb6d267495sm3576435b6e.12.2026.09.17.12.38.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2026 12:38:36 -0700 (PDT) From: Chris J Arges Date: Thu, 17 Sep 2026 14:38:24 -0500 Subject: [PATCH net-next v3 3/3] selftests: net: cover IPv6 uncached route device mismatch Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260917-hash-bucket-route-lists-v3-3-30493a37b6eb@cloudflare.com> References: <20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com> In-Reply-To: <20260917-hash-bucket-route-lists-v3-0-30493a37b6eb@cloudflare.com> To: David Ahern , Ido Schimmel , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@cloudflare.com, Chris J Arges X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1789673908; l=2338; i=carges@cloudflare.com; h=from:subject:message-id; bh=8WdFP1v7/Oa+dKIJynboqHB29B7wDjj6AhOsc16jvtI=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgaxY1IIT5oTohBZJmhnVgJo2HsM7Sv 9I0LdJCgpeGX6gAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QO1RwsZDwM/ssWO1I1sKt1WfupSScxRHMASpKykF6ut24vhcFE8p3yTn1fxnRNYklXMR/RihLXH dvewFKZS+iQc= X-Developer-Key: i=carges@cloudflare.com; a=openssh; fpr=SHA256:Cun99EBiH0EV7wvmfTBF9eDrld2NJx+aD4ScWZ45Q5M Local IPv6 routes through a VRF can use the VRF as dst.dev while retaining the VRF member interface in rt6i_idev. Exercise device teardown while such uncached routes are retained by a delayed qdisc. Reuse the existing VRF topology and msg_zerocopy raw-header sender, and verify route creation, qdisc retention, and prompt interface deletion. Signed-off-by: Chris J Arges --- tools/testing/selftests/net/vrf-xfrm-tests.sh | 35 +++++++++++++++++++++++= ++++ 1 file changed, 35 insertions(+) diff --git a/tools/testing/selftests/net/vrf-xfrm-tests.sh b/tools/testing/= selftests/net/vrf-xfrm-tests.sh index b64dd891699d..4f409d135a99 100755 --- a/tools/testing/selftests/net/vrf-xfrm-tests.sh +++ b/tools/testing/selftests/net/vrf-xfrm-tests.sh @@ -385,6 +385,37 @@ run_tests() cleanup_xfrm_dev } =20 +test_ipv6_uncached_mismatch() +{ + local sender_pid + local backlog + local rc + + # A local route through a VRF uses the VRF as dst.dev while retaining + # the VRF member interface in rt6i_idev. Raw header sends create uncached + # routes, and netem keeps them referenced while the interface is deleted. + run_cmd_host1 tc qdisc replace dev ${VRF} root netem limit 1 delay 10s + ip -6 -netns "$host1" route add local ${HOST1_6}/128 dev eth0 + ip netns exec "$host1" ./msg_zerocopy -6 \ + -S ${HOST1_6} -D ${HOST1_6} -s 1200 -t 0 raw_hdrincl \ + >/dev/null 2>&1 & + sender_pid=3D$! + wait "$sender_pid" + rc=3D$? + log_test $rc 0 "Create uncached IPv6 routes with mismatched devices" + [ $rc -ne 0 ] && return + + backlog=3D$(ip netns exec "$host1" tc -s qdisc show dev ${VRF}) + if ! echo "$backlog" | grep -Eq 'backlog .* [1-9][0-9]*p'; then + log_test 1 0 "Retain uncached IPv6 routes in VRF qdisc" + return + fi + log_test 0 0 "Retain uncached IPv6 routes in VRF qdisc" + + run_cmd_host1 timeout 2 ip link del eth0 + log_test $? 0 "Flush uncached IPv6 routes with mismatched devices" +} + ##########################################################################= ###### # usage =20 @@ -425,6 +456,10 @@ echo echo "netem qdisc on VRF device" run_tests =20 +echo +echo "Uncached IPv6 route with mismatched devices" +test_ipv6_uncached_mismatch + printf "\nTests passed: %3d\n" ${nsuccess} printf "Tests failed: %3d\n" ${nfail} =20 --=20 2.43.0