From nobody Sat Feb 7 21:14:52 2026 Received: from out-183.mta1.migadu.com (out-183.mta1.migadu.com [95.215.58.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C4DC2D94B2 for ; Sun, 26 Oct 2025 15:40:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.183 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493246; cv=none; b=EQ8ezQBKkWsFje7uehTzKcphHqcR8eC1sc1qwpDJM/eOfi7+HPpr0JzU9ekiBQyFqkzGXEHmwErVJKkst693p+m4kKFTwrEhGLri5fmSdSloXaKrc2BSBJ8Nn4as1izs2fEFOsb603zqBIQk623d15D35yTOtAO3crtAB2VKRO0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493246; c=relaxed/simple; bh=s3IR1Rj8Ag8368KG/w3P+8z2VWNFjQ++UP5BSrFxkz4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FCv9SzVUno5qYgO1CdU/kmq3Ldoyh4qQTCW+lN+qaSpwFDHdDX32bBFhLpm8vejg8gw481P3NSQ1u5Vg48qNS6mrui6i8pgjEEL7r8VRKHFvd3cRJD7ZQQu2IYWK+1aiXIMMRZf0zHQE2BdZ7OvqCO8rvEvA4W99vPpM0F28xto= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=UQaFgV0j; arc=none smtp.client-ip=95.215.58.183 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="UQaFgV0j" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1761493240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aN8oN34iZoLpzlmFeG3uFVvEYqjiK8nko/J9BQXfRs0=; b=UQaFgV0j7ML9bj3I2e0WrNsGpkRYULMaUbidt6LMoqhzh3nA3pqLkKV+S017bfuXE2cPyX PTQ2Hql8YFXCEEa1YQM3xHo8jm2cxTRr3zCF2H5aUUfpE6G0N/Kyfzptq4PDMYzoqeItBy yhU3rP3z9MXWPNiPAG5IB/gP/ue5QmM= From: Leon Hwang To: bpf@vger.kernel.org Cc: ast@kernel.org, andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, memxor@gmail.com, linux-kernel@vger.kernel.org, kernel-patches-bot@fb.com, Leon Hwang Subject: [PATCH bpf v3 1/4] bpf: Free special fields when update [lru_,]percpu_hash maps Date: Sun, 26 Oct 2025 23:39:57 +0800 Message-ID: <20251026154000.34151-2-leon.hwang@linux.dev> In-Reply-To: <20251026154000.34151-1-leon.hwang@linux.dev> References: <20251026154000.34151-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" As [lru_,]percpu_hash maps support BPF_KPTR_{REF,PERCPU}, missing calls to 'bpf_obj_free_fields()' in 'pcpu_copy_value()' could cause the memory referenced by BPF_KPTR_{REF,PERCPU} fields to be held until the map gets freed. Fix this by calling 'bpf_obj_free_fields()' after 'copy_map_value[,_long]()' in 'pcpu_copy_value()'. Fixes: 65334e64a493 ("bpf: Support kptrs in percpu hashmap and percpu LRU h= ashmap") Signed-off-by: Leon Hwang --- kernel/bpf/hashtab.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index c2fcd0cd51e51..26308adc9ccb3 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -950,12 +950,14 @@ static void pcpu_copy_value(struct bpf_htab *htab, vo= id __percpu *pptr, if (!onallcpus) { /* copy true value_size bytes */ copy_map_value(&htab->map, this_cpu_ptr(pptr), value); + bpf_obj_free_fields(htab->map.record, this_cpu_ptr(pptr)); } else { u32 size =3D round_up(htab->map.value_size, 8); int off =3D 0, cpu; =20 for_each_possible_cpu(cpu) { copy_map_value_long(&htab->map, per_cpu_ptr(pptr, cpu), value + off); + bpf_obj_free_fields(htab->map.record, per_cpu_ptr(pptr, cpu)); off +=3D size; } } --=20 2.51.0 From nobody Sat Feb 7 21:14:52 2026 Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [95.215.58.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F7722D12EC for ; Sun, 26 Oct 2025 15:40:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493251; cv=none; b=VOWKSgxvlqkuxmdqQTtl25jWvDVEKMbkuMwqJffT7AvT2vlqFUNDRN9Al4myE35TDi4P4T1XX0i0/aQAlPNuY3ddyn9P0Lr6xlXmLowM/rTRN0pmL3SsD/qoi21XKUmJsT4YUykgDuW43a9XBuwVJ4PpHflQTEdBruHn9DMJ2Gs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493251; c=relaxed/simple; bh=gaTmvfHNFqge4dQ7wx+Ukv4Yxs/3l/QORPaBttPJ8Jo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XB8jy+0t/4Qss1T4Wx5IvQdTm9VfzNAW0SuuezsjzPBZaFKjP0etnLHNSYk55FFYO+IB+kWhrcsGXyTyee2dFh05LJtPQrZe5i1seXje3mqukNLaWP8wce8st6+jo4/yjo2JvlZutBY8OvJ2nCOQBaP0ofECN1g4i7mNNOfBG3M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=aXPAr817; arc=none smtp.client-ip=95.215.58.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="aXPAr817" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1761493247; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ElRnlDcK8q21WZxBfi9bb0x6nOn4UNgvLQNmGbaQ58Q=; b=aXPAr817+KtXmB1j1jeq1VWjgf8ZRwo5QVlReVpTd1uETw2pD7ZZ406AdYo/97fuz6zVtz EeZlUuuHI+uXDNjPEvr39fU0EWjjmvjAJsFkCiwApsxl0deG14lzY94eCGfDXQGXp8eBOh M+bpAE7iL60/4XaJYUvs8WLDR1gQFqo= From: Leon Hwang To: bpf@vger.kernel.org Cc: ast@kernel.org, andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, memxor@gmail.com, linux-kernel@vger.kernel.org, kernel-patches-bot@fb.com, Leon Hwang Subject: [PATCH bpf v3 2/4] bpf: Free special fields when update hash maps with BPF_F_LOCK Date: Sun, 26 Oct 2025 23:39:58 +0800 Message-ID: <20251026154000.34151-3-leon.hwang@linux.dev> In-Reply-To: <20251026154000.34151-1-leon.hwang@linux.dev> References: <20251026154000.34151-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" When updating hash maps with BPF_F_LOCK, the special fields were not freed after being replaced. This could cause memory referenced by BPF_KPTR_{REF,PERCPU} fields to be held until the map gets freed. Fix this by calling 'check_and_free_fields()' after 'copy_map_value_locked()' to properly release the old fields. Fixes: 14a324f6a67e ("bpf: Wire up freeing of referenced kptr") Signed-off-by: Leon Hwang --- kernel/bpf/hashtab.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 26308adc9ccb3..65009ea3e9379 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -1124,6 +1124,7 @@ static long htab_map_update_elem(struct bpf_map *map,= void *key, void *value, copy_map_value_locked(map, htab_elem_value(l_old, key_size), value, false); + check_and_free_fields(htab, l_old); return 0; } /* fall through, grab the bucket lock and lookup again. @@ -1152,6 +1153,7 @@ static long htab_map_update_elem(struct bpf_map *map,= void *key, void *value, copy_map_value_locked(map, htab_elem_value(l_old, key_size), value, false); + check_and_free_fields(htab, l_old); ret =3D 0; goto err; } --=20 2.51.0 From nobody Sat Feb 7 21:14:52 2026 Received: from out-174.mta1.migadu.com (out-174.mta1.migadu.com [95.215.58.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F3FF253B58; Sun, 26 Oct 2025 15:40:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493259; cv=none; b=NQwrCrhx5M16MQJjs5LaPl0uVtTVSVh3kAdkd3KloBlpIuevkipV9W6wvLoU84kTkL8MBP9pSt9Q3h7ok99szmLuiazDUR3DM48Fw195Xe9c7sIZzjy0N+OeINgkO0dlVxS4hbACDC2k7uIPmY2In+k74/QM0yIo+ffll/7Itnk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493259; c=relaxed/simple; bh=X7SifmAEQtPtGTwptFSvQQj76o4yROrHVpi4rvR1QjQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h5cehDZoiHLWxv9ZRRIPkzgsf/FoFX+HWcKLRSAT7nXef/6ZEQ5AUAKF7GdUw44zD0d+lfu9J5WizdU5FJIOT0siORSxAnnh+Ks8SXND5ZtHEftti23ZO2gMCLrqa+yskHNlGySQmNKRMtJvEZ/0RbJ4mZQrhUA3/0GvhLOwNp0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Fx8yniAq; arc=none smtp.client-ip=95.215.58.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Fx8yniAq" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1761493256; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8AAgrBQP2Y1ONLw22Ac4cfKW6J+sJUDTFnmuPayNPe4=; b=Fx8yniAqIBRNggOhKdexNiDLm5Ri8VYEqPP7gmcl263G3X8A2OJ6OdKQFsH0FPTL6PBIlV hnGWGeFqF6WbU38if0GeGCNZxj3U6aLgLYbnhkOLgTmUYuhZY/C/zDE7NyLV4I0if5qYYu MhtLdklmKAkti5SAwmgHHqv0ef+mCgc= From: Leon Hwang To: bpf@vger.kernel.org Cc: ast@kernel.org, andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, memxor@gmail.com, linux-kernel@vger.kernel.org, kernel-patches-bot@fb.com, Leon Hwang Subject: [PATCH bpf v3 3/4] bpf: Free special fields when update local storage maps Date: Sun, 26 Oct 2025 23:39:59 +0800 Message-ID: <20251026154000.34151-4-leon.hwang@linux.dev> In-Reply-To: <20251026154000.34151-1-leon.hwang@linux.dev> References: <20251026154000.34151-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" When updating local storage maps with BPF_F_LOCK on the fast path, the special fields were not freed after being replaced. This could cause memory referenced by BPF_KPTR_{REF,PERCPU} fields to be held until the map gets freed. Similarly, on the other path, the old sdata's special fields were never freed regardless of whether BPF_F_LOCK was used, causing the same issue. Fix this by calling 'bpf_obj_free_fields()' after 'copy_map_value_locked()' to properly release the old fields. Fixes: 9db44fdd8105 ("bpf: Support kptrs in local storage maps") Signed-off-by: Leon Hwang --- kernel/bpf/bpf_local_storage.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index b931fbceb54da..8e3aea4e07c50 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -609,6 +609,7 @@ bpf_local_storage_update(void *owner, struct bpf_local_= storage_map *smap, if (old_sdata && selem_linked_to_storage_lockless(SELEM(old_sdata))) { copy_map_value_locked(&smap->map, old_sdata->data, value, false); + bpf_obj_free_fields(smap->map.record, old_sdata->data); return old_sdata; } } @@ -641,6 +642,7 @@ bpf_local_storage_update(void *owner, struct bpf_local_= storage_map *smap, if (old_sdata && (map_flags & BPF_F_LOCK)) { copy_map_value_locked(&smap->map, old_sdata->data, value, false); + bpf_obj_free_fields(smap->map.record, old_sdata->data); selem =3D SELEM(old_sdata); goto unlock; } @@ -654,6 +656,7 @@ bpf_local_storage_update(void *owner, struct bpf_local_= storage_map *smap, =20 /* Third, remove old selem, SELEM(old_sdata) */ if (old_sdata) { + bpf_obj_free_fields(smap->map.record, old_sdata->data); bpf_selem_unlink_map(SELEM(old_sdata)); bpf_selem_unlink_storage_nolock(local_storage, SELEM(old_sdata), true, &old_selem_free_list); --=20 2.51.0 From nobody Sat Feb 7 21:14:52 2026 Received: from out-180.mta1.migadu.com (out-180.mta1.migadu.com [95.215.58.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC78E253B58 for ; Sun, 26 Oct 2025 15:41:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493269; cv=none; b=HKsd+XK2EcNLhoGdF/pfZHlnTXQKHXYt98aODZI34XhswqBVdxbYQidl3eH58BAxwhaco5zuRIe28YwA6ksoPyjs+Q/OnSKrqtu0y5/wkffdSauUzl3zBWe6uMGjPCKNh40c+0499qQSLFIfHI9lNNLoGobqIEugVktyPa0MrVU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761493269; c=relaxed/simple; bh=C3zkqmEN+0VKTKhHfs/UaVL0bK8vWpsQGcqgmrbpIG4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U6ESj/z2KVK7HIhIgnHFbRhYjCNwOvnZjBJZPiZiQVy2rOGk5Fc5qufutof075AD4RMsZ924ALcSgPOa8CMpjklkXhHq27D2AKYSOq/pAc3SuRQQVpQ3FHZYUFhi/a2gW5NqzoLj1lGfQH4+fPAj0XsWkkn4HwRhUsQ+9qGAvh0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=sFrBEqMr; arc=none smtp.client-ip=95.215.58.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="sFrBEqMr" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1761493264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MqcmwyQ2OLmdmL3C4h4UHcEZUs1IdL0HZCQ+ZeNIF+g=; b=sFrBEqMrXOLWm23MyiVBMoVab2PmNuJdZHQvKcVwS1JKHDNsW1aO97lSJ+xl4T/jd6/TU9 adzf2hPhSzG7+2hLJ8ZzwLaAv5uQrrPAZLT6FXJHugjkNcyZNpvPdK5e9mkDkxYUJ2qSui v+StlZCos1zVEs66EM4OpKeTOYnawAE= From: Leon Hwang To: bpf@vger.kernel.org Cc: ast@kernel.org, andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, memxor@gmail.com, linux-kernel@vger.kernel.org, kernel-patches-bot@fb.com, Leon Hwang Subject: [PATCH bpf v3 4/4] selftests/bpf: Add tests to verify freeing the special fields when update hash and local storage maps Date: Sun, 26 Oct 2025 23:40:00 +0800 Message-ID: <20251026154000.34151-5-leon.hwang@linux.dev> In-Reply-To: <20251026154000.34151-1-leon.hwang@linux.dev> References: <20251026154000.34151-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Add tests to verify that updating hash and local storage maps decrements refcount when BPF_KPTR_REF objects are involved. The tests perform the following steps: 1. Call update_elem() to insert an initial value. 2. Use bpf_refcount_acquire() to increment the refcount. 3. Store the node pointer in the map value. 4. Add the node to a linked list. 5. Probe-read the refcount and verify it is *2*. 6. Call update_elem() again to trigger refcount decrement. 7. Probe-read the refcount and verify it is *1*. Signed-off-by: Leon Hwang --- .../bpf/prog_tests/refcounted_kptr.c | 178 +++++++++++++++++- .../selftests/bpf/progs/refcounted_kptr.c | 160 ++++++++++++++++ 2 files changed, 337 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/refcounted_kptr.c b/too= ls/testing/selftests/bpf/prog_tests/refcounted_kptr.c index d6bd5e16e6372..0a60330a1f4b3 100644 --- a/tools/testing/selftests/bpf/prog_tests/refcounted_kptr.c +++ b/tools/testing/selftests/bpf/prog_tests/refcounted_kptr.c @@ -3,7 +3,7 @@ =20 #include #include - +#include "cgroup_helpers.h" #include "refcounted_kptr.skel.h" #include "refcounted_kptr_fail.skel.h" =20 @@ -44,3 +44,179 @@ void test_refcounted_kptr_wrong_owner(void) ASSERT_OK(opts.retval, "rbtree_wrong_owner_remove_fail_a2 retval"); refcounted_kptr__destroy(skel); } + +static void test_refcnt_leak(void *values, size_t values_sz, u64 flags, st= ruct bpf_map *map, + struct bpf_program *prog_leak, struct bpf_program *prog_check) +{ + int ret, fd, key =3D 0; + LIBBPF_OPTS(bpf_test_run_opts, opts, + .data_in =3D &pkt_v4, + .data_size_in =3D sizeof(pkt_v4), + .repeat =3D 1, + ); + + ret =3D bpf_map__update_elem(map, &key, sizeof(key), values, values_sz, f= lags); + if (!ASSERT_OK(ret, "bpf_map__update_elem init")) + return; + + fd =3D bpf_program__fd(prog_leak); + ret =3D bpf_prog_test_run_opts(fd, &opts); + if (!ASSERT_OK(ret, "test_run_opts")) + return; + if (!ASSERT_EQ(opts.retval, 2, "retval refcount")) + return; + + ret =3D bpf_map__update_elem(map, &key, sizeof(key), values, values_sz, f= lags); + if (!ASSERT_OK(ret, "bpf_map__update_elem dec refcount")) + return; + + fd =3D bpf_program__fd(prog_check); + ret =3D bpf_prog_test_run_opts(fd, &opts); + ASSERT_OK(ret, "test_run_opts"); + ASSERT_EQ(opts.retval, 1, "retval"); +} + +static void test_percpu_hash_refcount_leak(void) +{ + struct refcounted_kptr *skel; + size_t values_sz; + u64 *values; + int cpu_nr; + + cpu_nr =3D libbpf_num_possible_cpus(); + if (!ASSERT_GT(cpu_nr, 0, "libbpf_num_possible_cpus")) + return; + + values =3D calloc(cpu_nr, sizeof(u64)); + if (!ASSERT_OK_PTR(values, "calloc values")) + return; + + skel =3D refcounted_kptr__open_and_load(); + if (!ASSERT_OK_PTR(skel, "refcounted_kptr__open_and_load")) { + free(values); + return; + } + + values_sz =3D cpu_nr * sizeof(u64); + memset(values, 0, values_sz); + + test_refcnt_leak(values, values_sz, 0, skel->maps.pcpu_hash, + skel->progs.pcpu_hash_refcount_leak, + skel->progs.check_pcpu_hash_refcount); + + refcounted_kptr__destroy(skel); + free(values); +} + +struct lock_map_value { + u64 kptr; + struct bpf_spin_lock lock; + int value; +}; + +static void test_hash_lock_refcount_leak(void) +{ + struct lock_map_value value =3D {}; + struct refcounted_kptr *skel; + + skel =3D refcounted_kptr__open_and_load(); + if (!ASSERT_OK_PTR(skel, "refcounted_kptr__open_and_load")) + return; + + test_refcnt_leak(&value, sizeof(value), BPF_F_LOCK, skel->maps.lock_hash, + skel->progs.hash_lock_refcount_leak, + skel->progs.check_hash_lock_refcount); + + refcounted_kptr__destroy(skel); +} + +static void test_cgrp_storage_refcount_leak(u64 flags) +{ + int server_fd =3D -1, client_fd =3D -1; + struct lock_map_value value =3D {}; + struct refcounted_kptr *skel; + struct bpf_link *link; + struct bpf_map *map; + int cgroup, err; + + cgroup =3D test__join_cgroup("/cg_refcount_leak"); + if (!ASSERT_GE(cgroup, 0, "test__join_cgroup")) + return; + + skel =3D refcounted_kptr__open_and_load(); + if (!ASSERT_OK_PTR(skel, "refcounted_kptr__open_and_load")) + goto out; + + link =3D bpf_program__attach_cgroup(skel->progs.cgroup_storage_refcount_l= eak, cgroup); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_cgroup")) + goto out; + skel->links.cgroup_storage_refcount_leak =3D link; + + server_fd =3D start_server(AF_INET6, SOCK_STREAM, "::1", 0, 0); + if (!ASSERT_GE(server_fd, 0, "start_server")) + goto out; + + client_fd =3D connect_to_fd(server_fd, 0); + if (!ASSERT_GE(client_fd, 0, "connect_to_fd")) + goto out; + + map =3D skel->maps.cgrp_strg; + err =3D bpf_map__lookup_elem(map, &cgroup, sizeof(cgroup), &value, sizeof= (value), flags); + if (!ASSERT_OK(err, "bpf_map__lookup_elem")) + goto out; + + ASSERT_EQ(value.value, 2, "refcount"); + + err =3D bpf_map__update_elem(map, &cgroup, sizeof(cgroup), &value, sizeof= (value), flags); + if (!ASSERT_OK(err, "bpf_map__update_elem")) + goto out; + + err =3D bpf_link__detach(skel->links.cgroup_storage_refcount_leak); + if (!ASSERT_OK(err, "bpf_link__detach")) + goto out; + + link =3D bpf_program__attach(skel->progs.check_cgroup_storage_refcount); + if (!ASSERT_OK_PTR(link, "bpf_program__attach")) + goto out; + skel->links.check_cgroup_storage_refcount =3D link; + + close(client_fd); + client_fd =3D connect_to_fd(server_fd, 0); + if (!ASSERT_GE(client_fd, 0, "connect_to_fd")) + goto out; + + err =3D bpf_map__lookup_elem(map, &cgroup, sizeof(cgroup), &value, sizeof= (value), flags); + if (!ASSERT_OK(err, "bpf_map__lookup_elem")) + goto out; + + ASSERT_EQ(value.value, 1, "refcount"); +out: + close(cgroup); + refcounted_kptr__destroy(skel); + if (client_fd >=3D 0) + close(client_fd); + if (server_fd >=3D 0) + close(server_fd); +} + +static void test_cgroup_storage_refcount_leak(void) +{ + test_cgrp_storage_refcount_leak(0); +} + +static void test_cgroup_storage_lock_refcount_leak(void) +{ + test_cgrp_storage_refcount_leak(BPF_F_LOCK); +} + +void test_kptr_refcount_leak(void) +{ + if (test__start_subtest("percpu_hash_refcount_leak")) + test_percpu_hash_refcount_leak(); + if (test__start_subtest("hash_lock_refcount_leak")) + test_hash_lock_refcount_leak(); + if (test__start_subtest("cgroup_storage_refcount_leak")) + test_cgroup_storage_refcount_leak(); + if (test__start_subtest("cgroup_storage_lock_refcount_leak")) + test_cgroup_storage_lock_refcount_leak(); +} diff --git a/tools/testing/selftests/bpf/progs/refcounted_kptr.c b/tools/te= sting/selftests/bpf/progs/refcounted_kptr.c index 893a4fdb4b6e9..09efae9537c9b 100644 --- a/tools/testing/selftests/bpf/progs/refcounted_kptr.c +++ b/tools/testing/selftests/bpf/progs/refcounted_kptr.c @@ -7,6 +7,7 @@ #include #include "bpf_misc.h" #include "bpf_experimental.h" +#include "bpf_tracing_net.h" =20 extern void bpf_rcu_read_lock(void) __ksym; extern void bpf_rcu_read_unlock(void) __ksym; @@ -568,4 +569,163 @@ int BPF_PROG(rbtree_sleepable_rcu_no_explicit_rcu_loc= k, return 0; } =20 +private(leak) u64 ref; + +static u32 probe_read_refcount(void) +{ + u32 refcnt; + + bpf_probe_read_kernel(&refcnt, sizeof(refcnt), (void *) ref); + return refcnt; +} + +static int __insert_in_list(struct bpf_list_head *head, struct bpf_spin_lo= ck *lock, + struct node_data __kptr **node) +{ + struct node_data *n, *m; + + n =3D bpf_obj_new(typeof(*n)); + if (!n) + return -1; + + m =3D bpf_refcount_acquire(n); + n =3D bpf_kptr_xchg(node, n); + if (n) { + bpf_obj_drop(n); + bpf_obj_drop(m); + return -2; + } + + bpf_spin_lock(lock); + bpf_list_push_front(head, &m->l); + ref =3D (u64)(void *) &m->ref; + bpf_spin_unlock(lock); + return probe_read_refcount(); +} + +static void *__lookup_map(void *map) +{ + int key =3D 0; + + return bpf_map_lookup_elem(map, &key); +} + +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_HASH); + __type(key, int); + __type(value, struct map_value); + __uint(max_entries, 1); +} pcpu_hash SEC(".maps"); + +SEC("tc") +int pcpu_hash_refcount_leak(void *ctx) +{ + struct map_value *v; + + v =3D __lookup_map(&pcpu_hash); + if (!v) + return 0; + + return __insert_in_list(&head, &lock, &v->node); +} + +SEC("tc") +int check_pcpu_hash_refcount(void *ctx) +{ + return probe_read_refcount(); +} + +struct lock_map_value { + struct node_data __kptr *node; + struct bpf_spin_lock lock; + int value; +}; + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __type(key, int); + __type(value, struct lock_map_value); + __uint(max_entries, 1); +} lock_hash SEC(".maps"); + +SEC("tc") +int hash_lock_refcount_leak(void *ctx) +{ + struct lock_map_value *v; + + v =3D __lookup_map(&lock_hash); + if (!v) + return 0; + + bpf_spin_lock(&v->lock); + v->value =3D 42; + bpf_spin_unlock(&v->lock); + return __insert_in_list(&head, &lock, &v->node); +} + +SEC("tc") +int check_hash_lock_refcount(void *ctx) +{ + return probe_read_refcount(); +} + +struct { + __uint(type, BPF_MAP_TYPE_CGRP_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct lock_map_value); +} cgrp_strg SEC(".maps"); + +SEC("cgroup/connect6") +int cgroup_storage_refcount_leak(struct bpf_sock_addr *ctx) +{ + struct lock_map_value *v; + struct tcp_sock *tsk; + struct bpf_sock *sk; + u32 refcnt; + + if (ctx->family !=3D AF_INET6 || ctx->user_family !=3D AF_INET6) + return 1; + + sk =3D ctx->sk; + if (!sk) + return 1; + + tsk =3D bpf_skc_to_tcp_sock(sk); + if (!tsk) + return 1; + + v =3D bpf_cgrp_storage_get(&cgrp_strg, tsk->inet_conn.icsk_inet.sk.sk_cgr= p_data.cgroup, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + if (!v) + return 1; + + refcnt =3D __insert_in_list(&head, &lock, &v->node); + bpf_spin_lock(&v->lock); + v->value =3D refcnt; + bpf_spin_unlock(&v->lock); + return 1; +} + +SEC("fexit/inet_stream_connect") +int BPF_PROG(check_cgroup_storage_refcount, struct socket *sock, struct so= ckaddr *uaddr, int addr_len, + int flags) +{ + struct lock_map_value *v; + u32 refcnt; + + if (uaddr->sa_family !=3D AF_INET6) + return 0; + + v =3D bpf_cgrp_storage_get(&cgrp_strg, sock->sk->sk_cgrp_data.cgroup, 0, = 0); + if (!v) + return 0; + + refcnt =3D probe_read_refcount(); + bpf_spin_lock(&v->lock); + v->value =3D refcnt; + bpf_spin_unlock(&v->lock); + return 0; +} + char _license[] SEC("license") =3D "GPL"; --=20 2.51.0