From nobody Mon Dec 15 11:04:16 2025 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15EE913BC0D; Sat, 22 Jun 2024 04:58:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719032294; cv=none; b=c2hg++aQH2NyA4LbmE6KGYA/ALbITAogLS2OJb2A4LCjkQQNsu3kjL5POYoevEZpf8s/2JQQRQgEnEoq3RASBx45b6yrzUAMwube+p5aPQ5rhMBd5JMQmoP0TxKD2L/uMC7+bg8j78aBbTYeG6mWoHqjGMk9lMy1u1PCf2zcEQc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719032294; c=relaxed/simple; bh=JQTkAAufPiH8cc+tCspR9B5nof5cegzV95T3T5qz0Sk=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=BJRw5RKQYS1IyoLGwc1pEg+gMccv/WhuTV5FcIAQ+omwdB+ZSm65xnUhPDuLSrbOrrz/TqlrfQdzYjK/WiCbKig+eCzbkz8F1HKI3PjvN0ERiM9eZDH8C8TgMfcp14B46ZS8UYFxDrifw7FYF4R0feG6rPh3yY/G1uQyFmnsokQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=d5d/lJHZ; arc=none smtp.client-ip=209.85.215.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="d5d/lJHZ" Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-6c53a315c6eso1916700a12.3; Fri, 21 Jun 2024 21:58:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719032292; x=1719637092; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ILiXramB5ZM1r7BijUZ/LciLF0vf3IBeREW+09u1Eb8=; b=d5d/lJHZwTQ+zj6MwAyEiD91Fz7ZG/mpQa0YItnyg80ZSj34BKPcTe9SThi1+YW4MD PGeWpbMwIHMVgQA3aiFdfR0Ep6k/BMPlsQAMf/+mTSpN3jQkMFI7fZNEzgyI/pRfCUdI TrFVXi+00wGSq+BHSoeuKYQmkiAqYPEfJtrwNM/QZ/U4GRLjINShkmP3qtvCkOgoKrIx 7n3kjoJd+62CmCajKelG69SiXx0967sjHkslFJ3IJYQwtn0SslyygJ+Us2e4tnMb65pc hqLEG9ISTIH6t7Ea4dQaV0sgpRXCEo/GrHBEPLJ3kJHU8Q7hgQ5YblCbL07gvvSb0kPJ A2og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719032292; x=1719637092; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ILiXramB5ZM1r7BijUZ/LciLF0vf3IBeREW+09u1Eb8=; b=Hixu5cr6czbqGxrihYr6ujcL5u3Y/NBPSXgfXu9iqWOtVWKzMA6Ggn2Tyvo5/D4Rth kPe36gt2acxInkZubcjmjc90HAEhbyBKX+eS84UJKA7CPgJwk1mcCAlYM7a6a+x/le/2 CTky6h+SMFUkHmxsMzbEH5kRxkIxZAvM30WFBs12gc5AzZmyQkb8FHUzjyvwKmNbfmTK f0g6by4IPCYTUfszvpdFMYmWHW6QRgBXVlYBoux/I66VE3xGVZ34qrkCGcah/zml+Q2h u3RjfYR+Huw2aAgnxzkYG25XmJ9HQ8q3UzYa2A1orpSBLzOgoAcjvPflquEhlUxlnnwc y4Rg== X-Forwarded-Encrypted: i=1; AJvYcCVrmS+xZCTIrQgM8SqBuyrB7FsGbWGlvJmMLAwI5JlOpLbNyqc/2r6/twVy8Kq/eJahPfjyTrY9i4Oz+LYD1nJu0tF1jIxus3FjTwq8lyLJxFPmbzHdNGJzPJpAfqbOV9M/lojfOpmqHsLY+OG8SB3l7qd6O8szrXXo7SgYvLZNBrIqxKbCS97h X-Gm-Message-State: AOJu0YzEiITvi5naAR9MMpzmd1tTUdTyIFTqUYrgvbk1j/rCanAme1xV 7ueS9q0qI0F86SX8ORx0VnH5GoxCzRRvJMoB7iKFn2WcF+BJYYbC X-Google-Smtp-Source: AGHT+IF1W3BhHK+/+cA/5F1jc1pMLbF1/HqHNrR8wYhpKybOGYQDGvwd7+1Gx6QdfJ7eMCBV6uU6jQ== X-Received: by 2002:a17:90a:3fc6:b0:2c7:da20:524b with SMTP id 98e67ed59e1d1-2c8504c7dbamr51506a91.10.1719032292063; Fri, 21 Jun 2024 21:58:12 -0700 (PDT) Received: from localhost.localdomain ([118.32.98.101]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c7e53e065bsm4523197a91.14.2024.06.21.21.58.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 21 Jun 2024 21:58:11 -0700 (PDT) From: yskelg@gmail.com To: Jakub Kicinski , Pedro Tammela Cc: netdev@vger.kernel.org, stable@vger.kernel.org, Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Takashi Iwai , "David S. Miller" , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , "Rafael J. Wysocki" , Jamal Hadi Salim , Cong Wang , Jiri Pirko , Eric Dumazet , Paolo Abeni , Taehee Yoo , Austin Kim , shjy180909@gmail.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, ppbuk5246@gmail.com, Yunseong Kim , Yeoreum Yun Subject: [PATCH v3] tracing/net_sched: NULL pointer dereference in perf_trace_qdisc_reset() Date: Sat, 22 Jun 2024 13:57:01 +0900 Message-ID: <20240622045701.8152-2-yskelg@gmail.com> X-Mailer: git-send-email 2.45.2 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Yunseong Kim In the TRACE_EVENT(qdisc_reset) NULL dereference occurred from qdisc->dev_queue->dev ->name This situation simulated from bunch of veths and Bluetooth dis/reconnection. During qdisc initialization, qdisc was being set to noop_queue. In veth_init_queue, the initial tx_num was reduced back to one, causing the qdisc reset to be called with noop, which led to the kernel pan= ic. I've attached the GitHub gist link that C converted syz-execprogram source code and 3 log of reproduced vmcore-dmesg. https://gist.github.com/yskelg/cc64562873ce249cdd0d5a358b77d740 Yeoreum and I use two fuzzing tool simultaneously. One process with syz-executor : https://github.com/google/syzkaller $ ./syz-execprog -executor=3D./syz-executor -repeat=3D1 -sandbox=3Dsetuid \ -enable=3Dnone -collide=3Dfalse log1 The other process with perf fuzzer: https://github.com/deater/perf_event_tests/tree/master/fuzzer $ perf_event_tests/fuzzer/perf_fuzzer I think this will happen on the kernel version. Linux kernel version +v6.7.10, +v6.8, +v6.9 and it could happen in v6.10. This occurred from 51270d573a8d. I think this patch is absolutely=20 necessary. Previously, It was showing not intended string value of name. I've reproduced 3 time from my fedora 40 Debug Kernel with any other module=20 or patched. version: 6.10.0-0.rc2.20240608gitdc772f8237f9.29.fc41.aarch64+debug [ 5287.164555] veth0_vlan: left promiscuous mode [ 5287.164929] veth1_macvtap: left promiscuous mode [ 5287.164950] veth0_macvtap: left promiscuous mode [ 5287.164983] veth1_vlan: left promiscuous mode [ 5287.165008] veth0_vlan: left promiscuous mode [ 5287.165450] veth1_macvtap: left promiscuous mode [ 5287.165472] veth0_macvtap: left promiscuous mode [ 5287.165502] veth1_vlan: left promiscuous mode =E2=80=A6 [ 5297.598240] bridge0: port 2(bridge_slave_1) entered blocking state [ 5297.598262] bridge0: port 2(bridge_slave_1) entered forwarding state [ 5297.598296] bridge0: port 1(bridge_slave_0) entered blocking state [ 5297.598313] bridge0: port 1(bridge_slave_0) entered forwarding state [ 5297.616090] 8021q: adding VLAN 0 to HW filter on device bond0 [ 5297.620405] bridge0: port 1(bridge_slave_0) entered disabled state [ 5297.620730] bridge0: port 2(bridge_slave_1) entered disabled state [ 5297.627247] 8021q: adding VLAN 0 to HW filter on device team0 [ 5297.629636] bridge0: port 1(bridge_slave_0) entered blocking state =E2=80=A6 [ 5298.002798] bridge_slave_0: left promiscuous mode [ 5298.002869] bridge0: port 1(bridge_slave_0) entered disabled state [ 5298.309444] bond0 (unregistering): (slave bond_slave_0): Releasing backu= p interface [ 5298.315206] bond0 (unregistering): (slave bond_slave_1): Releasing backu= p interface [ 5298.320207] bond0 (unregistering): Released all slaves [ 5298.354296] hsr_slave_0: left promiscuous mode [ 5298.360750] hsr_slave_1: left promiscuous mode [ 5298.374889] veth1_macvtap: left promiscuous mode [ 5298.374931] veth0_macvtap: left promiscuous mode [ 5298.374988] veth1_vlan: left promiscuous mode [ 5298.375024] veth0_vlan: left promiscuous mode [ 5299.109741] team0 (unregistering): Port device team_slave_1 removed [ 5299.185870] team0 (unregistering): Port device team_slave_0 removed =E2=80=A6 [ 5300.155443] Bluetooth: hci3: unexpected cc 0x0c03 length: 249 > 1 [ 5300.155724] Bluetooth: hci3: unexpected cc 0x1003 length: 249 > 9 [ 5300.155988] Bluetooth: hci3: unexpected cc 0x1001 length: 249 > 9 =E2=80=A6. [ 5301.075531] team0: Port device team_slave_1 added [ 5301.085515] bridge0: port 1(bridge_slave_0) entered blocking state [ 5301.085531] bridge0: port 1(bridge_slave_0) entered disabled state [ 5301.085588] bridge_slave_0: entered allmulticast mode [ 5301.085800] bridge_slave_0: entered promiscuous mode [ 5301.095617] bridge0: port 1(bridge_slave_0) entered blocking state [ 5301.095633] bridge0: port 1(bridge_slave_0) entered disabled state =E2=80=A6 [ 5301.149734] bond0: (slave bond_slave_0): Enslaving as an active interfac= e with an up link [ 5301.173234] bond0: (slave bond_slave_0): Enslaving as an active interfac= e with an up link [ 5301.180517] bond0: (slave bond_slave_1): Enslaving as an active interfac= e with an up link [ 5301.193481] hsr_slave_0: entered promiscuous mode [ 5301.204425] hsr_slave_1: entered promiscuous mode [ 5301.210172] debugfs: Directory 'hsr0' with parent 'hsr' already present! [ 5301.210185] Cannot create hsr debugfs directory [ 5301.224061] bond0: (slave bond_slave_1): Enslaving as an active interfac= e with an up link [ 5301.246901] bond0: (slave bond_slave_0): Enslaving as an active interfac= e with an up link [ 5301.255934] team0: Port device team_slave_0 added [ 5301.256480] team0: Port device team_slave_1 added [ 5301.256948] team0: Port device team_slave_0 added =E2=80=A6 [ 5301.435928] hsr_slave_0: entered promiscuous mode [ 5301.446029] hsr_slave_1: entered promiscuous mode [ 5301.455872] debugfs: Directory 'hsr0' with parent 'hsr' already present! [ 5301.455884] Cannot create hsr debugfs directory [ 5301.502664] hsr_slave_0: entered promiscuous mode [ 5301.513675] hsr_slave_1: entered promiscuous mode [ 5301.526155] debugfs: Directory 'hsr0' with parent 'hsr' already present! [ 5301.526164] Cannot create hsr debugfs directory [ 5301.563662] hsr_slave_0: entered promiscuous mode [ 5301.576129] hsr_slave_1: entered promiscuous mode [ 5301.580259] debugfs: Directory 'hsr0' with parent 'hsr' already present! [ 5301.580270] Cannot create hsr debugfs directory [ 5301.590269] 8021q: adding VLAN 0 to HW filter on device bond0 [ 5301.595872] KASAN: null-ptr-deref in range [0x0000000000000130-0x0000000= 000000137] [ 5301.595877] Mem abort info: [ 5301.595881] ESR =3D 0x0000000096000006 [ 5301.595885] EC =3D 0x25: DABT (current EL), IL =3D 32 bits [ 5301.595889] SET =3D 0, FnV =3D 0 [ 5301.595893] EA =3D 0, S1PTW =3D 0 [ 5301.595896] FSC =3D 0x06: level 2 translation fault [ 5301.595900] Data abort info: [ 5301.595903] ISV =3D 0, ISS =3D 0x00000006, ISS2 =3D 0x00000000 [ 5301.595907] CM =3D 0, WnR =3D 0, TnD =3D 0, TagAccess =3D 0 [ 5301.595911] GCS =3D 0, Overlay =3D 0, DirtyBit =3D 0, Xs =3D 0 [ 5301.595915] [dfff800000000026] address between user and kernel address r= anges [ 5301.595971] Internal error: Oops: 0000000096000006 [#1] SMP =E2=80=A6 [ 5301.596076] CPU: 2 PID: 102769 Comm: syz-executor.3 Kdump: loaded Tainted: G W ------- --- 6.10.0-0.rc2.20240608gitdc772f8237f9.29.f= c41.aarch64+debug #1 [ 5301.596080] Hardware name: VMware, Inc. VMware20,1/VBSA, BIOS VMW201.00V.21805430.BA64.2305221830 05/22/2023 [ 5301.596082] pstate: 01400005 (nzcv daif +PAN -UAO -TCO +DIT -SSBS BTYPE= =3D--) [ 5301.596085] pc : strnlen+0x40/0x88 [ 5301.596114] lr : trace_event_get_offsets_qdisc_reset+0x6c/0x2b0 [ 5301.596124] sp : ffff8000beef6b40 [ 5301.596126] x29: ffff8000beef6b40 x28: dfff800000000000 x27: 00000000000= 00001 [ 5301.596131] x26: 6de1800082c62bd0 x25: 1ffff000110aa9e0 x24: ffff8000885= 54f00 [ 5301.596136] x23: ffff800088554ec0 x22: 0000000000000130 x21: 00000000000= 00140 [ 5301.596140] x20: dfff800000000000 x19: ffff8000beef6c60 x18: ffff7000115= 106d8 [ 5301.596143] x17: ffff800121bad000 x16: ffff800080020000 x15: 00000000000= 00006 [ 5301.596147] x14: 0000000000000002 x13: ffff0001f3ed8d14 x12: ffff700017d= deda5 [ 5301.596151] x11: 1ffff00017ddeda4 x10: ffff700017ddeda4 x9 : ffff800082c= c5eec [ 5301.596155] x8 : 0000000000000004 x7 : 00000000f1f1f1f1 x6 : 00000000f2f= 2f200 [ 5301.596158] x5 : 00000000f3f3f3f3 x4 : ffff700017dded80 x3 : 00000000f20= 4f1f1 [ 5301.596162] x2 : 0000000000000026 x1 : 0000000000000000 x0 : 00000000000= 00130 [ 5301.596166] Call trace: [ 5301.596175] strnlen+0x40/0x88 [ 5301.596179] trace_event_get_offsets_qdisc_reset+0x6c/0x2b0 [ 5301.596182] perf_trace_qdisc_reset+0xb0/0x538 [ 5301.596184] __traceiter_qdisc_reset+0x68/0xc0 [ 5301.596188] qdisc_reset+0x43c/0x5e8 [ 5301.596190] netif_set_real_num_tx_queues+0x288/0x770 [ 5301.596194] veth_init_queues+0xfc/0x130 [veth] [ 5301.596198] veth_newlink+0x45c/0x850 [veth] [ 5301.596202] rtnl_newlink_create+0x2c8/0x798 [ 5301.596205] __rtnl_newlink+0x92c/0xb60 [ 5301.596208] rtnl_newlink+0xd8/0x130 [ 5301.596211] rtnetlink_rcv_msg+0x2e0/0x890 [ 5301.596214] netlink_rcv_skb+0x1c4/0x380 [ 5301.596225] rtnetlink_rcv+0x20/0x38 [ 5301.596227] netlink_unicast+0x3c8/0x640 [ 5301.596231] netlink_sendmsg+0x658/0xa60 [ 5301.596234] __sock_sendmsg+0xd0/0x180 [ 5301.596243] __sys_sendto+0x1c0/0x280 [ 5301.596246] __arm64_sys_sendto+0xc8/0x150 [ 5301.596249] invoke_syscall+0xdc/0x268 [ 5301.596256] el0_svc_common.constprop.0+0x16c/0x240 [ 5301.596259] do_el0_svc+0x48/0x68 [ 5301.596261] el0_svc+0x50/0x188 [ 5301.596265] el0t_64_sync_handler+0x120/0x130 [ 5301.596268] el0t_64_sync+0x194/0x198 [ 5301.596272] Code: eb15001f 54000120 d343fc02 12000801 (38f46842)=20 [ 5301.596285] SMP: stopping secondary CPUs [ 5301.597053] Starting crashdump kernel... [ 5301.597057] Bye! Yeoreum and I don't know if the patch we wrote will fix the underlying caus= e, but we think that priority is to prevent kernel panic happening. So, we're sending this patch. Link: https://lore.kernel.org/lkml/20240229143432.273b4871@gandalf.local.ho= me/t/ Fixes: 51270d573a8d ("tracing/net_sched: Fix tracepoints that save qdisc_de= v() as a string") Cc: netdev@vger.kernel.org Cc: stable@vger.kernel.org # +v6.7.10, +v6.8=20 Signed-off-by: Yunseong Kim Signed-off-by: Yeoreum Yun --- include/trace/events/qdisc.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/trace/events/qdisc.h b/include/trace/events/qdisc.h index f1b5e816e7e5..170b51fbe47a 100644 --- a/include/trace/events/qdisc.h +++ b/include/trace/events/qdisc.h @@ -81,7 +81,7 @@ TRACE_EVENT(qdisc_reset, TP_ARGS(q), =20 TP_STRUCT__entry( - __string( dev, qdisc_dev(q)->name ) + __string(dev, qdisc_dev(q) ? qdisc_dev(q)->name : "noop_queue") __string( kind, q->ops->id ) __field( u32, parent ) __field( u32, handle ) --=20 2.45.2