From nobody Sat Feb 7 23:48:17 2026 Received: from smtp.cecloud.com (unknown [1.203.97.240]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 545911BC099; Mon, 10 Feb 2025 07:47:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=1.203.97.240 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739173623; cv=none; b=DKcEGE/pY8dqIpUthqntaJMf61mZKiscydgeDWjO+IwjH5e8VCOqh/8bynBFLq8kJVgbXUOlTDidIXvEctz4WIXUWg0BbLgrMlx3+NDeT+ODH2oDlWc/rToZz6mO9XBoncCxsdoZwvqi8pO3QVgF5i1wtkGSnhlc+oef19QQyvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739173623; c=relaxed/simple; bh=lmFzZkHMcDBbZfoYden/WSPcx5iv6e/oJE9x9+Ol9A0=; h=Date:From:To:Cc:Subject:Mime-Version:Message-ID:Content-Type; b=ecqgMPz6DQvFz6/y2cs2YXkM7iTaDlTp3rxvYMAPk/jSZtMtoXIXRsgJEAXoKXaAVbPhe7jwBEFh2eGqOvjgCrXBCe3XAJTgHk4mS+lW8u7P/i3dWUhyVlftylKQucLWO0W8+AsCmiYS/RahGVR45OhTkciX2gPOUjiuIM6RDm0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=cestc.cn; spf=pass smtp.mailfrom=cestc.cn; arc=none smtp.client-ip=1.203.97.240 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=cestc.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cestc.cn Received: from localhost (localhost [127.0.0.1]) by smtp.cecloud.com (Postfix) with ESMTP id 9DE70900115; Mon, 10 Feb 2025 15:41:39 +0800 (CST) X-MAIL-GRAY: 0 X-MAIL-DELIVERY: 1 X-SKE-CHECKED: 1 X-ABS-CHECKED: 1 X-ANTISPAM-LEVEL: 2 Received: from desktop-n31qu50 (unknown [39.156.73.12]) by smtp.cecloud.com (postfix) whith ESMTP id P3907749T281458752745840S1739173298581651_; Mon, 10 Feb 2025 15:41:39 +0800 (CST) X-IP-DOMAINF: 1 X-RL-SENDER: zhang.guanghui@cestc.cn X-SENDER: zhang.guanghui@cestc.cn X-LOGIN-NAME: zhang.guanghui@cestc.cn X-FST-TO: sagi@grimberg.me X-RCPT-COUNT: 9 X-LOCAL-RCPT-COUNT: 1 X-MUTI-DOMAIN-COUNT: 0 X-SENDER-IP: 39.156.73.12 X-ATTACHMENT-NUM: 0 X-UNIQUE-TAG: X-System-Flag: 0 Date: Mon, 10 Feb 2025 15:41:38 +0800 From: "zhang.guanghui@cestc.cn" To: sagi , mgurtovoy , kbusch , sashal , chunguang.xu , zhang.guanghui Cc: linux-kernel , linux-nvme , linux-block Subject: nvme-tcp: fix a possible UAF when failing to send request X-Priority: 3 X-GUID: 9BDE04EE-39E0-4C6A-8A67-C818857406B3 X-Has-Attach: no X-Mailer: Foxmail 7.2.21.453[cn] Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 Message-ID: <2025021015413817916143@cestc.cn> Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hello=20 =C2=A0 =C2=A0=C2=A0When using the nvme-tcp driver in a storage cluster, the= driver may trigger a null pointer causing the host to crash several times. By analyzing the vmcore, we know the direct cause is that=C2=A0 the request= ->mq_hctx was used after free.=20 CPU1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0CPU2 nvme_tcp_poll=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nvme_tcp= _try_send=C2=A0 --failed to send reqrest 13=C2=A0 =C2=A0 =C2=A0=C2=A0nvme_tcp_try_recv=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 nvme_tcp_fail_request =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0nvme_tcp_recv_skb=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 nvme_tcp_end_request =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0nvme_tcp_recv_pdu= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nvme_complete_rq=C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0nvme_tcp_handle_comp=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0= =C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2= =A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 = =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0nvme_retry_req --=C2=A0request-= >mq_hctx have been freed, is NULL. =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 = =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0 =C2=A0=C2=A0nvme_tcp_process_nvme_cqe=C2=A0 =C2=A0=C2=A0=C2=A0 = =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2= =A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0= =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0 nvme_complete_rq =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0 nvme_end_req =C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0= =C2=A0=C2=A0 blk_mq_end_request=C2=A0 when nvme_tcp_try_send failed to send reqrest 13, it maybe be resulted by s= elinux or other reasons, this is a problem. then=C2=A0 the nvme_tcp_fail_re= quest would execute=E3=80=82 but the nvme_tcp_recv_pdu may have received the responding pdu and the nvme= _tcp_process_nvme_cqe would have completed the request.=C2=A0 request->mq_h= ctx was used after free.=20 the follow patch is to solve it.=20 can you give=C2=A0 some suggestions?=C2=A0 thanks! diff --git a/linux/drivers/nvme/host/core.c b/linux/drivers/nvme/host/core.c index a65b1dce8..417466674 100644 --- a/linux/drivers/nvme/host/core.c +++ b/linux/drivers/nvme/host/core.c @@ -288,6 +288,9 @@ static void nvme_retry_req(struct request *req) =C2=A0 =C2=A0 =C2=A0 =C2=A0 unsigned long delay =3D 0; =C2=A0 =C2=A0 =C2=A0 =C2=A0 u16 crd; +=C2=A0 =C2=A0 =C2=A0 =C2=A0if(!req->mq_hctx && req->state =3D=3D MQ_RQ_IDL= E) +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return; + =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* The mask and shift result must be <=3D 3 */ =C2=A0 =C2=A0 =C2=A0 =C2=A0 crd =3D (nvme_req(req)->status & NVME_SC_CRD) >= > 11; The details are as follows [35665.692310] nvme nvme2: failed to send request -13 [35665.692683] nvme nvme2: queue 1 failed to send request 00000000b42f4e2b = state 2 pdu 00000000d7fb8da3 type 4 rq_state 1 nrq_status 0 [35665.693323] nvme nvme2: failed to send rq 00000000f86a68c3 state 2 nrq_s= tatus 370 [35665.702265] nvme nvme2: unsupported pdu type (3) [35665.702272] BUG: kernel NULL pointer dereference, address: 0000000000000= 000 [35665.702542] nvme nvme2: queue 1 receive failed:=C2=A0 -22 [35665.703209] #PF: supervisor write access in kernel mode [35665.703213] #PF: error_code(0x0002) - not-present page [35665.703214] PGD 8000003801cce067 P4D 8000003801cce067 PUD 37e6f79067 PMD= 0 [35665.703220] Oops: 0002 [#1] SMP PTI [35665.703658] nvme nvme2: starting error recovery [35665.704442] CPU: 20 PID: 815 Comm: kworker/20:1H Kdump: loaded Not taint= ed 5.15.131-17.cl9.x86_64 #1 [35665.705168] nvme nvme2: queue 1 receive again after receive failed [35665.705809] Hardware name: Inspur aaabbb/YZMB-00882-104, BIOS 4.1.26 09/= 22/2022 [35665.705812] Workqueue: kblockd blk_mq_requeue_work [35665.709172] RIP: 0010:_raw_spin_lock+0xc/0x30 [35665.709606] Code: 05 c3 cc cc cc cc 89 c6 e8 31 05 68 ff 66 90 c3 cc cc = cc cc 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 31 c0 ba 01 00 00 00 0f b1 17 75 05 c3 cc cc cc cc 89 c6 e8 02 05 68 ff 66 90 c3 cc [35665.710470] RSP: 0000:ffffa67bcd797e08 EFLAGS: 00010246 [35665.710925] RAX: 0000000000000000 RBX: ffff92f6bbcc9840 RCX: ffff92f6bbc= c9888 [35665.711393] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 00000000000= 00000 [35665.711849] RBP: 0000000000000000 R08: ffffa67bcd797e48 R09: ffff932346d= 576f4 [35665.712275] R10: 0000000000000008 R11: 0000000000000008 R12: 00000000000= 00000 [35665.712725] R13: ffff92f6bbcc9888 R14: 0000000000000008 R15: 00000000000= 00000 [35665.713158] FS:=C2=A0 0000000000000000(0000) GS:ffff93527d400000(0000) k= nlGS:0000000000000000 [35665.713603] CS:=C2=A0 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [35665.714063] CR2: 0000000000000000 CR3: 000000371aa02006 CR4: 00000000007= 706e0 [35665.714534] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 00000000000= 00000 [35665.714961] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 00000000000= 00400 [35665.715359] PKRU: 55555554 [35665.715788] Call Trace: [35665.716201]=C2=A0 [35665.716613]=C2=A0 ? show_trace_log_lvl+0x1c1/0x2d9 [35665.717049]=C2=A0 ? show_trace_log_lvl+0x1c1/0x2d9 [35665.717457]=C2=A0 ? blk_mq_request_bypass_insert+0x2c/0xb0 [35665.717950]=C2=A0 ? __die_body.cold+0x8/0xd [35665.718361]=C2=A0 ? page_fault_oops+0xac/0x140 [35665.718749]=C2=A0 ? blk_mq_start_request+0x30/0xf0 [35665.719144]=C2=A0 ? nvme_tcp_queue_rq+0xc7/0x170 [nvme_tcp] [35665.719547]=C2=A0 ? exc_page_fault+0x62/0x130 [35665.719938]=C2=A0 ? asm_exc_page_fault+0x22/0x30 [35665.720333]=C2=A0 ? _raw_spin_lock+0xc/0x30 [35665.720723]=C2=A0 blk_mq_request_bypass_insert+0x2c/0xb0 [35665.721101]=C2=A0 blk_mq_requeue_work+0xa5/0x180 [35665.721451]=C2=A0 process_one_work+0x1e8/0x390 [35665.721809]=C2=A0 worker_thread+0x53/0x3d0 [35665.722159]=C2=A0 ? process_one_work+0x390/0x390 [35665.722501]=C2=A0 kthread+0x124/0x150 [35665.722849]=C2=A0 ? set_kthread_struct+0x50/0x50 [35665.723182]=C2=A0 ret_from_fork+0x1f/0x30 [35665.723508]=C2=A0 crash> struct nvme_tcp_request ffff92f6bbcc9950 struct nvme_tcp_request { =C2=A0 req =3D { =C2=A0=C2=A0=C2=A0 cmd =3D 0xffff92f5b83f6748, =C2=A0=C2=A0=C2=A0 result =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 u16 =3D 0, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 u32 =3D 0, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 u64 =3D 0 =C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0 genctr =3D 169 '\251', =C2=A0=C2=A0=C2=A0 retries =3D 1 '\001', =C2=A0=C2=A0=C2=A0 flags =3D 0 '\000', =C2=A0=C2=A0=C2=A0 status =3D 6, =C2=A0=C2=A0=C2=A0 ctrl =3D 0xffff92f5e5df7348 =C2=A0 }, =C2=A0 pdu =3D 0xffff92f5b83f6740, =C2=A0 queue =3D 0xffff92f407cc9128, =C2=A0 data_len =3D 4096, =C2=A0 pdu_len =3D 4096, =C2=A0 pdu_sent =3D 0, =C2=A0 h2cdata_left =3D 0, =C2=A0 h2cdata_offset =3D 0, =C2=A0 ttag =3D 62, =C2=A0 status =3D 12, =C2=A0 entry =3D { =C2=A0=C2=A0=C2=A0 next =3D 0xdead000000000100, =C2=A0=C2=A0=C2=A0 prev =3D 0xdead000000000122 =C2=A0 }, =C2=A0 lentry =3D { =C2=A0=C2=A0=C2=A0 next =3D 0x0 =C2=A0 }, =C2=A0 ddgst =3D 0, =C2=A0 curr_bio =3D 0xffff9324d639e240, =C2=A0 iter =3D { =C2=A0=C2=A0=C2=A0 iter_type =3D 2 '\002', =C2=A0=C2=A0=C2=A0 nofault =3D false, =C2=A0=C2=A0=C2=A0 data_source =3D true, =C2=A0=C2=A0=C2=A0 iov_offset =3D 0, =C2=A0=C2=A0=C2=A0 count =3D 4096, =C2=A0=C2=A0=C2=A0 { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 iov =3D 0xffff92f6bbcc98a8, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 kvec =3D 0xffff92f6bbcc98a8, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bvec =3D 0xffff92f6bbcc98a8, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xarray =3D 0xffff92f6bbcc98a8, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pipe =3D 0xffff92f6bbcc98a8 =C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0 { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 nr_segs =3D 1, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 head =3D 1, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 start_head =3D 0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xarray_start =3D 1 =C2=A0=C2=A0=C2=A0 } =C2=A0 }, =C2=A0 offset =3D 0, =C2=A0 data_sent =3D 0, =C2=A0 state =3D NVME_TCP_SEND_DATA } crash> nvme_tcp_hdr.type 0xffff92f5b83f6740 =C2=A0 type =3D 4 '\004', crash> crash> struct request=C2=A0 ffff92f6bbcc9840 struct request { =C2=A0 q =3D 0xffff92f59d55c240, =C2=A0 mq_ctx =3D 0xffffc67bb9a1f040, =C2=A0 mq_hctx =3D 0x0, =C2=A0 cmd_flags =3D 33556483, =C2=A0 rq_flags =3D 139456, =C2=A0 tag =3D 87, =C2=A0 internal_tag =3D -1, =C2=A0 __data_len =3D 0, =C2=A0 __sector =3D 66846720, =C2=A0 bio =3D 0x0, =C2=A0 biotail =3D 0x0, =C2=A0 queuelist =3D { =C2=A0=C2=A0=C2=A0 next =3D 0xffff92f6bbcc9888, =C2=A0=C2=A0=C2=A0 prev =3D 0xffff92f6bbcc9888 =C2=A0 }, =C2=A0 { =C2=A0=C2=A0=C2=A0 hash =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 next =3D 0x0, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pprev =3D 0x0 =C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0 ipi_list =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 next =3D 0x0 =C2=A0=C2=A0=C2=A0 } =C2=A0 }, =C2=A0 { =C2=A0=C2=A0=C2=A0 rb_node =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __rb_parent_color =3D 18446685131795018112, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rb_right =3D 0x1000, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rb_left =3D 0x0 =C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0 special_vec =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bv_page =3D 0xffffca64841f3180, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bv_len =3D 4096, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bv_offset =3D 0 =C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0 completion_data =3D 0xffffca64841f3180, =C2=A0=C2=A0=C2=A0 error_count =3D -2078330496 =C2=A0 }, =C2=A0 { =C2=A0=C2=A0=C2=A0 elv =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 icq =3D 0x0, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 priv =3D {0xffff92f6bbcc98c8, 0xffff92f6bbcc= 98c8} =C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0 flush =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 seq =3D 0, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 list =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 next =3D 0xffff92f6bbcc98c8, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 prev =3D 0xffff92f6bbcc98c8 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 saved_end_io =3D 0x0 =C2=A0=C2=A0=C2=A0 } =C2=A0 }, =C2=A0 rq_disk =3D 0xffff92f6bdbff600, =C2=A0 part =3D 0xffff92f59f557800, =C2=A0 start_time_ns =3D 35665692557229, =C2=A0 io_start_time_ns =3D 35665692566268, =C2=A0 wbt_flags =3D 0, =C2=A0 stats_sectors =3D 0, =C2=A0 nr_phys_segments =3D 1, =C2=A0 nr_integrity_segments =3D 0, =C2=A0 write_hint =3D 0, =C2=A0 ioprio =3D 0, =C2=A0 state =3D MQ_RQ_IDLE, =C2=A0 ref =3D { =C2=A0=C2=A0=C2=A0 refs =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 counter =3D 0 =C2=A0=C2=A0=C2=A0 } =C2=A0 }, =C2=A0 timeout =3D 180000, =C2=A0 deadline =3D 4330512774, =C2=A0 { =C2=A0=C2=A0=C2=A0 csd =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 node =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 llist =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 next =3D 0x0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 u_flags =3D 0, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 a_flags =3D { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 counter = =3D 0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 src =3D 0, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dst =3D 0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 func =3D 0xffffffff8eed62d0 <__blk_mq_comple= te_request_remote>, =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 info =3D 0xffff92f6bbcc9840 =C2=A0=C2=A0=C2=A0 }, =C2=A0=C2=A0=C2=A0 fifo_time =3D 0 =C2=A0 }, =C2=A0 end_io =3D 0x0, =C2=A0 end_io_data =3D 0x0 } Best regards zhang.guanghui@cestc.cn