From nobody Fri May 17 12:53:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1601212129; cv=none; d=zohomail.com; s=zohoarc; b=I/mlvzs6bTKS7BZ0FZLJahh6JOM3M7DHbd0HsfABeG/LmJZz1ZRv05HdCI3eiDbj3L4sFCAQFqectp0mc16B0d/0z2PqdyksCkBEPRq2P3IRMBpe+o3uR7t3Ubglmu2nm54m9/38sER8fckTq1Kvk9YVe/FQihAGbML0yWnPdto= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601212129; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JVVuL+0ra6Jc4uzKOyFimTraJfO+CodTT/mYhfd8R4k=; b=A8N0tXUTyc/VBLddtl5Tr/jlYkGtEH2Mu9E+cqY3hrxmg3a533zEQY0PFFWJRtiZwr/AEs/+oJGvkokELpabTdGnB9aQwI4LA8CCSzbo5/vpsTcaNjYTb8fzfTzvKE/1BRte4ZkVbmeMwTupQzBFZMVX64x+Mph0u6ox8qkQDIs= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601212129050820.4592658181558; Sun, 27 Sep 2020 06:08:49 -0700 (PDT) Received: from localhost ([::1]:36056 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMWQ3-0004NS-N9 for importer@patchew.org; Sun, 27 Sep 2020 09:08:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60868) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMS-0007qq-Ny for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:04 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:5149 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMQ-0003N5-Bk for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:04 -0400 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id D389B1639CE2190F26BD; Sun, 27 Sep 2020 21:04:48 +0800 (CST) Received: from localhost (10.174.185.104) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.487.0; Sun, 27 Sep 2020 21:04:38 +0800 From: Ying Fang To: Subject: [RFC PATCH 1/7] block-backend: introduce I/O rehandle info Date: Sun, 27 Sep 2020 21:04:14 +0800 Message-ID: <20200927130420.1095-2-fangying1@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20200927130420.1095-1-fangying1@huawei.com> References: <20200927130420.1095-1-fangying1@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.104] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.190; envelope-from=fangying1@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/27 09:04:49 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Ying Fang , Jiahui Cen , zhang.zhanghailiang@huawei.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" The I/O hang feature is realized based on a rehandle mechanism. Each block backend will have a list to store hanging block AIOs, and a timer to regularly resend these aios. In order to issue the AIOs again, each block AIOs also need to store its coroutine entry. Signed-off-by: Jiahui Cen Signed-off-by: Ying Fang --- block/block-backend.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index 24dd0670d1..bf104a7cf5 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -35,6 +35,18 @@ =20 static AioContext *blk_aiocb_get_aio_context(BlockAIOCB *acb); =20 +/* block backend rehandle timer interval 5s */ +#define BLOCK_BACKEND_REHANDLE_TIMER_INTERVAL 5000 + +typedef struct BlockBackendRehandleInfo { + bool enable; + QEMUTimer *ts; + unsigned timer_interval_ms; + + unsigned int in_flight; + QTAILQ_HEAD(, BlkAioEmAIOCB) re_aios; +} BlockBackendRehandleInfo; + typedef struct BlockBackendAioNotifier { void (*attached_aio_context)(AioContext *new_context, void *opaque); void (*detach_aio_context)(void *opaque); @@ -95,6 +107,8 @@ struct BlockBackend { * Accessed with atomic ops. */ unsigned int in_flight; + + BlockBackendRehandleInfo reinfo; }; =20 typedef struct BlockBackendAIOCB { @@ -350,6 +364,7 @@ BlockBackend *blk_new(AioContext *ctx, uint64_t perm, u= int64_t shared_perm) qemu_co_queue_init(&blk->queued_requests); notifier_list_init(&blk->remove_bs_notifiers); notifier_list_init(&blk->insert_bs_notifiers); + QLIST_INIT(&blk->aio_notifiers); =20 QTAILQ_INSERT_TAIL(&block_backends, blk, link); @@ -1392,6 +1407,10 @@ typedef struct BlkAioEmAIOCB { BlkRwCo rwco; int bytes; bool has_returned; + + /* for rehandle */ + CoroutineEntry *co_entry; + QTAILQ_ENTRY(BlkAioEmAIOCB) list; } BlkAioEmAIOCB; =20 static AioContext *blk_aio_em_aiocb_get_aio_context(BlockAIOCB *acb_) --=20 2.23.0 From nobody Fri May 17 12:53:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1601212133; cv=none; d=zohomail.com; s=zohoarc; b=AhtqECuISg+VhH8OgxkSoj8vAquGbIQ5AqbBYg9mjLcMP+F3BbC+uNeYzQHCnPl2fIQuiBT3ZCLRQ3Vf1YVCK0wtQBMK6QvvwcvGGU1KSoftn4AcQL4oPvAQ+Nh6g8xB9w8CpXv6JBfptEqQz0Eo5CbNnapBl1gum0Z/J7t45xk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601212133; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qXKwxmh8I2zA6RYfD4qN01PDt4WgaWPBDj4wLrb6XuI=; b=BKd2zMapfDTXJEPMJNAd7ccRXN/b3S1qpVfMAuHLVSHa66YmNdIFPUmbaQuN2Y4Cpb8atbD+Y0LwlxAxrk2+fA/GZSM+ZtojYulwQ+PWh42HG0DugWquC6ndMMXpK5SFso83/WZ5mrTWm94OxUmo48xtXtFUX9Mo0svmbqV7tz8= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 160121213393645.77582970480921; Sun, 27 Sep 2020 06:08:53 -0700 (PDT) Received: from localhost ([::1]:36460 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMWQ8-0004XO-Of for importer@patchew.org; Sun, 27 Sep 2020 09:08:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60872) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMS-0007qy-U9 for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:04 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:60246 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMQ-0003N2-C2 for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:04 -0400 Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id C68EDEBA499277E74A44; Sun, 27 Sep 2020 21:04:48 +0800 (CST) Received: from localhost (10.174.185.104) by DGGEMS402-HUB.china.huawei.com (10.3.19.202) with Microsoft SMTP Server id 14.3.487.0; Sun, 27 Sep 2020 21:04:39 +0800 From: Ying Fang To: Subject: [RFC PATCH 2/7] block-backend: rehandle block aios when EIO Date: Sun, 27 Sep 2020 21:04:15 +0800 Message-ID: <20200927130420.1095-3-fangying1@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20200927130420.1095-1-fangying1@huawei.com> References: <20200927130420.1095-1-fangying1@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.104] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.32; envelope-from=fangying1@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/27 09:04:49 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Ying Fang , Jiahui Cen , zhang.zhanghailiang@huawei.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" When a backend device temporarily does not response, like a network disk do= wn due to some network faults, any IO to the coresponding virtual block device in VM would return I/O error. If the hypervisor returns the error to VM, the filesystem on this block device may not work as usual. And in many situatio= ns, the returned error is often an EIO. To avoid this unavailablity, we can store the failed AIOs, and resend them later. If the error is temporary, the retries can succeed and the AIOs can be successfully completed. Signed-off-by: Ying Fang Signed-off-by: Jiahui Cen --- block/block-backend.c | 89 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index bf104a7cf5..90f1ca5753 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -365,6 +365,12 @@ BlockBackend *blk_new(AioContext *ctx, uint64_t perm, = uint64_t shared_perm) notifier_list_init(&blk->remove_bs_notifiers); notifier_list_init(&blk->insert_bs_notifiers); =20 + /* for rehandle */ + blk->reinfo.enable =3D false; + blk->reinfo.ts =3D NULL; + atomic_set(&blk->reinfo.in_flight, 0); + QTAILQ_INIT(&blk->reinfo.re_aios); + QLIST_INIT(&blk->aio_notifiers); =20 QTAILQ_INSERT_TAIL(&block_backends, blk, link); @@ -1425,8 +1431,16 @@ static const AIOCBInfo blk_aio_em_aiocb_info =3D { .get_aio_context =3D blk_aio_em_aiocb_get_aio_context, }; =20 +static void blk_rehandle_timer_cb(void *opaque); +static void blk_rehandle_aio_complete(BlkAioEmAIOCB *acb); + static void blk_aio_complete(BlkAioEmAIOCB *acb) { + if (acb->rwco.blk->reinfo.enable) { + blk_rehandle_aio_complete(acb); + return; + } + if (acb->has_returned) { acb->common.cb(acb->common.opaque, acb->rwco.ret); blk_dec_in_flight(acb->rwco.blk); @@ -1459,6 +1473,7 @@ static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, in= t64_t offset, int bytes, .ret =3D NOT_DONE, }; acb->bytes =3D bytes; + acb->co_entry =3D co_entry; acb->has_returned =3D false; =20 co =3D qemu_coroutine_create(co_entry, acb); @@ -2054,6 +2069,20 @@ static int blk_do_set_aio_context(BlockBackend *blk,= AioContext *new_context, throttle_group_attach_aio_context(tgm, new_context); bdrv_drained_end(bs); } + + if (blk->reinfo.enable) { + if (blk->reinfo.ts) { + timer_del(blk->reinfo.ts); + timer_free(blk->reinfo.ts); + } + blk->reinfo.ts =3D aio_timer_new(new_context, QEMU_CLOCK_REALT= IME, + SCALE_MS, blk_rehandle_timer_cb, + blk); + if (atomic_read(&blk->reinfo.in_flight)) { + timer_mod(blk->reinfo.ts, + qemu_clock_get_ms(QEMU_CLOCK_REALTIME)); + } + } } =20 blk->ctx =3D new_context; @@ -2405,6 +2434,66 @@ static void blk_root_drained_end(BdrvChild *child, i= nt *drained_end_counter) } } =20 +static void blk_rehandle_insert_aiocb(BlockBackend *blk, BlkAioEmAIOCB *ac= b) +{ + assert(blk->reinfo.enable); + + atomic_inc(&blk->reinfo.in_flight); + QTAILQ_INSERT_TAIL(&blk->reinfo.re_aios, acb, list); + timer_mod(blk->reinfo.ts, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + + blk->reinfo.timer_interval_ms); +} + +static void blk_rehandle_remove_aiocb(BlockBackend *blk, BlkAioEmAIOCB *ac= b) +{ + QTAILQ_REMOVE(&blk->reinfo.re_aios, acb, list); + atomic_dec(&blk->reinfo.in_flight); +} + +static void blk_rehandle_timer_cb(void *opaque) +{ + BlockBackend *blk =3D opaque; + BlockBackendRehandleInfo *reinfo =3D &blk->reinfo; + BlkAioEmAIOCB *acb, *tmp; + Coroutine *co; + + aio_context_acquire(blk_get_aio_context(blk)); + QTAILQ_FOREACH_SAFE(acb, &reinfo->re_aios, list, tmp) { + if (acb->rwco.ret =3D=3D NOT_DONE) { + continue; + } + + blk_inc_in_flight(acb->rwco.blk); + acb->rwco.ret =3D NOT_DONE; + acb->has_returned =3D false; + + co =3D qemu_coroutine_create(acb->co_entry, acb); + bdrv_coroutine_enter(blk_bs(blk), co); + + acb->has_returned =3D true; + if (acb->rwco.ret !=3D NOT_DONE) { + blk_rehandle_remove_aiocb(acb->rwco.blk, acb); + replay_bh_schedule_oneshot_event(blk_get_aio_context(blk), + blk_aio_complete_bh, acb); + } + } + aio_context_release(blk_get_aio_context(blk)); +} + +static void blk_rehandle_aio_complete(BlkAioEmAIOCB *acb) +{ + if (acb->has_returned) { + blk_dec_in_flight(acb->rwco.blk); + if (acb->rwco.ret =3D=3D -EIO) { + blk_rehandle_insert_aiocb(acb->rwco.blk, acb); + return; + } + + acb->common.cb(acb->common.opaque, acb->rwco.ret); + qemu_aio_unref(acb); + } +} + void blk_register_buf(BlockBackend *blk, void *host, size_t size) { bdrv_register_buf(blk_bs(blk), host, size); --=20 2.23.0 From nobody Fri May 17 12:53:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1601212010; cv=none; d=zohomail.com; s=zohoarc; b=MzYLfXHZ8w6JI6XiKum2VCnGlutMPMYOA8sVFIbkMsbUemuS7CXde9/lH1FmliK3TkpFjGtJ/aS6CU7EFArjfcwCYCY5M4TLlFkcpiVsAmu3+Lx4Nib7X//LCUusHxuwpOGgR7UQ94aB+1+BTYpmtLBxVx+cCMM4hsM7ODXfrlk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601212010; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=YJ5HJWJTAmdXxF24WSENRrtRNELmFQ2jyFqZaJIyN7o=; b=SOtivWhYAIL4Q1uDnPWZReKTegdT57ajwJ9+XhzJdWIAFnysE8HGIqq4pqWycFKG63XZCaS1fakaF9Td8AtO8lTBL0AluF6EIZrKY+Lo6+Tm1px5H45yAcJts7G5Pz81QgT5RnG9miOEAwYe6wskXOvMsQCogTDTh+DQysBQuBY= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601212010949982.2029008892865; Sun, 27 Sep 2020 06:06:50 -0700 (PDT) Received: from localhost ([::1]:56494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMWO9-0001Ev-NZ for importer@patchew.org; Sun, 27 Sep 2020 09:06:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMU-0007tk-PU for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:06 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:60240 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMQ-0003N0-Vy for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:06 -0400 Received: from DGGEMS401-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 2361F5E510B5427236C8; Sun, 27 Sep 2020 21:04:47 +0800 (CST) Received: from localhost (10.174.185.104) by DGGEMS401-HUB.china.huawei.com (10.3.19.201) with Microsoft SMTP Server id 14.3.487.0; Sun, 27 Sep 2020 21:04:40 +0800 From: Ying Fang To: Subject: [RFC PATCH 3/7] block-backend: add I/O hang timeout Date: Sun, 27 Sep 2020 21:04:16 +0800 Message-ID: <20200927130420.1095-4-fangying1@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20200927130420.1095-1-fangying1@huawei.com> References: <20200927130420.1095-1-fangying1@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.104] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.32; envelope-from=fangying1@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/27 09:04:49 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Ying Fang , Jiahui Cen , zhang.zhanghailiang@huawei.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Not all errors would be fixed, so it is better to add a rehandle timeout for I/O hang. Signed-off-by: Jiahui Cen Signed-off-by: Ying Fang --- block/block-backend.c | 99 +++++++++++++++++++++++++++++++++- include/sysemu/block-backend.h | 2 + 2 files changed, 100 insertions(+), 1 deletion(-) diff --git a/block/block-backend.c b/block/block-backend.c index 90f1ca5753..d0b2b59f55 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -38,6 +38,11 @@ static AioContext *blk_aiocb_get_aio_context(BlockAIOCB = *acb); /* block backend rehandle timer interval 5s */ #define BLOCK_BACKEND_REHANDLE_TIMER_INTERVAL 5000 =20 +enum BlockIOHangStatus { + BLOCK_IO_HANG_STATUS_NORMAL =3D 0, + BLOCK_IO_HANG_STATUS_HANG, +}; + typedef struct BlockBackendRehandleInfo { bool enable; QEMUTimer *ts; @@ -109,6 +114,11 @@ struct BlockBackend { unsigned int in_flight; =20 BlockBackendRehandleInfo reinfo; + + int64_t iohang_timeout; /* The I/O hang timeout value in sec. */ + int64_t iohang_time; /* The I/O hang start time */ + bool is_iohang_timeout; + int iohang_status; }; =20 typedef struct BlockBackendAIOCB { @@ -2480,20 +2490,107 @@ static void blk_rehandle_timer_cb(void *opaque) aio_context_release(blk_get_aio_context(blk)); } =20 +static bool blk_iohang_handle(BlockBackend *blk, int new_status) +{ + int64_t now; + int old_status =3D blk->iohang_status; + bool need_rehandle =3D false; + + switch (new_status) { + case BLOCK_IO_HANG_STATUS_NORMAL: + if (old_status =3D=3D BLOCK_IO_HANG_STATUS_HANG) { + /* Case when I/O Hang is recovered */ + blk->is_iohang_timeout =3D false; + blk->iohang_time =3D 0; + } + break; + case BLOCK_IO_HANG_STATUS_HANG: + if (old_status !=3D BLOCK_IO_HANG_STATUS_HANG) { + /* Case when I/O hang is first triggered */ + blk->iohang_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) / = 1000; + need_rehandle =3D true; + } else { + if (!blk->is_iohang_timeout) { + now =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) / 1000; + if (now >=3D (blk->iohang_time + blk->iohang_timeout)) { + /* Case when I/O hang is timeout */ + blk->is_iohang_timeout =3D true; + } else { + /* Case when I/O hang is continued */ + need_rehandle =3D true; + } + } + } + break; + default: + break; + } + + blk->iohang_status =3D new_status; + return need_rehandle; +} + +static bool blk_rehandle_aio(BlkAioEmAIOCB *acb, bool *has_timeout) +{ + bool need_rehandle =3D false; + + /* Rehandle aio which returns EIO before hang timeout */ + if (acb->rwco.ret =3D=3D -EIO) { + if (acb->rwco.blk->is_iohang_timeout) { + /* I/O hang has timeout and not recovered */ + *has_timeout =3D true; + } else { + need_rehandle =3D blk_iohang_handle(acb->rwco.blk, + BLOCK_IO_HANG_STATUS_HANG); + /* I/O hang timeout first trigger */ + if (acb->rwco.blk->is_iohang_timeout) { + *has_timeout =3D true; + } + } + } + + return need_rehandle; +} + static void blk_rehandle_aio_complete(BlkAioEmAIOCB *acb) { + bool has_timeout =3D false; + bool need_rehandle =3D false; + if (acb->has_returned) { blk_dec_in_flight(acb->rwco.blk); - if (acb->rwco.ret =3D=3D -EIO) { + need_rehandle =3D blk_rehandle_aio(acb, &has_timeout); + if (need_rehandle) { blk_rehandle_insert_aiocb(acb->rwco.blk, acb); return; } =20 acb->common.cb(acb->common.opaque, acb->rwco.ret); + + /* I/O hang return to normal status */ + if (!has_timeout) { + blk_iohang_handle(acb->rwco.blk, BLOCK_IO_HANG_STATUS_NORMAL); + } + qemu_aio_unref(acb); } } =20 +void blk_iohang_init(BlockBackend *blk, int64_t iohang_timeout) +{ + if (!blk) { + return; + } + + blk->is_iohang_timeout =3D false; + blk->iohang_time =3D 0; + blk->iohang_timeout =3D 0; + blk->iohang_status =3D BLOCK_IO_HANG_STATUS_NORMAL; + if (iohang_timeout > 0) { + blk->iohang_timeout =3D iohang_timeout; + } +} + void blk_register_buf(BlockBackend *blk, void *host, size_t size) { bdrv_register_buf(blk_bs(blk), host, size); diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index 8203d7f6f9..bfebe3a960 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -268,4 +268,6 @@ const BdrvChild *blk_root(BlockBackend *blk); =20 int blk_make_empty(BlockBackend *blk, Error **errp); =20 +void blk_iohang_init(BlockBackend *blk, int64_t iohang_timeout); + #endif --=20 2.23.0 From nobody Fri May 17 12:53:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1601212011; cv=none; d=zohomail.com; s=zohoarc; b=j+g90mz3PsfeO1QH2CYluVxBCZ662uGICuLTvKcKwrQiLXjz7mGwoZzzysD9idmE9PMzPUkUAIy+VKgBhRVSVt9nSCoBslS+QyxUnez+KpkJenYiTrq/fsTHWuhY2EKqSjsx1BFhK87foweO3HIHaTmyPpgg+EC1y251BlPXWkA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601212011; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hCg2TUsRM+MENtNvRADCYulQA6aN2linUWsHyq7e/o8=; b=BNWQgL1cS2UXOPNJ1AK35lRV5O1OmunFAEh61JqS6a5ipxVK9rRnh+BMVhGVn4FUUuyOYa9tw5Q1dxPT2pzw160SmCVOZ1NBxGES8rOG35v3rEGkK2EZ82r+bMGeEDAS47siFPUQlVatmZAsgzzojGmuBVqElYrsTcrvngeCrpA= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601212011588207.5117637544322; Sun, 27 Sep 2020 06:06:51 -0700 (PDT) Received: from localhost ([::1]:56506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMWOA-0001F8-5K for importer@patchew.org; Sun, 27 Sep 2020 09:06:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60858) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMS-0007qe-5d for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:04 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:40272 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMQ-0003Mz-1H for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:03 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id AFA9BD578B7977E05671; Sun, 27 Sep 2020 21:04:47 +0800 (CST) Received: from localhost (10.174.185.104) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.487.0; Sun, 27 Sep 2020 21:04:41 +0800 From: Ying Fang To: Subject: [RFC PATCH 4/7] block-backend: add I/O hang drain when disbale Date: Sun, 27 Sep 2020 21:04:17 +0800 Message-ID: <20200927130420.1095-5-fangying1@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20200927130420.1095-1-fangying1@huawei.com> References: <20200927130420.1095-1-fangying1@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.104] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.35; envelope-from=fangying1@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/27 09:04:48 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Ying Fang , Jiahui Cen , zhang.zhanghailiang@huawei.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" To disable I/O hang, all hanging AIOs need to be drained. A rehandle status field is introduced to notify rehandle mechanism not to rehandle failed AIOs when I/O hang is disabled. Signed-off-by: Ying Fang Signed-off-by: Jiahui Cen --- block/block-backend.c | 85 ++++++++++++++++++++++++++++++++-- include/sysemu/block-backend.h | 3 ++ 2 files changed, 84 insertions(+), 4 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index d0b2b59f55..95b2d6a679 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -37,6 +37,9 @@ static AioContext *blk_aiocb_get_aio_context(BlockAIOCB *= acb); =20 /* block backend rehandle timer interval 5s */ #define BLOCK_BACKEND_REHANDLE_TIMER_INTERVAL 5000 +#define BLOCK_BACKEND_REHANDLE_NORMAL 1 +#define BLOCK_BACKEND_REHANDLE_DRAIN_REQUESTED 2 +#define BLOCK_BACKEND_REHANDLE_DRAINED 3 =20 enum BlockIOHangStatus { BLOCK_IO_HANG_STATUS_NORMAL =3D 0, @@ -50,6 +53,8 @@ typedef struct BlockBackendRehandleInfo { =20 unsigned int in_flight; QTAILQ_HEAD(, BlkAioEmAIOCB) re_aios; + + int status; } BlockBackendRehandleInfo; =20 typedef struct BlockBackendAioNotifier { @@ -471,6 +476,8 @@ static void blk_delete(BlockBackend *blk) assert(!blk->refcnt); assert(!blk->name); assert(!blk->dev); + assert(atomic_read(&blk->reinfo.in_flight) =3D=3D 0); + blk_rehandle_disable(blk); if (blk->public.throttle_group_member.throttle_state) { blk_io_limits_disable(blk); } @@ -2460,6 +2467,37 @@ static void blk_rehandle_remove_aiocb(BlockBackend *= blk, BlkAioEmAIOCB *acb) atomic_dec(&blk->reinfo.in_flight); } =20 +static void blk_rehandle_drain(BlockBackend *blk) +{ + if (blk_bs(blk)) { + bdrv_drained_begin(blk_bs(blk)); + BDRV_POLL_WHILE(blk_bs(blk), atomic_read(&blk->reinfo.in_flight) >= 0); + bdrv_drained_end(blk_bs(blk)); + } +} + +static bool blk_rehandle_is_paused(BlockBackend *blk) +{ + return blk->reinfo.status =3D=3D BLOCK_BACKEND_REHANDLE_DRAIN_REQUESTE= D || + blk->reinfo.status =3D=3D BLOCK_BACKEND_REHANDLE_DRAINED; +} + +static void blk_rehandle_pause(BlockBackend *blk) +{ + BlockBackendRehandleInfo *reinfo =3D &blk->reinfo; + + aio_context_acquire(blk_get_aio_context(blk)); + if (!reinfo->enable || reinfo->status =3D=3D BLOCK_BACKEND_REHANDLE_DR= AINED) { + aio_context_release(blk_get_aio_context(blk)); + return; + } + + reinfo->status =3D BLOCK_BACKEND_REHANDLE_DRAIN_REQUESTED; + blk_rehandle_drain(blk); + reinfo->status =3D BLOCK_BACKEND_REHANDLE_DRAINED; + aio_context_release(blk_get_aio_context(blk)); +} + static void blk_rehandle_timer_cb(void *opaque) { BlockBackend *blk =3D opaque; @@ -2559,10 +2597,12 @@ static void blk_rehandle_aio_complete(BlkAioEmAIOCB= *acb) =20 if (acb->has_returned) { blk_dec_in_flight(acb->rwco.blk); - need_rehandle =3D blk_rehandle_aio(acb, &has_timeout); - if (need_rehandle) { - blk_rehandle_insert_aiocb(acb->rwco.blk, acb); - return; + if (!blk_rehandle_is_paused(acb->rwco.blk)) { + need_rehandle =3D blk_rehandle_aio(acb, &has_timeout); + if (need_rehandle) { + blk_rehandle_insert_aiocb(acb->rwco.blk, acb); + return; + } } =20 acb->common.cb(acb->common.opaque, acb->rwco.ret); @@ -2576,6 +2616,42 @@ static void blk_rehandle_aio_complete(BlkAioEmAIOCB = *acb) } } =20 +void blk_rehandle_enable(BlockBackend *blk) +{ + BlockBackendRehandleInfo *reinfo =3D &blk->reinfo; + + aio_context_acquire(blk_get_aio_context(blk)); + if (reinfo->enable) { + aio_context_release(blk_get_aio_context(blk)); + return; + } + + reinfo->ts =3D aio_timer_new(blk_get_aio_context(blk), QEMU_CLOCK_REAL= TIME, + SCALE_MS, blk_rehandle_timer_cb, blk); + reinfo->timer_interval_ms =3D BLOCK_BACKEND_REHANDLE_TIMER_INTERVAL; + reinfo->status =3D BLOCK_BACKEND_REHANDLE_NORMAL; + reinfo->enable =3D true; + aio_context_release(blk_get_aio_context(blk)); +} + +void blk_rehandle_disable(BlockBackend *blk) +{ + if (!blk->reinfo.enable) { + return; + } + + blk_rehandle_pause(blk); + timer_del(blk->reinfo.ts); + timer_free(blk->reinfo.ts); + blk->reinfo.ts =3D NULL; + blk->reinfo.enable =3D false; +} + +bool blk_iohang_is_enabled(BlockBackend *blk) +{ + return blk->iohang_timeout !=3D 0; +} + void blk_iohang_init(BlockBackend *blk, int64_t iohang_timeout) { if (!blk) { @@ -2588,6 +2664,7 @@ void blk_iohang_init(BlockBackend *blk, int64_t iohan= g_timeout) blk->iohang_status =3D BLOCK_IO_HANG_STATUS_NORMAL; if (iohang_timeout > 0) { blk->iohang_timeout =3D iohang_timeout; + blk_rehandle_enable(blk); } } =20 diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index bfebe3a960..375ae13b0b 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -268,6 +268,9 @@ const BdrvChild *blk_root(BlockBackend *blk); =20 int blk_make_empty(BlockBackend *blk, Error **errp); =20 +void blk_rehandle_enable(BlockBackend *blk); +void blk_rehandle_disable(BlockBackend *blk); +bool blk_iohang_is_enabled(BlockBackend *blk); void blk_iohang_init(BlockBackend *blk, int64_t iohang_timeout); =20 #endif --=20 2.23.0 From nobody Fri May 17 12:53:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1601212129; cv=none; d=zohomail.com; s=zohoarc; b=XuOxQP3PG4+me9hnUCHCqAAM2jiej3BPIx9Wtccp6KnWdeJZO76ebbihC9jI/3VzW//ii7XsxJcdXmTj/AgWeCw5qsoYEK9+KJk21wAp2qUHRw+2j0vShDbSNdNAKs51X4aXS1zy8KApvPiQBCEMnSpgWL3QpN0jBOxIPZK/X5s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601212129; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1e0ygzuxWrQYf/x6osumWOREM0nocZElA8ShNdsp5hE=; b=XKcjnPHYPEFm3U5qaMGLChvWdPOZ/HMMzffzN3MCBOSr7dYP4Szn55fltsA+lONkBQbDlqFnocMuaFhQQWsQ3RAihOJpxtbdE0Wp8n/DjshYfbc8J/aEtH4rjpzMTS2o/S8Ru695+S8XmBxNI/AZ0DjYfNkBlSa2CMxHJGulK9Q= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601212129839352.50348544959445; Sun, 27 Sep 2020 06:08:49 -0700 (PDT) Received: from localhost ([::1]:36186 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMWQ4-0004QW-Jp for importer@patchew.org; Sun, 27 Sep 2020 09:08:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60918) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMX-00081g-EL for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:09 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:60448 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMV-0003Nt-F2 for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:09 -0400 Received: from DGGEMS410-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 03E51313DBA182BE1FC0; Sun, 27 Sep 2020 21:04:50 +0800 (CST) Received: from localhost (10.174.185.104) by DGGEMS410-HUB.china.huawei.com (10.3.19.210) with Microsoft SMTP Server id 14.3.487.0; Sun, 27 Sep 2020 21:04:41 +0800 From: Ying Fang To: Subject: [RFC PATCH 5/7] virtio-blk: disable I/O hang when resetting Date: Sun, 27 Sep 2020 21:04:18 +0800 Message-ID: <20200927130420.1095-6-fangying1@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20200927130420.1095-1-fangying1@huawei.com> References: <20200927130420.1095-1-fangying1@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.104] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.32; envelope-from=fangying1@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/27 09:04:49 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Ying Fang , Jiahui Cen , zhang.zhanghailiang@huawei.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" All AIOs including the hanging AIOs need to be drained when resetting virtio-blk. So it is necessary to disable I/O hang before resetting and enable I/O hang again after resetting if I/O hang is enabled. Signed-off-by: Ying Fang Signed-off-by: Jiahui Cen --- hw/block/virtio-blk.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 2204ba149e..11837a54f5 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -892,6 +892,10 @@ static void virtio_blk_reset(VirtIODevice *vdev) AioContext *ctx; VirtIOBlockReq *req; =20 + if (blk_iohang_is_enabled(s->blk)) { + blk_rehandle_disable(s->blk); + } + ctx =3D blk_get_aio_context(s->blk); aio_context_acquire(ctx); blk_drain(s->blk); @@ -909,6 +913,10 @@ static void virtio_blk_reset(VirtIODevice *vdev) =20 assert(!s->dataplane_started); blk_set_enable_write_cache(s->blk, s->original_wce); + + if (blk_iohang_is_enabled(s->blk)) { + blk_rehandle_enable(s->blk); + } } =20 /* coalesce internal state, copy to pci i/o region 0 --=20 2.23.0 From nobody Fri May 17 12:53:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1601212007; cv=none; d=zohomail.com; s=zohoarc; b=NcKRGGmPPs6Wzg0xidNNys3Z+P1gELdjukFWnU64052ltMzwh4GDAItNm9OqyCVCs8rdZbTSbBikL78gnaDcmzAGGjuxmOfJ+SM5aouXBYWlxt49sxlS+gskpT1DwaDurj2v+kXUv1MMZlORbxH6Ob7+ONfbvq57lCW4DZ6zZcw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601212007; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2L20oynCUWqIG6MVT5aZlVplpG2vqhcmf54NRDBuB1k=; b=WQDel5oFIyxQXA01dQA76LKKbuWIfJLdINeVaJrH6voVXwzErQRXOkOa7rkV0HIYg/X622Ao4E/cwSaxLwiGGNU7UhWTHxwBYGh8miLPzyBZDsois/b0FN2Ql3zrYQZgC0z3SHvK5czgbbz0OGE1aEVJ6kazjPV3n6yAPaIpO3U= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 160121200769769.9959170975792; Sun, 27 Sep 2020 06:06:47 -0700 (PDT) Received: from localhost ([::1]:56088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMWO5-00013b-VT for importer@patchew.org; Sun, 27 Sep 2020 09:06:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60820) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMR-0007qX-DZ for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:03 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:60242 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMP-0003N3-5j for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:03 -0400 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id A1FD2B9EE6899670B86C; Sun, 27 Sep 2020 21:04:48 +0800 (CST) Received: from localhost (10.174.185.104) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.487.0; Sun, 27 Sep 2020 21:04:42 +0800 From: Ying Fang To: Subject: [RFC PATCH 6/7] qemu-option: add I/O hang timeout option Date: Sun, 27 Sep 2020 21:04:19 +0800 Message-ID: <20200927130420.1095-7-fangying1@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20200927130420.1095-1-fangying1@huawei.com> References: <20200927130420.1095-1-fangying1@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.104] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.32; envelope-from=fangying1@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/27 09:04:49 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Ying Fang , Jiahui Cen , zhang.zhanghailiang@huawei.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" I/O hang timeout should be different under different situations. So it is better to provide an option for user to determine I/O hang timeout for each block device. Signed-off-by: Jiahui Cen Signed-off-by: Ying Fang --- blockdev.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/blockdev.c b/blockdev.c index 7f2561081e..ff8cdcd497 100644 --- a/blockdev.c +++ b/blockdev.c @@ -500,6 +500,7 @@ static BlockBackend *blockdev_init(const char *file, QD= ict *bs_opts, BlockdevDetectZeroesOptions detect_zeroes =3D BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF; const char *throttling_group =3D NULL; + int64_t iohang_timeout =3D 0; =20 /* Check common options by copying from bs_opts to opts, all other opt= ions * stay in bs_opts for processing by bdrv_open(). */ @@ -622,6 +623,12 @@ static BlockBackend *blockdev_init(const char *file, Q= Dict *bs_opts, =20 bs->detect_zeroes =3D detect_zeroes; =20 + /* init timeout value for I/O Hang */ + iohang_timeout =3D qemu_opt_get_number(opts, "iohang-timeout", 0); + if (iohang_timeout > 0) { + blk_iohang_init(blk, iohang_timeout); + } + block_acct_setup(blk_get_stats(blk), account_invalid, account_fail= ed); =20 if (!parse_stats_intervals(blk_get_stats(blk), interval_list, errp= )) { @@ -3786,6 +3793,10 @@ QemuOptsList qemu_common_drive_opts =3D { .type =3D QEMU_OPT_BOOL, .help =3D "whether to account for failed I/O operations " "in the statistics", + },{ + .name =3D "iohang-timeout", + .type =3D QEMU_OPT_NUMBER, + .help =3D "timeout value for I/O Hang", }, { /* end of list */ } }, --=20 2.23.0 From nobody Fri May 17 12:53:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1601212267; cv=none; d=zohomail.com; s=zohoarc; b=bQPE0EO6aroB+ctdhmQqnQqePN5D8TBseRq7jvKN476sKl2cLCPKgdQRfnrfIxgh4ALtzmlcfS/M2HEnukuGV8dpbEZIVtwCW/y1OSh7QDJUchucyI7SZSHiMd6SUD6dNIP9/o/cwzcxiTox9Euneiq7lpgCvWHm27XCgTp0dqI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601212267; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AR2UFCKG4H1rMt+M+9/3Sx6HPIZqrTxz7RJjIKb1dR0=; b=RpmqyucOzVH4BYSdU5cSIannYqHTcBL37U0d7TS+TjQueZxnssqBy1MjlC9ypCPhXbJoNvkNBUjiheJrOmexth4PwKWaehc3VWyWKQPFxoEOp/Ni2nIVa/37vM+CIzcb+KCtwSFIXLEhVG+qVp+WZnunfQnujl5CDyRsc7TyWVE= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601212267290223.18490444565646; Sun, 27 Sep 2020 06:11:07 -0700 (PDT) Received: from localhost ([::1]:40558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMWSI-0006Ig-3T for importer@patchew.org; Sun, 27 Sep 2020 09:11:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60886) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMU-0007so-Ac for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:06 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:60250 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMWMQ-0003N4-NT for qemu-devel@nongnu.org; Sun, 27 Sep 2020 09:05:06 -0400 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 9CBBE948378066848F43; Sun, 27 Sep 2020 21:04:48 +0800 (CST) Received: from localhost (10.174.185.104) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.487.0; Sun, 27 Sep 2020 21:04:42 +0800 From: Ying Fang To: Subject: [RFC PATCH 7/7] qapi: add I/O hang and I/O hang timeout qapi event Date: Sun, 27 Sep 2020 21:04:20 +0800 Message-ID: <20200927130420.1095-8-fangying1@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20200927130420.1095-1-fangying1@huawei.com> References: <20200927130420.1095-1-fangying1@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.185.104] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.32; envelope-from=fangying1@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/27 09:04:49 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Ying Fang , Jiahui Cen , zhang.zhanghailiang@huawei.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Sometimes hypervisor management tools like libvirt may need to monitor I/O hang events. Let's report I/O hang and I/O hang timeout event via qapi. Signed-off-by: Jiahui Cen Signed-off-by: Ying Fang --- block/block-backend.c | 3 +++ qapi/block-core.json | 26 ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index 95b2d6a679..5dc5b11bcc 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -2540,6 +2540,7 @@ static bool blk_iohang_handle(BlockBackend *blk, int = new_status) /* Case when I/O Hang is recovered */ blk->is_iohang_timeout =3D false; blk->iohang_time =3D 0; + qapi_event_send_block_io_hang(false); } break; case BLOCK_IO_HANG_STATUS_HANG: @@ -2547,12 +2548,14 @@ static bool blk_iohang_handle(BlockBackend *blk, in= t new_status) /* Case when I/O hang is first triggered */ blk->iohang_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) / = 1000; need_rehandle =3D true; + qapi_event_send_block_io_hang(true); } else { if (!blk->is_iohang_timeout) { now =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) / 1000; if (now >=3D (blk->iohang_time + blk->iohang_timeout)) { /* Case when I/O hang is timeout */ blk->is_iohang_timeout =3D true; + qapi_event_send_block_io_hang_timeout(true); } else { /* Case when I/O hang is continued */ need_rehandle =3D true; diff --git a/qapi/block-core.json b/qapi/block-core.json index 3c16f1e11d..7bdf75c6d7 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -5535,3 +5535,29 @@ { 'command': 'blockdev-snapshot-delete-internal-sync', 'data': { 'device': 'str', '*id': 'str', '*name': 'str'}, 'returns': 'SnapshotInfo' } + +## +# @BLOCK_IO_HANG: +# +# Emitted when device I/O hang trigger event begin or end +# +# @set: true if I/O hang begin; false if I/O hang end. +# +# Since: 5.2 +# +## +{ 'event': 'BLOCK_IO_HANG', + 'data': { 'set': 'bool' }} + +## +# @BLOCK_IO_HANG_TIMEOUT: +# +# Emitted when device I/O hang timeout event set or clear +# +# @set: true if set; false if clear. +# +# Since: 5.2 +# +## +{ 'event': 'BLOCK_IO_HANG_TIMEOUT', + 'data': { 'set': 'bool' }} --=20 2.23.0