From nobody Mon Feb 9 07:31:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680711506; cv=none; d=zohomail.com; s=zohoarc; b=UhBrOifQY+YEmg+g4tLjYeHO2wrbkHVg12K7nT0m+uvDRpR5/ysob9+pIUEC02sKUVjLYmseQ+icc0CxVibQMvwFYUe2s2zd8POv19wjm3oxCTOkpe+ZNqIuWNpDwOUyJ517EHQLlILJwx2YuPwtMb3VOh/GdHqxDws49koytf8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680711506; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FZDKX6DN0GycYg9jzgH1X4Sy74NIHEOXcjZ7Jwmyr5s=; b=JUXbs6yJ2YIKU1alKrK9XQC2X3nWqnA7/DByE43P1eUbJ3m1fbTqcdL4U84ut0wfcET0DKnbaFA8G5nPWgJzUXDTYaJTNdOo90GiDf1+L67E+yPFB3XnIw8P1UJ2LneCZT4HldJ+hnLcmyS0oxP+eK+Y9KLK8DcM1hXKi3em6UM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1680711506427429.1032948465189; Wed, 5 Apr 2023 09:18:26 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pk5pr-000269-Bu; Wed, 05 Apr 2023 12:18:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk5pj-000250-UN for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:18:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk5pi-0007WY-5C for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:18:03 -0400 Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-421-gzzqLjpPPgCvQGy8CW_jRg-1; Wed, 05 Apr 2023 12:17:59 -0400 Received: by mail-ed1-f71.google.com with SMTP id x35-20020a50baa6000000b005021d1b1e9eso50790747ede.13 for ; Wed, 05 Apr 2023 09:17:59 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id e7-20020a170906c00700b0093f83562f09sm7551627ejz.87.2023.04.05.09.17.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 09:17:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680711481; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FZDKX6DN0GycYg9jzgH1X4Sy74NIHEOXcjZ7Jwmyr5s=; b=KxJRGphfmNwVzvkSQOCXaEoBSfQMlwtFf0/+H1YGiH8BubzvpmG/rmZK9QVooDvuzFtHCr KsSSLnk+/FcTOuK1PW8e6+MX60JUDCrfXe3e8QFCejLj+KqX3Y/IYswizmaN+fPAmO8xTp 2c4AhgpLv4GG+I6FAnWCvNd/eoES5I8= X-MC-Unique: gzzqLjpPPgCvQGy8CW_jRg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680711477; x=1683303477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FZDKX6DN0GycYg9jzgH1X4Sy74NIHEOXcjZ7Jwmyr5s=; b=FWazG+ZLVVXoo+pEtBqZEPqeDwY6DQxUrvOWCrQDi7DUKB6BfA40DzNVf3Qwgl4rS0 fzp3o/xrwIbRsOIAxXsbSiaAo50QinVsWVopGQQKIc01EFDO87kU8JlXreKs2pT+hMxA UiVJuq3XBzic0UvecS5KQuTtKOt0Zw7z1MHKLEFRvHRqNQ6fj7lOuvJ+QH/8WbSLb+oX w9YhjTLbi+pekiFaTflN44u0swWkupHr0PP7K0+YULCRRPJz3Nj14FJxNZiCGHXU22ce 0uxuYdxxrvvumjqwMbaCfNdf0SlY5+x9PWfqRsKNs7b5na3GuCUW6AvYEdVPS4FDqxQw mn0A== X-Gm-Message-State: AAQBX9eVqlC9EA+6o/wbhkzSFMKXN6T1OQC+uy4XFu2Jk8pY2avvodaH nibLMSynPm2A0/MgKnCgJCICFZ8RNeupQAGjp1m94zkruEjJdCzwh7Gm3meDtot0OYSh5oW6MUq GQYnaXERF4tp4FfuZIyCFKVejRLmRqRIb+AnyyM8Oh2AEVSEQYHqFA+uhAcv4uPf1IS1hb5qw3S E= X-Received: by 2002:a17:907:3d92:b0:93d:1c2b:bd23 with SMTP id he18-20020a1709073d9200b0093d1c2bbd23mr4715595ejc.39.1680711477641; Wed, 05 Apr 2023 09:17:57 -0700 (PDT) X-Google-Smtp-Source: AKy350ZPGbyFFfvGzxr0u4A753S0cbXqIZXFVa5F7SCn+VD4UXBUTXzD0bmfRYlzFRwxazfFJsqFHg== X-Received: by 2002:a17:907:3d92:b0:93d:1c2b:bd23 with SMTP id he18-20020a1709073d9200b0093d1c2bbd23mr4715571ejc.39.1680711477400; Wed, 05 Apr 2023 09:17:57 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, hreitz@redhat.com, kwolf@redhat.com, f.ebner@proxmox.com Subject: [PATCH 1/3] aio-posix: disable polling after aio_disable_external() Date: Wed, 5 Apr 2023 18:17:50 +0200 Message-Id: <20230405161752.194727-2-pbonzini@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230405161752.194727-1-pbonzini@redhat.com> References: <20230405161752.194727-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680711508719100003 Content-Type: text/plain; charset="utf-8" Polling can cause external requests to be picked up even if the AioContext is not looking at external file descriptors. Disable all polling between aio_disable_external() and aio_enable_external(), since aio_set_fd_poll() does not distinguish external handlers from those that in principle could run. Signed-off-by: Paolo Bonzini --- util/aio-posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/aio-posix.c b/util/aio-posix.c index a8be940f760d..0d22e3d6d37c 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -29,7 +29,7 @@ =20 bool aio_poll_disabled(AioContext *ctx) { - return qatomic_read(&ctx->poll_disable_cnt); + return qatomic_read(&ctx->poll_disable_cnt) || qatomic_read(&ctx->exte= rnal_disable_cnt); } =20 void aio_add_ready_handler(AioHandlerList *ready_list, --=20 2.39.2 From nobody Mon Feb 9 07:31:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680711540; cv=none; d=zohomail.com; s=zohoarc; b=ihQ98Z6PyUlNDp8VUccwDb+UXkRAfQ/tMiBdeWrPsDysAdD6be+WsNutDOOuqHrDQqngfBYS6htjIELI2RIv3zMDGs9SLJzIA4Ad1pkNgp+MtMgxU6oJkNN35co30mJnl+ZealRiKEI5lo1qZUZzdg3dz08PoEEF/lhPrdjoE1g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680711540; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wRv3d9XhhjNhjQXRivQph0G2zk9LhmJcwKMfnOGhgnU=; b=CbEdYrqlDITAn4fMx+xSzjhfsab3fMAqi/idtyA1JdVlS3i5vEv96mSXUVM+3IXKlgWuEem34fnEugDNd+SPgHeNdFllChcxfeHTX2UgeOgr32y/or4QUnfpWUvMF4VhLYzTF6lYKH9mfkwFXVS3RaMfiCKzyNtGuqSkI5jM51I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1680711540051133.7485195479071; Wed, 5 Apr 2023 09:19:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pk5pv-0002AG-GQ; Wed, 05 Apr 2023 12:18:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk5pp-000263-Ju for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:18:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk5pn-0007XZ-HZ for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:18:09 -0400 Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-18-4U0mPTQ2OaS2XwR6sxC3rQ-1; Wed, 05 Apr 2023 12:18:05 -0400 Received: by mail-ej1-f72.google.com with SMTP id a640c23a62f3a-930719372d5so45807866b.2 for ; Wed, 05 Apr 2023 09:18:05 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id s10-20020a056402036a00b004e48f8df7e2sm7443519edw.72.2023.04.05.09.18.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 09:18:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680711486; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wRv3d9XhhjNhjQXRivQph0G2zk9LhmJcwKMfnOGhgnU=; b=VL/xU4FDo7YZbYqTwcdnyjCEYh709oh2cpMsVE2OnXyIUP/L57yjpODlEUwyDeFRvdHzVM OqA3xo9FrMPSc2UBgs5ryw7pZWpMDlo38CL5VXTNJ7CJICnhciuFfZp4vu6u6iPGu+BqRZ 3afnZQC7oeByyjm/z+SyoBcTL8XtmWk= X-MC-Unique: 4U0mPTQ2OaS2XwR6sxC3rQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680711482; x=1683303482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wRv3d9XhhjNhjQXRivQph0G2zk9LhmJcwKMfnOGhgnU=; b=v6WREXpP2jzhaZ4EmHWZ6vlmDReNV2C/TthHuAztaozWsl2LydRmMi/NQQO/ZEzNBf HjPU++dN7wtOuT57UZet7EiG5NePhPX6oIZFUKYu3UOJotHdJ8qqGxUCdumKkaHlssMx QZNRuHbahvhGIUusmeVbp9zEw1N7naDr456H8iW/5WWn7m/WqAgA5GSjfW0ED6bw8poZ TcTJGXI8SyN4l6L8fejydiGZork8jD8YN6PlVXACreIgsWxUJpmV2OQ0CGMHlEiI+Cml pzzwcdnsStmkb7TEs+YpalgGlXehyGUvLZy0s7PjGaPGeuSgXUR7FI39bSkszYEBjzxX aD9w== X-Gm-Message-State: AAQBX9diF8kaAa2V9bujzl0lOvQN42EhRfXPEooxcDvh38hF30ozqHwH 8AZJEFrcHIucd9iFCK3OSPRwH689DH8HpmJPBvJx+1S/I1vH4ZuCnGLrTT9xGU5y9CPGClDVINS TJqQ/Vx1sEdLAQCcQQoh8ZDb16BA5fOxk2O3pZtBbdzKFRdTICfehYaanZW60nw+O8nhRq51RVK Q= X-Received: by 2002:aa7:d5c9:0:b0:4f9:deb4:b984 with SMTP id d9-20020aa7d5c9000000b004f9deb4b984mr2596315eds.9.1680711482534; Wed, 05 Apr 2023 09:18:02 -0700 (PDT) X-Google-Smtp-Source: AKy350aVxNJbFcpYBIn9pBt3c/IQs+BndlHoj8P3fr0PgkutghbxExubb314wBPjTQKtNFMeIt5nIw== X-Received: by 2002:aa7:d5c9:0:b0:4f9:deb4:b984 with SMTP id d9-20020aa7d5c9000000b004f9deb4b984mr2596286eds.9.1680711482177; Wed, 05 Apr 2023 09:18:02 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, hreitz@redhat.com, kwolf@redhat.com, f.ebner@proxmox.com Subject: [PATCH 2/3] block-backend: make global properties write-once Date: Wed, 5 Apr 2023 18:17:51 +0200 Message-Id: <20230405161752.194727-3-pbonzini@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230405161752.194727-1-pbonzini@redhat.com> References: <20230405161752.194727-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680711542438100001 Content-Type: text/plain; charset="utf-8" The three global properties allow_aio_context_change, disable_request_queuing and allow_write_before_eof are always set for the whole life of a BlockBackend. Make this clear by removing the possibility of clearing them, and by marking the corresponding function GLOBAL_STATE_CODE(). Signed-off-by: Paolo Bonzini --- block/block-backend.c | 27 +++++++++++++++------------ block/commit.c | 4 ++-- block/export/export.c | 2 +- block/mirror.c | 4 ++-- block/parallels.c | 2 +- block/qcow.c | 2 +- block/qcow2.c | 2 +- block/qed.c | 2 +- block/stream.c | 4 ++-- block/vdi.c | 2 +- block/vhdx.c | 2 +- block/vmdk.c | 4 ++-- block/vpc.c | 2 +- include/sysemu/block-backend-io.h | 6 +++--- nbd/server.c | 3 +-- tests/unit/test-bdrv-drain.c | 4 ++-- tests/unit/test-block-iothread.c | 2 +- 17 files changed, 38 insertions(+), 36 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index 9e0f48692a35..10419f8be91e 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -73,8 +73,13 @@ struct BlockBackend { uint64_t shared_perm; bool disable_perm; =20 + /* + * Can only become true; should be written before any requests is + * submitted to the BlockBackend. + */ bool allow_aio_context_change; bool allow_write_beyond_eof; + bool disable_request_queuing; =20 /* Protected by BQL */ NotifierList remove_bs_notifiers, insert_bs_notifiers; @@ -83,7 +88,6 @@ struct BlockBackend { int quiesce_counter; /* atomic: written under BQL, read by other threa= ds */ QemuMutex queued_requests_lock; /* protects queued_requests */ CoQueue queued_requests; - bool disable_request_queuing; /* atomic */ =20 VMChangeStateEntry *vmsh; bool force_allow_inactivate; @@ -1221,22 +1225,22 @@ void blk_iostatus_set_err(BlockBackend *blk, int er= ror) } } =20 -void blk_set_allow_write_beyond_eof(BlockBackend *blk, bool allow) +void blk_allow_write_beyond_eof(BlockBackend *blk) { - IO_CODE(); - blk->allow_write_beyond_eof =3D allow; + GLOBAL_STATE_CODE(); + blk->allow_write_beyond_eof =3D true; } =20 -void blk_set_allow_aio_context_change(BlockBackend *blk, bool allow) +void blk_allow_aio_context_change(BlockBackend *blk) { - IO_CODE(); - blk->allow_aio_context_change =3D allow; + GLOBAL_STATE_CODE(); + blk->allow_aio_context_change =3D true; } =20 -void blk_set_disable_request_queuing(BlockBackend *blk, bool disable) +void blk_disable_request_queuing(BlockBackend *blk) { - IO_CODE(); - qatomic_set(&blk->disable_request_queuing, disable); + GLOBAL_STATE_CODE(); + blk->disable_request_queuing =3D true; } =20 static int coroutine_fn GRAPH_RDLOCK @@ -1275,8 +1279,7 @@ static void coroutine_fn blk_wait_while_drained(Block= Backend *blk) { assert(blk->in_flight > 0); =20 - if (qatomic_read(&blk->quiesce_counter) && - !qatomic_read(&blk->disable_request_queuing)) { + if (qatomic_read(&blk->quiesce_counter) && !blk->disable_request_queui= ng) { /* * Take lock before decrementing in flight counter so main loop th= read * waits for us to enqueue ourselves before it can leave the drain= ed diff --git a/block/commit.c b/block/commit.c index 2b20fd0fd4d2..88e1d7373d36 100644 --- a/block/commit.c +++ b/block/commit.c @@ -379,7 +379,7 @@ void commit_start(const char *job_id, BlockDriverState = *bs, if (ret < 0) { goto fail; } - blk_set_disable_request_queuing(s->base, true); + blk_disable_request_queuing(s->base); s->base_bs =3D base; =20 /* Required permissions are already taken with block_job_add_bdrv() */ @@ -388,7 +388,7 @@ void commit_start(const char *job_id, BlockDriverState = *bs, if (ret < 0) { goto fail; } - blk_set_disable_request_queuing(s->top, true); + blk_disable_request_queuing(s->top); =20 s->backing_file_str =3D g_strdup(backing_file_str); s->on_error =3D on_error; diff --git a/block/export/export.c b/block/export/export.c index e3fee6061169..0a1336c07fed 100644 --- a/block/export/export.c +++ b/block/export/export.c @@ -155,7 +155,7 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Er= ror **errp) blk =3D blk_new(ctx, perm, BLK_PERM_ALL); =20 if (!fixed_iothread) { - blk_set_allow_aio_context_change(blk, true); + blk_allow_aio_context_change(blk); } =20 ret =3D blk_insert_bs(blk, bs, errp); diff --git a/block/mirror.c b/block/mirror.c index 1c46ad51bf50..93eda37660a3 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1787,8 +1787,8 @@ static BlockJob *mirror_start_job( * ensure that. */ blk_set_force_allow_inactivate(s->target); } - blk_set_allow_aio_context_change(s->target, true); - blk_set_disable_request_queuing(s->target, true); + blk_allow_aio_context_change(s->target); + blk_disable_request_queuing(s->target); =20 s->replaces =3D g_strdup(replaces); s->on_source_error =3D on_source_error; diff --git a/block/parallels.c b/block/parallels.c index 013684801a61..97a5c629bbab 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -578,7 +578,7 @@ static int coroutine_fn parallels_co_create(BlockdevCre= ateOptions* opts, ret =3D -EPERM; goto out; } - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); =20 /* Create image format */ bat_entries =3D DIV_ROUND_UP(total_size, cl_size); diff --git a/block/qcow.c b/block/qcow.c index 490e4f819ed1..5089dd0c6bf3 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -842,7 +842,7 @@ static int coroutine_fn qcow_co_create(BlockdevCreateOp= tions *opts, ret =3D -EPERM; goto exit; } - blk_set_allow_write_beyond_eof(qcow_blk, true); + blk_allow_write_beyond_eof(qcow_blk); =20 /* Create image format */ memset(&header, 0, sizeof(header)); diff --git a/block/qcow2.c b/block/qcow2.c index f8ea03a34515..761aa7e1555a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3643,7 +3643,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options= , Error **errp) ret =3D -EPERM; goto out; } - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); =20 /* Write the header */ QEMU_BUILD_BUG_ON((1 << MIN_CLUSTER_BITS) < sizeof(*header)); diff --git a/block/qed.c b/block/qed.c index 0705a7b4e25f..7fec1cabc4f6 100644 --- a/block/qed.c +++ b/block/qed.c @@ -690,7 +690,7 @@ static int coroutine_fn bdrv_qed_co_create(BlockdevCrea= teOptions *opts, ret =3D -EPERM; goto out; } - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); =20 /* Prepare image format */ header =3D (QEDHeader) { diff --git a/block/stream.c b/block/stream.c index d92a4c99d359..935e109a4e05 100644 --- a/block/stream.c +++ b/block/stream.c @@ -336,8 +336,8 @@ void stream_start(const char *job_id, BlockDriverState = *bs, * Disable request queuing in the BlockBackend to avoid deadlocks on d= rain: * The job reports that it's busy until it reaches a pause point. */ - blk_set_disable_request_queuing(s->blk, true); - blk_set_allow_aio_context_change(s->blk, true); + blk_disable_request_queuing(s->blk); + blk_allow_aio_context_change(s->blk); =20 /* * Prevent concurrent jobs trying to modify the graph structure here, = we diff --git a/block/vdi.c b/block/vdi.c index f2434d6153e1..1e4eb6a0bd0b 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -813,7 +813,7 @@ static int coroutine_fn vdi_co_do_create(BlockdevCreate= Options *create_options, goto exit; } =20 - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); =20 /* We need enough blocks to store the given disk size, so always round up. */ diff --git a/block/vhdx.c b/block/vhdx.c index 81420722a188..7f59b6cb0403 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -2003,7 +2003,7 @@ static int coroutine_fn vhdx_co_create(BlockdevCreate= Options *opts, ret =3D -EPERM; goto delete_and_exit; } - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); =20 /* Create (A) */ =20 diff --git a/block/vmdk.c b/block/vmdk.c index 3f8c731e32e8..08a009f527e1 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -2298,7 +2298,7 @@ vmdk_create_extent(const char *filename, int64_t file= size, bool flat, goto exit; } =20 - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); =20 ret =3D vmdk_init_extent(blk, filesize, flat, compress, zeroed_grain, = errp); exit: @@ -2796,7 +2796,7 @@ static BlockBackend * coroutine_fn vmdk_co_create_cb(= int64_t size, int idx, if (!blk) { return NULL; } - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); bdrv_unref(bs); =20 if (size !=3D -1) { diff --git a/block/vpc.c b/block/vpc.c index b89b0ff8e275..1dc9a86c6aa2 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -1016,7 +1016,7 @@ static int coroutine_fn vpc_co_create(BlockdevCreateO= ptions *opts, ret =3D -EPERM; goto out; } - blk_set_allow_write_beyond_eof(blk, true); + blk_allow_write_beyond_eof(blk); =20 /* Get geometry and check that it matches the image size*/ ret =3D calculate_rounded_image_size(vpc_opts, &cyls, &heads, &secs_pe= r_cyl, diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backe= nd-io.h index db29c164997d..1a55d25c133a 100644 --- a/include/sysemu/block-backend-io.h +++ b/include/sysemu/block-backend-io.h @@ -27,9 +27,9 @@ const char *blk_name(const BlockBackend *blk); =20 BlockDriverState *blk_bs(BlockBackend *blk); =20 -void blk_set_allow_write_beyond_eof(BlockBackend *blk, bool allow); -void blk_set_allow_aio_context_change(BlockBackend *blk, bool allow); -void blk_set_disable_request_queuing(BlockBackend *blk, bool disable); +void blk_allow_write_beyond_eof(BlockBackend *blk); +void blk_allow_aio_context_change(BlockBackend *blk); +void blk_disable_request_queuing(BlockBackend *blk); bool blk_iostatus_is_enabled(const BlockBackend *blk); =20 char *blk_get_attached_dev_id(BlockBackend *blk); diff --git a/nbd/server.c b/nbd/server.c index cb41b56095ee..423dc2d2517e 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -1777,7 +1777,7 @@ static int nbd_export_create(BlockExport *blk_exp, Bl= ockExportOptions *exp_args, * be properly quiesced when entering a drained section, as our corout= ines * servicing pending requests might enter blk_pread(). */ - blk_set_disable_request_queuing(blk, true); + blk_disable_request_queuing(blk); =20 blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, ex= p); =20 @@ -1853,7 +1853,6 @@ static void nbd_export_delete(BlockExport *blk_exp) } blk_remove_aio_context_notifier(exp->common.blk, blk_aio_attached, blk_aio_detach, exp); - blk_set_disable_request_queuing(exp->common.blk, false); =20 for (i =3D 0; i < exp->nr_export_bitmaps; i++) { bdrv_dirty_bitmap_set_busy(exp->export_bitmaps[i], false); diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index d9d38070621a..9484e194d6f9 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -513,7 +513,7 @@ static void test_iothread_common(enum drain_type drain_= type, int drain_thread) &error_abort); s =3D bs->opaque; blk_insert_bs(blk, bs, &error_abort); - blk_set_disable_request_queuing(blk, true); + blk_disable_request_queuing(blk); =20 blk_set_aio_context(blk, ctx_a, &error_abort); aio_context_acquire(ctx_a); @@ -739,7 +739,7 @@ static void test_blockjob_common_drain_node(enum drain_= type drain_type, &error_abort); blk_target =3D blk_new(qemu_get_aio_context(), BLK_PERM_ALL, BLK_PERM_= ALL); blk_insert_bs(blk_target, target, &error_abort); - blk_set_allow_aio_context_change(blk_target, true); + blk_allow_aio_context_change(blk_target); =20 aio_context_acquire(ctx); tjob =3D block_job_create("job0", &test_job_driver, NULL, src, diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothr= ead.c index 3a5e1eb2c413..90b60ce32c68 100644 --- a/tests/unit/test-block-iothread.c +++ b/tests/unit/test-block-iothread.c @@ -795,7 +795,7 @@ static void test_propagate_mirror(void) =20 /* ...unless we explicitly allow it */ aio_context_acquire(ctx); - blk_set_allow_aio_context_change(blk, true); + blk_allow_aio_context_change(blk); bdrv_try_change_aio_context(target, ctx, NULL, &error_abort); aio_context_release(ctx); =20 --=20 2.39.2 From nobody Mon Feb 9 07:31:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680712301; cv=none; d=zohomail.com; s=zohoarc; b=BE3vzE0FIcB0OLPUqScigUUJMqCjAg5R9MM0rVUXkQbsakrppCzKUbZpUGJtXrsowCK2CtDM3IvmEPSW0daxOIKxkfDGm04DgjNIlf1ArH2h/rpucj+wVPRFgMlAlTsVTyJXZEq+ME1phJLHs0hQet1YKo7Nz4DrFbs5cWT+dJM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680712301; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PD1i5VlxjmCGb03EV7N5z9TRh9g2GyvKM4Zs6uY6moc=; b=KTFZrvm3YyWqSUT7QqKgsTUEWwV5+bZVG+J9sT+E9TN2q14eTOlhy6xKKE4BaZysPWfp1XkapmTfSoafOZjmDm0boqmzcuAGij/xOBvrHQB+I5lsBxgIRp1llEiQl7+U0/75bh7hvtxsbsey+v0NzWriALIGe9H0KXjpMvfv1/0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1680712301731685.6822149311985; Wed, 5 Apr 2023 09:31:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pk62Y-0006Gy-7v; Wed, 05 Apr 2023 12:31:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk62W-0006Fj-Kc for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:31:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk62U-0006ys-Tv for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:31:16 -0400 Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-664-XG27DRO3MNOpqDkwT2QWrA-1; Wed, 05 Apr 2023 12:31:12 -0400 Received: by mail-ed1-f72.google.com with SMTP id v1-20020a50d581000000b0050291cda08aso16932340edi.15 for ; Wed, 05 Apr 2023 09:31:12 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id l16-20020a170906939000b00930d505a567sm7498234ejx.128.2023.04.05.09.31.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 09:31:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680712274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PD1i5VlxjmCGb03EV7N5z9TRh9g2GyvKM4Zs6uY6moc=; b=Oau2BaPUkfL1hf2eeiN1tFO8cukdwQrV83dvdXy6/Gt/YrfpHv/F+DYlrMI8bDKMX4ubj3 zXLVqfyZKgkz3kq6lu1hQOlFF/cJwQJzZq2+oQmLBTKDNmlIR6M4gNhOIMMeLBSzultuUT CkC/W95fOn1fAmyaGJYHzDzpQlccLss= X-MC-Unique: XG27DRO3MNOpqDkwT2QWrA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680712271; x=1683304271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PD1i5VlxjmCGb03EV7N5z9TRh9g2GyvKM4Zs6uY6moc=; b=mHGjikaWagm6DFL2ZVNpLZje/0RQTn7dm21LJp/5vfd7St+kmoGPJiL5qtDF9U5sA6 BtQNObsYCP4ZJlVtffT8EK/Gz5rqMhosma9MEsSMxyx3j5u/YBH5JKV6EncSvT14uQXj MFRJItBUPI8IeHFAFwaBdJeSRXjS9LJbALhmfPALpdkwho/lxcEZUOaPe1FSuJaRs4mU ptGtLqApCgI0sioWZZVZoCJIXgdRN6h8RieD/zVZ89ng9R0FDjQyjDKc3JwJp+iYKE9c Es+6bVRrYTCiZWX+Pc4pV/RJvU/YjHeSg0eNqo0kwzYwtoyQJbKrVwGyVttd78yzvKk1 O6AQ== X-Gm-Message-State: AAQBX9eBDW8KQ9bVvXb+4pO93aTmmIRenD3xchbhggvPICYfWQmVQSbP aQRan+I/ss22/vYRQ5KUg/dGBHLeFwByAU3QMset/pp3dzIpNfwefxLIV2jLwGZCaNSdHEzbX8d gMNaMgap6U8D7HjK0DHDURDXXbOd9NZiMBa/RQnghxRo1QsOa/6fbngu5UD+p6Hijp6gr41vmoO M= X-Received: by 2002:a17:907:3201:b0:8b1:2eef:154c with SMTP id xg1-20020a170907320100b008b12eef154cmr3650191ejb.0.1680712270904; Wed, 05 Apr 2023 09:31:10 -0700 (PDT) X-Google-Smtp-Source: AKy350YgmmrstwLM9er6whhqISlnaFTBQG3GVZsWRo5UEspLnypzmK6pqX3Tnkx5QhnCZpKUpwGWCw== X-Received: by 2002:a17:907:3201:b0:8b1:2eef:154c with SMTP id xg1-20020a170907320100b008b12eef154cmr3650166ejb.0.1680712270499; Wed, 05 Apr 2023 09:31:10 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Fiona Ebner Subject: [PATCH 3/3] block-backend: delay application of request queuing Date: Wed, 5 Apr 2023 18:31:09 +0200 Message-Id: <20230405163109.197876-1-pbonzini@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230405161752.194727-1-pbonzini@redhat.com> References: <20230405161752.194727-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680712302757100005 Content-Type: text/plain; charset="utf-8" Request queuing prevents new requests from being submitted to the BlockDriverState, not allowing them to start instead of just letting them complete before bdrv_drained_begin() returns. The reason for this was to ensure progress and avoid a livelock in blk_drain(), blk_drain_all_begin(), bdrv_drained_begin() or bdrv_drain_all_begin(), if there is an endless stream of requests to a BlockBackend. However, this is prone to deadlocks. In particular, IDE TRIM wants to elevate its BB's in-flight counter for a "macro" operation that consists of several actual I/O operations. Each of those operations is individually started and awaited. It does this so that blk_drain() will drain the whole TRIM, and not just a single one of the many discard operations it may encompass. When request queuing is enabled, this leads to a deadlock: The currently ongoing discard is drained, and the next one is queued, waiting for the drain to stop. Meanwhile, TRIM still keeps the in-flight counter elevated, waiting for all discards to stop -- which will never happen, because with the in-flight counter elevated, the BB is never considered drained, so the drained section does not begin and cannot end. Fixing the implementation of request queuing is hard to do in general, and even harder to do without adding more hacks. As the name suggests, deadlocks are worse than livelocks :) so let's avoid them: turn the request queuing on only after the BlockBackend has quiesced, and leave the second functionality of bdrv_drained_begin() to the BQL or to the individual BlockDevOps implementations. In fact, devices such as IDE that run in the vCPU thread do not suffer from this livelock because they only submit requests while they are allowed to hold the big QEMU lock (i.e., not during bdrv_drained_begin() or bdrv_drain_all_begin(). Other devices can avoid it through external file descriptor (so that aio_disable_external() will prevent submission of new requests) or with a .drained_begin callback in their BlockDevOps. Note that this change does not affect the safety of bdrv_drained_begin(), since the patch does not completely get away with request queuing. Reported-by: Fiona Ebner Fixes: 7e5cdb345f77 ("ide: Increment BB in-flight counter for TRIM BH") Signed-off-by: Paolo Bonzini --- block/block-backend.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index 10419f8be91e..acb4cb91a5ee 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -42,6 +42,12 @@ typedef struct BlockBackendAioNotifier { QLIST_ENTRY(BlockBackendAioNotifier) list; } BlockBackendAioNotifier; =20 +typedef enum { + BLK_QUEUE_READY, + BLK_QUEUE_DISABLED, + BLK_QUEUE_QUIESCENT, +} BlockBackendQueueState; + struct BlockBackend { char *name; int refcnt; @@ -79,13 +85,14 @@ struct BlockBackend { */ bool allow_aio_context_change; bool allow_write_beyond_eof; - bool disable_request_queuing; =20 /* Protected by BQL */ NotifierList remove_bs_notifiers, insert_bs_notifiers; QLIST_HEAD(, BlockBackendAioNotifier) aio_notifiers; =20 int quiesce_counter; /* atomic: written under BQL, read by other threa= ds */ + BlockBackendQueueState request_queuing; + QemuMutex queued_requests_lock; /* protects queued_requests */ CoQueue queued_requests; =20 @@ -368,6 +375,7 @@ BlockBackend *blk_new(AioContext *ctx, uint64_t perm, u= int64_t shared_perm) blk->shared_perm =3D shared_perm; blk_set_enable_write_cache(blk, true); =20 + blk->request_queuing =3D BLK_QUEUE_READY; blk->on_read_error =3D BLOCKDEV_ON_ERROR_REPORT; blk->on_write_error =3D BLOCKDEV_ON_ERROR_ENOSPC; =20 @@ -1240,7 +1248,7 @@ void blk_allow_aio_context_change(BlockBackend *blk) void blk_disable_request_queuing(BlockBackend *blk) { GLOBAL_STATE_CODE(); - blk->disable_request_queuing =3D true; + blk->request_queuing =3D BLK_QUEUE_DISABLED; } =20 static int coroutine_fn GRAPH_RDLOCK @@ -1279,16 +1287,18 @@ static void coroutine_fn blk_wait_while_drained(Blo= ckBackend *blk) { assert(blk->in_flight > 0); =20 - if (qatomic_read(&blk->quiesce_counter) && !blk->disable_request_queui= ng) { + if (qatomic_read(&blk->request_queuing) =3D=3D BLK_QUEUE_QUIESCENT) { /* * Take lock before decrementing in flight counter so main loop th= read * waits for us to enqueue ourselves before it can leave the drain= ed * section. */ qemu_mutex_lock(&blk->queued_requests_lock); - blk_dec_in_flight(blk); - qemu_co_queue_wait(&blk->queued_requests, &blk->queued_requests_lo= ck); - blk_inc_in_flight(blk); + if (qatomic_read(&blk->request_queuing) =3D=3D BLK_QUEUE_QUIESCENT= ) { + blk_dec_in_flight(blk); + qemu_co_queue_wait(&blk->queued_requests, &blk->queued_request= s_lock); + blk_inc_in_flight(blk); + } qemu_mutex_unlock(&blk->queued_requests_lock); } } @@ -2600,7 +2610,14 @@ static bool blk_root_drained_poll(BdrvChild *child) if (blk->dev_ops && blk->dev_ops->drained_poll) { busy =3D blk->dev_ops->drained_poll(blk->dev_opaque); } - return busy || !!blk->in_flight; + if (busy || blk->in_flight) { + return true; + } + + if (qatomic_read(&blk->request_queuing) =3D=3D BLK_QUEUE_READY) { + qatomic_set(&blk->request_queuing, BLK_QUEUE_QUIESCENT); + } + return false; } =20 static void blk_root_drained_end(BdrvChild *child) @@ -2616,9 +2633,12 @@ static void blk_root_drained_end(BdrvChild *child) blk->dev_ops->drained_end(blk->dev_opaque); } qemu_mutex_lock(&blk->queued_requests_lock); - while (qemu_co_enter_next(&blk->queued_requests, - &blk->queued_requests_lock)) { - /* Resume all queued requests */ + if (qatomic_read(&blk->request_queuing) !=3D BLK_QUEUE_DISABLED) { + qatomic_set(&blk->request_queuing, BLK_QUEUE_READY); + while (qemu_co_enter_next(&blk->queued_requests, + &blk->queued_requests_lock)) { + /* Resume all queued requests */ + } } qemu_mutex_unlock(&blk->queued_requests_lock); } --=20 2.39.2 From nobody Mon Feb 9 07:31:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680712301; cv=none; d=zohomail.com; s=zohoarc; b=UfOL3EO6esYcbWgx+uHB9we5w+RwbZR6vgdpRzeBAoa6RCqjviAynbGGDXJXn9tI3XxL3lnrqZu7CdbuZCwQEcUjkvjQZyggPRQqdGVtxaKCn0SzsIStc33+hgOlqUQQF/RFukSxvvSNuBXwHo3Ji5Nyy5uqpTtmJ/PYr63UV1E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680712301; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PD1i5VlxjmCGb03EV7N5z9TRh9g2GyvKM4Zs6uY6moc=; b=RorKmPwI5z5LsSJU9RYtP3Rnh/8pp6LU6UxAyq2BK6+AXhNVjZH8pLmnhXIY2m9aYwa1vmidSpRsfTIkEYxrN+3L86Eg6eylkW+CXMcmLK4u7c1/5ulsOjH6uYnmZ33k7FLpSexNBRjXGMYqvMRyMRdNcONPXo/eZT1N1d3CrbM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1680712301688487.7656645439197; Wed, 5 Apr 2023 09:31:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pk62B-0005kL-JQ; Wed, 05 Apr 2023 12:30:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk628-0005j5-R8 for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:30:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk627-0006Z2-4Z for qemu-devel@nongnu.org; Wed, 05 Apr 2023 12:30:52 -0400 Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-589-xe5rtiMMNiqGbW7_pMIYrQ-1; Wed, 05 Apr 2023 12:30:48 -0400 Received: by mail-ed1-f71.google.com with SMTP id u30-20020a50c05e000000b0050299de3f82so14570790edd.10 for ; Wed, 05 Apr 2023 09:30:48 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id x16-20020a170906711000b008e53874f8d8sm7512488ejj.180.2023.04.05.09.30.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 09:30:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680712249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PD1i5VlxjmCGb03EV7N5z9TRh9g2GyvKM4Zs6uY6moc=; b=US/2dlnvxlv8SI27uev1DEQRgnG1ccQlQsWtMIPaJREUdwnKXJ9Lmf5P+QpI+gW4hpFP7t 5jed1JxzItbSZPCAlVnAtBc8ffFJIMuKbiZxuYCqfmcltbvOcyJsqrpPfdk6PxlGM2Gw/h vG5ArMWddCzrrdWIKx4sNlNW/vA9lqU= X-MC-Unique: xe5rtiMMNiqGbW7_pMIYrQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680712246; x=1683304246; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PD1i5VlxjmCGb03EV7N5z9TRh9g2GyvKM4Zs6uY6moc=; b=DwY4kfQYT3u/pKqaWbP/olKmdF/IcyrzgaCxsoQqhT4mB7gc1YIIritetQHhr+xs9G vJ4Mdy++7pJOC2tqU0vQVrXMkzuWnZNy6xmNilqDXkNcqOh486n34u7oJHC9m4CSdmxQ PD43i6sAZi7Bh3zYBRyS+/uIqEGyz2dxTskggaJXNahz9BdZpkESZpN/yw+/2lqGrA25 GEWPHNE8rISF0dwQFSxCsSjN43wNThirKlIsvbJ51BvujjKl8s99sjGRL3CUjZQjHRe9 mKMHViroFU1IxnWfBfOfZPgbHA02Dj7Ky+waT/dY0SfbSUb5lQ4n969fb95sDnloey0Y i+aQ== X-Gm-Message-State: AAQBX9ewOH250jgIMkPDCFYJUAeYDGqxNAvZBvruhnOapyd5ydSVax0T yiY4ckjauXSagns6T+LfcCqZH/ayEWQvGJvON+HiANgiMJ5fdGI8nNkmtrdsTDr1U3T62Cx/CVa p13d2lfTtYvOMukylutDaTduv4s4xGNNax/cHmesXC0LKbhFEcgHf1NyD6i9vpInHNTzzrL3v4g g= X-Received: by 2002:a17:906:6981:b0:932:4cbf:5bbb with SMTP id i1-20020a170906698100b009324cbf5bbbmr3464166ejr.19.1680712246263; Wed, 05 Apr 2023 09:30:46 -0700 (PDT) X-Google-Smtp-Source: AKy350ZJHMHIUA0k9vNCjr5e4Fio2B/Tfvi6s/za+4bChc2SslvK6IeLF0QRJk5mBwI3l8szT04raw== X-Received: by 2002:a17:906:6981:b0:932:4cbf:5bbb with SMTP id i1-20020a170906698100b009324cbf5bbbmr3464150ejr.19.1680712245923; Wed, 05 Apr 2023 09:30:45 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Fiona Ebner Subject: [PATCH] block-backend: delay application of request queuing Date: Wed, 5 Apr 2023 18:30:44 +0200 Message-Id: <20230405163044.197796-1-pbonzini@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230405161752.194727-1-pbonzini@redhat.com> References: <20230405161752.194727-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680712303987100008 Content-Type: text/plain; charset="utf-8" Request queuing prevents new requests from being submitted to the BlockDriverState, not allowing them to start instead of just letting them complete before bdrv_drained_begin() returns. The reason for this was to ensure progress and avoid a livelock in blk_drain(), blk_drain_all_begin(), bdrv_drained_begin() or bdrv_drain_all_begin(), if there is an endless stream of requests to a BlockBackend. However, this is prone to deadlocks. In particular, IDE TRIM wants to elevate its BB's in-flight counter for a "macro" operation that consists of several actual I/O operations. Each of those operations is individually started and awaited. It does this so that blk_drain() will drain the whole TRIM, and not just a single one of the many discard operations it may encompass. When request queuing is enabled, this leads to a deadlock: The currently ongoing discard is drained, and the next one is queued, waiting for the drain to stop. Meanwhile, TRIM still keeps the in-flight counter elevated, waiting for all discards to stop -- which will never happen, because with the in-flight counter elevated, the BB is never considered drained, so the drained section does not begin and cannot end. Fixing the implementation of request queuing is hard to do in general, and even harder to do without adding more hacks. As the name suggests, deadlocks are worse than livelocks :) so let's avoid them: turn the request queuing on only after the BlockBackend has quiesced, and leave the second functionality of bdrv_drained_begin() to the BQL or to the individual BlockDevOps implementations. In fact, devices such as IDE that run in the vCPU thread do not suffer from this livelock because they only submit requests while they are allowed to hold the big QEMU lock (i.e., not during bdrv_drained_begin() or bdrv_drain_all_begin(). Other devices can avoid it through external file descriptor (so that aio_disable_external() will prevent submission of new requests) or with a .drained_begin callback in their BlockDevOps. Note that this change does not affect the safety of bdrv_drained_begin(), since the patch does not completely get away with request queuing. Reported-by: Fiona Ebner Fixes: 7e5cdb345f77 ("ide: Increment BB in-flight counter for TRIM BH") Signed-off-by: Paolo Bonzini --- block/block-backend.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index 10419f8be91e..acb4cb91a5ee 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -42,6 +42,12 @@ typedef struct BlockBackendAioNotifier { QLIST_ENTRY(BlockBackendAioNotifier) list; } BlockBackendAioNotifier; =20 +typedef enum { + BLK_QUEUE_READY, + BLK_QUEUE_DISABLED, + BLK_QUEUE_QUIESCENT, +} BlockBackendQueueState; + struct BlockBackend { char *name; int refcnt; @@ -79,13 +85,14 @@ struct BlockBackend { */ bool allow_aio_context_change; bool allow_write_beyond_eof; - bool disable_request_queuing; =20 /* Protected by BQL */ NotifierList remove_bs_notifiers, insert_bs_notifiers; QLIST_HEAD(, BlockBackendAioNotifier) aio_notifiers; =20 int quiesce_counter; /* atomic: written under BQL, read by other threa= ds */ + BlockBackendQueueState request_queuing; + QemuMutex queued_requests_lock; /* protects queued_requests */ CoQueue queued_requests; =20 @@ -368,6 +375,7 @@ BlockBackend *blk_new(AioContext *ctx, uint64_t perm, u= int64_t shared_perm) blk->shared_perm =3D shared_perm; blk_set_enable_write_cache(blk, true); =20 + blk->request_queuing =3D BLK_QUEUE_READY; blk->on_read_error =3D BLOCKDEV_ON_ERROR_REPORT; blk->on_write_error =3D BLOCKDEV_ON_ERROR_ENOSPC; =20 @@ -1240,7 +1248,7 @@ void blk_allow_aio_context_change(BlockBackend *blk) void blk_disable_request_queuing(BlockBackend *blk) { GLOBAL_STATE_CODE(); - blk->disable_request_queuing =3D true; + blk->request_queuing =3D BLK_QUEUE_DISABLED; } =20 static int coroutine_fn GRAPH_RDLOCK @@ -1279,16 +1287,18 @@ static void coroutine_fn blk_wait_while_drained(Blo= ckBackend *blk) { assert(blk->in_flight > 0); =20 - if (qatomic_read(&blk->quiesce_counter) && !blk->disable_request_queui= ng) { + if (qatomic_read(&blk->request_queuing) =3D=3D BLK_QUEUE_QUIESCENT) { /* * Take lock before decrementing in flight counter so main loop th= read * waits for us to enqueue ourselves before it can leave the drain= ed * section. */ qemu_mutex_lock(&blk->queued_requests_lock); - blk_dec_in_flight(blk); - qemu_co_queue_wait(&blk->queued_requests, &blk->queued_requests_lo= ck); - blk_inc_in_flight(blk); + if (qatomic_read(&blk->request_queuing) =3D=3D BLK_QUEUE_QUIESCENT= ) { + blk_dec_in_flight(blk); + qemu_co_queue_wait(&blk->queued_requests, &blk->queued_request= s_lock); + blk_inc_in_flight(blk); + } qemu_mutex_unlock(&blk->queued_requests_lock); } } @@ -2600,7 +2610,14 @@ static bool blk_root_drained_poll(BdrvChild *child) if (blk->dev_ops && blk->dev_ops->drained_poll) { busy =3D blk->dev_ops->drained_poll(blk->dev_opaque); } - return busy || !!blk->in_flight; + if (busy || blk->in_flight) { + return true; + } + + if (qatomic_read(&blk->request_queuing) =3D=3D BLK_QUEUE_READY) { + qatomic_set(&blk->request_queuing, BLK_QUEUE_QUIESCENT); + } + return false; } =20 static void blk_root_drained_end(BdrvChild *child) @@ -2616,9 +2633,12 @@ static void blk_root_drained_end(BdrvChild *child) blk->dev_ops->drained_end(blk->dev_opaque); } qemu_mutex_lock(&blk->queued_requests_lock); - while (qemu_co_enter_next(&blk->queued_requests, - &blk->queued_requests_lock)) { - /* Resume all queued requests */ + if (qatomic_read(&blk->request_queuing) !=3D BLK_QUEUE_DISABLED) { + qatomic_set(&blk->request_queuing, BLK_QUEUE_READY); + while (qemu_co_enter_next(&blk->queued_requests, + &blk->queued_requests_lock)) { + /* Resume all queued requests */ + } } qemu_mutex_unlock(&blk->queued_requests_lock); } --=20 2.39.2