From nobody Sun May 5 17:45:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1494426862220255.0354432331983; Wed, 10 May 2017 07:34:22 -0700 (PDT) Received: from localhost ([::1]:43089 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sh1-0002PU-V4 for importer@patchew.org; Wed, 10 May 2017 10:34:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56232) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sf1-00015V-9H for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8Sf0-00028h-CI for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:15 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38730) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d8Sew-00026t-7l; Wed, 10 May 2017 10:32:10 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2FBB4C04B92A; Wed, 10 May 2017 14:32:09 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-116-59.ams2.redhat.com [10.36.116.59]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v4AEW5WO005842; Wed, 10 May 2017 10:32:07 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2FBB4C04B92A Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=pbonzini@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 2FBB4C04B92A From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 10 May 2017 16:31:59 +0200 Message-Id: <20170510143205.32013-2-pbonzini@redhat.com> In-Reply-To: <20170510143205.32013-1-pbonzini@redhat.com> References: <20170510143205.32013-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 10 May 2017 14:32:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 1/7] curl: strengthen assertion in curl_clean_state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jcody@redhat.com, qemu-stable@nongnu.org, qemu-block@nongnu.org, rjones@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" curl_clean_state should only be called after all AIOCBs have been completed. This is not so obvious for the call from curl_detach_aio_contex= t, so assert that. Cc: qemu-stable@nongnu.org Cc: jcody@redhat.com Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody Reviewed-by: Max Reitz --- block/curl.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/block/curl.c b/block/curl.c index 2708d57c2f..25a301e7b4 100644 --- a/block/curl.c +++ b/block/curl.c @@ -532,6 +532,11 @@ static CURLState *curl_init_state(BlockDriverState *bs= , BDRVCURLState *s) =20 static void curl_clean_state(CURLState *s) { + int j; + for (j=3D0; jacb[j]); + } + if (s->s->multi) curl_multi_remove_handle(s->s->multi, s->curl); =20 --=20 2.12.2 From nobody Sun May 5 17:45:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1494427016864530.1259035859331; Wed, 10 May 2017 07:36:56 -0700 (PDT) Received: from localhost ([::1]:43108 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8SjX-0004Xo-CK for importer@patchew.org; Wed, 10 May 2017 10:36:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56238) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sf1-00015a-Eh for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8Sf0-00028t-I3 for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:15 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38820) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d8Sey-00027Y-28; Wed, 10 May 2017 10:32:12 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0EF11C04B941; Wed, 10 May 2017 14:32:11 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-116-59.ams2.redhat.com [10.36.116.59]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v4AEW5WP005842; Wed, 10 May 2017 10:32:09 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0EF11C04B941 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=pbonzini@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 0EF11C04B941 From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 10 May 2017 16:32:00 +0200 Message-Id: <20170510143205.32013-3-pbonzini@redhat.com> In-Reply-To: <20170510143205.32013-1-pbonzini@redhat.com> References: <20170510143205.32013-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 10 May 2017 14:32:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 2/7] curl: never invoke callbacks with s->mutex held X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jcody@redhat.com, qemu-stable@nongnu.org, qemu-block@nongnu.org, rjones@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" All curl callbacks go through curl_multi_do, and hence are called with s->mutex held. Note that with comments, and make curl_read_cb drop the lock before invoking the callback. Likewise for curl_find_buf, where the callback can be invoked by the caller. Cc: qemu-stable@nongnu.org Cc: jcody@redhat.com Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody Reviewed-by: Max Reitz --- block/curl.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/block/curl.c b/block/curl.c index 25a301e7b4..9a00fdc28e 100644 --- a/block/curl.c +++ b/block/curl.c @@ -147,6 +147,7 @@ static void curl_multi_do(void *arg); static void curl_multi_read(void *arg); =20 #ifdef NEED_CURL_TIMER_CALLBACK +/* Called from curl_multi_do_locked, with s->mutex held. */ static int curl_timer_cb(CURLM *multi, long timeout_ms, void *opaque) { BDRVCURLState *s =3D opaque; @@ -163,6 +164,7 @@ static int curl_timer_cb(CURLM *multi, long timeout_ms,= void *opaque) } #endif =20 +/* Called from curl_multi_do_locked, with s->mutex held. */ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action, void *userp, void *sp) { @@ -212,6 +214,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, i= nt action, return 0; } =20 +/* Called from curl_multi_do_locked, with s->mutex held. */ static size_t curl_header_cb(void *ptr, size_t size, size_t nmemb, void *o= paque) { BDRVCURLState *s =3D opaque; @@ -226,6 +229,7 @@ static size_t curl_header_cb(void *ptr, size_t size, si= ze_t nmemb, void *opaque) return realsize; } =20 +/* Called from curl_multi_do_locked, with s->mutex held. */ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opa= que) { CURLState *s =3D ((CURLState*)opaque); @@ -264,7 +268,9 @@ static size_t curl_read_cb(void *ptr, size_t size, size= _t nmemb, void *opaque) request_length - offset); } =20 + qemu_mutex_unlock(&s->s->mutex); acb->common.cb(acb->common.opaque, 0); + qemu_mutex_lock(&s->s->mutex); qemu_aio_unref(acb); s->acb[i] =3D NULL; } @@ -305,8 +311,6 @@ static int curl_find_buf(BDRVCURLState *s, size_t start= , size_t len, if (clamped_len < len) { qemu_iovec_memset(acb->qiov, clamped_len, 0, len - clamped= _len); } - acb->common.cb(acb->common.opaque, 0); - return FIND_RET_OK; } =20 @@ -832,8 +836,8 @@ static void curl_readv_bh_cb(void *p) // we can just call the callback and be done. switch (curl_find_buf(s, start, acb->nb_sectors * BDRV_SECTOR_SIZE, ac= b)) { case FIND_RET_OK: - qemu_aio_unref(acb); - // fall through + ret =3D 0; + goto out; case FIND_RET_WAIT: goto out; default: --=20 2.12.2 From nobody Sun May 5 17:45:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1494427114889351.23996207039045; Wed, 10 May 2017 07:38:34 -0700 (PDT) Received: from localhost ([::1]:43114 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sl5-00061Z-8Z for importer@patchew.org; Wed, 10 May 2017 10:38:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56301) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sf4-00017V-Bj for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8Sf3-0002D5-AC for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54596) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d8Sez-000284-T6; Wed, 10 May 2017 10:32:14 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C51E080E7C; Wed, 10 May 2017 14:32:12 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-116-59.ams2.redhat.com [10.36.116.59]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v4AEW5WQ005842; Wed, 10 May 2017 10:32:11 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C51E080E7C Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=pbonzini@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C51E080E7C From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 10 May 2017 16:32:01 +0200 Message-Id: <20170510143205.32013-4-pbonzini@redhat.com> In-Reply-To: <20170510143205.32013-1-pbonzini@redhat.com> References: <20170510143205.32013-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 10 May 2017 14:32:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 3/7] curl: avoid recursive locking of BDRVCURLState mutex X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jcody@redhat.com, qemu-stable@nongnu.org, qemu-block@nongnu.org, rjones@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The curl driver has a ugly hack where, if it cannot find an empty CURLState, it just uses aio_poll to wait for one to be empty. This is probably buggy when used together with dataplane, and the simplest way to fix it is to use coroutines instead. A more immediate effect of the bug however is that it can cause a recursive call to curl_readv_bh_cb and recursively taking the BDRVCURLState mutex. This causes a deadlock. The fix is to unlock the mutex around aio_poll, but for cleanliness we should also take the mutex around all calls to curl_init_state, even if reaching the unlock/lock pair is impossible. The same is true for curl_clean_state. Reported-by: Richard W.M. Jones Cc: jcody@redhat.com Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini Reviewed-by: Max Reitz --- block/curl.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/block/curl.c b/block/curl.c index 9a00fdc28e..b18e79bf54 100644 --- a/block/curl.c +++ b/block/curl.c @@ -281,6 +281,7 @@ read_end: return size * nmemb; } =20 +/* Called with s->mutex held. */ static int curl_find_buf(BDRVCURLState *s, size_t start, size_t len, CURLAIOCB *acb) { @@ -453,6 +454,7 @@ static void curl_multi_timeout_do(void *arg) #endif } =20 +/* Called with s->mutex held. */ static CURLState *curl_init_state(BlockDriverState *bs, BDRVCURLState *s) { CURLState *state =3D NULL; @@ -471,7 +473,9 @@ static CURLState *curl_init_state(BlockDriverState *bs,= BDRVCURLState *s) break; } if (!state) { + qemu_mutex_unlock(&s->mutex); aio_poll(bdrv_get_aio_context(bs), true); + qemu_mutex_lock(&s->mutex); } } while(!state); =20 @@ -534,6 +538,7 @@ static CURLState *curl_init_state(BlockDriverState *bs,= BDRVCURLState *s) return state; } =20 +/* Called with s->mutex held. */ static void curl_clean_state(CURLState *s) { int j; @@ -565,6 +570,7 @@ static void curl_detach_aio_context(BlockDriverState *b= s) BDRVCURLState *s =3D bs->opaque; int i; =20 + qemu_mutex_lock(&s->mutex); for (i =3D 0; i < CURL_NUM_STATES; i++) { if (s->states[i].in_use) { curl_clean_state(&s->states[i]); @@ -580,6 +586,7 @@ static void curl_detach_aio_context(BlockDriverState *b= s) curl_multi_cleanup(s->multi); s->multi =3D NULL; } + qemu_mutex_unlock(&s->mutex); =20 timer_del(&s->timer); } @@ -745,9 +752,12 @@ static int curl_open(BlockDriverState *bs, QDict *opti= ons, int flags, } =20 DPRINTF("CURL: Opening %s\n", file); + qemu_mutex_init(&s->mutex); s->aio_context =3D bdrv_get_aio_context(bs); s->url =3D g_strdup(file); + qemu_mutex_lock(&s->mutex); state =3D curl_init_state(bs, s); + qemu_mutex_unlock(&s->mutex); if (!state) goto out_noclean; =20 @@ -791,11 +801,12 @@ static int curl_open(BlockDriverState *bs, QDict *opt= ions, int flags, } DPRINTF("CURL: Size =3D %zd\n", s->len); =20 + qemu_mutex_lock(&s->mutex); curl_clean_state(state); + qemu_mutex_unlock(&s->mutex); curl_easy_cleanup(state->curl); state->curl =3D NULL; =20 - qemu_mutex_init(&s->mutex); curl_attach_aio_context(bs, bdrv_get_aio_context(bs)); =20 qemu_opts_del(opts); --=20 2.12.2 From nobody Sun May 5 17:45:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1494426860143918.628044508617; Wed, 10 May 2017 07:34:20 -0700 (PDT) Received: from localhost ([::1]:43086 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sh0-0002O0-5Y for importer@patchew.org; Wed, 10 May 2017 10:34:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56322) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sf5-000183-9Z for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8Sf4-0002E6-C5 for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:23493) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d8Sf1-00029p-PZ; Wed, 10 May 2017 10:32:15 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B9E607A49B; Wed, 10 May 2017 14:32:14 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-116-59.ams2.redhat.com [10.36.116.59]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v4AEW5WR005842; Wed, 10 May 2017 10:32:12 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B9E607A49B Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=pbonzini@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B9E607A49B From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 10 May 2017 16:32:02 +0200 Message-Id: <20170510143205.32013-5-pbonzini@redhat.com> In-Reply-To: <20170510143205.32013-1-pbonzini@redhat.com> References: <20170510143205.32013-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 10 May 2017 14:32:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 4/7] curl: split curl_find_state/curl_init_state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jcody@redhat.com, qemu-stable@nongnu.org, qemu-block@nongnu.org, rjones@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If curl_easy_init fails, a CURLState is left with s->in_use =3D 1. Split curl_init_state in two, so that we can distinguish the two failures and call curl_clean_state if needed. While at it, simplify curl_find_state, removing a dummy loop. The aio_poll loop is moved to the sole caller that needs it. Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody Reviewed-by: Max Reitz --- block/curl.c | 52 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 30 insertions(+), 22 deletions(-) diff --git a/block/curl.c b/block/curl.c index b18e79bf54..4b4d5a2389 100644 --- a/block/curl.c +++ b/block/curl.c @@ -455,34 +455,27 @@ static void curl_multi_timeout_do(void *arg) } =20 /* Called with s->mutex held. */ -static CURLState *curl_init_state(BlockDriverState *bs, BDRVCURLState *s) +static CURLState *curl_find_state(BDRVCURLState *s) { CURLState *state =3D NULL; - int i, j; - - do { - for (i=3D0; istates[i].acb[j]) - continue; - if (s->states[i].in_use) - continue; + int i; =20 + for (i=3D0; istates[i].in_use) { state =3D &s->states[i]; state->in_use =3D 1; break; } - if (!state) { - qemu_mutex_unlock(&s->mutex); - aio_poll(bdrv_get_aio_context(bs), true); - qemu_mutex_lock(&s->mutex); - } - } while(!state); + } + return state; +} =20 +static int curl_init_state(BDRVCURLState *s, CURLState *state) +{ if (!state->curl) { state->curl =3D curl_easy_init(); if (!state->curl) { - return NULL; + return -EIO; } curl_easy_setopt(state->curl, CURLOPT_URL, s->url); curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER, @@ -535,7 +528,7 @@ static CURLState *curl_init_state(BlockDriverState *bs,= BDRVCURLState *s) QLIST_INIT(&state->sockets); state->s =3D s; =20 - return state; + return 0; } =20 /* Called with s->mutex held. */ @@ -756,13 +749,18 @@ static int curl_open(BlockDriverState *bs, QDict *opt= ions, int flags, s->aio_context =3D bdrv_get_aio_context(bs); s->url =3D g_strdup(file); qemu_mutex_lock(&s->mutex); - state =3D curl_init_state(bs, s); + state =3D curl_find_state(s); qemu_mutex_unlock(&s->mutex); - if (!state) + if (!state) { goto out_noclean; + } =20 // Get file size =20 + if (curl_init_state(s, state) < 0) { + goto out; + } + s->accept_range =3D false; curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1); curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION, @@ -856,8 +854,18 @@ static void curl_readv_bh_cb(void *p) } =20 // No cache found, so let's start a new request - state =3D curl_init_state(acb->common.bs, s); - if (!state) { + for (;;) { + state =3D curl_find_state(s); + if (state) { + break; + } + qemu_mutex_unlock(&s->mutex); + aio_poll(bdrv_get_aio_context(bs), true); + qemu_mutex_lock(&s->mutex); + } + + if (curl_init_state(s, state) < 0) { + curl_clean_state(state); ret =3D -EIO; goto out; } --=20 2.12.2 From nobody Sun May 5 17:45:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1494427255682344.3442811658522; Wed, 10 May 2017 07:40:55 -0700 (PDT) Received: from localhost ([::1]:43134 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8SnM-000801-Eb for importer@patchew.org; Wed, 10 May 2017 10:40:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56404) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8SfC-0001E3-Jx for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8SfB-0002JT-Dt for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59572) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d8Sf3-0002CY-NA; Wed, 10 May 2017 10:32:17 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ACE7E61D22; Wed, 10 May 2017 14:32:16 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-116-59.ams2.redhat.com [10.36.116.59]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v4AEW5WS005842; Wed, 10 May 2017 10:32:14 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com ACE7E61D22 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=pbonzini@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com ACE7E61D22 From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 10 May 2017 16:32:03 +0200 Message-Id: <20170510143205.32013-6-pbonzini@redhat.com> In-Reply-To: <20170510143205.32013-1-pbonzini@redhat.com> References: <20170510143205.32013-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 10 May 2017 14:32:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 5/7] curl: convert CURLAIOCB to byte values X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jcody@redhat.com, qemu-stable@nongnu.org, qemu-block@nongnu.org, rjones@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This is in preparation for the conversion from bdrv_aio_readv to bdrv_co_preadv, and it also requires changing some of the size_t values to uint64_t. This was broken before for disks > 2TB, but now it would break at 4GB. Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody Reviewed-by: Max Reitz --- block/curl.c | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/block/curl.c b/block/curl.c index 4b4d5a2389..3e288f2bc7 100644 --- a/block/curl.c +++ b/block/curl.c @@ -96,8 +96,8 @@ typedef struct CURLAIOCB { BlockAIOCB common; QEMUIOVector *qiov; =20 - int64_t sector_num; - int nb_sectors; + uint64_t offset; + uint64_t bytes; =20 size_t start; size_t end; @@ -115,7 +115,7 @@ typedef struct CURLState CURL *curl; QLIST_HEAD(, CURLSocket) sockets; char *orig_buf; - size_t buf_start; + uint64_t buf_start; size_t buf_off; size_t buf_len; char range[128]; @@ -126,7 +126,7 @@ typedef struct CURLState typedef struct BDRVCURLState { CURLM *multi; QEMUTimer timer; - size_t len; + uint64_t len; CURLState states[CURL_NUM_STATES]; char *url; size_t readahead_size; @@ -257,7 +257,7 @@ static size_t curl_read_cb(void *ptr, size_t size, size= _t nmemb, void *opaque) continue; =20 if ((s->buf_off >=3D acb->end)) { - size_t request_length =3D acb->nb_sectors * BDRV_SECTOR_SIZE; + size_t request_length =3D acb->bytes; =20 qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start, acb->end - acb->start); @@ -282,18 +282,18 @@ read_end: } =20 /* Called with s->mutex held. */ -static int curl_find_buf(BDRVCURLState *s, size_t start, size_t len, +static int curl_find_buf(BDRVCURLState *s, uint64_t start, uint64_t len, CURLAIOCB *acb) { int i; - size_t end =3D start + len; - size_t clamped_end =3D MIN(end, s->len); - size_t clamped_len =3D clamped_end - start; + uint64_t end =3D start + len; + uint64_t clamped_end =3D MIN(end, s->len); + uint64_t clamped_len =3D clamped_end - start; =20 for (i=3D0; istates[i]; - size_t buf_end =3D (state->buf_start + state->buf_off); - size_t buf_fend =3D (state->buf_start + state->buf_len); + uint64_t buf_end =3D (state->buf_start + state->buf_off); + uint64_t buf_fend =3D (state->buf_start + state->buf_len); =20 if (!state->orig_buf) continue; @@ -788,7 +788,7 @@ static int curl_open(BlockDriverState *bs, QDict *optio= ns, int flags, } #endif =20 - s->len =3D (size_t)d; + s->len =3D d; =20 if ((!strncasecmp(s->url, "http://", strlen("http://")) || !strncasecmp(s->url, "https://", strlen("https://"))) @@ -797,7 +797,7 @@ static int curl_open(BlockDriverState *bs, QDict *optio= ns, int flags, "Server does not support 'range' (byte ranges)."); goto out; } - DPRINTF("CURL: Size =3D %zd\n", s->len); + DPRINTF("CURL: Size =3D %" PRIu64 "\n", s->len); =20 qemu_mutex_lock(&s->mutex); curl_clean_state(state); @@ -836,14 +836,14 @@ static void curl_readv_bh_cb(void *p) BlockDriverState *bs =3D acb->common.bs; BDRVCURLState *s =3D bs->opaque; =20 - size_t start =3D acb->sector_num * BDRV_SECTOR_SIZE; - size_t end; + uint64_t start =3D acb->offset; + uint64_t end; =20 qemu_mutex_lock(&s->mutex); =20 // In case we have the requested data already (e.g. read-ahead), // we can just call the callback and be done. - switch (curl_find_buf(s, start, acb->nb_sectors * BDRV_SECTOR_SIZE, ac= b)) { + switch (curl_find_buf(s, start, acb->bytes, acb)) { case FIND_RET_OK: ret =3D 0; goto out; @@ -871,7 +871,7 @@ static void curl_readv_bh_cb(void *p) } =20 acb->start =3D 0; - acb->end =3D MIN(acb->nb_sectors * BDRV_SECTOR_SIZE, s->len - start); + acb->end =3D MIN(acb->bytes, s->len - start); =20 state->buf_off =3D 0; g_free(state->orig_buf); @@ -886,9 +886,9 @@ static void curl_readv_bh_cb(void *p) } state->acb[0] =3D acb; =20 - snprintf(state->range, 127, "%zd-%zd", start, end); - DPRINTF("CURL (AIO): Reading %llu at %zd (%s)\n", - (acb->nb_sectors * BDRV_SECTOR_SIZE), start, state->range); + snprintf(state->range, 127, "%" PRIu64 "-%" PRIu64, start, end); + DPRINTF("CURL (AIO): Reading %" PRIu64 " at %" PRIu64 " (%s)\n", + acb->bytes, start, state->range); curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range); =20 curl_multi_add_handle(s->multi, state->curl); @@ -913,8 +913,8 @@ static BlockAIOCB *curl_aio_readv(BlockDriverState *bs, acb =3D qemu_aio_get(&curl_aiocb_info, bs, cb, opaque); =20 acb->qiov =3D qiov; - acb->sector_num =3D sector_num; - acb->nb_sectors =3D nb_sectors; + acb->offset =3D sector_num * BDRV_SECTOR_SIZE; + acb->bytes =3D nb_sectors * BDRV_SECTOR_SIZE; =20 aio_bh_schedule_oneshot(bdrv_get_aio_context(bs), curl_readv_bh_cb, ac= b); return &acb->common; --=20 2.12.2 From nobody Sun May 5 17:45:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1494427017616519.1922348332978; Wed, 10 May 2017 07:36:57 -0700 (PDT) Received: from localhost ([::1]:43107 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8SjV-0004Wh-8q for importer@patchew.org; Wed, 10 May 2017 10:36:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56403) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8SfC-0001E2-Jp for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8SfB-0002JD-AN for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38990) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d8Sf5-0002EI-6r; Wed, 10 May 2017 10:32:19 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2CF73C04B955; Wed, 10 May 2017 14:32:18 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-116-59.ams2.redhat.com [10.36.116.59]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v4AEW5WT005842; Wed, 10 May 2017 10:32:16 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2CF73C04B955 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=pbonzini@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 2CF73C04B955 From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 10 May 2017 16:32:04 +0200 Message-Id: <20170510143205.32013-7-pbonzini@redhat.com> In-Reply-To: <20170510143205.32013-1-pbonzini@redhat.com> References: <20170510143205.32013-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 10 May 2017 14:32:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 6/7] curl: convert readv to coroutines X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jcody@redhat.com, qemu-stable@nongnu.org, qemu-block@nongnu.org, rjones@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This is pretty simple. The bottom half goes away because, unlike bdrv_aio_readv, coroutine-based read can return immediately without yielding. However, for simplicity I kept the former bottom half handler in a separate function. Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody --- block/curl.c | 94 ++++++++++++++++++++++++--------------------------------= ---- 1 file changed, 38 insertions(+), 56 deletions(-) diff --git a/block/curl.c b/block/curl.c index 3e288f2bc7..80870bd60c 100644 --- a/block/curl.c +++ b/block/curl.c @@ -76,10 +76,6 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi= _handle, #define CURL_TIMEOUT_DEFAULT 5 #define CURL_TIMEOUT_MAX 10000 =20 -#define FIND_RET_NONE 0 -#define FIND_RET_OK 1 -#define FIND_RET_WAIT 2 - #define CURL_BLOCK_OPT_URL "url" #define CURL_BLOCK_OPT_READAHEAD "readahead" #define CURL_BLOCK_OPT_SSLVERIFY "sslverify" @@ -93,11 +89,12 @@ static CURLMcode __curl_multi_socket_action(CURLM *mult= i_handle, struct BDRVCURLState; =20 typedef struct CURLAIOCB { - BlockAIOCB common; + Coroutine *co; QEMUIOVector *qiov; =20 uint64_t offset; uint64_t bytes; + int ret; =20 size_t start; size_t end; @@ -268,11 +265,11 @@ static size_t curl_read_cb(void *ptr, size_t size, si= ze_t nmemb, void *opaque) request_length - offset); } =20 + acb->ret =3D 0; + s->acb[i] =3D NULL; qemu_mutex_unlock(&s->s->mutex); - acb->common.cb(acb->common.opaque, 0); + aio_co_wake(acb->co); qemu_mutex_lock(&s->s->mutex); - qemu_aio_unref(acb); - s->acb[i] =3D NULL; } } =20 @@ -282,8 +279,8 @@ read_end: } =20 /* Called with s->mutex held. */ -static int curl_find_buf(BDRVCURLState *s, uint64_t start, uint64_t len, - CURLAIOCB *acb) +static bool curl_find_buf(BDRVCURLState *s, uint64_t start, uint64_t len, + CURLAIOCB *acb) { int i; uint64_t end =3D start + len; @@ -312,7 +309,8 @@ static int curl_find_buf(BDRVCURLState *s, uint64_t sta= rt, uint64_t len, if (clamped_len < len) { qemu_iovec_memset(acb->qiov, clamped_len, 0, len - clamped= _len); } - return FIND_RET_OK; + acb->ret =3D 0; + return true; } =20 // Wait for unfinished chunks @@ -330,13 +328,13 @@ static int curl_find_buf(BDRVCURLState *s, uint64_t s= tart, uint64_t len, for (j=3D0; jacb[j]) { state->acb[j] =3D acb; - return FIND_RET_WAIT; + return true; } } } } =20 - return FIND_RET_NONE; + return false; } =20 /* Called with s->mutex held. */ @@ -381,11 +379,11 @@ static void curl_multi_check_completion(BDRVCURLState= *s) continue; } =20 + acb->ret =3D -EIO; + state->acb[i] =3D NULL; qemu_mutex_unlock(&s->mutex); - acb->common.cb(acb->common.opaque, -EIO); + aio_co_wake(acb->co); qemu_mutex_lock(&s->mutex); - qemu_aio_unref(acb); - state->acb[i] =3D NULL; } } =20 @@ -821,19 +819,11 @@ out_noclean: return -EINVAL; } =20 -static const AIOCBInfo curl_aiocb_info =3D { - .aiocb_size =3D sizeof(CURLAIOCB), -}; - - -static void curl_readv_bh_cb(void *p) +static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb) { CURLState *state; int running; - int ret =3D -EINPROGRESS; =20 - CURLAIOCB *acb =3D p; - BlockDriverState *bs =3D acb->common.bs; BDRVCURLState *s =3D bs->opaque; =20 uint64_t start =3D acb->offset; @@ -843,14 +833,8 @@ static void curl_readv_bh_cb(void *p) =20 // In case we have the requested data already (e.g. read-ahead), // we can just call the callback and be done. - switch (curl_find_buf(s, start, acb->bytes, acb)) { - case FIND_RET_OK: - ret =3D 0; - goto out; - case FIND_RET_WAIT: - goto out; - default: - break; + if (curl_find_buf(s, start, acb->bytes, acb)) { + goto out; } =20 // No cache found, so let's start a new request @@ -866,7 +850,7 @@ static void curl_readv_bh_cb(void *p) =20 if (curl_init_state(s, state) < 0) { curl_clean_state(state); - ret =3D -EIO; + acb->ret =3D -EIO; goto out; } =20 @@ -881,7 +865,7 @@ static void curl_readv_bh_cb(void *p) state->orig_buf =3D g_try_malloc(state->buf_len); if (state->buf_len && state->orig_buf =3D=3D NULL) { curl_clean_state(state); - ret =3D -ENOMEM; + acb->ret =3D -ENOMEM; goto out; } state->acb[0] =3D acb; @@ -898,26 +882,24 @@ static void curl_readv_bh_cb(void *p) =20 out: qemu_mutex_unlock(&s->mutex); - if (ret !=3D -EINPROGRESS) { - acb->common.cb(acb->common.opaque, ret); - qemu_aio_unref(acb); - } } =20 -static BlockAIOCB *curl_aio_readv(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, int nb_sectors, - BlockCompletionFunc *cb, void *opaque) +static int coroutine_fn curl_co_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - CURLAIOCB *acb; - - acb =3D qemu_aio_get(&curl_aiocb_info, bs, cb, opaque); - - acb->qiov =3D qiov; - acb->offset =3D sector_num * BDRV_SECTOR_SIZE; - acb->bytes =3D nb_sectors * BDRV_SECTOR_SIZE; - - aio_bh_schedule_oneshot(bdrv_get_aio_context(bs), curl_readv_bh_cb, ac= b); - return &acb->common; + CURLAIOCB acb =3D { + .co =3D qemu_coroutine_self(), + .ret =3D -EINPROGRESS, + .qiov =3D qiov, + .offset =3D offset, + .bytes =3D bytes + }; + + curl_setup_preadv(bs, &acb); + while (acb.ret =3D=3D -EINPROGRESS) { + qemu_coroutine_yield(); + } + return acb.ret; } =20 static void curl_close(BlockDriverState *bs) @@ -948,7 +930,7 @@ static BlockDriver bdrv_http =3D { .bdrv_close =3D curl_close, .bdrv_getlength =3D curl_getlength, =20 - .bdrv_aio_readv =3D curl_aio_readv, + .bdrv_co_preadv =3D curl_co_preadv, =20 .bdrv_detach_aio_context =3D curl_detach_aio_context, .bdrv_attach_aio_context =3D curl_attach_aio_context, @@ -964,7 +946,7 @@ static BlockDriver bdrv_https =3D { .bdrv_close =3D curl_close, .bdrv_getlength =3D curl_getlength, =20 - .bdrv_aio_readv =3D curl_aio_readv, + .bdrv_co_preadv =3D curl_co_preadv, =20 .bdrv_detach_aio_context =3D curl_detach_aio_context, .bdrv_attach_aio_context =3D curl_attach_aio_context, @@ -980,7 +962,7 @@ static BlockDriver bdrv_ftp =3D { .bdrv_close =3D curl_close, .bdrv_getlength =3D curl_getlength, =20 - .bdrv_aio_readv =3D curl_aio_readv, + .bdrv_co_preadv =3D curl_co_preadv, =20 .bdrv_detach_aio_context =3D curl_detach_aio_context, .bdrv_attach_aio_context =3D curl_attach_aio_context, @@ -996,7 +978,7 @@ static BlockDriver bdrv_ftps =3D { .bdrv_close =3D curl_close, .bdrv_getlength =3D curl_getlength, =20 - .bdrv_aio_readv =3D curl_aio_readv, + .bdrv_co_preadv =3D curl_co_preadv, =20 .bdrv_detach_aio_context =3D curl_detach_aio_context, .bdrv_attach_aio_context =3D curl_attach_aio_context, --=20 2.12.2 From nobody Sun May 5 17:45:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1494426988449441.76029856034734; Wed, 10 May 2017 07:36:28 -0700 (PDT) Received: from localhost ([::1]:43106 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8Sj5-0004Ad-4D for importer@patchew.org; Wed, 10 May 2017 10:36:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56401) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d8SfC-0001DY-8l for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d8SfB-0002JK-Az for qemu-devel@nongnu.org; Wed, 10 May 2017 10:32:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54718) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d8Sf6-0002FL-SI; Wed, 10 May 2017 10:32:20 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C62FA81233; Wed, 10 May 2017 14:32:19 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-116-59.ams2.redhat.com [10.36.116.59]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v4AEW5WU005842; Wed, 10 May 2017 10:32:18 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C62FA81233 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=pbonzini@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C62FA81233 From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 10 May 2017 16:32:05 +0200 Message-Id: <20170510143205.32013-8-pbonzini@redhat.com> In-Reply-To: <20170510143205.32013-1-pbonzini@redhat.com> References: <20170510143205.32013-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 10 May 2017 14:32:19 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 7/7] curl: do not do aio_poll when waiting for a free CURLState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jcody@redhat.com, qemu-stable@nongnu.org, qemu-block@nongnu.org, rjones@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Instead, put the CURLAIOCB on a wait list; curl_clean_state will wake the corresponding coroutine. Because of CURL's callback-based structure, we cannot easily convert everything to CoMutex/CoQueue; keeping the QemuMutex is simpler. However, CoQueue is a simple wrapper around a linked list, so we can use QSIMPLEQ easily to open-code a CoQueue that has a QemuMutex's protection instead of a CoMutex's. Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody Reviewed-by: Max Reitz --- block/curl.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/block/curl.c b/block/curl.c index 80870bd60c..4ccdf63510 100644 --- a/block/curl.c +++ b/block/curl.c @@ -98,6 +98,8 @@ typedef struct CURLAIOCB { =20 size_t start; size_t end; + + QSIMPLEQ_ENTRY(CURLAIOCB) next; } CURLAIOCB; =20 typedef struct CURLSocket { @@ -133,6 +135,7 @@ typedef struct BDRVCURLState { bool accept_range; AioContext *aio_context; QemuMutex mutex; + QSIMPLEQ_HEAD(, CURLAIOCB) free_state_waitq; char *username; char *password; char *proxyusername; @@ -532,6 +535,7 @@ static int curl_init_state(BDRVCURLState *s, CURLState = *state) /* Called with s->mutex held. */ static void curl_clean_state(CURLState *s) { + CURLAIOCB *next; int j; for (j=3D0; jacb[j]); @@ -548,6 +552,14 @@ static void curl_clean_state(CURLState *s) } =20 s->in_use =3D 0; + + next =3D QSIMPLEQ_FIRST(&s->s->free_state_waitq); + if (next) { + QSIMPLEQ_REMOVE_HEAD(&s->s->free_state_waitq, next); + qemu_mutex_unlock(&s->s->mutex); + aio_co_wake(next->co); + qemu_mutex_lock(&s->s->mutex); + } } =20 static void curl_parse_filename(const char *filename, QDict *options, @@ -744,6 +756,7 @@ static int curl_open(BlockDriverState *bs, QDict *optio= ns, int flags, =20 DPRINTF("CURL: Opening %s\n", file); qemu_mutex_init(&s->mutex); + QSIMPLEQ_INIT(&s->free_state_waitq); s->aio_context =3D bdrv_get_aio_context(bs); s->url =3D g_strdup(file); qemu_mutex_lock(&s->mutex); @@ -843,8 +856,9 @@ static void curl_setup_preadv(BlockDriverState *bs, CUR= LAIOCB *acb) if (state) { break; } + QSIMPLEQ_INSERT_TAIL(&s->free_state_waitq, acb, next); qemu_mutex_unlock(&s->mutex); - aio_poll(bdrv_get_aio_context(bs), true); + qemu_coroutine_yield(); qemu_mutex_lock(&s->mutex); } =20 --=20 2.12.2