From nobody Mon Apr 29 01:45:06 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680007049; cv=none; d=zohomail.com; s=zohoarc; b=gcRRqC+/VsXjTlCYsRjtduB+QzIJkX4rfQTzwDNeJZ+UTHtSVWZU1HKn1PnOggVyVWKZ33eZ93+Wzn21ufUc4jEjwbihZ0bBlcP7h4usfrxL4GcE9k6E9q6SxQ13CufOfXV1jy2L1NX2/pEwYK0Ui1x/h8pFmwESVtQOl7hySY0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680007049; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2rTzy4TesxCg60LU6meuGfbZE05Jvzsyuo/AaMdHhJ8=; b=Th59o595fnbcb3CRgIPCoSPuVT2uSBXHKa/lZuX/xRF3xFLXh2l05D/kCVg9pRpa42flS4ORQN2MFyuAeH9GGJ9M+4DgUIOFU3KFFeSpRaWt4A7+gOxpWAimfEWddeuE6QswPFWFqzI1EyLtRr6NnBBIaKiRXuP5jGymtfqUA+c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1680007049387381.8522229747085; Tue, 28 Mar 2023 05:37:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ph8YR-0004EZ-KK; Tue, 28 Mar 2023 08:35:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YO-0004BY-6l for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YM-0006CV-QU for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:55 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-27-R3vaBVX1NH6zthQCtUzxRQ-1; Tue, 28 Mar 2023 08:35:51 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A862E385556A; Tue, 28 Mar 2023 12:35:50 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.193.57]) by smtp.corp.redhat.com (Postfix) with ESMTP id EDA78C15BA0; Tue, 28 Mar 2023 12:35:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680006954; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2rTzy4TesxCg60LU6meuGfbZE05Jvzsyuo/AaMdHhJ8=; b=LRvl+oFzBzUzft/Dm+Hdl1hyvNnG3/YJHLaII7TkQaGDu1D+SWRmBXkMed7bSCHEtU+5df pRJQbCxKIzyWVdbT5iqVulLQWTvLtsEFrSgsrWNw3lib2W6Sal23+/o9Ntb7Z7BudZHCay 0sYi3eUwm4ZJoBYa98t8QM5fKMFPdd8= X-MC-Unique: R3vaBVX1NH6zthQCtUzxRQ-1 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Subject: [PULL 1/4] nbd/server: push pending frames after sending reply Date: Tue, 28 Mar 2023 14:35:39 +0200 Message-Id: <20230328123542.222022-2-kwolf@redhat.com> In-Reply-To: <20230328123542.222022-1-kwolf@redhat.com> References: <20230328123542.222022-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680007050105100003 Content-Type: text/plain; charset="utf-8" From: Florian Westphal qemu-nbd doesn't set TCP_NODELAY on the tcp socket. Kernel waits for more data and avoids transmission of small packets. Without TLS this is barely noticeable, but with TLS this really shows. Booting a VM via qemu-nbd on localhost (with tls) takes more than 2 minutes on my system. tcpdump shows frequent wait periods, where no packets get sent for a 40ms period. Add explicit (un)corking when processing (and responding to) requests. "TCP_CORK, &zero" after earlier "CORK, &one" will flush pending data. VM Boot time: main: no tls: 23s, with tls: 2m45s patched: no tls: 14s, with tls: 15s VM Boot time, qemu-nbd via network (same lan): main: no tls: 18s, with tls: 1m50s patched: no tls: 17s, with tls: 18s Future optimization: if we could detect if there is another pending request we could defer the uncork operation because more data would be appended. Signed-off-by: Florian Westphal Message-Id: <20230324104720.2498-1-fw@strlen.de> Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- nbd/server.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/nbd/server.c b/nbd/server.c index a4750e4188..848836d414 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -2667,6 +2667,8 @@ static coroutine_fn void nbd_trip(void *opaque) goto disconnect; } =20 + qio_channel_set_cork(client->ioc, true); + if (ret < 0) { /* It wasn't -EIO, so, according to nbd_co_receive_request() * semantics, we should return the error to the client. */ @@ -2692,6 +2694,7 @@ static coroutine_fn void nbd_trip(void *opaque) goto disconnect; } =20 + qio_channel_set_cork(client->ioc, false); done: nbd_request_put(req); nbd_client_put(client); --=20 2.39.2 From nobody Mon Apr 29 01:45:06 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680007030; cv=none; d=zohomail.com; s=zohoarc; b=J7WKgclrl18klQx2tqIqueTiOEfFQ+kf7W0pxLg1X5T0K17Rl8Cs0pvqsgq57ewfFSg9H2fE4wwqGOnZ18v//J53hapsKdxl8Yl4WdwTYKmMHt/bAepM4l9+1ugwJX6nsRUakPv6RBRGJ+rXZjsMOkzd7CMS4mYc63xlXPPguQY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680007030; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RA1ZnMKSxLra8z0VlFyzeY9VJiVlM6sAzM3t1q82S6M=; b=hpGHW1/uRY05AGTYpptZkCtRDc+G4WNcilTaIre3D7mY5yxN10/3qaNDq3blgb99xMMKufye9PDrf5cwi0GtJpHVFXRmrfFATX23FDt5nvBXWcAmW4zkXo1oN8gKMS4X5laFHHKEyqKRhqygR1zEhSIeMpyK9fREfurD4N7tA8o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1680007030680950.3564167588169; Tue, 28 Mar 2023 05:37:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ph8YQ-0004DM-Us; Tue, 28 Mar 2023 08:35:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YO-0004BX-6k for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YM-0006CN-Ay for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:55 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-481-7Ec6T_y4NeuTFxScQUTQrA-1; Tue, 28 Mar 2023 08:35:52 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 972FF1C27D85; Tue, 28 Mar 2023 12:35:51 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.193.57]) by smtp.corp.redhat.com (Postfix) with ESMTP id DF12FC15BA0; Tue, 28 Mar 2023 12:35:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680006953; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RA1ZnMKSxLra8z0VlFyzeY9VJiVlM6sAzM3t1q82S6M=; b=h5qR3LJZKhnI738AR4LHu0tlNXuW9ijRTJ5t6xqmr465ssWKD1umQFO0OuzUqa6timAl7U xMYNy8NatF2GP6ZulZEx+2fEsPmVzTwxVL5dJhepGPDWO71pLopaY+cl+ojmuvsqr5pHe/ RYYBNnMz8zQE1Mbj4dDoWk097PLtV1E= X-MC-Unique: 7Ec6T_y4NeuTFxScQUTQrA-1 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Subject: [PULL 2/4] block/export: only acquire AioContext once for vhost_user_server_stop() Date: Tue, 28 Mar 2023 14:35:40 +0200 Message-Id: <20230328123542.222022-3-kwolf@redhat.com> In-Reply-To: <20230328123542.222022-1-kwolf@redhat.com> References: <20230328123542.222022-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680007032036100003 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi vhost_user_server_stop() uses AIO_WAIT_WHILE(). AIO_WAIT_WHILE() requires that AioContext is only acquired once. Since blk_exp_request_shutdown() already acquires the AioContext it shouldn't be acquired again in vhost_user_server_stop(). Signed-off-by: Stefan Hajnoczi Message-Id: <20230323145853.1345527-1-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- util/vhost-user-server.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c index 40f36ea214..5b6216069c 100644 --- a/util/vhost-user-server.c +++ b/util/vhost-user-server.c @@ -346,10 +346,9 @@ static void vu_accept(QIONetListener *listener, QIOCha= nnelSocket *sioc, aio_context_release(server->ctx); } =20 +/* server->ctx acquired by caller */ void vhost_user_server_stop(VuServer *server) { - aio_context_acquire(server->ctx); - qemu_bh_delete(server->restart_listener_bh); server->restart_listener_bh =3D NULL; =20 @@ -366,8 +365,6 @@ void vhost_user_server_stop(VuServer *server) AIO_WAIT_WHILE(server->ctx, server->co_trip); } =20 - aio_context_release(server->ctx); - if (server->listener) { qio_net_listener_disconnect(server->listener); object_unref(OBJECT(server->listener)); --=20 2.39.2 From nobody Mon Apr 29 01:45:06 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680007021; cv=none; d=zohomail.com; s=zohoarc; b=PeClPP3JP9013g2qj3gnSWsDON+YOB8ai3aWp5p8HtyON9Hh2sw4w5EOKyzn0zYO1SK7dvQyRA3q9i1hEU5GgiLaIqB/wBwy/08GAAipNj2ItcQ/4Sdg3z1rcnt1grP9B/KSvYy5KId8DyQKqJro06ExI003reQx4A1OimFkLWQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680007021; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=i5lexZTwCcBrivawXz8aAWf8TkRaCTm0+xUS7YYfiFs=; b=PXm2VXteqlrzAn0Lk0x9vHZUuLu7s/Ziw1yWL5qWxhOICvws103pycNZ+TWxQlLPpsmh8/o1wJocxnmdoEIENh497w7IFaWIHN9e55B6v/e6cIqP3TsscS8Fv0IaK2kACaAqlzL84AXBMJXySERUU9Fvt4gn/PSr3vgpUX4sxS4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 168000702149581.51618668747642; Tue, 28 Mar 2023 05:37:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ph8YS-0004Em-8g; Tue, 28 Mar 2023 08:36:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YQ-0004DI-R7 for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YP-0006DE-2y for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:58 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-169-KoxwuCLwPcyM_aBD159YrQ-1; Tue, 28 Mar 2023 08:35:52 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 85566101A54F; Tue, 28 Mar 2023 12:35:52 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.193.57]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD3F5C15BA0; Tue, 28 Mar 2023 12:35:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680006956; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i5lexZTwCcBrivawXz8aAWf8TkRaCTm0+xUS7YYfiFs=; b=QTqmnz0QeLw7PfbfA4b3l6i6Se5CTDmlhpH9RM+FJ/KCD0AakYV8m/ZIljCLm6N3SLhZ5I l/+rRFoPPPWOTowW9YWiqNV23WojbS5jEhFijS/b5RRsC6wHm4SNwKU5co150E7OlBecwd 5tPBleOeJC36bzLKibraylN/HeTeXUI= X-MC-Unique: KoxwuCLwPcyM_aBD159YrQ-1 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Subject: [PULL 3/4] aio-posix: fix race between epoll upgrade and aio_set_fd_handler() Date: Tue, 28 Mar 2023 14:35:41 +0200 Message-Id: <20230328123542.222022-4-kwolf@redhat.com> In-Reply-To: <20230328123542.222022-1-kwolf@redhat.com> References: <20230328123542.222022-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680007022209100003 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi If another thread calls aio_set_fd_handler() while the IOThread event loop is upgrading from ppoll(2) to epoll(7) then we might miss new AioHandlers. The epollfd will not monitor the new AioHandler's fd, resulting in hangs. Take the AioHandler list lock while upgrading to epoll. This prevents AioHandlers from changing while epoll is being set up. If we cannot lock because we're in a nested event loop, then don't upgrade to epoll (it will happen next time we're not in a nested call). The downside to taking the lock is that the aio_set_fd_handler() thread has to wait until the epoll upgrade is finished, which involves many epoll_ctl(2) system calls. However, this scenario is rare and I couldn't think of another solution that is still simple. Reported-by: Qing Wang Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=3D2090998 Cc: Paolo Bonzini Cc: Fam Zheng Signed-off-by: Stefan Hajnoczi Message-Id: <20230323144859.1338495-1-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- util/fdmon-epoll.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/util/fdmon-epoll.c b/util/fdmon-epoll.c index e11a8a022e..1683aa1105 100644 --- a/util/fdmon-epoll.c +++ b/util/fdmon-epoll.c @@ -127,6 +127,8 @@ static bool fdmon_epoll_try_enable(AioContext *ctx) =20 bool fdmon_epoll_try_upgrade(AioContext *ctx, unsigned npfd) { + bool ok; + if (ctx->epollfd < 0) { return false; } @@ -136,14 +138,23 @@ bool fdmon_epoll_try_upgrade(AioContext *ctx, unsigne= d npfd) return false; } =20 - if (npfd >=3D EPOLL_ENABLE_THRESHOLD) { - if (fdmon_epoll_try_enable(ctx)) { - return true; - } else { - fdmon_epoll_disable(ctx); - } + if (npfd < EPOLL_ENABLE_THRESHOLD) { + return false; + } + + /* The list must not change while we add fds to epoll */ + if (!qemu_lockcnt_dec_if_lock(&ctx->list_lock)) { + return false; + } + + ok =3D fdmon_epoll_try_enable(ctx); + + qemu_lockcnt_inc_and_unlock(&ctx->list_lock); + + if (!ok) { + fdmon_epoll_disable(ctx); } - return false; + return ok; } =20 void fdmon_epoll_setup(AioContext *ctx) --=20 2.39.2 From nobody Mon Apr 29 01:45:06 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1680007041; cv=none; d=zohomail.com; s=zohoarc; b=LtZFDUngBa6tORYctlPF93BGn/xdXX7UaCqQw835d0AUM5Xocl2ZZDANxATz/OzQJQXIZetiP5U2NQaUHtvZYodsHxAq4fAsvLdKGND7xFsJSqui2Jzwo7qEZfUcTFj7/wf1sLNZu0Qys7B5OL7JTX5Mz04Na6doUKGs70sr3XE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1680007041; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ku5fOyVZexgOKjdYqYqFbffPqSSHEYxcOzG3H24s3ZE=; b=WB2gvKZm5xBs3EBpMYVhehUfpXRCmvELPrFz9rubMaf5mUkAyLgrYh9Arv1ITHWTQxjM8USRfNLmEV2YKxomFhGOzbzZFwBmS5MMuRiki7402FDbBugTny8AL2Qt6Lct6hyRFStetooMqigm+LolFxUi4PHMbOXxQuZrzTiEqe4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1680007041917821.7290429701907; Tue, 28 Mar 2023 05:37:21 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ph8YT-0004FM-Jj; Tue, 28 Mar 2023 08:36:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YR-0004Ee-Qo for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ph8YQ-0006DT-27 for qemu-devel@nongnu.org; Tue, 28 Mar 2023 08:35:59 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-228-i3ySLcWfPGO3KD9PMK6nMQ-1; Tue, 28 Mar 2023 08:35:53 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 727E2101A531; Tue, 28 Mar 2023 12:35:53 +0000 (UTC) Received: from merkur.fritz.box (unknown [10.39.193.57]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBAE4C15BA0; Tue, 28 Mar 2023 12:35:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680006957; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ku5fOyVZexgOKjdYqYqFbffPqSSHEYxcOzG3H24s3ZE=; b=V+Xg0obgkDtez0XncptVoQwTe9lL/WeKTUCmYeXCW+R+PhdcAOcsLXPSSp4LYh26J4lj5r vsYvyyLr2JCtKlkxQIAbAACjC1GMVn0Y+BmtNmd99Z4UjP4nmoGGVy5rEiXHxoIJp6CoIe CVwemeQ9L53zQVRDRvwnxCOkw3w7SMw= X-MC-Unique: i3ySLcWfPGO3KD9PMK6nMQ-1 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Subject: [PULL 4/4] block/export: Fix graph locking in blk_get_geometry() call Date: Tue, 28 Mar 2023 14:35:42 +0200 Message-Id: <20230328123542.222022-5-kwolf@redhat.com> In-Reply-To: <20230328123542.222022-1-kwolf@redhat.com> References: <20230328123542.222022-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1680007044074100003 blk_get_geometry() eventually calls bdrv_nb_sectors(), which is a co_wrapper_mixed_bdrv_rdlock. This means that when it is called from coroutine context, it already assume to have the graph locked. However, virtio_blk_sect_range_ok() in block/export/virtio-blk-handler.c (used by vhost-user-blk and VDUSE exports) runs in a coroutine, but doesn't take the graph lock - blk_*() functions are generally expected to do that internally. This causes an assertion failure when accessing an export for the first time if it runs in an iothread. This is an example of the crash: $ ./storage-daemon/qemu-storage-daemon --object iothread,id=3Dth0 --block= dev file,filename=3D/home/kwolf/images/hd.img,node-name=3Ddisk --export vho= st-user-blk,addr.type=3Dunix,addr.path=3D/tmp/vhost.sock,node-name=3Ddisk,i= d=3Dexp0,iothread=3Dth0 qemu-storage-daemon: ../block/graph-lock.c:268: void assert_bdrv_graph_re= adable(void): Assertion `qemu_in_main_thread() || reader_count()' failed. (gdb) bt #0 0x00007ffff6eafe5c in __pthread_kill_implementation () from /lib64/li= bc.so.6 #1 0x00007ffff6e5fa76 in raise () from /lib64/libc.so.6 #2 0x00007ffff6e497fc in abort () from /lib64/libc.so.6 #3 0x00007ffff6e4971b in __assert_fail_base.cold () from /lib64/libc.so.6 #4 0x00007ffff6e58656 in __assert_fail () from /lib64/libc.so.6 #5 0x00005555556337a3 in assert_bdrv_graph_readable () at ../block/graph= -lock.c:268 #6 0x00005555555fd5a2 in bdrv_co_nb_sectors (bs=3D0x5555564c5ef0) at ../= block.c:5847 #7 0x00005555555ee949 in bdrv_nb_sectors (bs=3D0x5555564c5ef0) at block/= block-gen.c:256 #8 0x00005555555fd6b9 in bdrv_get_geometry (bs=3D0x5555564c5ef0, nb_sect= ors_ptr=3D0x7fffef7fedd0) at ../block.c:5884 #9 0x000055555562ad6d in blk_get_geometry (blk=3D0x5555564cb200, nb_sect= ors_ptr=3D0x7fffef7fedd0) at ../block/block-backend.c:1624 #10 0x00005555555ddb74 in virtio_blk_sect_range_ok (blk=3D0x5555564cb200,= block_size=3D512, sector=3D0, size=3D512) at ../block/export/virtio-blk-ha= ndler.c:44 #11 0x00005555555dd80d in virtio_blk_process_req (handler=3D0x5555564cbb9= 8, in_iov=3D0x7fffe8003830, out_iov=3D0x7fffe8003860, in_num=3D1, out_num= =3D0) at ../block/export/virtio-blk-handler.c:189 #12 0x00005555555dd546 in vu_blk_virtio_process_req (opaque=3D0x7fffe8003= 800) at ../block/export/vhost-user-blk-server.c:66 #13 0x00005555557bf4a1 in coroutine_trampoline (i0=3D-402635264, i1=3D327= 67) at ../util/coroutine-ucontext.c:177 #14 0x00007ffff6e75c20 in ?? () from /lib64/libc.so.6 #15 0x00007fffefffa870 in ?? () #16 0x0000000000000000 in ?? () Fix this by creating a new blk_co_get_geometry() that takes the lock, and changing blk_get_geometry() to be a co_wrapper_mixed around it. To make the resulting code cleaner, virtio-blk-handler.c can directly call the coroutine version now (though that wouldn't be necessary for fixing the bug, taking the lock in blk_co_get_geometry() is what fixes it). Fixes: 8ab8140a04cf771d63e9754d6ba6c1e676bfe507 Reported-by: Luk=C3=A1=C5=A1 Doktor Signed-off-by: Kevin Wolf Message-Id: <20230327113959.60071-1-kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf --- include/block/block-io.h | 4 +++- include/sysemu/block-backend-io.h | 5 ++++- block.c | 5 +++-- block/block-backend.c | 7 +++++-- block/export/virtio-blk-handler.c | 7 ++++--- 5 files changed, 19 insertions(+), 9 deletions(-) diff --git a/include/block/block-io.h b/include/block/block-io.h index 5da99d4d60..dbc034b728 100644 --- a/include/block/block-io.h +++ b/include/block/block-io.h @@ -89,7 +89,9 @@ int64_t co_wrapper bdrv_get_allocated_file_size(BlockDriv= erState *bs); =20 BlockMeasureInfo *bdrv_measure(BlockDriver *drv, QemuOpts *opts, BlockDriverState *in_bs, Error **errp); -void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr); + +void coroutine_fn GRAPH_RDLOCK +bdrv_co_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr); =20 int coroutine_fn GRAPH_RDLOCK bdrv_co_delete_file(BlockDriverState *bs, Error **errp); diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backe= nd-io.h index 40ab178719..c672b77247 100644 --- a/include/sysemu/block-backend-io.h +++ b/include/sysemu/block-backend-io.h @@ -70,7 +70,10 @@ void co_wrapper blk_eject(BlockBackend *blk, bool eject_= flag); int64_t coroutine_fn blk_co_getlength(BlockBackend *blk); int64_t co_wrapper_mixed blk_getlength(BlockBackend *blk); =20 -void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr); +void coroutine_fn blk_co_get_geometry(BlockBackend *blk, + uint64_t *nb_sectors_ptr); +void co_wrapper_mixed blk_get_geometry(BlockBackend *blk, + uint64_t *nb_sectors_ptr); =20 int64_t coroutine_fn blk_co_nb_sectors(BlockBackend *blk); int64_t co_wrapper_mixed blk_nb_sectors(BlockBackend *blk); diff --git a/block.c b/block.c index 0dd604d0f6..e0c6c648b1 100644 --- a/block.c +++ b/block.c @@ -5879,9 +5879,10 @@ int64_t coroutine_fn bdrv_co_getlength(BlockDriverSt= ate *bs) } =20 /* return 0 as number of sectors if no device present or error */ -void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr) +void coroutine_fn bdrv_co_get_geometry(BlockDriverState *bs, + uint64_t *nb_sectors_ptr) { - int64_t nb_sectors =3D bdrv_nb_sectors(bs); + int64_t nb_sectors =3D bdrv_co_nb_sectors(bs); IO_CODE(); =20 *nb_sectors_ptr =3D nb_sectors < 0 ? 0 : nb_sectors; diff --git a/block/block-backend.c b/block/block-backend.c index 278b04ce69..2ee39229e4 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1615,13 +1615,16 @@ int64_t coroutine_fn blk_co_getlength(BlockBackend = *blk) return bdrv_co_getlength(blk_bs(blk)); } =20 -void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr) +void coroutine_fn blk_co_get_geometry(BlockBackend *blk, + uint64_t *nb_sectors_ptr) { IO_CODE(); + GRAPH_RDLOCK_GUARD(); + if (!blk_bs(blk)) { *nb_sectors_ptr =3D 0; } else { - bdrv_get_geometry(blk_bs(blk), nb_sectors_ptr); + bdrv_co_get_geometry(blk_bs(blk), nb_sectors_ptr); } } =20 diff --git a/block/export/virtio-blk-handler.c b/block/export/virtio-blk-ha= ndler.c index 313666e8ab..bc1cec6757 100644 --- a/block/export/virtio-blk-handler.c +++ b/block/export/virtio-blk-handler.c @@ -22,8 +22,9 @@ struct virtio_blk_inhdr { unsigned char status; }; =20 -static bool virtio_blk_sect_range_ok(BlockBackend *blk, uint32_t block_siz= e, - uint64_t sector, size_t size) +static bool coroutine_fn +virtio_blk_sect_range_ok(BlockBackend *blk, uint32_t block_size, + uint64_t sector, size_t size) { uint64_t nb_sectors; uint64_t total_sectors; @@ -41,7 +42,7 @@ static bool virtio_blk_sect_range_ok(BlockBackend *blk, u= int32_t block_size, if ((sector << VIRTIO_BLK_SECTOR_BITS) % block_size) { return false; } - blk_get_geometry(blk, &total_sectors); + blk_co_get_geometry(blk, &total_sectors); if (sector > total_sectors || nb_sectors > total_sectors - sector) { return false; } --=20 2.39.2