From nobody Tue Nov 26 22:23:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1703173037; cv=none; d=zohomail.com; s=zohoarc; b=LZuHHNuFx1J2lAiWpwdQTH2GmintwikljL2GLObh0uxbPkuejmH7KFLOVDltrQ4TSft3eTvy2SV6DsTrPmAibA/yB4t77HsSpIjBCH6ObNPscraV41R5q7+Ea8Q+CLSrgWEP2epAmjlW/Ux0X5ylH/qXUOTa7TOW5WDIIjr0h5c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703173037; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Z6GWwX5cBUe5INCQZNIbh12/Fu0Hcv2jrD8zOggAzlc=; b=NSgjpBqYXAtPJmGdz69i+0uYZ+JPS8x+pDqZdvZ8zKOGHJk+rBaKbnVP5ZhrhLmlu283o6o8zVS/c0exX8dT1BbiJZzQKJjPhPq5s683Ni1vZfcIgkZzGIBsh0umNJlITkM74toMWHwKONtcKtRPTSrP4kMuebV1rXxGaHsbyPU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1703173037865651.8316251736615; Thu, 21 Dec 2023 07:37:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGL5c-0005s0-HI; Thu, 21 Dec 2023 10:36:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5a-0005rH-Pn for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:35:58 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5Z-0005Yk-5w for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:35:58 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-140-lTdCP7uVMTiZu-G2UsUkSA-1; Thu, 21 Dec 2023 10:35:53 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1DED7185A789; Thu, 21 Dec 2023 15:35:53 +0000 (UTC) Received: from localhost (unknown [10.39.194.145]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E61D40C6E2B; Thu, 21 Dec 2023 15:35:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703172955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z6GWwX5cBUe5INCQZNIbh12/Fu0Hcv2jrD8zOggAzlc=; b=BQu5JxxjRu2D36WGQ3QOvIwZyxXCmfHV+RAihzUZTHLw/KGAp2Bf9QZ1q3a0SG4dCdDamq cosPA42HGWHJ+YcMB5StVvGI8UTD8yLVy5IGWVPdDHF+TJtWb0qV2T6QVFnwnZjlgrpynw KFWoYgRowkZA/sHK7guqynfljkBtPps= X-MC-Unique: lTdCP7uVMTiZu-G2UsUkSA-1 From: Stefan Hajnoczi To: Kevin Wolf , qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , Vladimir Sementsov-Ogievskiy , Hanna Reitz , Paolo Bonzini , Leonardo Bras , Stefan Hajnoczi , Peter Xu , Fam Zheng , Eric Blake , qemu-block@nongnu.org Subject: [PATCH v2 1/6] fixup block-coroutine-wrapper: use qemu_get_current_aio_context() Date: Thu, 21 Dec 2023 10:35:43 -0500 Message-ID: <20231221153548.1752005-2-stefanha@redhat.com> In-Reply-To: <20231221153548.1752005-1-stefanha@redhat.com> References: <20231221153548.1752005-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.061, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1703173039036100003 Content-Type: text/plain; charset="utf-8" qemu-iotests 051 fails on my machine so the change to 051.pc.out made by the above commit appears to be incorrect, at least against the current QEMU source tree. Revert it so that 051 passes again. Signed-off-by: Stefan Hajnoczi --- tests/qemu-iotests/051.pc.out | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out index 86a3e113c4..7e10c5fa1b 100644 --- a/tests/qemu-iotests/051.pc.out +++ b/tests/qemu-iotests/051.pc.out @@ -177,11 +177,11 @@ QEMU X.Y.Z monitor - type 'help' for more information =20 Testing: -drive file=3DTEST_DIR/t.qcow2,if=3Dnone,node-name=3Ddisk -object= iothread,id=3Dthread0 -device virtio-scsi,iothread=3Dthread0,id=3Dvirtio-s= csi0 -device scsi-hd,bus=3Dvirtio-scsi0.0,drive=3Ddisk,share-rw=3Don -devic= e lsi53c895a,id=3Dlsi0 -device scsi-hd,bus=3Dlsi0.0,drive=3Ddisk,share-rw= =3Don QEMU X.Y.Z monitor - type 'help' for more information -QEMU_PROG: -device scsi-hd,bus=3Dlsi0.0,drive=3Ddisk,share-rw=3Don: HBA do= es not support iothreads +(qemu) QEMU_PROG: -device scsi-hd,bus=3Dlsi0.0,drive=3Ddisk,share-rw=3Don:= HBA does not support iothreads =20 Testing: -drive file=3DTEST_DIR/t.qcow2,if=3Dnone,node-name=3Ddisk -object= iothread,id=3Dthread0 -device virtio-scsi,iothread=3Dthread0,id=3Dvirtio-s= csi0 -device scsi-hd,bus=3Dvirtio-scsi0.0,drive=3Ddisk,share-rw=3Don -devic= e virtio-scsi,id=3Dvirtio-scsi1 -device scsi-hd,bus=3Dvirtio-scsi1.0,drive= =3Ddisk,share-rw=3Don QEMU X.Y.Z monitor - type 'help' for more information -QEMU_PROG: -device scsi-hd,bus=3Dvirtio-scsi1.0,drive=3Ddisk,share-rw=3Don= : Cannot change iothread of active block backend +(qemu) QEMU_PROG: -device scsi-hd,bus=3Dvirtio-scsi1.0,drive=3Ddisk,share-= rw=3Don: Cannot change iothread of active block backend =20 Testing: -drive file=3DTEST_DIR/t.qcow2,if=3Dnone,node-name=3Ddisk -object= iothread,id=3Dthread0 -device virtio-scsi,iothread=3Dthread0,id=3Dvirtio-s= csi0 -device scsi-hd,bus=3Dvirtio-scsi0.0,drive=3Ddisk,share-rw=3Don -devic= e virtio-blk-pci,drive=3Ddisk,iothread=3Dthread0,share-rw=3Don QEMU X.Y.Z monitor - type 'help' for more information --=20 2.43.0 From nobody Tue Nov 26 22:23:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1703173017; cv=none; d=zohomail.com; s=zohoarc; b=iLhdC/YRLmRp93OzLnOQtchlBcuTjJbuFCfCwh1KJWUwRdzDaNHM5/9yh3eYj3HKU/QFCV9VrjKNZXWIoAMT08abGRukeuEojn6Wpa7USo/PzEoPEqVETfJ93vtw7nU+QSfY4Rnvx0a6r5ArcL5pGJLDy5c8IHFygjOPZhYjJ4k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703173017; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mLpAPhOLexRIuQHAy6nH/DLSJaRi5up7/+uWx1QOq3Y=; b=UDnbG35BoQAdGwn/yiDfmfd4X0vOn9Hg4lzz38QqJjUhAv44IcLWJUSmGMyxKUvGyQK/Wy6IhpvaV+XcR9hHfzQSx/L8+vW3qBVlEhDyqoc7bq/jcF3gIIzmk8zYIo1cdFDy8wTyLRkPa0u+KiQsNWRUO7FCcWuCtcVMvPwvqI8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 170317301782441.85542460715283; Thu, 21 Dec 2023 07:36:57 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGL5f-0005t2-Aa; Thu, 21 Dec 2023 10:36:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5d-0005sM-TW for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5c-0005ZW-IY for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:01 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-49-tJhHthZeM_S8h5hqvTd28g-1; Thu, 21 Dec 2023 10:35:55 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A3B6185A784; Thu, 21 Dec 2023 15:35:55 +0000 (UTC) Received: from localhost (unknown [10.39.194.145]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E907C159B0; Thu, 21 Dec 2023 15:35:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703172960; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mLpAPhOLexRIuQHAy6nH/DLSJaRi5up7/+uWx1QOq3Y=; b=IfflUSAMekZnB5gUiXAZBZlYwVIvojQ6pyAzZ8klzlPEYfgsO560Pe/CwT7YMZmof1Swba 6jB5BNEs6UpuWc/PKGha0otQCxM5h2NRpytSoYZOvNMXN0c1pIsNk2qGkYd1m/kVaDkSD8 uBHREls96npnf/eTS8yvAzhjYZDjXvc= X-MC-Unique: tJhHthZeM_S8h5hqvTd28g-1 From: Stefan Hajnoczi To: Kevin Wolf , qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , Vladimir Sementsov-Ogievskiy , Hanna Reitz , Paolo Bonzini , Leonardo Bras , Stefan Hajnoczi , Peter Xu , Fam Zheng , Eric Blake , qemu-block@nongnu.org Subject: [PATCH v2 2/6] fixup block: remove AioContext locking Date: Thu, 21 Dec 2023 10:35:44 -0500 Message-ID: <20231221153548.1752005-3-stefanha@redhat.com> In-Reply-To: <20231221153548.1752005-1-stefanha@redhat.com> References: <20231221153548.1752005-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.061, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1703173019444100002 Content-Type: text/plain; charset="utf-8" Signed-off-by: Stefan Hajnoczi --- migration/block.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/migration/block.c b/migration/block.c index 2bcfcbfdf6..6ec6a1d6e6 100644 --- a/migration/block.c +++ b/migration/block.c @@ -311,10 +311,17 @@ static int mig_save_device_bulk(QEMUFile *f, BlkMigDe= vState *bmds) block_mig_state.submitted++; blk_mig_unlock(); =20 + /* + * The migration thread does not have an AioContext. Lock the BQL so t= hat + * I/O runs in the main loop AioContext (see + * qemu_get_current_aio_context()). + */ + qemu_mutex_lock_iothread(); bdrv_reset_dirty_bitmap(bmds->dirty_bitmap, cur_sector * BDRV_SECTOR_S= IZE, nr_sectors * BDRV_SECTOR_SIZE); blk->aiocb =3D blk_aio_preadv(bb, cur_sector * BDRV_SECTOR_SIZE, &blk-= >qiov, 0, blk_mig_read_cb, blk); + qemu_mutex_unlock_iothread(); =20 bmds->cur_sector =3D cur_sector + nr_sectors; return (bmds->cur_sector >=3D total_sectors); --=20 2.43.0 From nobody Tue Nov 26 22:23:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1703173018; cv=none; d=zohomail.com; s=zohoarc; b=KUS49MqzWQOvF+XHjSHZSohGZPzr44x/H5g9hevHAxKIjvKHr7Bo9vpLE/fBNxZV5V5U4gzNL4QEJZ0F/anZzPQV+mz72fj/SzBKaa3SmhG6TQWlDrIMExUEPLAsECkLf5lBAA0ohsA9ko+6OnMQmrDf315qtxOmIpiICETom1I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703173018; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TWS9d0MNzNV2xV2VNA1Huwo4h7Tta7bQDm7YpCyuZCc=; b=MmzbQwmfOkMDRxbfV0fjONuOoHNpxGMv2mIThw12hkvEiK8EIH75/SUv/4UzA3oqThHXL2hJnpMosRImusFPcsvnUGNyB0yUynFV5iz1UwSWkqTNEA7JlJQX2LKwnkoXBhYT4x3KN3O643l8uuOegvvIca8iPoiQdk1GQdbVSNk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1703173018175670.3837715201387; Thu, 21 Dec 2023 07:36:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGL5h-0005uK-UA; Thu, 21 Dec 2023 10:36:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5g-0005tT-3X for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5e-0005Zp-IE for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:03 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-675-bFO7PipmPeWbdPYegjkbdQ-1; Thu, 21 Dec 2023 10:35:58 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C899983BA8F; Thu, 21 Dec 2023 15:35:57 +0000 (UTC) Received: from localhost (unknown [10.39.194.145]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2E4F81C060AF; Thu, 21 Dec 2023 15:35:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703172961; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TWS9d0MNzNV2xV2VNA1Huwo4h7Tta7bQDm7YpCyuZCc=; b=NJrWyiLfs/BNBmGZ4uN3FK/s+Yim6TICEUTANOYU4nZfPRVVwzw98ixVyJixRztLGyZa7m i+C4+qgjKamHTTBT5BVllbAaekjCGT7fxbIo7IipzfN6gH7zZWdr/LZAQZ7q5w3rhqkETU p/9oV61V8B1ovXem9wcbnXM0G+GPQ1M= X-MC-Unique: bFO7PipmPeWbdPYegjkbdQ-1 From: Stefan Hajnoczi To: Kevin Wolf , qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , Vladimir Sementsov-Ogievskiy , Hanna Reitz , Paolo Bonzini , Leonardo Bras , Stefan Hajnoczi , Peter Xu , Fam Zheng , Eric Blake , qemu-block@nongnu.org Subject: [PATCH v2 3/6] fixup scsi: only access SCSIDevice->requests from one thread Date: Thu, 21 Dec 2023 10:35:45 -0500 Message-ID: <20231221153548.1752005-4-stefanha@redhat.com> In-Reply-To: <20231221153548.1752005-1-stefanha@redhat.com> References: <20231221153548.1752005-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.061, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1703173019441100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini --- hw/scsi/scsi-bus.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c index df68a44b6a..5b08cbf60a 100644 --- a/hw/scsi/scsi-bus.c +++ b/hw/scsi/scsi-bus.c @@ -127,7 +127,8 @@ static void scsi_device_for_each_req_async_bh(void *opa= que) */ ctx =3D blk_get_aio_context(s->conf.blk); if (ctx !=3D qemu_get_current_aio_context()) { - aio_bh_schedule_oneshot(ctx, scsi_device_for_each_req_async_bh, da= ta); + aio_bh_schedule_oneshot(ctx, scsi_device_for_each_req_async_bh, + g_steal_pointer(&data)); return; } =20 --=20 2.43.0 From nobody Tue Nov 26 22:23:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1703173063; cv=none; d=zohomail.com; s=zohoarc; b=QN9LAayquNIpTfQb6ed0CVCoGCaOJwYw9y+eCfzvqX+sBctkHN8dJ41AjsxtP0iDKcGsTSGmLhatbV0s8Ju7Xysx7K4LXM6pwjVoYqYwXCRWiqdnmHGglUMGeCwT/ScbvHb1uz0iYwdDBrhdyTqNhk9QbNtbsW2Kn1Ew4fXGOTk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703173063; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=yqk4OFQOKT4s9q1FhkTw1+IEsq2kegWN4COgjx1UDyc=; b=Nlo4RFME5KqhaWdCvv3VsSDBHm/l27wYtO1ONlYRpccIMWLC1ZxlFymRQQIC55E0T/ZzsHiTLeGcJetkztk8rproqn7rRICK4UaDm7KXndHPKeWKa9iG4wgUEHSpgM1SKtxNN8hD7JevfGKk2wKDVfQXS26QKJrkaI2IsLgd24s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1703173063168962.7666521823971; Thu, 21 Dec 2023 07:37:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGL5l-0005vp-Ka; Thu, 21 Dec 2023 10:36:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5i-0005ud-3x for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5g-0005aW-Jx for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:05 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-622-LuOAwn7fPASlaIHSV3yKbw-1; Thu, 21 Dec 2023 10:36:00 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EA3C783BA91; Thu, 21 Dec 2023 15:35:59 +0000 (UTC) Received: from localhost (unknown [10.39.194.145]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B3761121313; Thu, 21 Dec 2023 15:35:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703172964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yqk4OFQOKT4s9q1FhkTw1+IEsq2kegWN4COgjx1UDyc=; b=fcd2xT/PqvZC8EWrHzp/cwWg6i6GFK9aScy6Q1zY8i424xkeJ6Dmvq5cpsfUmfK5M647bN 0iYb7buLCHUkRSh6Arul2pqKI0rgufWO/Ct4iRQcRN/AO55B3CfwE5bsJ8h6ZLUY/Mbb6l afjiIa/pQ+NzrJeKBKUSzgUR5K9W6sA= X-MC-Unique: LuOAwn7fPASlaIHSV3yKbw-1 From: Stefan Hajnoczi To: Kevin Wolf , qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , Vladimir Sementsov-Ogievskiy , Hanna Reitz , Paolo Bonzini , Leonardo Bras , Stefan Hajnoczi , Peter Xu , Fam Zheng , Eric Blake , qemu-block@nongnu.org Subject: [PATCH v2 4/6] nbd/server: avoid per-NBDRequest nbd_client_get/put() Date: Thu, 21 Dec 2023 10:35:46 -0500 Message-ID: <20231221153548.1752005-5-stefanha@redhat.com> In-Reply-To: <20231221153548.1752005-1-stefanha@redhat.com> References: <20231221153548.1752005-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.061, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1703173065151100003 Content-Type: text/plain; charset="utf-8" nbd_trip() processes a single NBD request from start to finish and holds an NBDClient reference throughout. NBDRequest does not outlive the scope of nbd_trip(). Therefore it is unnecessary to ref/unref NBDClient for each NBDRequest. Removing these nbd_client_get()/nbd_client_put() calls will make thread-safety easier in the commits that follow. Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini --- nbd/server.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index 895cf0a752..0b09ccc8dc 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -1557,7 +1557,6 @@ static NBDRequestData *nbd_request_get(NBDClient *cli= ent) client->nb_requests++; =20 req =3D g_new0(NBDRequestData, 1); - nbd_client_get(client); req->client =3D client; return req; } @@ -1578,8 +1577,6 @@ static void nbd_request_put(NBDRequestData *req) } =20 nbd_client_receive_next_request(client); - - nbd_client_put(client); } =20 static void blk_aio_attached(AioContext *ctx, void *opaque) --=20 2.43.0 From nobody Tue Nov 26 22:23:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1703173090; cv=none; d=zohomail.com; s=zohoarc; b=ek5kANIa54ofqhIJjm/Pm3YdOMKqZ9QU8PaIiWv28rXoqHCHEdV+DHgqlxf+m2hHjxWmwf7QWWK8Roz6RwXQysFiAoHrLWC8AQhSZoQ2b8ka/zXZ6kSULGTNRXPbHffmcKwBoB0VJKgOv7gZmbWz5r5ShSfCIQB4zWlGjP4ti/M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703173090; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mK+NLIDFhmr0TUKTwrjSQoiP7pMtYM6wCDVaFG6Vy34=; b=WZF2jmEoO1RQXbX1jOozjgmM/Ryrhs2Kq/tUJo1GQvTNnDd8jjvlGwHqrtaslsKRQIdqjVLPcd7A+Ump5dO+cktRss0+vLyEX7sGnXXHyu2CCv/F6brfK9A9GZ0siMsA8QThKgQclslM0OOQv2y2tDGZ0DMji4JTynkayHMUtiE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 170317309076564.96253541936119; Thu, 21 Dec 2023 07:38:10 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGL5x-0005y4-Jw; Thu, 21 Dec 2023 10:36:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5w-0005xd-9U for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:20 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5i-0005az-7T for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:19 -0500 Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-494-l9vTLMsMN86Rkf14PpPTzg-1; Thu, 21 Dec 2023 10:36:02 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D5CEE3C29A6F; Thu, 21 Dec 2023 15:36:01 +0000 (UTC) Received: from localhost (unknown [10.39.194.145]) by smtp.corp.redhat.com (Postfix) with ESMTP id 573FA51D5; Thu, 21 Dec 2023 15:36:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703172965; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mK+NLIDFhmr0TUKTwrjSQoiP7pMtYM6wCDVaFG6Vy34=; b=Tbv+2zG8GyJU5gt/jRlF344So06LcFFFGqvvNRWcDrv3cUCOe6UNPXzeil1KIkGAYcqegx ts8YaJ95akOOhWeVYeHZkyE/Zle9zCre6piAfwLM6OaMBXpnji+5AyuS/aqfeQ7Otj83pq xZg1FpMWDTs43o4576Qos8JgYrRnQrk= X-MC-Unique: l9vTLMsMN86Rkf14PpPTzg-1 From: Stefan Hajnoczi To: Kevin Wolf , qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , Vladimir Sementsov-Ogievskiy , Hanna Reitz , Paolo Bonzini , Leonardo Bras , Stefan Hajnoczi , Peter Xu , Fam Zheng , Eric Blake , qemu-block@nongnu.org Subject: [PATCH v2 5/6] nbd/server: only traverse NBDExport->clients from main loop thread Date: Thu, 21 Dec 2023 10:35:47 -0500 Message-ID: <20231221153548.1752005-6-stefanha@redhat.com> In-Reply-To: <20231221153548.1752005-1-stefanha@redhat.com> References: <20231221153548.1752005-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.061, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1703173091287100003 Content-Type: text/plain; charset="utf-8" The NBD clients list is currently accessed from both the export AioContext and the main loop thread. When the AioContext lock is removed there will be nothing protecting the clients list. Adding a lock around the clients list is tricky because NBDClient structs are refcounted and may be freed from the export AioContext or the main loop thread. nbd_export_request_shutdown() -> client_close() -> nbd_client_put() is also tricky because the list lock would be held while indirectly dropping references to NDBClients. A simpler approach is to only allow nbd_client_put() and client_close() calls from the main loop thread. Then the NBD clients list is only accessed from the main loop thread and no fancy locking is needed. nbd_trip() just needs to reschedule itself in the main loop AioContext before calling nbd_client_put() and client_close(). This costs more CPU cycles per NBD request so add nbd_client_put_nonzero() to optimize the common case where more references to NBDClient remain. Note that nbd_client_get() can still be called from either thread, so make NBDClient->refcount atomic. Signed-off-by: Stefan Hajnoczi --- nbd/server.c | 61 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 51 insertions(+), 10 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index 0b09ccc8dc..e91e2e0903 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -122,7 +122,7 @@ struct NBDMetaContexts { }; =20 struct NBDClient { - int refcount; + int refcount; /* atomic */ void (*close_fn)(NBDClient *client, bool negotiated); =20 NBDExport *exp; @@ -1501,14 +1501,17 @@ static int coroutine_fn nbd_receive_request(NBDClie= nt *client, NBDRequest *reque =20 #define MAX_NBD_REQUESTS 16 =20 +/* Runs in export AioContext and main loop thread */ void nbd_client_get(NBDClient *client) { - client->refcount++; + qatomic_inc(&client->refcount); } =20 void nbd_client_put(NBDClient *client) { - if (--client->refcount =3D=3D 0) { + assert(qemu_in_main_thread()); + + if (qatomic_fetch_dec(&client->refcount) =3D=3D 1) { /* The last reference should be dropped by client->close, * which is called by client_close. */ @@ -1529,8 +1532,35 @@ void nbd_client_put(NBDClient *client) } } =20 +/* + * Tries to release the reference to @client, but only if other references + * remain. This is an optimization for the common case where we want to av= oid + * the expense of scheduling nbd_client_put() in the main loop thread. + * + * Returns true upon success or false if the reference was not released be= cause + * it is the last reference. + */ +static bool nbd_client_put_nonzero(NBDClient *client) +{ + int old =3D qatomic_read(&client->refcount); + int expected; + + do { + if (old =3D=3D 1) { + return false; + } + + expected =3D old; + old =3D qatomic_cmpxchg(&client->refcount, expected, expected - 1); + } while (old !=3D expected); + + return true; +} + static void client_close(NBDClient *client, bool negotiated) { + assert(qemu_in_main_thread()); + if (client->closing) { return; } @@ -2933,15 +2963,20 @@ static coroutine_fn int nbd_handle_request(NBDClien= t *client, static coroutine_fn void nbd_trip(void *opaque) { NBDClient *client =3D opaque; - NBDRequestData *req; + NBDRequestData *req =3D NULL; NBDRequest request =3D { 0 }; /* GCC thinks it can be used uninitia= lized */ int ret; Error *local_err =3D NULL; =20 + /* + * Note that nbd_client_put() and client_close() must be called from t= he + * main loop thread. Use aio_co_reschedule_self() to switch AioContext + * before calling these functions. + */ + trace_nbd_trip(); if (client->closing) { - nbd_client_put(client); - return; + goto done; } =20 if (client->quiescing) { @@ -2949,10 +2984,9 @@ static coroutine_fn void nbd_trip(void *opaque) * We're switching between AIO contexts. Don't attempt to receive = a new * request and kick the main context which may be waiting for us. */ - nbd_client_put(client); client->recv_coroutine =3D NULL; aio_wait_kick(); - return; + goto done; } =20 req =3D nbd_request_get(client); @@ -3012,8 +3046,13 @@ static coroutine_fn void nbd_trip(void *opaque) =20 qio_channel_set_cork(client->ioc, false); done: - nbd_request_put(req); - nbd_client_put(client); + if (req) { + nbd_request_put(req); + } + if (!nbd_client_put_nonzero(client)) { + aio_co_reschedule_self(qemu_get_aio_context()); + nbd_client_put(client); + } return; =20 disconnect: @@ -3021,6 +3060,8 @@ disconnect: error_reportf_err(local_err, "Disconnect client, due to: "); } nbd_request_put(req); + + aio_co_reschedule_self(qemu_get_aio_context()); client_close(client, true); nbd_client_put(client); } --=20 2.43.0 From nobody Tue Nov 26 22:23:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1703173017; cv=none; d=zohomail.com; s=zohoarc; b=Nhzs/Dix5hDP67IgYgqNZ+wyKTRxUdjQQ0Ia7rWKyQMFVVeelDJjsODKsZIoKiiE/ZPVIjdS6iOJP1b8n0jWCYyEfJZ6+QAWnU0GK7wAslB5ecfIwbVwbc31Rtdxz6SpbZ9S952w9ko8DRS5HRes6ABy9k+x+gg5VNWxdZAk0eI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703173017; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=1iZrZv2b3ZJWeeh57u0iL4YdxghYpsfHyb6rxL9N0Mg=; b=a+hrwVI+/3KMhi1h0cilsvqah8MVpTsG7BM8Y5HKU8/36ntZky1MpgcTvBi+kvIdIhyUBbjOOiPjeBj8iWZb5DnA+FYlhpA92jcC6L3jPfko7boMl9YMOiabovBVUXqTwek3bvEo3lI6D0WTgApkjcy+3hoQA2aa+fxh3Z9jmuI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 170317301783110.614713898934838; Thu, 21 Dec 2023 07:36:57 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGL5p-0005wS-V6; Thu, 21 Dec 2023 10:36:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5n-0005w7-61 for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:11 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGL5l-0005be-5y for qemu-devel@nongnu.org; Thu, 21 Dec 2023 10:36:10 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-117-hTz9hYKIP3me7OaTkW5big-1; Thu, 21 Dec 2023 10:36:04 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D7703879842; Thu, 21 Dec 2023 15:36:03 +0000 (UTC) Received: from localhost (unknown [10.39.194.145]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2C0B71121313; Thu, 21 Dec 2023 15:36:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703172968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1iZrZv2b3ZJWeeh57u0iL4YdxghYpsfHyb6rxL9N0Mg=; b=TDDwQodHa8e+F30LQv5xjZqtpwCIlFv7IhTt2/Wwr0f2jLrnpqa2JOEIVj84Qt9tzGXCzi 6uGuVYNMer4sZezI0eObGXo6DhJITv2Ei0SVBPEEwcQbpbOCGYO5B5AKuwjLa/tg17n9pz DoD1RQdhbjYeK4EcjNNKt7d4mptJFe0= X-MC-Unique: hTz9hYKIP3me7OaTkW5big-1 From: Stefan Hajnoczi To: Kevin Wolf , qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , Vladimir Sementsov-Ogievskiy , Hanna Reitz , Paolo Bonzini , Leonardo Bras , Stefan Hajnoczi , Peter Xu , Fam Zheng , Eric Blake , qemu-block@nongnu.org Subject: [PATCH v2 6/6] nbd/server: introduce NBDClient->lock to protect fields Date: Thu, 21 Dec 2023 10:35:48 -0500 Message-ID: <20231221153548.1752005-7-stefanha@redhat.com> In-Reply-To: <20231221153548.1752005-1-stefanha@redhat.com> References: <20231221153548.1752005-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.061, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1703173019493100007 Content-Type: text/plain; charset="utf-8" NBDClient has a number of fields that are accessed by both the export AioContext and the main loop thread. When the AioContext lock is removed these fields will need another form of protection. Add NBDClient->lock and protect fields that are accessed by both threads. Also add assertions where possible and otherwise add doc comments stating assumptions about which thread and lock holding. Note this patch moves the client->recv_coroutine assertion from nbd_co_receive_request() to nbd_trip() where client->lock is held. Signed-off-by: Stefan Hajnoczi --- nbd/server.c | 141 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 108 insertions(+), 33 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index e91e2e0903..5b08aae535 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -125,23 +125,25 @@ struct NBDClient { int refcount; /* atomic */ void (*close_fn)(NBDClient *client, bool negotiated); =20 + QemuMutex lock; + NBDExport *exp; QCryptoTLSCreds *tlscreds; char *tlsauthz; QIOChannelSocket *sioc; /* The underlying data channel */ QIOChannel *ioc; /* The current I/O channel which may differ (eg TLS) = */ =20 - Coroutine *recv_coroutine; + Coroutine *recv_coroutine; /* protected by lock */ =20 CoMutex send_lock; Coroutine *send_coroutine; =20 - bool read_yielding; - bool quiescing; + bool read_yielding; /* protected by lock */ + bool quiescing; /* protected by lock */ =20 QTAILQ_ENTRY(NBDClient) next; - int nb_requests; - bool closing; + int nb_requests; /* protected by lock */ + bool closing; /* protected by lock */ =20 uint32_t check_align; /* If non-zero, check for aligned client request= s */ =20 @@ -1415,11 +1417,15 @@ nbd_read_eof(NBDClient *client, void *buffer, size_= t size, Error **errp) =20 len =3D qio_channel_readv(client->ioc, &iov, 1, errp); if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { - client->read_yielding =3D true; + WITH_QEMU_LOCK_GUARD(&client->lock) { + client->read_yielding =3D true; + } qio_channel_yield(client->ioc, G_IO_IN); - client->read_yielding =3D false; - if (client->quiescing) { - return -EAGAIN; + WITH_QEMU_LOCK_GUARD(&client->lock) { + client->read_yielding =3D false; + if (client->quiescing) { + return -EAGAIN; + } } continue; } else if (len < 0) { @@ -1528,6 +1534,7 @@ void nbd_client_put(NBDClient *client) blk_exp_unref(&client->exp->common); } g_free(client->contexts.bitmaps); + qemu_mutex_destroy(&client->lock); g_free(client); } } @@ -1561,11 +1568,13 @@ static void client_close(NBDClient *client, bool ne= gotiated) { assert(qemu_in_main_thread()); =20 - if (client->closing) { - return; - } + WITH_QEMU_LOCK_GUARD(&client->lock) { + if (client->closing) { + return; + } =20 - client->closing =3D true; + client->closing =3D true; + } =20 /* Force requests to finish. They will drop their own references, * then we'll close the socket and free the NBDClient. @@ -1579,6 +1588,7 @@ static void client_close(NBDClient *client, bool nego= tiated) } } =20 +/* Runs in export AioContext with client->lock held */ static NBDRequestData *nbd_request_get(NBDClient *client) { NBDRequestData *req; @@ -1591,6 +1601,7 @@ static NBDRequestData *nbd_request_get(NBDClient *cli= ent) return req; } =20 +/* Runs in export AioContext with client->lock held */ static void nbd_request_put(NBDRequestData *req) { NBDClient *client =3D req->client; @@ -1614,14 +1625,18 @@ static void blk_aio_attached(AioContext *ctx, void = *opaque) NBDExport *exp =3D opaque; NBDClient *client; =20 + assert(qemu_in_main_thread()); + trace_nbd_blk_aio_attached(exp->name, ctx); =20 exp->common.ctx =3D ctx; =20 QTAILQ_FOREACH(client, &exp->clients, next) { - assert(client->nb_requests =3D=3D 0); - assert(client->recv_coroutine =3D=3D NULL); - assert(client->send_coroutine =3D=3D NULL); + WITH_QEMU_LOCK_GUARD(&client->lock) { + assert(client->nb_requests =3D=3D 0); + assert(client->recv_coroutine =3D=3D NULL); + assert(client->send_coroutine =3D=3D NULL); + } } } =20 @@ -1629,6 +1644,8 @@ static void blk_aio_detach(void *opaque) { NBDExport *exp =3D opaque; =20 + assert(qemu_in_main_thread()); + trace_nbd_blk_aio_detach(exp->name, exp->common.ctx); =20 exp->common.ctx =3D NULL; @@ -1639,8 +1656,12 @@ static void nbd_drained_begin(void *opaque) NBDExport *exp =3D opaque; NBDClient *client; =20 + assert(qemu_in_main_thread()); + QTAILQ_FOREACH(client, &exp->clients, next) { - client->quiescing =3D true; + WITH_QEMU_LOCK_GUARD(&client->lock) { + client->quiescing =3D true; + } } } =20 @@ -1649,28 +1670,48 @@ static void nbd_drained_end(void *opaque) NBDExport *exp =3D opaque; NBDClient *client; =20 + assert(qemu_in_main_thread()); + QTAILQ_FOREACH(client, &exp->clients, next) { - client->quiescing =3D false; - nbd_client_receive_next_request(client); + WITH_QEMU_LOCK_GUARD(&client->lock) { + client->quiescing =3D false; + nbd_client_receive_next_request(client); + } } } =20 +/* Runs in export AioContext */ +static void nbd_wake_read_bh(void *opaque) +{ + NBDClient *client =3D opaque; + qio_channel_wake_read(client->ioc); +} + static bool nbd_drained_poll(void *opaque) { NBDExport *exp =3D opaque; NBDClient *client; =20 + assert(qemu_in_main_thread()); + QTAILQ_FOREACH(client, &exp->clients, next) { - if (client->nb_requests !=3D 0) { - /* - * If there's a coroutine waiting for a request on nbd_read_eo= f() - * enter it here so we don't depend on the client to wake it u= p. - */ - if (client->recv_coroutine !=3D NULL && client->read_yielding)= { - qio_channel_wake_read(client->ioc); + WITH_QEMU_LOCK_GUARD(&client->lock) { + if (client->nb_requests !=3D 0) { + /* + * If there's a coroutine waiting for a request on nbd_rea= d_eof() + * enter it here so we don't depend on the client to wake = it up. + * + * Schedule a BH in the export AioContext to avoid missing= the + * wake up due to the race between qio_channel_wake_read()= and + * qio_channel_yield(). + */ + if (client->recv_coroutine !=3D NULL && client->read_yield= ing) { + aio_bh_schedule_oneshot(nbd_export_aio_context(client-= >exp), + nbd_wake_read_bh, client); + } + + return true; } - - return true; } } =20 @@ -1681,6 +1722,8 @@ static void nbd_eject_notifier(Notifier *n, void *dat= a) { NBDExport *exp =3D container_of(n, NBDExport, eject_notifier); =20 + assert(qemu_in_main_thread()); + blk_exp_request_shutdown(&exp->common); } =20 @@ -2566,7 +2609,6 @@ static int coroutine_fn nbd_co_receive_request(NBDReq= uestData *req, int ret; =20 g_assert(qemu_in_coroutine()); - assert(client->recv_coroutine =3D=3D qemu_coroutine_self()); ret =3D nbd_receive_request(client, request, errp); if (ret < 0) { return ret; @@ -2975,6 +3017,9 @@ static coroutine_fn void nbd_trip(void *opaque) */ =20 trace_nbd_trip(); + + qemu_mutex_lock(&client->lock); + if (client->closing) { goto done; } @@ -2990,7 +3035,21 @@ static coroutine_fn void nbd_trip(void *opaque) } =20 req =3D nbd_request_get(client); - ret =3D nbd_co_receive_request(req, &request, &local_err); + + /* + * nbd_co_receive_request() returns -EAGAIN when nbd_drained_begin() h= as + * set client->quiescing but by the time we get back nbd_drained_end()= may + * have already cleared client->quiescing. In that case we try again + * because nothing else will spawn an nbd_trip() coroutine until we set + * client->recv_coroutine =3D NULL further down. + */ + do { + assert(client->recv_coroutine =3D=3D qemu_coroutine_self()); + qemu_mutex_unlock(&client->lock); + ret =3D nbd_co_receive_request(req, &request, &local_err); + qemu_mutex_lock(&client->lock); + } while (ret =3D=3D -EAGAIN && !client->quiescing); + client->recv_coroutine =3D NULL; =20 if (client->closing) { @@ -3002,15 +3061,16 @@ static coroutine_fn void nbd_trip(void *opaque) } =20 if (ret =3D=3D -EAGAIN) { - assert(client->quiescing); goto done; } =20 nbd_client_receive_next_request(client); + if (ret =3D=3D -EIO) { goto disconnect; } =20 + qemu_mutex_unlock(&client->lock); qio_channel_set_cork(client->ioc, true); =20 if (ret < 0) { @@ -3030,6 +3090,10 @@ static coroutine_fn void nbd_trip(void *opaque) g_free(request.contexts->bitmaps); g_free(request.contexts); } + + qio_channel_set_cork(client->ioc, false); + qemu_mutex_lock(&client->lock); + if (ret < 0) { error_prepend(&local_err, "Failed to send reply: "); goto disconnect; @@ -3044,11 +3108,13 @@ static coroutine_fn void nbd_trip(void *opaque) goto disconnect; } =20 - qio_channel_set_cork(client->ioc, false); done: if (req) { nbd_request_put(req); } + + qemu_mutex_unlock(&client->lock); + if (!nbd_client_put_nonzero(client)) { aio_co_reschedule_self(qemu_get_aio_context()); nbd_client_put(client); @@ -3059,13 +3125,19 @@ disconnect: if (local_err) { error_reportf_err(local_err, "Disconnect client, due to: "); } + nbd_request_put(req); + qemu_mutex_unlock(&client->lock); =20 aio_co_reschedule_self(qemu_get_aio_context()); client_close(client, true); nbd_client_put(client); } =20 +/* + * Runs in export AioContext and main loop thread. Caller must hold + * client->lock. + */ static void nbd_client_receive_next_request(NBDClient *client) { if (!client->recv_coroutine && client->nb_requests < MAX_NBD_REQUESTS = && @@ -3091,7 +3163,9 @@ static coroutine_fn void nbd_co_client_start(void *op= aque) return; } =20 - nbd_client_receive_next_request(client); + WITH_QEMU_LOCK_GUARD(&client->lock) { + nbd_client_receive_next_request(client); + } } =20 /* @@ -3108,6 +3182,7 @@ void nbd_client_new(QIOChannelSocket *sioc, Coroutine *co; =20 client =3D g_new0(NBDClient, 1); + qemu_mutex_init(&client->lock); client->refcount =3D 1; client->tlscreds =3D tlscreds; if (tlscreds) { --=20 2.43.0