From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699714; cv=none; d=zohomail.com; s=zohoarc; b=SZp1wOZRbUnmeQ42SP9+7OV1bIiXQjZ4F2FG0qq00ryt2uNkhrwSPSqQTdloisP3ZO2aef/MN906igTlMXA9q7lHqjwff7sHB7ZGBmJcyG7RuPPwnQQJCdW4Rc4KRgqbGg1XbjI+QEo5uZ+LjyOPLMyM05+z7wbn2WLYem5OVe0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699714; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=KCMsquC2+bHD6SzCsnLY4O2720WG63JefOfU9U9aC90=; b=WMv1lAyd1hvxCK6dU1BW733E1PB/LhiJ10Sqn7tCBuRFo35Jn2al62u0vWVr3B2PimfeZQI2Zkob4GeFNF8Wfk0Fl7PqfHk9CNpkTkHvtF7UD5emPMZB6UMldchKLv2lOL3IaOPB8sa+cGRCJTB89WcP8cHOMeh9pw6fGmEbZXs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699714890223.5830673494189; Tue, 11 Mar 2025 06:28:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzcz-0001VU-Oi; Tue, 11 Mar 2025 09:26:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzcx-0001O6-6Z for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:26:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzcv-00059Q-FY for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:26:34 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-642-fLzxOZnjMDeZqaM-QneaDg-1; Tue, 11 Mar 2025 09:26:29 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EC8A61828AA2; Tue, 11 Mar 2025 13:26:27 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1B6521955BCB; Tue, 11 Mar 2025 13:26:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699592; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KCMsquC2+bHD6SzCsnLY4O2720WG63JefOfU9U9aC90=; b=TUWFB/lEtSv7Z30FkJ/ZrU7Jw9cVYTXll4h/hxZRlImcejgPF8k7dY3rmIozwdOZxBrmtg f7vpCHqe2rot6tiddmKVDnFWXedQxuj04zrGbnwF4lUU++pqrUMl6NbqnZTAPYtLWzvRTF xaoX4WIsca//WWdstWzV2tCv5oSFm08= X-MC-Unique: fLzxOZnjMDeZqaM-QneaDg-1 X-Mimecast-MFC-AGG-ID: fLzxOZnjMDeZqaM-QneaDg_1741699588 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 01/13] scsi-disk: drop unused SCSIDiskState->bh field Date: Tue, 11 Mar 2025 21:26:04 +0800 Message-ID: <20250311132616.1049687-2-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699716857019000 Commit 71544d30a6f8 ("scsi: push request restart to SCSIDevice") removed the only user of SCSIDiskState->bh. Signed-off-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- hw/scsi/scsi-disk.c | 1 - 1 file changed, 1 deletion(-) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index e7f738b484..caf6c1437f 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/scsi-disk.c @@ -106,7 +106,6 @@ struct SCSIDiskState { uint64_t max_unmap_size; uint64_t max_io_size; uint32_t quirks; - QEMUBH *bh; char *version; char *serial; char *vendor; --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699721; cv=none; d=zohomail.com; s=zohoarc; b=AY0VHXLWSEY0eQdAoMh/B+vHWZT0yKk2iFDTrwuNKBy5oBG7YgZkMXG4tvyFyqkjsz2uTBVI6Zzk/RokIDBIRvqHQomOsAbpMc3WgwqRAMEMkfL6sx2z4Wr+TE6fovnhTGw0dzv0DX8x5+Fc6wV/xEnHjU5YEUEPqAUOMUhBKmc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699721; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6u6rIEzeUsAlrV4pzWADYSFrJAIwCpUshbKF+5Fyp10=; b=ZOOLTrq9LnWuMdowiNpMAflGZLxMP6ifMPEaX7tBvDDmr0UCdH5Mz3QXGEIB2857NEqfeg2TlBF0MKQ+GQrwNsgNgQptJtkNik8/q0rN1o0G/H/i6c5l9ApbGnNS5CTvTHODhz7HfVHum6OBH854d9Vnu7aW1fNSF9Bo9WTLPxo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699721475142.77361601097073; Tue, 11 Mar 2025 06:28:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzdP-0002H7-Dg; Tue, 11 Mar 2025 09:27:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzd6-0001r0-Pr for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:26:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzd0-0005BU-O3 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:26:44 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-651-J80qW3xaNSevFf1CL-77EA-1; Tue, 11 Mar 2025 09:26:33 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 43C77180AF6E; Tue, 11 Mar 2025 13:26:32 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DDB641955F0F; Tue, 11 Mar 2025 13:26:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699597; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u6rIEzeUsAlrV4pzWADYSFrJAIwCpUshbKF+5Fyp10=; b=V5RoYN/JW9LGb7azElx2PuotDfvSZ0UsTJ7b5HecvqdyIV4HL9jBKRKbn2kwfACv7I23Jv ACDCuruhLQ5uoMaYICeiGCGb7K7lV17qwqVrBW3wNA/jLHP8JWkUgnF3xGN5Hmw0NjIy04 U2Dne/IkHovxnJsemrHOvJwdqVY4CUo= X-MC-Unique: J80qW3xaNSevFf1CL-77EA-1 X-Mimecast-MFC-AGG-ID: J80qW3xaNSevFf1CL-77EA_1741699592 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 02/13] dma: use current AioContext for dma_blk_io() Date: Tue, 11 Mar 2025 21:26:05 +0800 Message-ID: <20250311132616.1049687-3-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699723915019100 Content-Type: text/plain; charset="utf-8" In the past a single AioContext was used for block I/O and it was fetched using blk_get_aio_context(). Nowadays the block layer supports running I/O from any AioContext and multiple AioContexts at the same time. Remove the dma_blk_io() AioContext argument and use the current AioContext instead. This makes calling the function easier and enables multiple IOThreads to use dma_blk_io() concurrently for the same block device. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- include/system/dma.h | 3 +-- hw/ide/core.c | 3 +-- hw/ide/macio.c | 3 +-- hw/scsi/scsi-disk.c | 6 ++---- system/dma-helpers.c | 8 ++++---- 5 files changed, 9 insertions(+), 14 deletions(-) diff --git a/include/system/dma.h b/include/system/dma.h index 5a49a30628..e142f7efa6 100644 --- a/include/system/dma.h +++ b/include/system/dma.h @@ -290,8 +290,7 @@ typedef BlockAIOCB *DMAIOFunc(int64_t offset, QEMUIOVec= tor *iov, BlockCompletionFunc *cb, void *cb_opaque, void *opaque); =20 -BlockAIOCB *dma_blk_io(AioContext *ctx, - QEMUSGList *sg, uint64_t offset, uint32_t align, +BlockAIOCB *dma_blk_io(QEMUSGList *sg, uint64_t offset, uint32_t align, DMAIOFunc *io_func, void *io_func_opaque, BlockCompletionFunc *cb, void *opaque, DMADirection= dir); BlockAIOCB *dma_blk_read(BlockBackend *blk, diff --git a/hw/ide/core.c b/hw/ide/core.c index f9baba59e9..b14983ec54 100644 --- a/hw/ide/core.c +++ b/hw/ide/core.c @@ -968,8 +968,7 @@ static void ide_dma_cb(void *opaque, int ret) BDRV_SECTOR_SIZE, ide_dma_cb, s= ); break; case IDE_DMA_TRIM: - s->bus->dma->aiocb =3D dma_blk_io(blk_get_aio_context(s->blk), - &s->sg, offset, BDRV_SECTOR_SIZE, + s->bus->dma->aiocb =3D dma_blk_io(&s->sg, offset, BDRV_SECTOR_SIZE, ide_issue_trim, s, ide_dma_cb, s, DMA_DIRECTION_TO_DEVICE); break; diff --git a/hw/ide/macio.c b/hw/ide/macio.c index 5fe764b49b..c8e8e44cc9 100644 --- a/hw/ide/macio.c +++ b/hw/ide/macio.c @@ -187,8 +187,7 @@ static void pmac_ide_transfer_cb(void *opaque, int ret) pmac_ide_transfer_cb, io); break; case IDE_DMA_TRIM: - s->bus->dma->aiocb =3D dma_blk_io(blk_get_aio_context(s->blk), &s-= >sg, - offset, 0x1, ide_issue_trim, s, + s->bus->dma->aiocb =3D dma_blk_io(&s->sg, offset, 0x1, ide_issue_t= rim, s, pmac_ide_transfer_cb, io, DMA_DIRECTION_TO_DEVICE); break; diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index caf6c1437f..f049a20275 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/scsi-disk.c @@ -487,8 +487,7 @@ static void scsi_do_read(SCSIDiskReq *r, int ret) if (r->req.sg) { dma_acct_start(s->qdev.conf.blk, &r->acct, r->req.sg, BLOCK_ACCT_R= EAD); r->req.residual -=3D r->req.sg->size; - r->req.aiocb =3D dma_blk_io(blk_get_aio_context(s->qdev.conf.blk), - r->req.sg, r->sector << BDRV_SECTOR_BITS, + r->req.aiocb =3D dma_blk_io(r->req.sg, r->sector << BDRV_SECTOR_BI= TS, BDRV_SECTOR_SIZE, sdc->dma_readv, r, scsi_dma_complete, r, DMA_DIRECTION_FROM_DEVICE); @@ -650,8 +649,7 @@ static void scsi_write_data(SCSIRequest *req) if (r->req.sg) { dma_acct_start(s->qdev.conf.blk, &r->acct, r->req.sg, BLOCK_ACCT_W= RITE); r->req.residual -=3D r->req.sg->size; - r->req.aiocb =3D dma_blk_io(blk_get_aio_context(s->qdev.conf.blk), - r->req.sg, r->sector << BDRV_SECTOR_BITS, + r->req.aiocb =3D dma_blk_io(r->req.sg, r->sector << BDRV_SECTOR_BI= TS, BDRV_SECTOR_SIZE, sdc->dma_writev, r, scsi_dma_complete, r, DMA_DIRECTION_TO_DEVICE); diff --git a/system/dma-helpers.c b/system/dma-helpers.c index f6403242f5..6bad75876f 100644 --- a/system/dma-helpers.c +++ b/system/dma-helpers.c @@ -211,7 +211,7 @@ static const AIOCBInfo dma_aiocb_info =3D { .cancel_async =3D dma_aio_cancel, }; =20 -BlockAIOCB *dma_blk_io(AioContext *ctx, +BlockAIOCB *dma_blk_io( QEMUSGList *sg, uint64_t offset, uint32_t align, DMAIOFunc *io_func, void *io_func_opaque, BlockCompletionFunc *cb, @@ -223,7 +223,7 @@ BlockAIOCB *dma_blk_io(AioContext *ctx, =20 dbs->acb =3D NULL; dbs->sg =3D sg; - dbs->ctx =3D ctx; + dbs->ctx =3D qemu_get_current_aio_context(); dbs->offset =3D offset; dbs->align =3D align; dbs->sg_cur_index =3D 0; @@ -251,7 +251,7 @@ BlockAIOCB *dma_blk_read(BlockBackend *blk, QEMUSGList *sg, uint64_t offset, uint32_t align, void (*cb)(void *opaque, int ret), void *opaque) { - return dma_blk_io(blk_get_aio_context(blk), sg, offset, align, + return dma_blk_io(sg, offset, align, dma_blk_read_io_func, blk, cb, opaque, DMA_DIRECTION_FROM_DEVICE); } @@ -269,7 +269,7 @@ BlockAIOCB *dma_blk_write(BlockBackend *blk, QEMUSGList *sg, uint64_t offset, uint32_t align, void (*cb)(void *opaque, int ret), void *opaque) { - return dma_blk_io(blk_get_aio_context(blk), sg, offset, align, + return dma_blk_io(sg, offset, align, dma_blk_write_io_func, blk, cb, opaque, DMA_DIRECTION_TO_DEVICE); } --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699866; cv=none; d=zohomail.com; s=zohoarc; b=EsX7Ib3WpbUSniq3CQPoIanH+n6TO39lEZOMX4pGNIZYId9eg/JmQQ2ovCMeM2bU0yEoD4hnIxQWDSXbK5RevVDEbIDU68srEo/H6EV3Rydy50kSCn8fUOkyDfOv0F5wiXthcL4encpB0MIP0FV6vhwklPZtPbNKuAtTa4hfJsk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699866; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=owj2HZv9sPJ7eC1JcJnAuK8sPXLAi7OHMr4StlZBrM4=; b=BpWgqKaP7+BqUSnw0jBWv6SVYI1sJhu0L8jQ3c4MFMVTShrkZ2tausBRsQJ5j87+d177hN47YLbLYO0WsZ/9ol8WATjHnBkHMd8a4fxVOV726Cl9Qo/h7OWdUCgQIdB9aBgzZVVmo3fIQsaiqpmRC21bHGMqhkUEySrIbH3Njgk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699866665406.64716697295523; Tue, 11 Mar 2025 06:31:06 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzdH-00028Z-Nw; Tue, 11 Mar 2025 09:26:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzd6-0001nr-03 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:26:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzd4-0005Bv-08 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:26:43 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-571-2dzTinyWOne6lJvuByXT_g-1; Tue, 11 Mar 2025 09:26:37 -0400 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 413A8180AF55; Tue, 11 Mar 2025 13:26:36 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6650119560AB; Tue, 11 Mar 2025 13:26:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=owj2HZv9sPJ7eC1JcJnAuK8sPXLAi7OHMr4StlZBrM4=; b=dV9nzxT8vga0ZZV4XoMRP4l6DjisxiYP60zBXEJKAPkFEoNQDm+0Cua3mN6+0i5DuB55+W UEMdasIgtgDJUb1+1SozbRXr+1NZFOLFua769Akvs1XTL0O4b8DidyGn7ARCpkwd3zD9Nw 5ky/AMFEVbJCnX7LkP2EsG6wV0mL6NQ= X-MC-Unique: 2dzTinyWOne6lJvuByXT_g-1 X-Mimecast-MFC-AGG-ID: 2dzTinyWOne6lJvuByXT_g_1741699596 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 03/13] scsi: track per-SCSIRequest AioContext Date: Tue, 11 Mar 2025 21:26:06 +0800 Message-ID: <20250311132616.1049687-4-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699868786019100 Content-Type: text/plain; charset="utf-8" Until now, a SCSIDevice's I/O requests have run in a single AioContext. In order to support multiple IOThreads it will be necessary to move to the concept of a per-SCSIRequest AioContext. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- include/hw/scsi/scsi.h | 1 + hw/scsi/scsi-bus.c | 1 + hw/scsi/scsi-disk.c | 17 ++++++----------- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h index c3d5e17e38..ffc48203f9 100644 --- a/include/hw/scsi/scsi.h +++ b/include/hw/scsi/scsi.h @@ -24,6 +24,7 @@ struct SCSIRequest { SCSIBus *bus; SCSIDevice *dev; const SCSIReqOps *ops; + AioContext *ctx; uint32_t refcount; uint32_t tag; uint32_t lun; diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c index 7d4546800f..846bbbf0ec 100644 --- a/hw/scsi/scsi-bus.c +++ b/hw/scsi/scsi-bus.c @@ -868,6 +868,7 @@ invalid_opcode: } } =20 + req->ctx =3D qemu_get_current_aio_context(); req->cmd =3D cmd; req->residual =3D req->cmd.xfer; =20 diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index f049a20275..7cf8c31b98 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/scsi-disk.c @@ -328,9 +328,8 @@ static void scsi_aio_complete(void *opaque, int ret) SCSIDiskReq *r =3D (SCSIDiskReq *)opaque; SCSIDiskState *s =3D DO_UPCAST(SCSIDiskState, qdev, r->req.dev); =20 - /* The request must only run in the BlockBackend's AioContext */ - assert(blk_get_aio_context(s->qdev.conf.blk) =3D=3D - qemu_get_current_aio_context()); + /* The request must run in its AioContext */ + assert(r->req.ctx =3D=3D qemu_get_current_aio_context()); =20 assert(r->req.aiocb !=3D NULL); r->req.aiocb =3D NULL; @@ -430,12 +429,10 @@ static void scsi_dma_complete(void *opaque, int ret) =20 static void scsi_read_complete_noio(SCSIDiskReq *r, int ret) { - SCSIDiskState *s =3D DO_UPCAST(SCSIDiskState, qdev, r->req.dev); uint32_t n; =20 - /* The request must only run in the BlockBackend's AioContext */ - assert(blk_get_aio_context(s->qdev.conf.blk) =3D=3D - qemu_get_current_aio_context()); + /* The request must run in its AioContext */ + assert(r->req.ctx =3D=3D qemu_get_current_aio_context()); =20 assert(r->req.aiocb =3D=3D NULL); if (scsi_disk_req_check_error(r, ret, ret > 0)) { @@ -562,12 +559,10 @@ static void scsi_read_data(SCSIRequest *req) =20 static void scsi_write_complete_noio(SCSIDiskReq *r, int ret) { - SCSIDiskState *s =3D DO_UPCAST(SCSIDiskState, qdev, r->req.dev); uint32_t n; =20 - /* The request must only run in the BlockBackend's AioContext */ - assert(blk_get_aio_context(s->qdev.conf.blk) =3D=3D - qemu_get_current_aio_context()); + /* The request must run in its AioContext */ + assert(r->req.ctx =3D=3D qemu_get_current_aio_context()); =20 assert (r->req.aiocb =3D=3D NULL); if (scsi_disk_req_check_error(r, ret, ret > 0)) { --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699867; cv=none; d=zohomail.com; s=zohoarc; b=ULmS2Da7nEhHUDzf18FClM8BGu5gNXnDE4SazP6hio9xJQwYRXi81Sl4IuCCi8ItQTYjhBlmXYEkStbHbRjRqZYVb1VXdYZ7kro6JEU+GxcNWTufvyIN4OFiax7YPfp75PUzIYQhcnaZeBG1YQVvrZfnIlVWYd5qcvU3AvLBP5Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699867; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=rLmh01chR4pJz9pfvW5Ge56La3He1Nnf711wKnicLcw=; b=NNY8N9+OIPoHOiDKJ7+lnUaNS/+tyXMHGezO0e4bIjQxW0GvDAznyerwuYDtDthyV84gt0gePEJYTZwwHXn4h2rcXteNQA2vj5oP68hMxnroo5OnpQ1gWBuR1klUHQZs7tsOTBrF97IeJ3uv5eOZrFLQJ5FyX0VbNK3qHocSjOQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699867321313.3137302852574; Tue, 11 Mar 2025 06:31:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzeB-0002ui-Ay; Tue, 11 Mar 2025 09:27:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdK-0002Hb-61 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdA-0005DM-2K for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:26:57 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-552-MEn9xl2LMqKT58FLIMKEqQ-1; Tue, 11 Mar 2025 09:26:42 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D3F3B1800A36; Tue, 11 Mar 2025 13:26:40 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 19C831828A92; Tue, 11 Mar 2025 13:26:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699606; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rLmh01chR4pJz9pfvW5Ge56La3He1Nnf711wKnicLcw=; b=bOkl/pfhliZce9msEa3biopDA3ZUIzuAuVE6Apg5F+4r+fF9nQ7I6Cq/dBNbDBWxfUxJ5s HyEIE32IhM433Tdlh0NB2BdyGtxGEeoBUitu0A7pm8U9oBMVnzeYJRfu6rBODHOPS+R1H5 KW8L2zyut8s0LOQtGKJRsF++xPAMAd0= X-MC-Unique: MEn9xl2LMqKT58FLIMKEqQ-1 X-Mimecast-MFC-AGG-ID: MEn9xl2LMqKT58FLIMKEqQ_1741699600 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 04/13] scsi: introduce requests_lock Date: Tue, 11 Mar 2025 21:26:07 +0800 Message-ID: <20250311132616.1049687-5-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699868398019000 Content-Type: text/plain; charset="utf-8" SCSIDevice keeps track of in-flight requests for device reset and Task Management Functions (TMFs). The request list requires protection so that multi-threaded SCSI emulation can be implemented in commits that follow. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- include/hw/scsi/scsi.h | 7 ++- hw/scsi/scsi-bus.c | 120 +++++++++++++++++++++++++++++------------ 2 files changed, 88 insertions(+), 39 deletions(-) diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h index ffc48203f9..90ee192b4d 100644 --- a/include/hw/scsi/scsi.h +++ b/include/hw/scsi/scsi.h @@ -49,6 +49,8 @@ struct SCSIRequest { bool dma_started; BlockAIOCB *aiocb; QEMUSGList *sg; + + /* Protected by SCSIDevice->requests_lock */ QTAILQ_ENTRY(SCSIRequest) next; }; =20 @@ -77,10 +79,7 @@ struct SCSIDevice uint8_t sense[SCSI_SENSE_BUF_SIZE]; uint32_t sense_len; =20 - /* - * The requests list is only accessed from the AioContext that executes - * requests or from the main loop when IOThread processing is stopped. - */ + QemuMutex requests_lock; /* protects the requests list */ QTAILQ_HEAD(, SCSIRequest) requests; =20 uint32_t channel; diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c index 846bbbf0ec..ece1107ee8 100644 --- a/hw/scsi/scsi-bus.c +++ b/hw/scsi/scsi-bus.c @@ -100,8 +100,15 @@ static void scsi_device_for_each_req_sync(SCSIDevice *= s, assert(!runstate_is_running()); assert(qemu_in_main_thread()); =20 - QTAILQ_FOREACH_SAFE(req, &s->requests, next, next_req) { - fn(req, opaque); + /* + * Locking is not necessary because the guest is stopped and no other + * threads can be accessing the requests list, but take the lock for + * consistency. + */ + WITH_QEMU_LOCK_GUARD(&s->requests_lock) { + QTAILQ_FOREACH_SAFE(req, &s->requests, next, next_req) { + fn(req, opaque); + } } } =20 @@ -115,21 +122,29 @@ static void scsi_device_for_each_req_async_bh(void *o= paque) { g_autofree SCSIDeviceForEachReqAsyncData *data =3D opaque; SCSIDevice *s =3D data->s; - AioContext *ctx; - SCSIRequest *req; - SCSIRequest *next; + g_autoptr(GList) reqs =3D NULL; =20 /* - * The BB cannot have changed contexts between this BH being scheduled= and - * now: BBs' AioContexts, when they have a node attached, can only be - * changed via bdrv_try_change_aio_context(), in a drained section. W= hile - * we have the in-flight counter incremented, that drain must block. + * Build a list of requests in this AioContext so fn() can be invoked = later + * outside requests_lock. */ - ctx =3D blk_get_aio_context(s->conf.blk); - assert(ctx =3D=3D qemu_get_current_aio_context()); + WITH_QEMU_LOCK_GUARD(&s->requests_lock) { + AioContext *ctx =3D qemu_get_current_aio_context(); + SCSIRequest *req; + SCSIRequest *next; =20 - QTAILQ_FOREACH_SAFE(req, &s->requests, next, next) { - data->fn(req, data->fn_opaque); + QTAILQ_FOREACH_SAFE(req, &s->requests, next, next) { + if (req->ctx =3D=3D ctx) { + scsi_req_ref(req); /* dropped after calling fn() */ + reqs =3D g_list_prepend(reqs, req); + } + } + } + + /* Call fn() on each request */ + for (GList *elem =3D g_list_first(reqs); elem; elem =3D g_list_next(el= em)) { + data->fn(elem->data, data->fn_opaque); + scsi_req_unref(elem->data); } =20 /* Drop the reference taken by scsi_device_for_each_req_async() */ @@ -139,9 +154,35 @@ static void scsi_device_for_each_req_async_bh(void *op= aque) blk_dec_in_flight(s->conf.blk); } =20 +static void scsi_device_for_each_req_async_do_ctx(gpointer key, gpointer v= alue, + gpointer user_data) +{ + AioContext *ctx =3D key; + SCSIDeviceForEachReqAsyncData *params =3D user_data; + SCSIDeviceForEachReqAsyncData *data; + + data =3D g_new(SCSIDeviceForEachReqAsyncData, 1); + data->s =3D params->s; + data->fn =3D params->fn; + data->fn_opaque =3D params->fn_opaque; + + /* + * Hold a reference to the SCSIDevice until + * scsi_device_for_each_req_async_bh() finishes. + */ + object_ref(OBJECT(data->s)); + + /* Paired with scsi_device_for_each_req_async_bh() */ + blk_inc_in_flight(data->s->conf.blk); + + aio_bh_schedule_oneshot(ctx, scsi_device_for_each_req_async_bh, data); +} + /* * Schedule @fn() to be invoked for each enqueued request in device @s. @f= n() - * runs in the AioContext that is executing the request. + * must be thread-safe because it runs concurrently in each AioContext tha= t is + * executing a request. + * * Keeps the BlockBackend's in-flight counter incremented until everything= is * done, so draining it will settle all scheduled @fn() calls. */ @@ -151,24 +192,26 @@ static void scsi_device_for_each_req_async(SCSIDevice= *s, { assert(qemu_in_main_thread()); =20 - SCSIDeviceForEachReqAsyncData *data =3D - g_new(SCSIDeviceForEachReqAsyncData, 1); + /* The set of AioContexts where the requests are being processed */ + g_autoptr(GHashTable) aio_contexts =3D g_hash_table_new(NULL, NULL); + WITH_QEMU_LOCK_GUARD(&s->requests_lock) { + SCSIRequest *req; + QTAILQ_FOREACH(req, &s->requests, next) { + g_hash_table_add(aio_contexts, req->ctx); + } + } =20 - data->s =3D s; - data->fn =3D fn; - data->fn_opaque =3D opaque; - - /* - * Hold a reference to the SCSIDevice until - * scsi_device_for_each_req_async_bh() finishes. - */ - object_ref(OBJECT(s)); - - /* Paired with blk_dec_in_flight() in scsi_device_for_each_req_async_b= h() */ - blk_inc_in_flight(s->conf.blk); - aio_bh_schedule_oneshot(blk_get_aio_context(s->conf.blk), - scsi_device_for_each_req_async_bh, - data); + /* Schedule a BH for each AioContext */ + SCSIDeviceForEachReqAsyncData params =3D { + .s =3D s, + .fn =3D fn, + .fn_opaque =3D opaque, + }; + g_hash_table_foreach( + aio_contexts, + scsi_device_for_each_req_async_do_ctx, + ¶ms + ); } =20 static void scsi_device_realize(SCSIDevice *s, Error **errp) @@ -349,6 +392,7 @@ static void scsi_qdev_realize(DeviceState *qdev, Error = **errp) dev->lun =3D lun; } =20 + qemu_mutex_init(&dev->requests_lock); QTAILQ_INIT(&dev->requests); scsi_device_realize(dev, &local_err); if (local_err) { @@ -369,6 +413,8 @@ static void scsi_qdev_unrealize(DeviceState *qdev) =20 scsi_device_purge_requests(dev, SENSE_CODE(NO_SENSE)); =20 + qemu_mutex_destroy(&dev->requests_lock); + scsi_device_unrealize(dev); =20 blockdev_mark_auto_del(dev->conf.blk); @@ -965,7 +1011,10 @@ static void scsi_req_enqueue_internal(SCSIRequest *re= q) req->sg =3D NULL; } req->enqueued =3D true; - QTAILQ_INSERT_TAIL(&req->dev->requests, req, next); + + WITH_QEMU_LOCK_GUARD(&req->dev->requests_lock) { + QTAILQ_INSERT_TAIL(&req->dev->requests, req, next); + } } =20 int32_t scsi_req_enqueue(SCSIRequest *req) @@ -985,7 +1034,9 @@ static void scsi_req_dequeue(SCSIRequest *req) trace_scsi_req_dequeue(req->dev->id, req->lun, req->tag); req->retry =3D false; if (req->enqueued) { - QTAILQ_REMOVE(&req->dev->requests, req, next); + WITH_QEMU_LOCK_GUARD(&req->dev->requests_lock) { + QTAILQ_REMOVE(&req->dev->requests, req, next); + } req->enqueued =3D false; scsi_req_unref(req); } @@ -1962,8 +2013,7 @@ static void scsi_device_class_init(ObjectClass *klass= , void *data) =20 static void scsi_dev_instance_init(Object *obj) { - DeviceState *dev =3D DEVICE(obj); - SCSIDevice *s =3D SCSI_DEVICE(dev); + SCSIDevice *s =3D SCSI_DEVICE(obj); =20 device_add_bootindex_property(obj, &s->conf.bootindex, "bootindex", NULL, --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699759; cv=none; d=zohomail.com; s=zohoarc; b=Kx0+gwKdnX+1KxVnS22kyH9a03+LuUJ/Cy85vKTvHUW+BD6qUK4tM4OD1CUpspwvn4gsuWPsLu6O0deKkXWyHjSYsfG7gppP3EePZ/s5UMvO2AYoT5Zd/pauG3fTL3tb3oHq1i1+VM3Tyq68LphQ5jXd+Zc4lBgfUoG7d3yFmyQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699759; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/+CBBHVFHYNuPG5khaRJmuyKUdSakM0KMD2Bj8O8nm4=; b=RaQxtcjZ0kaT65Pl7aqr1y98RZkcbgLODAvNyvuUtPIa03bRPggQJh2gqiVWzfkUyQr0H0knx8zqopSOtbs8gmH0ccbCotT0yFPyVyoO9/HxhJvzriGafXMeu7PGKnajzvwJlGMllVVh9P2fw5RIhwhH+ukAVBt5fKGUcF9TNwY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699759684189.65242116463583; Tue, 11 Mar 2025 06:29:19 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzdX-0002a8-SF; Tue, 11 Mar 2025 09:27:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdN-0002JC-PK for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdF-0005E8-8F for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:01 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-618-KbnpOiUTM1GOrTYbqx87TQ-1; Tue, 11 Mar 2025 09:26:46 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 90A7A1955D50; Tue, 11 Mar 2025 13:26:45 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E95A91800945; Tue, 11 Mar 2025 13:26:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/+CBBHVFHYNuPG5khaRJmuyKUdSakM0KMD2Bj8O8nm4=; b=ek7z+EyLLKXO2x5xRPkWntTS1cWIqP5afn1PV6snSVn3qJy/45vw9Z8U6ii0M/eLqzCb36 qK9G8unsvhtQqzDhqK/cFwU8HcAUwodOF4CIn1PA3xMsLGYI4lS3S8vLOGWPafqchuZvRW ElINlcBqJMnrPMClOT2Ldl6wZcYaRSM= X-MC-Unique: KbnpOiUTM1GOrTYbqx87TQ-1 X-Mimecast-MFC-AGG-ID: KbnpOiUTM1GOrTYbqx87TQ_1741699605 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 05/13] virtio-scsi: introduce event and ctrl virtqueue locks Date: Tue, 11 Mar 2025 21:26:08 +0800 Message-ID: <20250311132616.1049687-6-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699761893019100 Content-Type: text/plain; charset="utf-8" Virtqueues are not thread-safe. Until now this was not a major issue since all virtqueue processing happened in the same thread. The ctrl queue's Task Management Function (TMF) requests sometimes need the main loop, so a BH was used to schedule the virtqueue completion back in the thread that has virtqueue access. When IOThread Virtqueue Mapping is introduced in later commits, event and ctrl virtqueue accesses from other threads will become necessary. Introduce an optional per-virtqueue lock so the event and ctrl virtqueues can be protected in the commits that follow. The addition of the ctrl virtqueue lock makes virtio_scsi_complete_req_from_main_loop() and its BH unnecessary. Instead, take the ctrl virtqueue lock from the main loop thread. The cmd virtqueue does not have a lock because the entirety of SCSI command processing happens in one thread. Only one thread accesses the cmd virtqueue and a lock is unnecessary. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- include/hw/virtio/virtio-scsi.h | 3 ++ hw/scsi/virtio-scsi.c | 90 ++++++++++++++++++--------------- 2 files changed, 52 insertions(+), 41 deletions(-) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scs= i.h index be230cd4bf..4ee98ebf63 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -84,6 +84,9 @@ struct VirtIOSCSI { int resetting; /* written from main loop thread, read from any thread = */ bool events_dropped; =20 + QemuMutex ctrl_lock; /* protects ctrl_vq */ + QemuMutex event_lock; /* protects event_vq */ + /* * TMFs deferred to main loop BH. These fields are protected by * tmf_bh_lock. diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 7d094e1881..073ccd3d5b 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -102,13 +102,18 @@ static void virtio_scsi_free_req(VirtIOSCSIReq *req) g_free(req); } =20 -static void virtio_scsi_complete_req(VirtIOSCSIReq *req) +static void virtio_scsi_complete_req(VirtIOSCSIReq *req, QemuMutex *vq_loc= k) { VirtIOSCSI *s =3D req->dev; VirtQueue *vq =3D req->vq; VirtIODevice *vdev =3D VIRTIO_DEVICE(s); =20 qemu_iovec_from_buf(&req->resp_iov, 0, &req->resp, req->resp_size); + + if (vq_lock) { + qemu_mutex_lock(vq_lock); + } + virtqueue_push(vq, &req->elem, req->qsgl.size + req->resp_iov.size); if (s->dataplane_started && !s->dataplane_fenced) { virtio_notify_irqfd(vdev, vq); @@ -116,6 +121,10 @@ static void virtio_scsi_complete_req(VirtIOSCSIReq *re= q) virtio_notify(vdev, vq); } =20 + if (vq_lock) { + qemu_mutex_unlock(vq_lock); + } + if (req->sreq) { req->sreq->hba_private =3D NULL; scsi_req_unref(req->sreq); @@ -123,34 +132,20 @@ static void virtio_scsi_complete_req(VirtIOSCSIReq *r= eq) virtio_scsi_free_req(req); } =20 -static void virtio_scsi_complete_req_bh(void *opaque) -{ - VirtIOSCSIReq *req =3D opaque; - - virtio_scsi_complete_req(req); -} - -/* - * Called from virtio_scsi_do_one_tmf_bh() in main loop thread. The main l= oop - * thread cannot touch the virtqueue since that could race with an IOThrea= d. - */ -static void virtio_scsi_complete_req_from_main_loop(VirtIOSCSIReq *req) -{ - VirtIOSCSI *s =3D req->dev; - - if (!s->ctx || s->ctx =3D=3D qemu_get_aio_context()) { - /* No need to schedule a BH when there is no IOThread */ - virtio_scsi_complete_req(req); - } else { - /* Run request completion in the IOThread */ - aio_wait_bh_oneshot(s->ctx, virtio_scsi_complete_req_bh, req); - } -} - -static void virtio_scsi_bad_req(VirtIOSCSIReq *req) +static void virtio_scsi_bad_req(VirtIOSCSIReq *req, QemuMutex *vq_lock) { virtio_error(VIRTIO_DEVICE(req->dev), "wrong size for virtio-scsi head= ers"); + + if (vq_lock) { + qemu_mutex_lock(vq_lock); + } + virtqueue_detach_element(req->vq, &req->elem, 0); + + if (vq_lock) { + qemu_mutex_unlock(vq_lock); + } + virtio_scsi_free_req(req); } =20 @@ -235,12 +230,21 @@ static int virtio_scsi_parse_req(VirtIOSCSIReq *req, return 0; } =20 -static VirtIOSCSIReq *virtio_scsi_pop_req(VirtIOSCSI *s, VirtQueue *vq) +static VirtIOSCSIReq *virtio_scsi_pop_req(VirtIOSCSI *s, VirtQueue *vq, Qe= muMutex *vq_lock) { VirtIOSCSICommon *vs =3D (VirtIOSCSICommon *)s; VirtIOSCSIReq *req; =20 + if (vq_lock) { + qemu_mutex_lock(vq_lock); + } + req =3D virtqueue_pop(vq, sizeof(VirtIOSCSIReq) + vs->cdb_size); + + if (vq_lock) { + qemu_mutex_unlock(vq_lock); + } + if (!req) { return NULL; } @@ -305,7 +309,7 @@ static void virtio_scsi_cancel_notify(Notifier *notifie= r, void *data) =20 trace_virtio_scsi_tmf_resp(virtio_scsi_get_lun(req->req.tmf.lun), req->req.tmf.tag, req->resp.tmf.respons= e); - virtio_scsi_complete_req(req); + virtio_scsi_complete_req(req, &req->dev->ctrl_lock); } g_free(n); } @@ -361,7 +365,7 @@ static void virtio_scsi_do_one_tmf_bh(VirtIOSCSIReq *re= q) =20 out: object_unref(OBJECT(d)); - virtio_scsi_complete_req_from_main_loop(req); + virtio_scsi_complete_req(req, &s->ctrl_lock); } =20 /* Some TMFs must be processed from the main loop thread */ @@ -408,7 +412,7 @@ static void virtio_scsi_reset_tmf_bh(VirtIOSCSI *s) =20 /* SAM-6 6.3.2 Hard reset */ req->resp.tmf.response =3D VIRTIO_SCSI_S_TARGET_FAILURE; - virtio_scsi_complete_req(req); + virtio_scsi_complete_req(req, &req->dev->ctrl_lock); } } =20 @@ -562,7 +566,7 @@ static void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, = VirtIOSCSIReq *req) =20 if (iov_to_buf(req->elem.out_sg, req->elem.out_num, 0, &type, sizeof(type)) < sizeof(type)) { - virtio_scsi_bad_req(req); + virtio_scsi_bad_req(req, &s->ctrl_lock); return; } =20 @@ -570,7 +574,7 @@ static void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, = VirtIOSCSIReq *req) if (type =3D=3D VIRTIO_SCSI_T_TMF) { if (virtio_scsi_parse_req(req, sizeof(VirtIOSCSICtrlTMFReq), sizeof(VirtIOSCSICtrlTMFResp)) < 0) { - virtio_scsi_bad_req(req); + virtio_scsi_bad_req(req, &s->ctrl_lock); return; } else { r =3D virtio_scsi_do_tmf(s, req); @@ -580,7 +584,7 @@ static void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, = VirtIOSCSIReq *req) type =3D=3D VIRTIO_SCSI_T_AN_SUBSCRIBE) { if (virtio_scsi_parse_req(req, sizeof(VirtIOSCSICtrlANReq), sizeof(VirtIOSCSICtrlANResp)) < 0) { - virtio_scsi_bad_req(req); + virtio_scsi_bad_req(req, &s->ctrl_lock); return; } else { req->req.an.event_requested =3D @@ -600,7 +604,7 @@ static void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, = VirtIOSCSIReq *req) type =3D=3D VIRTIO_SCSI_T_AN_SUBSCRIBE) trace_virtio_scsi_an_resp(virtio_scsi_get_lun(req->req.an.lun), req->resp.an.response); - virtio_scsi_complete_req(req); + virtio_scsi_complete_req(req, &s->ctrl_lock); } else { assert(r =3D=3D -EINPROGRESS); } @@ -610,7 +614,7 @@ static void virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, V= irtQueue *vq) { VirtIOSCSIReq *req; =20 - while ((req =3D virtio_scsi_pop_req(s, vq))) { + while ((req =3D virtio_scsi_pop_req(s, vq, &s->ctrl_lock))) { virtio_scsi_handle_ctrl_req(s, req); } } @@ -654,7 +658,7 @@ static void virtio_scsi_complete_cmd_req(VirtIOSCSIReq = *req) * in virtio_scsi_command_complete. */ req->resp_size =3D sizeof(VirtIOSCSICmdResp); - virtio_scsi_complete_req(req); + virtio_scsi_complete_req(req, NULL); } =20 static void virtio_scsi_command_failed(SCSIRequest *r) @@ -788,7 +792,7 @@ static int virtio_scsi_handle_cmd_req_prepare(VirtIOSCS= I *s, VirtIOSCSIReq *req) virtio_scsi_fail_cmd_req(req); return -ENOTSUP; } else { - virtio_scsi_bad_req(req); + virtio_scsi_bad_req(req, NULL); return -EINVAL; } } @@ -843,7 +847,7 @@ static void virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, Vi= rtQueue *vq) virtio_queue_set_notification(vq, 0); } =20 - while ((req =3D virtio_scsi_pop_req(s, vq))) { + while ((req =3D virtio_scsi_pop_req(s, vq, NULL))) { ret =3D virtio_scsi_handle_cmd_req_prepare(s, req); if (!ret) { QTAILQ_INSERT_TAIL(&reqs, req, next); @@ -973,7 +977,7 @@ static void virtio_scsi_push_event(VirtIOSCSI *s, return; } =20 - req =3D virtio_scsi_pop_req(s, vs->event_vq); + req =3D virtio_scsi_pop_req(s, vs->event_vq, &s->event_lock); if (!req) { s->events_dropped =3D true; return; @@ -985,7 +989,7 @@ static void virtio_scsi_push_event(VirtIOSCSI *s, } =20 if (virtio_scsi_parse_req(req, 0, sizeof(VirtIOSCSIEvent))) { - virtio_scsi_bad_req(req); + virtio_scsi_bad_req(req, &s->event_lock); return; } =20 @@ -1005,7 +1009,7 @@ static void virtio_scsi_push_event(VirtIOSCSI *s, } trace_virtio_scsi_event(virtio_scsi_get_lun(evt->lun), event, reason); =20 - virtio_scsi_complete_req(req); + virtio_scsi_complete_req(req, &s->event_lock); } =20 static void virtio_scsi_handle_event_vq(VirtIOSCSI *s, VirtQueue *vq) @@ -1236,6 +1240,8 @@ static void virtio_scsi_device_realize(DeviceState *d= ev, Error **errp) Error *err =3D NULL; =20 QTAILQ_INIT(&s->tmf_bh_list); + qemu_mutex_init(&s->ctrl_lock); + qemu_mutex_init(&s->event_lock); qemu_mutex_init(&s->tmf_bh_lock); =20 virtio_scsi_common_realize(dev, @@ -1280,6 +1286,8 @@ static void virtio_scsi_device_unrealize(DeviceState = *dev) qbus_set_hotplug_handler(BUS(&s->bus), NULL); virtio_scsi_common_unrealize(dev); qemu_mutex_destroy(&s->tmf_bh_lock); + qemu_mutex_destroy(&s->event_lock); + qemu_mutex_destroy(&s->ctrl_lock); } =20 static const Property virtio_scsi_properties[] =3D { --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699767; cv=none; d=zohomail.com; s=zohoarc; b=hCI+w6fv90wXAx+mXnTbajcpCNyB5vaFu8T6hVHnnKgbA4OS4R8h0+P7o4nBN/1FCbQyvPKdM+tUI4mbKBhAo//UrSEsdVzidZZgXIHdSl7DkXVkwBVDHQa5SOvBcnU74h0bYGRSqUn30yrRfsYRX+pibiB4tFNqA+waDnR3Sns= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699767; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CE1im9Bjq31X9Eaxj30dsdZEIRgev+UbxVXubrTF5n4=; b=IBvNU/GKpfFATVKq546L1BRxuNIhbLQ425BLweC/2fQay/CmVi8EtZTutgKNsw/KZeBOBgD8CTdL9l6nTtss4qqFhBz1OVxEs6AbC2I7NjOMOZWtdxZAHrhXl223i8Vt5Z+6aWerQrXCCCFOK/uhg2LQRmYiKqTuHLcMP9sSFKE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699767341587.2505749878513; Tue, 11 Mar 2025 06:29:27 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzeI-0003Na-Iv; Tue, 11 Mar 2025 09:27:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdP-0002Mf-LN for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdK-0005Eb-NZ for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:03 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-668-07ky7UkHO3aMNU2DoXvYwg-1; Tue, 11 Mar 2025 09:26:51 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F35491955D67; Tue, 11 Mar 2025 13:26:49 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CBBE518001DE; Tue, 11 Mar 2025 13:26:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699614; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CE1im9Bjq31X9Eaxj30dsdZEIRgev+UbxVXubrTF5n4=; b=MflfqOafsB9ZQ6eqK/goquqzFfLzTJfqotLeMQrkhNaaKBGty8o3RjyNWK0YF+74yQnrTS l+LfBomPmVK2vTpXJt+LZcCh8e+CrPm3zUDNT5R6R1u7D/Qu165TsdAVCvQ4u/Y6u8dMlv 1TGPi8HXQeBPjj5ljOTMmNlIZitHcZo= X-MC-Unique: 07ky7UkHO3aMNU2DoXvYwg-1 X-Mimecast-MFC-AGG-ID: 07ky7UkHO3aMNU2DoXvYwg_1741699610 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 06/13] virtio-scsi: protect events_dropped field Date: Tue, 11 Mar 2025 21:26:09 +0800 Message-ID: <20250311132616.1049687-7-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699769578019000 Content-Type: text/plain; charset="utf-8" The block layer can invoke the resize callback from any AioContext that is processing requests. The virtqueue is already protected but the events_dropped field also needs to be protected against races. Cover it using the event virtqueue lock because it is closely associated with accesses to the virtqueue. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- include/hw/virtio/virtio-scsi.h | 3 ++- hw/scsi/virtio-scsi.c | 29 ++++++++++++++++++++--------- 2 files changed, 22 insertions(+), 10 deletions(-) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scs= i.h index 4ee98ebf63..7b7e3ced7a 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -82,10 +82,11 @@ struct VirtIOSCSI { =20 SCSIBus bus; int resetting; /* written from main loop thread, read from any thread = */ + + QemuMutex event_lock; /* protects event_vq and events_dropped */ bool events_dropped; =20 QemuMutex ctrl_lock; /* protects ctrl_vq */ - QemuMutex event_lock; /* protects event_vq */ =20 /* * TMFs deferred to main loop BH. These fields are protected by diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 073ccd3d5b..2d796a861b 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -948,7 +948,10 @@ static void virtio_scsi_reset(VirtIODevice *vdev) =20 vs->sense_size =3D VIRTIO_SCSI_SENSE_DEFAULT_SIZE; vs->cdb_size =3D VIRTIO_SCSI_CDB_DEFAULT_SIZE; - s->events_dropped =3D false; + + WITH_QEMU_LOCK_GUARD(&s->event_lock) { + s->events_dropped =3D false; + } } =20 typedef struct { @@ -978,14 +981,16 @@ static void virtio_scsi_push_event(VirtIOSCSI *s, } =20 req =3D virtio_scsi_pop_req(s, vs->event_vq, &s->event_lock); - if (!req) { - s->events_dropped =3D true; - return; - } + WITH_QEMU_LOCK_GUARD(&s->event_lock) { + if (!req) { + s->events_dropped =3D true; + return; + } =20 - if (s->events_dropped) { - event |=3D VIRTIO_SCSI_T_EVENTS_MISSED; - s->events_dropped =3D false; + if (s->events_dropped) { + event |=3D VIRTIO_SCSI_T_EVENTS_MISSED; + s->events_dropped =3D false; + } } =20 if (virtio_scsi_parse_req(req, 0, sizeof(VirtIOSCSIEvent))) { @@ -1014,7 +1019,13 @@ static void virtio_scsi_push_event(VirtIOSCSI *s, =20 static void virtio_scsi_handle_event_vq(VirtIOSCSI *s, VirtQueue *vq) { - if (s->events_dropped) { + bool events_dropped; + + WITH_QEMU_LOCK_GUARD(&s->event_lock) { + events_dropped =3D s->events_dropped; + } + + if (events_dropped) { VirtIOSCSIEventInfo info =3D { .event =3D VIRTIO_SCSI_T_NO_EVENT, }; --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699867; cv=none; d=zohomail.com; s=zohoarc; b=QUVcnCEs09TlgXforyOhn0cm9xQ4VErg/9l3Z4ze1mLBPZhdZ41sLQdNvzvLuvfDvTPylBs4WWdBDB18+okYED/UMal1pnEsL2oc5jGMNfIYKdgZFQkD6OxA73MjaU72Wf6mKhIm2jyPPqJzjPTFfOlnxKH4fszotHUFi8mBuR4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699867; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GAHz6RB4kDNJyJvHeHywpNJORyXczYHYXtfqU2j/QsE=; b=Ygp4qR2nZzaHcfXWkMqp01nCNMYUv+Zf21q2+qhyhBXUZBBSI78l6roauJ3Sc8HObq3q6HKf8dvRUACvUNea2l2FVEO9G64Sco1Ec8RV3tpk0H9boIVvfJyOc3KPJ4UmZu6NEONOQ9Fwl/x9FSYjc9Zgqi+B+ElcnaFDEmBeAF8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699867465534.4825584825675; Tue, 11 Mar 2025 06:31:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzeM-0003Xi-OF; Tue, 11 Mar 2025 09:28:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdQ-0002Mt-VT for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdN-0005Fc-40 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:04 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-486-WgYToQbmPU2RuJPQmkgq4w-1; Tue, 11 Mar 2025 09:26:55 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D5CBB18001E1; Tue, 11 Mar 2025 13:26:53 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 083A81801747; Tue, 11 Mar 2025 13:26:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699619; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GAHz6RB4kDNJyJvHeHywpNJORyXczYHYXtfqU2j/QsE=; b=R9ybre5MTKYYMAuN0bGX5ER3QysZtGVeYu1FxHbyeQvthHcOkwCHoWhlJu84PA/aSELzTq lhiKvocn7KWtOYHq72XFrHky7FFVnIugmTl121hKY4edU1dDPj0Q2t30dwg1OnwKYWveB4 RsQZWvZR1UjRvkvE/IAmkDV4xEr3NVM= X-MC-Unique: WgYToQbmPU2RuJPQmkgq4w-1 X-Mimecast-MFC-AGG-ID: WgYToQbmPU2RuJPQmkgq4w_1741699614 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 07/13] virtio-scsi: perform TMFs in appropriate AioContexts Date: Tue, 11 Mar 2025 21:26:10 +0800 Message-ID: <20250311132616.1049687-8-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699870498019000 Content-Type: text/plain; charset="utf-8" With IOThread Virtqueue Mapping there will be multiple AioContexts processing SCSI requests. scsi_req_cancel() and other SCSI request operations must be performed from the AioContext where the request is running. Introduce a virtio_scsi_defer_tmf_to_aio_context() function and the necessary VirtIOSCSIReq->remaining refcount infrastructure to move the TMF code into the AioContext where the request is running. For the time being there is still just one AioContext: the main loop or the IOThread. When the iothread-vq-mapping parameter is added in a later patch this will be changed to per-virtqueue AioContexts. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- hw/scsi/virtio-scsi.c | 266 ++++++++++++++++++++++++++++++++---------- 1 file changed, 204 insertions(+), 62 deletions(-) diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 2d796a861b..2045d27289 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -47,7 +47,7 @@ typedef struct VirtIOSCSIReq { /* Used for two-stage request submission and TMFs deferred to BH */ QTAILQ_ENTRY(VirtIOSCSIReq) next; =20 - /* Used for cancellation of request during TMFs */ + /* Used for cancellation of request during TMFs. Atomic. */ int remaining; =20 SCSIRequest *sreq; @@ -298,19 +298,23 @@ typedef struct { VirtIOSCSIReq *tmf_req; } VirtIOSCSICancelNotifier; =20 +static void virtio_scsi_tmf_dec_remaining(VirtIOSCSIReq *tmf) +{ + if (qatomic_fetch_dec(&tmf->remaining) =3D=3D 1) { + trace_virtio_scsi_tmf_resp(virtio_scsi_get_lun(tmf->req.tmf.lun), + tmf->req.tmf.tag, tmf->resp.tmf.respons= e); + + virtio_scsi_complete_req(tmf, &tmf->dev->ctrl_lock); + } +} + static void virtio_scsi_cancel_notify(Notifier *notifier, void *data) { VirtIOSCSICancelNotifier *n =3D container_of(notifier, VirtIOSCSICancelNotifier, notifier); =20 - if (--n->tmf_req->remaining =3D=3D 0) { - VirtIOSCSIReq *req =3D n->tmf_req; - - trace_virtio_scsi_tmf_resp(virtio_scsi_get_lun(req->req.tmf.lun), - req->req.tmf.tag, req->resp.tmf.respons= e); - virtio_scsi_complete_req(req, &req->dev->ctrl_lock); - } + virtio_scsi_tmf_dec_remaining(n->tmf_req); g_free(n); } =20 @@ -416,7 +420,7 @@ static void virtio_scsi_reset_tmf_bh(VirtIOSCSI *s) } } =20 -static void virtio_scsi_defer_tmf_to_bh(VirtIOSCSIReq *req) +static void virtio_scsi_defer_tmf_to_main_loop(VirtIOSCSIReq *req) { VirtIOSCSI *s =3D req->dev; =20 @@ -430,6 +434,137 @@ static void virtio_scsi_defer_tmf_to_bh(VirtIOSCSIReq= *req) } } =20 +static void virtio_scsi_tmf_cancel_req(VirtIOSCSIReq *tmf, SCSIRequest *r) +{ + VirtIOSCSICancelNotifier *notifier; + + assert(r->ctx =3D=3D qemu_get_current_aio_context()); + + /* Decremented in virtio_scsi_cancel_notify() */ + qatomic_inc(&tmf->remaining); + + notifier =3D g_new(VirtIOSCSICancelNotifier, 1); + notifier->notifier.notify =3D virtio_scsi_cancel_notify; + notifier->tmf_req =3D tmf; + scsi_req_cancel_async(r, ¬ifier->notifier); +} + +/* Execute a TMF on the requests in the current AioContext */ +static void virtio_scsi_do_tmf_aio_context(void *opaque) +{ + AioContext *ctx =3D qemu_get_current_aio_context(); + VirtIOSCSIReq *tmf =3D opaque; + VirtIOSCSI *s =3D tmf->dev; + SCSIDevice *d =3D virtio_scsi_device_get(s, tmf->req.tmf.lun); + SCSIRequest *r; + bool match_tag; + + if (!d) { + tmf->resp.tmf.response =3D VIRTIO_SCSI_S_BAD_TARGET; + virtio_scsi_tmf_dec_remaining(tmf); + return; + } + + /* + * This function could handle other subtypes that need to be processed= in + * the request's AioContext in the future, but for now only request + * cancelation subtypes are performed here. + */ + switch (tmf->req.tmf.subtype) { + case VIRTIO_SCSI_T_TMF_ABORT_TASK: + match_tag =3D true; + break; + case VIRTIO_SCSI_T_TMF_ABORT_TASK_SET: + case VIRTIO_SCSI_T_TMF_CLEAR_TASK_SET: + match_tag =3D false; + break; + default: + g_assert_not_reached(); + } + + WITH_QEMU_LOCK_GUARD(&d->requests_lock) { + QTAILQ_FOREACH(r, &d->requests, next) { + VirtIOSCSIReq *cmd_req =3D r->hba_private; + assert(cmd_req); /* request has hba_private while enqueued */ + + if (r->ctx !=3D ctx) { + continue; + } + if (match_tag && cmd_req->req.cmd.tag !=3D tmf->req.tmf.tag) { + continue; + } + virtio_scsi_tmf_cancel_req(tmf, r); + } + } + + /* Incremented by virtio_scsi_do_tmf() */ + virtio_scsi_tmf_dec_remaining(tmf); + + object_unref(d); +} + +static void dummy_bh(void *opaque) +{ + /* Do nothing */ +} + +/* + * Wait for pending virtio_scsi_defer_tmf_to_aio_context() BHs. + */ +static void virtio_scsi_flush_defer_tmf_to_aio_context(VirtIOSCSI *s) +{ + GLOBAL_STATE_CODE(); + + assert(!s->dataplane_started); + + if (s->ctx) { + /* Our BH only runs after previously scheduled BHs */ + aio_wait_bh_oneshot(s->ctx, dummy_bh, NULL); + } +} + +/* + * Run the TMF in a specific AioContext, handling only requests in that + * AioContext. This is necessary because requests can run in different + * AioContext and it is only possible to cancel them from the AioContext w= here + * they are running. + */ +static void virtio_scsi_defer_tmf_to_aio_context(VirtIOSCSIReq *tmf, + AioContext *ctx) +{ + /* Decremented in virtio_scsi_do_tmf_aio_context() */ + qatomic_inc(&tmf->remaining); + + /* See virtio_scsi_flush_defer_tmf_to_aio_context() cleanup during res= et */ + aio_bh_schedule_oneshot(ctx, virtio_scsi_do_tmf_aio_context, tmf); +} + +/* + * Returns the AioContext for a given TMF's tag field or NULL. Note that t= he + * request identified by the tag may have completed by the time you can ex= ecute + * a BH in the AioContext, so don't assume the request still exists in you= r BH. + */ +static AioContext *find_aio_context_for_tmf_tag(SCSIDevice *d, + VirtIOSCSIReq *tmf) +{ + WITH_QEMU_LOCK_GUARD(&d->requests_lock) { + SCSIRequest *r; + SCSIRequest *next; + + QTAILQ_FOREACH_SAFE(r, &d->requests, next, next) { + VirtIOSCSIReq *cmd_req =3D r->hba_private; + + /* hba_private is non-NULL while the request is enqueued */ + assert(cmd_req); + + if (cmd_req->req.cmd.tag =3D=3D tmf->req.tmf.tag) { + return r->ctx; + } + } + } + return NULL; +} + /* Return 0 if the request is ready to be completed and return to guest; * -EINPROGRESS if the request is submitted and will be completed later, i= n the * case of async cancellation. */ @@ -437,6 +572,7 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSCSI= Req *req) { SCSIDevice *d =3D virtio_scsi_device_get(s, req->req.tmf.lun); SCSIRequest *r, *next; + AioContext *ctx; int ret =3D 0; =20 virtio_scsi_ctx_check(s, d); @@ -454,7 +590,22 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSCS= IReq *req) req->req.tmf.tag, req->req.tmf.subtype); =20 switch (req->req.tmf.subtype) { - case VIRTIO_SCSI_T_TMF_ABORT_TASK: + case VIRTIO_SCSI_T_TMF_ABORT_TASK: { + if (!d) { + goto fail; + } + if (d->lun !=3D virtio_scsi_get_lun(req->req.tmf.lun)) { + goto incorrect_lun; + } + + ctx =3D find_aio_context_for_tmf_tag(d, req); + if (ctx) { + virtio_scsi_defer_tmf_to_aio_context(req, ctx); + ret =3D -EINPROGRESS; + } + break; + } + case VIRTIO_SCSI_T_TMF_QUERY_TASK: if (!d) { goto fail; @@ -462,44 +613,49 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSC= SIReq *req) if (d->lun !=3D virtio_scsi_get_lun(req->req.tmf.lun)) { goto incorrect_lun; } - QTAILQ_FOREACH_SAFE(r, &d->requests, next, next) { - VirtIOSCSIReq *cmd_req =3D r->hba_private; - if (cmd_req && cmd_req->req.cmd.tag =3D=3D req->req.tmf.tag) { - break; - } - } - if (r) { - /* - * Assert that the request has not been completed yet, we - * check for it in the loop above. - */ - assert(r->hba_private); - if (req->req.tmf.subtype =3D=3D VIRTIO_SCSI_T_TMF_QUERY_TASK) { - /* "If the specified command is present in the task set, t= hen - * return a service response set to FUNCTION SUCCEEDED". - */ - req->resp.tmf.response =3D VIRTIO_SCSI_S_FUNCTION_SUCCEEDE= D; - } else { - VirtIOSCSICancelNotifier *notifier; =20 - req->remaining =3D 1; - notifier =3D g_new(VirtIOSCSICancelNotifier, 1); - notifier->tmf_req =3D req; - notifier->notifier.notify =3D virtio_scsi_cancel_notify; - scsi_req_cancel_async(r, ¬ifier->notifier); - ret =3D -EINPROGRESS; + WITH_QEMU_LOCK_GUARD(&d->requests_lock) { + QTAILQ_FOREACH(r, &d->requests, next) { + VirtIOSCSIReq *cmd_req =3D r->hba_private; + assert(cmd_req); /* request has hba_private while enqueued= */ + + if (cmd_req->req.cmd.tag =3D=3D req->req.tmf.tag) { + /* + * "If the specified command is present in the task se= t, + * then return a service response set to FUNCTION + * SUCCEEDED". + */ + req->resp.tmf.response =3D VIRTIO_SCSI_S_FUNCTION_SUCC= EEDED; + } } } break; =20 case VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET: case VIRTIO_SCSI_T_TMF_I_T_NEXUS_RESET: - virtio_scsi_defer_tmf_to_bh(req); + virtio_scsi_defer_tmf_to_main_loop(req); ret =3D -EINPROGRESS; break; =20 case VIRTIO_SCSI_T_TMF_ABORT_TASK_SET: - case VIRTIO_SCSI_T_TMF_CLEAR_TASK_SET: + case VIRTIO_SCSI_T_TMF_CLEAR_TASK_SET: { + if (!d) { + goto fail; + } + if (d->lun !=3D virtio_scsi_get_lun(req->req.tmf.lun)) { + goto incorrect_lun; + } + + qatomic_inc(&req->remaining); + + ctx =3D s->ctx ?: qemu_get_aio_context(); + virtio_scsi_defer_tmf_to_aio_context(req, ctx); + + virtio_scsi_tmf_dec_remaining(req); + ret =3D -EINPROGRESS; + break; + } + case VIRTIO_SCSI_T_TMF_QUERY_TASK_SET: if (!d) { goto fail; @@ -508,34 +664,19 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSC= SIReq *req) goto incorrect_lun; } =20 - /* Add 1 to "remaining" until virtio_scsi_do_tmf returns. - * This way, if the bus starts calling back to the notifiers - * even before we finish the loop, virtio_scsi_cancel_notify - * will not complete the TMF too early. - */ - req->remaining =3D 1; - QTAILQ_FOREACH_SAFE(r, &d->requests, next, next) { - if (r->hba_private) { - if (req->req.tmf.subtype =3D=3D VIRTIO_SCSI_T_TMF_QUERY_TA= SK_SET) { - /* "If there is any command present in the task set, t= hen - * return a service response set to FUNCTION SUCCEEDED= ". - */ - req->resp.tmf.response =3D VIRTIO_SCSI_S_FUNCTION_SUCC= EEDED; - break; - } else { - VirtIOSCSICancelNotifier *notifier; + WITH_QEMU_LOCK_GUARD(&d->requests_lock) { + QTAILQ_FOREACH_SAFE(r, &d->requests, next, next) { + /* Request has hba_private while enqueued */ + assert(r->hba_private); =20 - req->remaining++; - notifier =3D g_new(VirtIOSCSICancelNotifier, 1); - notifier->notifier.notify =3D virtio_scsi_cancel_notif= y; - notifier->tmf_req =3D req; - scsi_req_cancel_async(r, ¬ifier->notifier); - } + /* + * "If there is any command present in the task set, then + * return a service response set to FUNCTION SUCCEEDED". + */ + req->resp.tmf.response =3D VIRTIO_SCSI_S_FUNCTION_SUCCEEDE= D; + break; } } - if (--req->remaining > 0) { - ret =3D -EINPROGRESS; - } break; =20 case VIRTIO_SCSI_T_TMF_CLEAR_ACA: @@ -941,6 +1082,7 @@ static void virtio_scsi_reset(VirtIODevice *vdev) assert(!s->dataplane_started); =20 virtio_scsi_reset_tmf_bh(s); + virtio_scsi_flush_defer_tmf_to_aio_context(s); =20 qatomic_inc(&s->resetting); bus_cold_reset(BUS(&s->bus)); --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699922; cv=none; d=zohomail.com; s=zohoarc; b=LyuIhDnSrKvscUYe8tiVOs5lsQVWD29UyefYSidJtVOH9NKcUAJQKB7QKlRY3ngkih7do4nbRK1shFrUSfYlquUuLFexIQRNnxmvoZNxgRFxXbTAWmJzaQPLsv3mc5/ollXLOb3IxbnCmKTkhNjCFyGHCQ0h0wLv8HnZc/hzoXA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699922; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LEAblqs+/QeL64NIpBD9nW8zWVA/2GdqUlR+17Zc9i4=; b=XjHX5uON66UCtRfu7yukSRTQ9Yrf5+gzSRY8XqepzzCoDjwGwTDQ07zFy4DZUnpwhg0FWUAmheRRY7lC0hxG2P2nLQqaIH64oPaexCKuj3Npdn1/TicGwMwvL0kURpj2B2qxVV1iZac8yHEhlUgvKCWLW7VG4JmfJpAhkFPCU9w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699922605495.4173468990209; Tue, 11 Mar 2025 06:32:02 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzeS-0003e4-6I; Tue, 11 Mar 2025 09:28:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdS-0002Ol-D5 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdQ-0005GL-F0 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:06 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-552-kGSmP0wyMMmia9dXoFBmqA-1; Tue, 11 Mar 2025 09:27:01 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E061C18001E2; Tue, 11 Mar 2025 13:26:59 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 18788195608F; Tue, 11 Mar 2025 13:26:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699622; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LEAblqs+/QeL64NIpBD9nW8zWVA/2GdqUlR+17Zc9i4=; b=JN5x/aXBDeZnnNZDqokQKO/GPlIzDEBjkB97K83AojercHqW9HAloMbMSe/hk9Y9eZkCPT freYXm8bXglYeDp/MHw/M5nFFsAKu8J4d87gqFlk+Pw9+L1pdjFPz2eCgz54szj9FewHA1 VorngtR1tOO3BbioJzRYdtkXdCp7e+U= X-MC-Unique: kGSmP0wyMMmia9dXoFBmqA-1 X-Mimecast-MFC-AGG-ID: kGSmP0wyMMmia9dXoFBmqA_1741699620 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 08/13] virtio-blk: extract cleanup_iothread_vq_mapping() function Date: Tue, 11 Mar 2025 21:26:11 +0800 Message-ID: <20250311132616.1049687-9-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699924542019000 Content-Type: text/plain; charset="utf-8" This is the cleanup function that must be called after apply_iothread_vq_mapping() succeeds. virtio-scsi will need this function too, so extract it. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- hw/block/virtio-blk.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 5135b4d8f1..21b1b768ed 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1495,6 +1495,9 @@ validate_iothread_vq_mapping_list(IOThreadVirtQueueMa= ppingList *list, * Fill in the AioContext for each virtqueue in the @vq_aio_context array = given * the iothread-vq-mapping parameter in @iothread_vq_mapping_list. * + * cleanup_iothread_vq_mapping() must be called to free IOThread object + * references after this function returns success. + * * Returns: %true on success, %false on failure. **/ static bool apply_iothread_vq_mapping( @@ -1545,6 +1548,23 @@ static bool apply_iothread_vq_mapping( return true; } =20 +/** + * cleanup_iothread_vq_mapping: + * @list: The mapping of virtqueues to IOThreads. + * + * Release IOThread object references that were acquired by + * apply_iothread_vq_mapping(). + */ +static void cleanup_iothread_vq_mapping(IOThreadVirtQueueMappingList *list) +{ + IOThreadVirtQueueMappingList *node; + + for (node =3D list; node; node =3D node->next) { + IOThread *iothread =3D iothread_by_id(node->value->iothread); + object_unref(OBJECT(iothread)); + } +} + /* Context: BQL held */ static bool virtio_blk_vq_aio_context_init(VirtIOBlock *s, Error **errp) { @@ -1611,12 +1631,7 @@ static void virtio_blk_vq_aio_context_cleanup(VirtIO= Block *s) assert(!s->ioeventfd_started); =20 if (conf->iothread_vq_mapping_list) { - IOThreadVirtQueueMappingList *node; - - for (node =3D conf->iothread_vq_mapping_list; node; node =3D node-= >next) { - IOThread *iothread =3D iothread_by_id(node->value->iothread); - object_unref(OBJECT(iothread)); - } + cleanup_iothread_vq_mapping(conf->iothread_vq_mapping_list); } =20 if (conf->iothread) { --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699753; cv=none; d=zohomail.com; s=zohoarc; b=hBwPbYOZZRli7FS08P1l6Fdd1AEBHAfV4vS9x1axs5hVKRmxjh3FKoymtaauKVDkGa6o/sI26NlR3jezWwXjtGpRSDSJu4Kd71hRCdKIhpwQH57Ugn5poaVc+TM7uP7x5R9sYjANtlCtBqbkC9m55KfzRuxli202uByXNazYDoE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699753; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=X3EZmGUp+mWY/UUw13gvrqgslZdCixQZtPNNeV29Mkk=; b=JVzjdMlVoVOfPPlcTuJ+GWXVEif1UxohrhQSrCi5aEl9SSOYRa5U4+P8b8NqX8LRcVcuFXqZ6EDFZHiehkIjbokbDdZkijusMSV7EqT20z0mOO+fZSqWSfpSW89euqeRfFk4JuUFjvODhciOyblEaH3OWItUqeNOPVfivbabMm8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699753134442.8675280436489; Tue, 11 Mar 2025 06:29:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzek-0004Vm-9g; Tue, 11 Mar 2025 09:28:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdc-0002kP-4n for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdV-0005Hl-Pz for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:15 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-654-Ud084b90PMS3uRH-OaFBig-1; Tue, 11 Mar 2025 09:27:05 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9A890180AF7B; Tue, 11 Mar 2025 13:27:04 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 103F73001D12; Tue, 11 Mar 2025 13:27:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X3EZmGUp+mWY/UUw13gvrqgslZdCixQZtPNNeV29Mkk=; b=Mhn3nLrMCT7UyGKHRJqA+pRYZCF1v1Zuo9gqyMm30BNDs8cocbIknR5fIpaI4TxDnruhRC eFY4ehWHUUCdleNAVgwpxC9viE+2f2M9PZcEOZE5v2UkjYUPHSvXkLvlyweZ4zFL37uNNZ ueRkcoE4Col/uzF4yDg2DzL9xGhUgAk= X-MC-Unique: Ud084b90PMS3uRH-OaFBig-1 X-Mimecast-MFC-AGG-ID: Ud084b90PMS3uRH-OaFBig_1741699624 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 09/13] virtio-blk: tidy up iothread_vq_mapping functions Date: Tue, 11 Mar 2025 21:26:12 +0800 Message-ID: <20250311132616.1049687-10-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699754057019100 Content-Type: text/plain; charset="utf-8" Use noun_verb() function naming instead of verb_noun() because the former is the most common naming style for APIs. The next commit will move these functions into a header file so that virtio-scsi can call them. Shorten iothread_vq_mapping_apply()'s iothread_vq_mapping_list argument to just "list" like in the other functions. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- hw/block/virtio-blk.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 21b1b768ed..6bf7b50520 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1424,8 +1424,8 @@ static const BlockDevOps virtio_block_ops =3D { }; =20 static bool -validate_iothread_vq_mapping_list(IOThreadVirtQueueMappingList *list, - uint16_t num_queues, Error **errp) +iothread_vq_mapping_validate(IOThreadVirtQueueMappingList *list, uint16_t + num_queues, Error **errp) { g_autofree unsigned long *vqs =3D bitmap_new(num_queues); g_autoptr(GHashTable) iothreads =3D @@ -1486,22 +1486,22 @@ validate_iothread_vq_mapping_list(IOThreadVirtQueue= MappingList *list, } =20 /** - * apply_iothread_vq_mapping: - * @iothread_vq_mapping_list: The mapping of virtqueues to IOThreads. + * iothread_vq_mapping_apply: + * @list: The mapping of virtqueues to IOThreads. * @vq_aio_context: The array of AioContext pointers to fill in. * @num_queues: The length of @vq_aio_context. * @errp: If an error occurs, a pointer to the area to store the error. * * Fill in the AioContext for each virtqueue in the @vq_aio_context array = given - * the iothread-vq-mapping parameter in @iothread_vq_mapping_list. + * the iothread-vq-mapping parameter in @list. * - * cleanup_iothread_vq_mapping() must be called to free IOThread object + * iothread_vq_mapping_cleanup() must be called to free IOThread object * references after this function returns success. * * Returns: %true on success, %false on failure. **/ -static bool apply_iothread_vq_mapping( - IOThreadVirtQueueMappingList *iothread_vq_mapping_list, +static bool iothread_vq_mapping_apply( + IOThreadVirtQueueMappingList *list, AioContext **vq_aio_context, uint16_t num_queues, Error **errp) @@ -1510,16 +1510,15 @@ static bool apply_iothread_vq_mapping( size_t num_iothreads =3D 0; size_t cur_iothread =3D 0; =20 - if (!validate_iothread_vq_mapping_list(iothread_vq_mapping_list, - num_queues, errp)) { + if (!iothread_vq_mapping_validate(list, num_queues, errp)) { return false; } =20 - for (node =3D iothread_vq_mapping_list; node; node =3D node->next) { + for (node =3D list; node; node =3D node->next) { num_iothreads++; } =20 - for (node =3D iothread_vq_mapping_list; node; node =3D node->next) { + for (node =3D list; node; node =3D node->next) { IOThread *iothread =3D iothread_by_id(node->value->iothread); AioContext *ctx =3D iothread_get_aio_context(iothread); =20 @@ -1549,13 +1548,13 @@ static bool apply_iothread_vq_mapping( } =20 /** - * cleanup_iothread_vq_mapping: + * iothread_vq_mapping_cleanup: * @list: The mapping of virtqueues to IOThreads. * * Release IOThread object references that were acquired by - * apply_iothread_vq_mapping(). + * iothread_vq_mapping_apply(). */ -static void cleanup_iothread_vq_mapping(IOThreadVirtQueueMappingList *list) +static void iothread_vq_mapping_cleanup(IOThreadVirtQueueMappingList *list) { IOThreadVirtQueueMappingList *node; =20 @@ -1597,7 +1596,7 @@ static bool virtio_blk_vq_aio_context_init(VirtIOBloc= k *s, Error **errp) s->vq_aio_context =3D g_new(AioContext *, conf->num_queues); =20 if (conf->iothread_vq_mapping_list) { - if (!apply_iothread_vq_mapping(conf->iothread_vq_mapping_list, + if (!iothread_vq_mapping_apply(conf->iothread_vq_mapping_list, s->vq_aio_context, conf->num_queues, errp)) { @@ -1631,7 +1630,7 @@ static void virtio_blk_vq_aio_context_cleanup(VirtIOB= lock *s) assert(!s->ioeventfd_started); =20 if (conf->iothread_vq_mapping_list) { - cleanup_iothread_vq_mapping(conf->iothread_vq_mapping_list); + iothread_vq_mapping_cleanup(conf->iothread_vq_mapping_list); } =20 if (conf->iothread) { --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699739; cv=none; d=zohomail.com; s=zohoarc; b=M58P8P11UdhpaGj/ubGa0djF9VKV4FtkAhfk3MEE/VrXeEg8YegWyf3WoqO+L22qqpdCHvNXH2Il1C5xCzG5291k8nxU1mOEicf7UFu03ATku5z0S6LD8TRfowQpF/E1+yjEEGJ81yflVrftsPx4tYYBWDOEOkKNmI7ENh2fYnY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699739; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=1pCN17umfgpCN1HAoUixnFiYVxtWYNvop2Aer+fkivc=; b=kUwrdNVpbydOyFerwAzgNOfwgnshgPfxNRVsijg13WS8eqsFoDcxrkKSlElERfDqVeKEahRmNsiB0KUaegBQvIv3SmHUcE3kK9lchblM50sPOaInOV0z9vmAf1AlCyjbcTdFg7fULLvyTviBePMoHJLPbDSTqjG89qO3Sw7VIqg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699738993625.3147050857855; Tue, 11 Mar 2025 06:28:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzee-0003um-TP; Tue, 11 Mar 2025 09:28:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzde-0002mS-Dr for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdb-0005JT-R1 for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:18 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-308-7Mk664qdM7S3LJAtKm383A-1; Tue, 11 Mar 2025 09:27:10 -0400 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5BC6D18001E1; Tue, 11 Mar 2025 13:27:09 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D738D19560AB; Tue, 11 Mar 2025 13:27:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1pCN17umfgpCN1HAoUixnFiYVxtWYNvop2Aer+fkivc=; b=Du+uJIGcFMVr6zB35outplpgaMQXVy9DJJzQHJAaUTGwg4OIDv7hoeh9tv7LHdx/sq5H72 7rG9+4a0uxd7HEkbUQqV9u15drCJP61pN4fkWC0jGRIuNps9paPx85rKUarXuGhxaJ8qKB gTgIAakbpiP/aS2XHZqU4LUd8uU8kzA= X-MC-Unique: 7Mk664qdM7S3LJAtKm383A-1 X-Mimecast-MFC-AGG-ID: 7Mk664qdM7S3LJAtKm383A_1741699629 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 10/13] virtio: extract iothread-vq-mapping.h API Date: Tue, 11 Mar 2025 21:26:13 +0800 Message-ID: <20250311132616.1049687-11-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699741170019000 Content-Type: text/plain; charset="utf-8" The code that builds an array of AioContext pointers indexed by the virtqueue is not specific to virtio-blk. virtio-scsi will need to do the same thing, so extract the functions. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf Tested-by: Peter Krempa --- include/hw/virtio/iothread-vq-mapping.h | 45 ++++++++ hw/block/virtio-blk.c | 142 +----------------------- hw/virtio/iothread-vq-mapping.c | 131 ++++++++++++++++++++++ hw/virtio/meson.build | 1 + 4 files changed, 178 insertions(+), 141 deletions(-) create mode 100644 include/hw/virtio/iothread-vq-mapping.h create mode 100644 hw/virtio/iothread-vq-mapping.c diff --git a/include/hw/virtio/iothread-vq-mapping.h b/include/hw/virtio/io= thread-vq-mapping.h new file mode 100644 index 0000000000..57335c3703 --- /dev/null +++ b/include/hw/virtio/iothread-vq-mapping.h @@ -0,0 +1,45 @@ +/* + * IOThread Virtqueue Mapping + * + * Copyright Red Hat, Inc + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef HW_VIRTIO_IOTHREAD_VQ_MAPPING_H +#define HW_VIRTIO_IOTHREAD_VQ_MAPPING_H + +#include "qapi/error.h" +#include "qapi/qapi-types-virtio.h" + +/** + * iothread_vq_mapping_apply: + * @list: The mapping of virtqueues to IOThreads. + * @vq_aio_context: The array of AioContext pointers to fill in. + * @num_queues: The length of @vq_aio_context. + * @errp: If an error occurs, a pointer to the area to store the error. + * + * Fill in the AioContext for each virtqueue in the @vq_aio_context array = given + * the iothread-vq-mapping parameter in @list. + * + * iothread_vq_mapping_cleanup() must be called to free IOThread object + * references after this function returns success. + * + * Returns: %true on success, %false on failure. + **/ +bool iothread_vq_mapping_apply( + IOThreadVirtQueueMappingList *list, + AioContext **vq_aio_context, + uint16_t num_queues, + Error **errp); + +/** + * iothread_vq_mapping_cleanup: + * @list: The mapping of virtqueues to IOThreads. + * + * Release IOThread object references that were acquired by + * iothread_vq_mapping_apply(). + */ +void iothread_vq_mapping_cleanup(IOThreadVirtQueueMappingList *list); + +#endif /* HW_VIRTIO_IOTHREAD_VQ_MAPPING_H */ diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 6bf7b50520..5077793e5e 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -33,6 +33,7 @@ #endif #include "hw/virtio/virtio-bus.h" #include "migration/qemu-file-types.h" +#include "hw/virtio/iothread-vq-mapping.h" #include "hw/virtio/virtio-access.h" #include "hw/virtio/virtio-blk-common.h" #include "qemu/coroutine.h" @@ -1423,147 +1424,6 @@ static const BlockDevOps virtio_block_ops =3D { .drained_end =3D virtio_blk_drained_end, }; =20 -static bool -iothread_vq_mapping_validate(IOThreadVirtQueueMappingList *list, uint16_t - num_queues, Error **errp) -{ - g_autofree unsigned long *vqs =3D bitmap_new(num_queues); - g_autoptr(GHashTable) iothreads =3D - g_hash_table_new(g_str_hash, g_str_equal); - - for (IOThreadVirtQueueMappingList *node =3D list; node; node =3D node-= >next) { - const char *name =3D node->value->iothread; - uint16List *vq; - - if (!iothread_by_id(name)) { - error_setg(errp, "IOThread \"%s\" object does not exist", name= ); - return false; - } - - if (!g_hash_table_add(iothreads, (gpointer)name)) { - error_setg(errp, - "duplicate IOThread name \"%s\" in iothread-vq-mapping= ", - name); - return false; - } - - if (node !=3D list) { - if (!!node->value->vqs !=3D !!list->value->vqs) { - error_setg(errp, "either all items in iothread-vq-mapping " - "must have vqs or none of them must have = it"); - return false; - } - } - - for (vq =3D node->value->vqs; vq; vq =3D vq->next) { - if (vq->value >=3D num_queues) { - error_setg(errp, "vq index %u for IOThread \"%s\" must be " - "less than num_queues %u in iothread-vq-mapping", - vq->value, name, num_queues); - return false; - } - - if (test_and_set_bit(vq->value, vqs)) { - error_setg(errp, "cannot assign vq %u to IOThread \"%s\" " - "because it is already assigned", vq->value, name); - return false; - } - } - } - - if (list->value->vqs) { - for (uint16_t i =3D 0; i < num_queues; i++) { - if (!test_bit(i, vqs)) { - error_setg(errp, - "missing vq %u IOThread assignment in iothread-vq-= mapping", - i); - return false; - } - } - } - - return true; -} - -/** - * iothread_vq_mapping_apply: - * @list: The mapping of virtqueues to IOThreads. - * @vq_aio_context: The array of AioContext pointers to fill in. - * @num_queues: The length of @vq_aio_context. - * @errp: If an error occurs, a pointer to the area to store the error. - * - * Fill in the AioContext for each virtqueue in the @vq_aio_context array = given - * the iothread-vq-mapping parameter in @list. - * - * iothread_vq_mapping_cleanup() must be called to free IOThread object - * references after this function returns success. - * - * Returns: %true on success, %false on failure. - **/ -static bool iothread_vq_mapping_apply( - IOThreadVirtQueueMappingList *list, - AioContext **vq_aio_context, - uint16_t num_queues, - Error **errp) -{ - IOThreadVirtQueueMappingList *node; - size_t num_iothreads =3D 0; - size_t cur_iothread =3D 0; - - if (!iothread_vq_mapping_validate(list, num_queues, errp)) { - return false; - } - - for (node =3D list; node; node =3D node->next) { - num_iothreads++; - } - - for (node =3D list; node; node =3D node->next) { - IOThread *iothread =3D iothread_by_id(node->value->iothread); - AioContext *ctx =3D iothread_get_aio_context(iothread); - - /* Released in virtio_blk_vq_aio_context_cleanup() */ - object_ref(OBJECT(iothread)); - - if (node->value->vqs) { - uint16List *vq; - - /* Explicit vq:IOThread assignment */ - for (vq =3D node->value->vqs; vq; vq =3D vq->next) { - assert(vq->value < num_queues); - vq_aio_context[vq->value] =3D ctx; - } - } else { - /* Round-robin vq:IOThread assignment */ - for (unsigned i =3D cur_iothread; i < num_queues; - i +=3D num_iothreads) { - vq_aio_context[i] =3D ctx; - } - } - - cur_iothread++; - } - - return true; -} - -/** - * iothread_vq_mapping_cleanup: - * @list: The mapping of virtqueues to IOThreads. - * - * Release IOThread object references that were acquired by - * iothread_vq_mapping_apply(). - */ -static void iothread_vq_mapping_cleanup(IOThreadVirtQueueMappingList *list) -{ - IOThreadVirtQueueMappingList *node; - - for (node =3D list; node; node =3D node->next) { - IOThread *iothread =3D iothread_by_id(node->value->iothread); - object_unref(OBJECT(iothread)); - } -} - /* Context: BQL held */ static bool virtio_blk_vq_aio_context_init(VirtIOBlock *s, Error **errp) { diff --git a/hw/virtio/iothread-vq-mapping.c b/hw/virtio/iothread-vq-mappin= g.c new file mode 100644 index 0000000000..15909eb933 --- /dev/null +++ b/hw/virtio/iothread-vq-mapping.c @@ -0,0 +1,131 @@ +/* + * IOThread Virtqueue Mapping + * + * Copyright Red Hat, Inc + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#include "qemu/osdep.h" +#include "system/iothread.h" +#include "hw/virtio/iothread-vq-mapping.h" + +static bool +iothread_vq_mapping_validate(IOThreadVirtQueueMappingList *list, uint16_t + num_queues, Error **errp) +{ + g_autofree unsigned long *vqs =3D bitmap_new(num_queues); + g_autoptr(GHashTable) iothreads =3D + g_hash_table_new(g_str_hash, g_str_equal); + + for (IOThreadVirtQueueMappingList *node =3D list; node; node =3D node-= >next) { + const char *name =3D node->value->iothread; + uint16List *vq; + + if (!iothread_by_id(name)) { + error_setg(errp, "IOThread \"%s\" object does not exist", name= ); + return false; + } + + if (!g_hash_table_add(iothreads, (gpointer)name)) { + error_setg(errp, + "duplicate IOThread name \"%s\" in iothread-vq-mapping= ", + name); + return false; + } + + if (node !=3D list) { + if (!!node->value->vqs !=3D !!list->value->vqs) { + error_setg(errp, "either all items in iothread-vq-mapping " + "must have vqs or none of them must have = it"); + return false; + } + } + + for (vq =3D node->value->vqs; vq; vq =3D vq->next) { + if (vq->value >=3D num_queues) { + error_setg(errp, "vq index %u for IOThread \"%s\" must be " + "less than num_queues %u in iothread-vq-mapping", + vq->value, name, num_queues); + return false; + } + + if (test_and_set_bit(vq->value, vqs)) { + error_setg(errp, "cannot assign vq %u to IOThread \"%s\" " + "because it is already assigned", vq->value, name); + return false; + } + } + } + + if (list->value->vqs) { + for (uint16_t i =3D 0; i < num_queues; i++) { + if (!test_bit(i, vqs)) { + error_setg(errp, + "missing vq %u IOThread assignment in iothread-vq-= mapping", + i); + return false; + } + } + } + + return true; +} + +bool iothread_vq_mapping_apply( + IOThreadVirtQueueMappingList *list, + AioContext **vq_aio_context, + uint16_t num_queues, + Error **errp) +{ + IOThreadVirtQueueMappingList *node; + size_t num_iothreads =3D 0; + size_t cur_iothread =3D 0; + + if (!iothread_vq_mapping_validate(list, num_queues, errp)) { + return false; + } + + for (node =3D list; node; node =3D node->next) { + num_iothreads++; + } + + for (node =3D list; node; node =3D node->next) { + IOThread *iothread =3D iothread_by_id(node->value->iothread); + AioContext *ctx =3D iothread_get_aio_context(iothread); + + /* Released in virtio_blk_vq_aio_context_cleanup() */ + object_ref(OBJECT(iothread)); + + if (node->value->vqs) { + uint16List *vq; + + /* Explicit vq:IOThread assignment */ + for (vq =3D node->value->vqs; vq; vq =3D vq->next) { + assert(vq->value < num_queues); + vq_aio_context[vq->value] =3D ctx; + } + } else { + /* Round-robin vq:IOThread assignment */ + for (unsigned i =3D cur_iothread; i < num_queues; + i +=3D num_iothreads) { + vq_aio_context[i] =3D ctx; + } + } + + cur_iothread++; + } + + return true; +} + +void iothread_vq_mapping_cleanup(IOThreadVirtQueueMappingList *list) +{ + IOThreadVirtQueueMappingList *node; + + for (node =3D list; node; node =3D node->next) { + IOThread *iothread =3D iothread_by_id(node->value->iothread); + object_unref(OBJECT(iothread)); + } +} + diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index a5f9f7999d..19b04c4d9c 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -1,5 +1,6 @@ system_virtio_ss =3D ss.source_set() system_virtio_ss.add(files('virtio-bus.c')) +system_virtio_ss.add(files('iothread-vq-mapping.c')) system_virtio_ss.add(when: 'CONFIG_VIRTIO_PCI', if_true: files('virtio-pci= .c')) system_virtio_ss.add(when: 'CONFIG_VIRTIO_MMIO', if_true: files('virtio-mm= io.c')) system_virtio_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('virtio-= crypto.c')) --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699863; cv=none; d=zohomail.com; s=zohoarc; b=Nclls/YGlb9W1IDnJxxWgjCYxLxDt8yFBWN/+A3ZVbwL99OI50C5+00KKGKICMTvFGkmXkgVLFUTOhUnuPrarSlKICruYlWsWar9Rd+ugSPCN5ck4fYT2oO+TB6OW/S6Ll02tx1TF4nw+MtVbzjUn5z2nDU+lBNqebhErgF8Uro= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699863; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fV0JIVBXuvMhoH8sIqb5X136kajioqVDUIP6dv4zU1I=; b=S753+JFPc0+jt46INyfsj2MgM5cXkr7Iy0ndyu/6Rw3idFdWW+pvk/qjTXcNMmNa03s2/3utEBfQ/fBoa63wf70ukvzWk5842JMCa6AnEUD4v6HNtobXJs2CGpuaL4wKJ7VfcyfrJF5pqkI5EglPCMP1q3nlRbpj2Qu2bXnFW2Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699863352150.68860096007802; Tue, 11 Mar 2025 06:31:03 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzen-0004sg-9L; Tue, 11 Mar 2025 09:28:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdl-0002qh-Nt for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdh-0005Kb-MO for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:24 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-663-v4c7DUTUMXSFOxulEdO9VQ-1; Tue, 11 Mar 2025 09:27:14 -0400 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 625701800A3A; Tue, 11 Mar 2025 13:27:13 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 76DC619560AB; Tue, 11 Mar 2025 13:27:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699637; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fV0JIVBXuvMhoH8sIqb5X136kajioqVDUIP6dv4zU1I=; b=adb0eOmXvnODb5ApKbJE5xamGTs9Z1Ma/2vpYmcGbNI9+5HLLrH4X7xPSUe3n87muCMsXh 0OrkMukXR/iiI/SlX79FZnxTeK/eSos+fdZIBZjvmFJLET18eUqBvWj/93Qnx2fpKSDDwt HsYJkufROvBanIy6kdUd3QDKLHULX88= X-MC-Unique: v4c7DUTUMXSFOxulEdO9VQ-1 X-Mimecast-MFC-AGG-ID: v4c7DUTUMXSFOxulEdO9VQ_1741699633 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 11/13] virtio-scsi: add iothread-vq-mapping parameter Date: Tue, 11 Mar 2025 21:26:14 +0800 Message-ID: <20250311132616.1049687-12-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699865131019100 Content-Type: text/plain; charset="utf-8" Allow virtio-scsi virtqueues to be assigned to different IOThreads. This makes it possible to take advantage of host multi-queue block layer scalability by assigning virtqueues that have affinity with vCPUs to different IOThreads that have affinity with host CPUs. The same feature was introduced for virtio-blk in the past: https://developers.redhat.com/articles/2024/09/05/scaling-virtio-blk-disk-i= o-iothread-virtqueue-mapping Here are fio randread 4k iodepth=3D64 results from a 4 vCPU guest with an Intel P4800X SSD: iothreads IOPS Tested-by: Peter Krempa ------------------------------ 1 189576 2 312698 4 346744 Signed-off-by: Stefan Hajnoczi --- include/hw/virtio/virtio-scsi.h | 5 +- hw/scsi/virtio-scsi-dataplane.c | 90 ++++++++++++++++++++++++--------- hw/scsi/virtio-scsi.c | 63 ++++++++++++++--------- 3 files changed, 107 insertions(+), 51 deletions(-) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scs= i.h index 7b7e3ced7a..086201efa2 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -22,6 +22,7 @@ #include "hw/virtio/virtio.h" #include "hw/scsi/scsi.h" #include "chardev/char-fe.h" +#include "qapi/qapi-types-virtio.h" #include "system/iothread.h" =20 #define TYPE_VIRTIO_SCSI_COMMON "virtio-scsi-common" @@ -60,6 +61,7 @@ struct VirtIOSCSIConf { CharBackend chardev; uint32_t boot_tpgt; IOThread *iothread; + IOThreadVirtQueueMappingList *iothread_vq_mapping_list; }; =20 struct VirtIOSCSI; @@ -97,7 +99,7 @@ struct VirtIOSCSI { QTAILQ_HEAD(, VirtIOSCSIReq) tmf_bh_list; =20 /* Fields for dataplane below */ - AioContext *ctx; /* one iothread per virtio-scsi-pci for now */ + AioContext **vq_aio_context; /* per-virtqueue AioContext pointer */ =20 bool dataplane_started; bool dataplane_starting; @@ -115,6 +117,7 @@ void virtio_scsi_common_realize(DeviceState *dev, void virtio_scsi_common_unrealize(DeviceState *dev); =20 void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp); +void virtio_scsi_dataplane_cleanup(VirtIOSCSI *s); int virtio_scsi_dataplane_start(VirtIODevice *s); void virtio_scsi_dataplane_stop(VirtIODevice *s); =20 diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplan= e.c index f49ab98ecc..6bb368c8a5 100644 --- a/hw/scsi/virtio-scsi-dataplane.c +++ b/hw/scsi/virtio-scsi-dataplane.c @@ -18,6 +18,7 @@ #include "system/block-backend.h" #include "hw/scsi/scsi.h" #include "scsi/constants.h" +#include "hw/virtio/iothread-vq-mapping.h" #include "hw/virtio/virtio-bus.h" =20 /* Context: BQL held */ @@ -27,8 +28,16 @@ void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **= errp) VirtIODevice *vdev =3D VIRTIO_DEVICE(s); BusState *qbus =3D qdev_get_parent_bus(DEVICE(vdev)); VirtioBusClass *k =3D VIRTIO_BUS_GET_CLASS(qbus); + uint16_t num_vqs =3D vs->conf.num_queues + VIRTIO_SCSI_VQ_NUM_FIXED; =20 - if (vs->conf.iothread) { + if (vs->conf.iothread && vs->conf.iothread_vq_mapping_list) { + error_setg(errp, + "iothread and iothread-vq-mapping properties cannot be = set " + "at the same time"); + return; + } + + if (vs->conf.iothread || vs->conf.iothread_vq_mapping_list) { if (!k->set_guest_notifiers || !k->ioeventfd_assign) { error_setg(errp, "device is incompatible with iothread " @@ -39,15 +48,50 @@ void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error *= *errp) error_setg(errp, "ioeventfd is required for iothread"); return; } - s->ctx =3D iothread_get_aio_context(vs->conf.iothread); - } else { - if (!virtio_device_ioeventfd_enabled(vdev)) { + } + + s->vq_aio_context =3D g_new(AioContext *, num_vqs); + + if (vs->conf.iothread_vq_mapping_list) { + if (!iothread_vq_mapping_apply(vs->conf.iothread_vq_mapping_list, + s->vq_aio_context, num_vqs, errp)) { + g_free(s->vq_aio_context); + s->vq_aio_context =3D NULL; return; } - s->ctx =3D qemu_get_aio_context(); + } else if (vs->conf.iothread) { + AioContext *ctx =3D iothread_get_aio_context(vs->conf.iothread); + for (uint16_t i =3D 0; i < num_vqs; i++) { + s->vq_aio_context[i] =3D ctx; + } + + /* Released in virtio_scsi_dataplane_cleanup() */ + object_ref(OBJECT(vs->conf.iothread)); + } else { + AioContext *ctx =3D qemu_get_aio_context(); + for (unsigned i =3D 0; i < num_vqs; i++) { + s->vq_aio_context[i] =3D ctx; + } } } =20 +/* Context: BQL held */ +void virtio_scsi_dataplane_cleanup(VirtIOSCSI *s) +{ + VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(s); + + if (vs->conf.iothread_vq_mapping_list) { + iothread_vq_mapping_cleanup(vs->conf.iothread_vq_mapping_list); + } + + if (vs->conf.iothread) { + object_unref(OBJECT(vs->conf.iothread)); + } + + g_free(s->vq_aio_context); + s->vq_aio_context =3D NULL; +} + static int virtio_scsi_set_host_notifier(VirtIOSCSI *s, VirtQueue *vq, int= n) { BusState *qbus =3D BUS(qdev_get_parent_bus(DEVICE(s))); @@ -66,31 +110,20 @@ static int virtio_scsi_set_host_notifier(VirtIOSCSI *s= , VirtQueue *vq, int n) } =20 /* Context: BH in IOThread */ -static void virtio_scsi_dataplane_stop_bh(void *opaque) +static void virtio_scsi_dataplane_stop_vq_bh(void *opaque) { - VirtIOSCSI *s =3D opaque; - VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(s); + AioContext *ctx =3D qemu_get_current_aio_context(); + VirtQueue *vq =3D opaque; EventNotifier *host_notifier; - int i; =20 - virtio_queue_aio_detach_host_notifier(vs->ctrl_vq, s->ctx); - host_notifier =3D virtio_queue_get_host_notifier(vs->ctrl_vq); + virtio_queue_aio_detach_host_notifier(vq, ctx); + host_notifier =3D virtio_queue_get_host_notifier(vq); =20 /* * Test and clear notifier after disabling event, in case poll callback * didn't have time to run. */ virtio_queue_host_notifier_read(host_notifier); - - virtio_queue_aio_detach_host_notifier(vs->event_vq, s->ctx); - host_notifier =3D virtio_queue_get_host_notifier(vs->event_vq); - virtio_queue_host_notifier_read(host_notifier); - - for (i =3D 0; i < vs->conf.num_queues; i++) { - virtio_queue_aio_detach_host_notifier(vs->cmd_vqs[i], s->ctx); - host_notifier =3D virtio_queue_get_host_notifier(vs->cmd_vqs[i]); - virtio_queue_host_notifier_read(host_notifier); - } } =20 /* Context: BQL held */ @@ -154,11 +187,14 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev) smp_wmb(); /* paired with aio_notify_accept() */ =20 if (s->bus.drain_count =3D=3D 0) { - virtio_queue_aio_attach_host_notifier(vs->ctrl_vq, s->ctx); - virtio_queue_aio_attach_host_notifier_no_poll(vs->event_vq, s->ctx= ); + virtio_queue_aio_attach_host_notifier(vs->ctrl_vq, + s->vq_aio_context[0]); + virtio_queue_aio_attach_host_notifier_no_poll(vs->event_vq, + s->vq_aio_context[1]= ); =20 for (i =3D 0; i < vs->conf.num_queues; i++) { - virtio_queue_aio_attach_host_notifier(vs->cmd_vqs[i], s->ctx); + AioContext *ctx =3D s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED= + i]; + virtio_queue_aio_attach_host_notifier(vs->cmd_vqs[i], ctx); } } return 0; @@ -207,7 +243,11 @@ void virtio_scsi_dataplane_stop(VirtIODevice *vdev) s->dataplane_stopping =3D true; =20 if (s->bus.drain_count =3D=3D 0) { - aio_wait_bh_oneshot(s->ctx, virtio_scsi_dataplane_stop_bh, s); + for (i =3D 0; i < vs->conf.num_queues + VIRTIO_SCSI_VQ_NUM_FIXED; = i++) { + VirtQueue *vq =3D virtio_get_queue(&vs->parent_obj, i); + AioContext *ctx =3D s->vq_aio_context[i]; + aio_wait_bh_oneshot(ctx, virtio_scsi_dataplane_stop_vq_bh, vq); + } } =20 blk_drain_all(); /* ensure there are no in-flight requests */ diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 2045d27289..9f61eb97db 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -27,6 +27,7 @@ #include "hw/qdev-properties.h" #include "hw/scsi/scsi.h" #include "scsi/constants.h" +#include "hw/virtio/iothread-vq-mapping.h" #include "hw/virtio/virtio-bus.h" #include "hw/virtio/virtio-access.h" #include "trace.h" @@ -318,13 +319,6 @@ static void virtio_scsi_cancel_notify(Notifier *notifi= er, void *data) g_free(n); } =20 -static inline void virtio_scsi_ctx_check(VirtIOSCSI *s, SCSIDevice *d) -{ - if (s->dataplane_started && d && blk_is_available(d->conf.blk)) { - assert(blk_get_aio_context(d->conf.blk) =3D=3D s->ctx); - } -} - static void virtio_scsi_do_one_tmf_bh(VirtIOSCSIReq *req) { VirtIOSCSI *s =3D req->dev; @@ -517,9 +511,11 @@ static void virtio_scsi_flush_defer_tmf_to_aio_context= (VirtIOSCSI *s) =20 assert(!s->dataplane_started); =20 - if (s->ctx) { + for (uint32_t i =3D 0; i < s->parent_obj.conf.num_queues; i++) { + AioContext *ctx =3D s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED + i= ]; + /* Our BH only runs after previously scheduled BHs */ - aio_wait_bh_oneshot(s->ctx, dummy_bh, NULL); + aio_wait_bh_oneshot(ctx, dummy_bh, NULL); } } =20 @@ -575,7 +571,6 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSCSI= Req *req) AioContext *ctx; int ret =3D 0; =20 - virtio_scsi_ctx_check(s, d); /* Here VIRTIO_SCSI_S_OK means "FUNCTION COMPLETE". */ req->resp.tmf.response =3D VIRTIO_SCSI_S_OK; =20 @@ -639,6 +634,8 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSCSI= Req *req) =20 case VIRTIO_SCSI_T_TMF_ABORT_TASK_SET: case VIRTIO_SCSI_T_TMF_CLEAR_TASK_SET: { + g_autoptr(GHashTable) aio_contexts =3D g_hash_table_new(NULL, NULL= ); + if (!d) { goto fail; } @@ -648,8 +645,15 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSCS= IReq *req) =20 qatomic_inc(&req->remaining); =20 - ctx =3D s->ctx ?: qemu_get_aio_context(); - virtio_scsi_defer_tmf_to_aio_context(req, ctx); + for (uint32_t i =3D 0; i < s->parent_obj.conf.num_queues; i++) { + ctx =3D s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED + i]; + + if (!g_hash_table_add(aio_contexts, ctx)) { + continue; /* skip previously added AioContext */ + } + + virtio_scsi_defer_tmf_to_aio_context(req, ctx); + } =20 virtio_scsi_tmf_dec_remaining(req); ret =3D -EINPROGRESS; @@ -770,9 +774,12 @@ static void virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, = VirtQueue *vq) */ static bool virtio_scsi_defer_to_dataplane(VirtIOSCSI *s) { - if (!s->ctx || s->dataplane_started) { + if (s->dataplane_started) { return false; } + if (s->vq_aio_context[0] =3D=3D qemu_get_aio_context()) { + return false; /* not using IOThreads */ + } =20 virtio_device_start_ioeventfd(&s->parent_obj.parent_obj); return !s->dataplane_fenced; @@ -946,7 +953,6 @@ static int virtio_scsi_handle_cmd_req_prepare(VirtIOSCS= I *s, VirtIOSCSIReq *req) virtio_scsi_complete_cmd_req(req); return -ENOENT; } - virtio_scsi_ctx_check(s, d); req->sreq =3D scsi_req_new(d, req->req.cmd.tag, virtio_scsi_get_lun(req->req.cmd.lun), req->req.cmd.cdb, vs->cdb_size, req); @@ -1218,14 +1224,16 @@ static void virtio_scsi_hotplug(HotplugHandler *hot= plug_dev, DeviceState *dev, { VirtIODevice *vdev =3D VIRTIO_DEVICE(hotplug_dev); VirtIOSCSI *s =3D VIRTIO_SCSI(vdev); + AioContext *ctx =3D s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED]; SCSIDevice *sd =3D SCSI_DEVICE(dev); - int ret; =20 - if (s->ctx && !s->dataplane_fenced) { - ret =3D blk_set_aio_context(sd->conf.blk, s->ctx, errp); - if (ret < 0) { - return; - } + if (ctx !=3D qemu_get_aio_context() && !s->dataplane_fenced) { + /* + * Try to make the BlockBackend's AioContext match ours. Ignore fa= ilure + * because I/O will still work although block jobs and other users + * might be slower when multiple AioContexts use a BlockBackend. + */ + blk_set_aio_context(sd->conf.blk, ctx, errp); } =20 if (virtio_vdev_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG)) { @@ -1260,7 +1268,7 @@ static void virtio_scsi_hotunplug(HotplugHandler *hot= plug_dev, DeviceState *dev, =20 qdev_simple_device_unplug_cb(hotplug_dev, dev, errp); =20 - if (s->ctx) { + if (s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED] !=3D qemu_get_aio_cont= ext()) { /* If other users keep the BlockBackend in the iothread, that's ok= */ blk_set_aio_context(sd->conf.blk, qemu_get_aio_context(), NULL); } @@ -1294,7 +1302,7 @@ static void virtio_scsi_drained_begin(SCSIBus *bus) =20 for (uint32_t i =3D 0; i < total_queues; i++) { VirtQueue *vq =3D virtio_get_queue(vdev, i); - virtio_queue_aio_detach_host_notifier(vq, s->ctx); + virtio_queue_aio_detach_host_notifier(vq, s->vq_aio_context[i]); } } =20 @@ -1320,10 +1328,12 @@ static void virtio_scsi_drained_end(SCSIBus *bus) =20 for (uint32_t i =3D 0; i < total_queues; i++) { VirtQueue *vq =3D virtio_get_queue(vdev, i); + AioContext *ctx =3D s->vq_aio_context[i]; + if (vq =3D=3D vs->event_vq) { - virtio_queue_aio_attach_host_notifier_no_poll(vq, s->ctx); + virtio_queue_aio_attach_host_notifier_no_poll(vq, ctx); } else { - virtio_queue_aio_attach_host_notifier(vq, s->ctx); + virtio_queue_aio_attach_host_notifier(vq, ctx); } } } @@ -1430,12 +1440,13 @@ void virtio_scsi_common_unrealize(DeviceState *dev) virtio_cleanup(vdev); } =20 +/* main loop */ static void virtio_scsi_device_unrealize(DeviceState *dev) { VirtIOSCSI *s =3D VIRTIO_SCSI(dev); =20 virtio_scsi_reset_tmf_bh(s); - + virtio_scsi_dataplane_cleanup(s); qbus_set_hotplug_handler(BUS(&s->bus), NULL); virtio_scsi_common_unrealize(dev); qemu_mutex_destroy(&s->tmf_bh_lock); @@ -1460,6 +1471,8 @@ static const Property virtio_scsi_properties[] =3D { VIRTIO_SCSI_F_CHANGE, true= ), DEFINE_PROP_LINK("iothread", VirtIOSCSI, parent_obj.conf.iothread, TYPE_IOTHREAD, IOThread *), + DEFINE_PROP_IOTHREAD_VQ_MAPPING_LIST("iothread-vq-mapping", VirtIOSCSI, + parent_obj.conf.iothread_vq_mapping_list), }; =20 static const VMStateDescription vmstate_virtio_scsi =3D { --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699918; cv=none; d=zohomail.com; s=zohoarc; b=T9HppRDnDOTO1Tt3YVUKXED8RYwtZufnPAFnY7PdXya6RFoxjwmBdjEjoVuWEAiUBj/2NygTRt7cfJdIA5Bp8s1c4SBSZDvd0nHCmXF2QTpnVxzi6yVxBZQH7UYpB1Si/9BOzC8vQMm/LX4cV0XR6OmQPuedtksW9Ua3PE5R12Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699918; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=w9m9I6ZVlZyAekweXONgksNVJiGxrQd+tXwkplLtEsg=; b=C0211GpDyjD6KQxKM3BBx05cMCVeLk5Nw/YozCIkah/x3Z6nuAs7FI/QvyFlRYtCrIQMLMmIoIXqtd2KkklZCRJShnf8R4+EeMtiqIEiWTKWu0l2Vo9T34GVfGEYwdZqay85o4kk6NvRzzSIONuyY6N5sgEjV3y63IPFAPOBoTc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 174169991806843.461581685499596; Tue, 11 Mar 2025 06:31:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzeq-00056h-Vc; Tue, 11 Mar 2025 09:28:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdt-0002s7-7Y for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdl-0005Ln-Gx for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:30 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-141-I5tcQ69NP8Wb3T6S2P9YgA-1; Tue, 11 Mar 2025 09:27:20 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3CA58180AF49; Tue, 11 Mar 2025 13:27:18 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5CEC21955BCB; Tue, 11 Mar 2025 13:27:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699643; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w9m9I6ZVlZyAekweXONgksNVJiGxrQd+tXwkplLtEsg=; b=BL0SL8hbMpf5lgqm9WmzIULTEMkTlKSx9mIMrE7nWbHWWWvuCj7m1TQTBujON1phe2RhUn nKwXvplhLPJ3HmTU67xAGMQhY4KW1Us25K8UAvRKPlqIeOuBlJe0uF3QVqMi7b0uv6kIZI r2EZdbEf3nLgqfc/Uj4ALyGUP/12UE8= X-MC-Unique: I5tcQ69NP8Wb3T6S2P9YgA-1 X-Mimecast-MFC-AGG-ID: I5tcQ69NP8Wb3T6S2P9YgA_1741699638 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 12/13] virtio-scsi: handle ctrl virtqueue in main loop Date: Tue, 11 Mar 2025 21:26:15 +0800 Message-ID: <20250311132616.1049687-13-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699918443019000 Content-Type: text/plain; charset="utf-8" Previously the ctrl virtqueue was handled in the AioContext where SCSI requests are processed. When IOThread Virtqueue Mapping was added things become more complicated because SCSI requests could run in other AioContexts. Simplify by handling the ctrl virtqueue in the main loop where reset operations can be performed. Note that BHs are still used canceling SCSI requests in their AioContexts but at least the mean loop activity doesn't need BHs anymore. Signed-off-by: Stefan Hajnoczi Tested-by: Peter Krempa --- include/hw/virtio/virtio-scsi.h | 8 -- hw/scsi/virtio-scsi-dataplane.c | 6 ++ hw/scsi/virtio-scsi.c | 144 ++++++-------------------------- 3 files changed, 33 insertions(+), 125 deletions(-) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scs= i.h index 086201efa2..31e852ed6c 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -90,14 +90,6 @@ struct VirtIOSCSI { =20 QemuMutex ctrl_lock; /* protects ctrl_vq */ =20 - /* - * TMFs deferred to main loop BH. These fields are protected by - * tmf_bh_lock. - */ - QemuMutex tmf_bh_lock; - QEMUBH *tmf_bh; - QTAILQ_HEAD(, VirtIOSCSIReq) tmf_bh_list; - /* Fields for dataplane below */ AioContext **vq_aio_context; /* per-virtqueue AioContext pointer */ =20 diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplan= e.c index 6bb368c8a5..2d37fa6712 100644 --- a/hw/scsi/virtio-scsi-dataplane.c +++ b/hw/scsi/virtio-scsi-dataplane.c @@ -73,6 +73,12 @@ void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **= errp) s->vq_aio_context[i] =3D ctx; } } + + /* + * Always handle the ctrl virtqueue in the main loop thread where devi= ce + * resets can be performed. + */ + s->vq_aio_context[0] =3D qemu_get_aio_context(); } =20 /* Context: BQL held */ diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 9f61eb97db..d9c32c3245 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -319,115 +319,6 @@ static void virtio_scsi_cancel_notify(Notifier *notif= ier, void *data) g_free(n); } =20 -static void virtio_scsi_do_one_tmf_bh(VirtIOSCSIReq *req) -{ - VirtIOSCSI *s =3D req->dev; - SCSIDevice *d =3D virtio_scsi_device_get(s, req->req.tmf.lun); - BusChild *kid; - int target; - - switch (req->req.tmf.subtype) { - case VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET: - if (!d) { - req->resp.tmf.response =3D VIRTIO_SCSI_S_BAD_TARGET; - goto out; - } - if (d->lun !=3D virtio_scsi_get_lun(req->req.tmf.lun)) { - req->resp.tmf.response =3D VIRTIO_SCSI_S_INCORRECT_LUN; - goto out; - } - qatomic_inc(&s->resetting); - device_cold_reset(&d->qdev); - qatomic_dec(&s->resetting); - break; - - case VIRTIO_SCSI_T_TMF_I_T_NEXUS_RESET: - target =3D req->req.tmf.lun[1]; - qatomic_inc(&s->resetting); - - rcu_read_lock(); - QTAILQ_FOREACH_RCU(kid, &s->bus.qbus.children, sibling) { - SCSIDevice *d1 =3D SCSI_DEVICE(kid->child); - if (d1->channel =3D=3D 0 && d1->id =3D=3D target) { - device_cold_reset(&d1->qdev); - } - } - rcu_read_unlock(); - - qatomic_dec(&s->resetting); - break; - - default: - g_assert_not_reached(); - } - -out: - object_unref(OBJECT(d)); - virtio_scsi_complete_req(req, &s->ctrl_lock); -} - -/* Some TMFs must be processed from the main loop thread */ -static void virtio_scsi_do_tmf_bh(void *opaque) -{ - VirtIOSCSI *s =3D opaque; - QTAILQ_HEAD(, VirtIOSCSIReq) reqs =3D QTAILQ_HEAD_INITIALIZER(reqs); - VirtIOSCSIReq *req; - VirtIOSCSIReq *tmp; - - GLOBAL_STATE_CODE(); - - WITH_QEMU_LOCK_GUARD(&s->tmf_bh_lock) { - QTAILQ_FOREACH_SAFE(req, &s->tmf_bh_list, next, tmp) { - QTAILQ_REMOVE(&s->tmf_bh_list, req, next); - QTAILQ_INSERT_TAIL(&reqs, req, next); - } - - qemu_bh_delete(s->tmf_bh); - s->tmf_bh =3D NULL; - } - - QTAILQ_FOREACH_SAFE(req, &reqs, next, tmp) { - QTAILQ_REMOVE(&reqs, req, next); - virtio_scsi_do_one_tmf_bh(req); - } -} - -static void virtio_scsi_reset_tmf_bh(VirtIOSCSI *s) -{ - VirtIOSCSIReq *req; - VirtIOSCSIReq *tmp; - - GLOBAL_STATE_CODE(); - - /* Called after ioeventfd has been stopped, so tmf_bh_lock is not need= ed */ - if (s->tmf_bh) { - qemu_bh_delete(s->tmf_bh); - s->tmf_bh =3D NULL; - } - - QTAILQ_FOREACH_SAFE(req, &s->tmf_bh_list, next, tmp) { - QTAILQ_REMOVE(&s->tmf_bh_list, req, next); - - /* SAM-6 6.3.2 Hard reset */ - req->resp.tmf.response =3D VIRTIO_SCSI_S_TARGET_FAILURE; - virtio_scsi_complete_req(req, &req->dev->ctrl_lock); - } -} - -static void virtio_scsi_defer_tmf_to_main_loop(VirtIOSCSIReq *req) -{ - VirtIOSCSI *s =3D req->dev; - - WITH_QEMU_LOCK_GUARD(&s->tmf_bh_lock) { - QTAILQ_INSERT_TAIL(&s->tmf_bh_list, req, next); - - if (!s->tmf_bh) { - s->tmf_bh =3D qemu_bh_new(virtio_scsi_do_tmf_bh, s); - qemu_bh_schedule(s->tmf_bh); - } - } -} - static void virtio_scsi_tmf_cancel_req(VirtIOSCSIReq *tmf, SCSIRequest *r) { VirtIOSCSICancelNotifier *notifier; @@ -627,11 +518,35 @@ static int virtio_scsi_do_tmf(VirtIOSCSI *s, VirtIOSC= SIReq *req) break; =20 case VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET: - case VIRTIO_SCSI_T_TMF_I_T_NEXUS_RESET: - virtio_scsi_defer_tmf_to_main_loop(req); - ret =3D -EINPROGRESS; + if (!d) { + goto fail; + } + if (d->lun !=3D virtio_scsi_get_lun(req->req.tmf.lun)) { + goto incorrect_lun; + } + qatomic_inc(&s->resetting); + device_cold_reset(&d->qdev); + qatomic_dec(&s->resetting); break; =20 + case VIRTIO_SCSI_T_TMF_I_T_NEXUS_RESET: { + BusChild *kid; + int target =3D req->req.tmf.lun[1]; + qatomic_inc(&s->resetting); + + rcu_read_lock(); + QTAILQ_FOREACH_RCU(kid, &s->bus.qbus.children, sibling) { + SCSIDevice *d1 =3D SCSI_DEVICE(kid->child); + if (d1->channel =3D=3D 0 && d1->id =3D=3D target) { + device_cold_reset(&d1->qdev); + } + } + rcu_read_unlock(); + + qatomic_dec(&s->resetting); + break; + } + case VIRTIO_SCSI_T_TMF_ABORT_TASK_SET: case VIRTIO_SCSI_T_TMF_CLEAR_TASK_SET: { g_autoptr(GHashTable) aio_contexts =3D g_hash_table_new(NULL, NULL= ); @@ -1087,7 +1002,6 @@ static void virtio_scsi_reset(VirtIODevice *vdev) =20 assert(!s->dataplane_started); =20 - virtio_scsi_reset_tmf_bh(s); virtio_scsi_flush_defer_tmf_to_aio_context(s); =20 qatomic_inc(&s->resetting); @@ -1402,10 +1316,8 @@ static void virtio_scsi_device_realize(DeviceState *= dev, Error **errp) VirtIOSCSI *s =3D VIRTIO_SCSI(dev); Error *err =3D NULL; =20 - QTAILQ_INIT(&s->tmf_bh_list); qemu_mutex_init(&s->ctrl_lock); qemu_mutex_init(&s->event_lock); - qemu_mutex_init(&s->tmf_bh_lock); =20 virtio_scsi_common_realize(dev, virtio_scsi_handle_ctrl, @@ -1445,11 +1357,9 @@ static void virtio_scsi_device_unrealize(DeviceState= *dev) { VirtIOSCSI *s =3D VIRTIO_SCSI(dev); =20 - virtio_scsi_reset_tmf_bh(s); virtio_scsi_dataplane_cleanup(s); qbus_set_hotplug_handler(BUS(&s->bus), NULL); virtio_scsi_common_unrealize(dev); - qemu_mutex_destroy(&s->tmf_bh_lock); qemu_mutex_destroy(&s->event_lock); qemu_mutex_destroy(&s->ctrl_lock); } --=20 2.48.1 From nobody Thu Apr 3 10:06:31 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741699922; cv=none; d=zohomail.com; s=zohoarc; b=mmZq1k3w+H9uLj04hGDgsTfXkuPi/HqYggT1NxkpA+opzGP6/SwnUcZWQYelAyyH0p2YdLsVD4pQ+j5N0Trv0jko0JnBXh40jNpCvWzOKT/ghDZV1U16NH13Vdg9PfS88UST8hLanNuHXGePKE+J/+V7sEd/N/5L6GMtHuWGfXo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741699922; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DLBedgRZsFWvjgHM/ekL46tibum9+c7DeNR4LfA+/ng=; b=fW2M+XqJQpak+gJWnEraD6YBObmEE7u7VI3w2HkFB8M/Q3AYF7VItAE8MaaPdcymiJaaloMEx4QPOYVwKzzQytm249M4l0hyYMdafCQUDrLXUT6ZvM2SfBhwzFXj8NdPdnLQ7RwXBoigDXR7Tu0b/DgBI9CCxiF8nETo9aVtnpA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741699922288843.0340820744734; Tue, 11 Mar 2025 06:32:02 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trzek-0004Nw-2F; Tue, 11 Mar 2025 09:28:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdt-0002s8-7d for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trzdp-0005Mk-9D for qemu-devel@nongnu.org; Tue, 11 Mar 2025 09:27:31 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-593-Iub_z2EcPDCSevDEUinA-g-1; Tue, 11 Mar 2025 09:27:24 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EBD73180049D; Tue, 11 Mar 2025 13:27:22 +0000 (UTC) Received: from localhost (unknown [10.2.16.118]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2F3E51801747; Tue, 11 Mar 2025 13:27:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741699648; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DLBedgRZsFWvjgHM/ekL46tibum9+c7DeNR4LfA+/ng=; b=Ci+ojZlVWYIH1bU1LjHsk8Z2dk6DN2DQfbLpwTjnjMyvkXbHsxsNA6DxT2dJwbEt++MdqM E5tKHpmQvjNyxFTL5GDZA0dgxqjCZMr1/JxdIhvv8evZyv1JVNd/hu2d+aMapm5ACq05u/ CWhLBpTpr3QgDLZGEUL7Hsq8LhYxdgc= X-MC-Unique: Iub_z2EcPDCSevDEUinA-g-1 X-Mimecast-MFC-AGG-ID: Iub_z2EcPDCSevDEUinA-g_1741699643 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: John Snow , Paolo Bonzini , "Michael S. Tsirkin" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Kevin Wolf , pkrempa@redhat.com, Fam Zheng , , David Hildenbrand , Peter Xu , Hanna Reitz Subject: [PATCH v4 13/13] virtio-scsi: only expose cmd vqs via iothread-vq-mapping Date: Tue, 11 Mar 2025 21:26:16 +0800 Message-ID: <20250311132616.1049687-14-stefanha@redhat.com> In-Reply-To: <20250311132616.1049687-1-stefanha@redhat.com> References: <20250311132616.1049687-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741699922544019000 Content-Type: text/plain; charset="utf-8" Peter Krempa and Kevin Wolf observed that iothread-vq-mapping is confusing to use because the control and event virtqueues have a fixed location before the command virtqueues but need to be treated differently. Only expose the command virtqueues via iothread-vq-mapping so that the command-line parameter is intuitive: it controls where SCSI requests are processed. The control virtqueue needs to be hardcoded to the main loop thread for technical reasons anyway. Kevin also pointed out that it's better to place the event virtqueue in the main loop thread since its no poll behavior would prevent polling if assigned to an IOThread. This change is its own commit to avoid squashing the previous commit. Suggested-by: Kevin Wolf Suggested-by: Peter Krempa Signed-off-by: Stefan Hajnoczi Tested-by: Peter Krempa --- hw/scsi/virtio-scsi-dataplane.c | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplan= e.c index 2d37fa6712..95f13fb7c2 100644 --- a/hw/scsi/virtio-scsi-dataplane.c +++ b/hw/scsi/virtio-scsi-dataplane.c @@ -28,7 +28,6 @@ void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **e= rrp) VirtIODevice *vdev =3D VIRTIO_DEVICE(s); BusState *qbus =3D qdev_get_parent_bus(DEVICE(vdev)); VirtioBusClass *k =3D VIRTIO_BUS_GET_CLASS(qbus); - uint16_t num_vqs =3D vs->conf.num_queues + VIRTIO_SCSI_VQ_NUM_FIXED; =20 if (vs->conf.iothread && vs->conf.iothread_vq_mapping_list) { error_setg(errp, @@ -50,35 +49,43 @@ void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error *= *errp) } } =20 - s->vq_aio_context =3D g_new(AioContext *, num_vqs); + s->vq_aio_context =3D g_new(AioContext *, vs->conf.num_queues + + VIRTIO_SCSI_VQ_NUM_FIXED); + + /* + * Handle the ctrl virtqueue in the main loop thread where device rese= ts + * can be performed. + */ + s->vq_aio_context[0] =3D qemu_get_aio_context(); + + /* + * Handle the event virtqueue in the main loop thread where its no_poll + * behavior won't stop IOThread polling. + */ + s->vq_aio_context[1] =3D qemu_get_aio_context(); =20 if (vs->conf.iothread_vq_mapping_list) { if (!iothread_vq_mapping_apply(vs->conf.iothread_vq_mapping_list, - s->vq_aio_context, num_vqs, errp)) { + &s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED], + vs->conf.num_queues, errp)) { g_free(s->vq_aio_context); s->vq_aio_context =3D NULL; return; } } else if (vs->conf.iothread) { AioContext *ctx =3D iothread_get_aio_context(vs->conf.iothread); - for (uint16_t i =3D 0; i < num_vqs; i++) { - s->vq_aio_context[i] =3D ctx; + for (uint16_t i =3D 0; i < vs->conf.num_queues; i++) { + s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED + i] =3D ctx; } =20 /* Released in virtio_scsi_dataplane_cleanup() */ object_ref(OBJECT(vs->conf.iothread)); } else { AioContext *ctx =3D qemu_get_aio_context(); - for (unsigned i =3D 0; i < num_vqs; i++) { - s->vq_aio_context[i] =3D ctx; + for (unsigned i =3D 0; i < vs->conf.num_queues; i++) { + s->vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED + i] =3D ctx; } } - - /* - * Always handle the ctrl virtqueue in the main loop thread where devi= ce - * resets can be performed. - */ - s->vq_aio_context[0] =3D qemu_get_aio_context(); } =20 /* Context: BQL held */ --=20 2.48.1