From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896818; cv=none; d=zohomail.com; s=zohoarc; b=QgiFvP4q3KZ24s7EKxbabxhX9jMQ0rMwxm9UzvyWxDlOUZEE0MBmu0PBtm3fgEPtUHJ2rSTvuG8Dco1NUYmpTBHQTBxVNW1LDol6NMumlZur1S4lEqcy3GTgmnHw8c5n6hOOvcBGXKhSbvctkCXEUejp47/GChTBLtuHv0hTgrs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896818; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=d540xW7kcf/XZsdXpMIXfUvUMGaAPCvGJQycBSdnHGA=; b=IySlLMbA+/Tf0XMyajH6HLG3VgaNBSKw4Xv6bKtcrcxo6N+3/fWOx4QtDLxmYMWJDKaCKaxAF2EqTLbhfKsU2+TA0Ax+1HBZq9/rwC47eSjxF7/7SjlR76Nhe0gFqDMuJu0VRpWwepIqwZYgbkwitTkbFTpHfO3WIeB5YYo/IYQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896818928994.7678818526373; Tue, 11 Nov 2025 13:33:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzC-0006dw-9h; Tue, 11 Nov 2025 16:33:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzA-0006am-0M for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:08 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvz8-0001oP-JL for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:07 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-660-LaDJ8OmuM9mzwD-j5ZF1Og-1; Tue, 11 Nov 2025 16:33:02 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DC73C195608E; Tue, 11 Nov 2025 21:33:01 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2A9A730044E0; Tue, 11 Nov 2025 21:32:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896786; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d540xW7kcf/XZsdXpMIXfUvUMGaAPCvGJQycBSdnHGA=; b=JGHLoQuzLHs/tEuXXexk+QtcFVYnkILQvWffI71Wjr/e6aa4l49VKpATZwLIt6vp60CXFW OJ7iTKMGIQ8qf3OUaeQg9BwlEIT3kRorx00KEYvtyJvbUI/xQGfDkLnBg4jS1Ua0uQ4sNK w6PusKlC/nLF7TcBJdlcHeaL3nx1El0= X-MC-Unique: LaDJ8OmuM9mzwD-j5ZF1Og-1 X-Mimecast-MFC-AGG-ID: LaDJ8OmuM9mzwD-j5ZF1Og_1762896782 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 01/28] aio-posix: fix race between io_uring CQE and AioHandler deletion Date: Tue, 11 Nov 2025 22:32:11 +0100 Message-ID: <20251111213238.181992-2-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896820947153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi When an AioHandler is enqueued on ctx->submit_list for removal, the fill_sq_ring() function will submit an io_uring POLL_REMOVE operation to cancel the in-flight POLL_ADD operation. There is a race when another thread enqueues an AioHandler for deletion on ctx->submit_list when the POLL_ADD CQE has already appeared. In that case POLL_REMOVE is unnecessary. The code already handled this, but forgot that the AioHandler itself is still on ctx->submit_list when the POLL_ADD CQE is being processed. It's unsafe to delete the AioHandler at that point in time (use-after-free). Solve this problem by keeping the AioHandler alive but setting a flag so that it will be deleted by fill_sq_ring() when it runs. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Message-ID: <20251104022933.618123-2-stefanha@redhat.com> Signed-off-by: Kevin Wolf --- util/fdmon-io_uring.c | 33 ++++++++++++++++++++++++++------- 1 file changed, 26 insertions(+), 7 deletions(-) diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index b0d68bdc44..ad89160f31 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -52,9 +52,10 @@ enum { FDMON_IO_URING_ENTRIES =3D 128, /* sq/cq ring size */ =20 /* AioHandler::flags */ - FDMON_IO_URING_PENDING =3D (1 << 0), - FDMON_IO_URING_ADD =3D (1 << 1), - FDMON_IO_URING_REMOVE =3D (1 << 2), + FDMON_IO_URING_PENDING =3D (1 << 0), + FDMON_IO_URING_ADD =3D (1 << 1), + FDMON_IO_URING_REMOVE =3D (1 << 2), + FDMON_IO_URING_DELETE_AIO_HANDLER =3D (1 << 3), }; =20 static inline int poll_events_from_pfd(int pfd_events) @@ -218,6 +219,16 @@ static void fill_sq_ring(AioContext *ctx) if (flags & FDMON_IO_URING_REMOVE) { add_poll_remove_sqe(ctx, node); } + if (flags & FDMON_IO_URING_DELETE_AIO_HANDLER) { + /* + * process_cqe() sets this flag after ADD and REMOVE have been + * cleared. They cannot be set again, so they must be clear. + */ + assert(!(flags & FDMON_IO_URING_ADD)); + assert(!(flags & FDMON_IO_URING_REMOVE)); + + QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, node, node_d= eleted); + } } } =20 @@ -241,7 +252,12 @@ static bool process_cqe(AioContext *ctx, */ flags =3D qatomic_fetch_and(&node->flags, ~FDMON_IO_URING_REMOVE); if (flags & FDMON_IO_URING_REMOVE) { - QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, node, node_delet= ed); + if (flags & FDMON_IO_URING_PENDING) { + /* Still on ctx->submit_list, defer deletion until fill_sq_rin= g() */ + qatomic_or(&node->flags, FDMON_IO_URING_DELETE_AIO_HANDLER); + } else { + QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, node, node_d= eleted); + } return false; } =20 @@ -347,10 +363,13 @@ void fdmon_io_uring_destroy(AioContext *ctx) unsigned flags =3D qatomic_fetch_and(&node->flags, ~(FDMON_IO_URING_PENDING | FDMON_IO_URING_ADD | - FDMON_IO_URING_REMOVE)); + FDMON_IO_URING_REMOVE | + FDMON_IO_URING_DELETE_AIO_HANDLER)); =20 - if (flags & FDMON_IO_URING_REMOVE) { - QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, node, no= de_deleted); + if ((flags & FDMON_IO_URING_REMOVE) || + (flags & FDMON_IO_URING_DELETE_AIO_HANDLER)) { + QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, + node, node_deleted); } =20 QSLIST_REMOVE_HEAD_RCU(&ctx->submit_list, node_submitted); --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897024; cv=none; d=zohomail.com; s=zohoarc; b=oEUCY/k/43YqkY5/xUbBq3M814o16sFAyWd93ms9ZgUe71IFgEMbvSilUYw8cTWiMwVyb4KKX9bU39tW8/8r4tQ//+QRZXUmzeTcINpWvd1c+Z6ssdreqdX8Tx8/dggLCzwrSXA6zjxgr1TJzW6Tyoh2Slt2jiKZnpFjJw6xJ4s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897024; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=l+CpWO3JCH0B/CwO232sJN6EEvF1wViIZzhnu7FRhIc=; b=cRS0iOFarV/BFC6bsLoQ6B4YcFoAIBSca8pHlIMxXd8xTftYtpOM4rf/VKW5gkMLU7nVWdx89jO//7J+mfqopJOD2N2pnrx//DKaiNewYrlW82GkaDRoT9mt8Wj3F/Au/lwux9IS0be+KPfw5FkvYMCbVdOU4VOjlast9gMh8qc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897024710180.5252749335474; Tue, 11 Nov 2025 13:37:04 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzE-0006jV-IE; Tue, 11 Nov 2025 16:33:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzC-0006eJ-Tg for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:10 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzB-0001ot-9g for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:10 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-UCRwQINCMvSxJ3IO_6t95w-1; Tue, 11 Nov 2025 16:33:05 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 017D5195608F; Tue, 11 Nov 2025 21:33:04 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 641FC30044E5; Tue, 11 Nov 2025 21:33:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896788; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l+CpWO3JCH0B/CwO232sJN6EEvF1wViIZzhnu7FRhIc=; b=iMWI5ZZ4mtlCN/FKRlfEYMiWdCS5zAd/FHcDiLLHg3ct5gSkmAMY87PPmzHWAsMJMKX7DM Oz29djvosdYAl/38x8mhtioiVdIvUe1gzX8aPpCxqtur9xzGgnDjT25LqHkjxdYcE9ESdz E4w/Rw2JMqSCcZFXvejZMLq+M6bGQCQ= X-MC-Unique: UCRwQINCMvSxJ3IO_6t95w-1 X-Mimecast-MFC-AGG-ID: UCRwQINCMvSxJ3IO_6t95w_1762896784 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 02/28] aio-posix: fix fdmon-io_uring.c timeout stack variable lifetime Date: Tue, 11 Nov 2025 22:32:12 +0100 Message-ID: <20251111213238.181992-3-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897027118153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi io_uring_prep_timeout() stashes a pointer to the timespec struct rather than copying its fields. That means the struct must live until after the SQE has been submitted by io_uring_enter(2). add_timeout_sqe() violates this constraint because the SQE is not submitted within the function. Inline add_timeout_sqe() into fdmon_io_uring_wait() so that the struct lives at least as long as io_uring_enter(2). This fixes random hangs (bogus timeout values) when the kernel loads undefined timespec struct values from userspace after the original struct on the stack has been destroyed. Reported-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-ID: <20251104022933.618123-3-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- util/fdmon-io_uring.c | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index ad89160f31..b64ce42513 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -188,20 +188,6 @@ static void add_poll_remove_sqe(AioContext *ctx, AioHa= ndler *node) io_uring_sqe_set_data(sqe, NULL); } =20 -/* Add a timeout that self-cancels when another cqe becomes ready */ -static void add_timeout_sqe(AioContext *ctx, int64_t ns) -{ - struct io_uring_sqe *sqe; - struct __kernel_timespec ts =3D { - .tv_sec =3D ns / NANOSECONDS_PER_SECOND, - .tv_nsec =3D ns % NANOSECONDS_PER_SECOND, - }; - - sqe =3D get_sqe(ctx); - io_uring_prep_timeout(sqe, &ts, 1, 0); - io_uring_sqe_set_data(sqe, NULL); -} - /* Add sqes from ctx->submit_list for submission */ static void fill_sq_ring(AioContext *ctx) { @@ -291,13 +277,24 @@ static int process_cq_ring(AioContext *ctx, AioHandle= rList *ready_list) static int fdmon_io_uring_wait(AioContext *ctx, AioHandlerList *ready_list, int64_t timeout) { + struct __kernel_timespec ts; unsigned wait_nr =3D 1; /* block until at least one cqe is ready */ int ret; =20 if (timeout =3D=3D 0) { wait_nr =3D 0; /* non-blocking */ } else if (timeout > 0) { - add_timeout_sqe(ctx, timeout); + /* Add a timeout that self-cancels when another cqe becomes ready = */ + struct io_uring_sqe *sqe; + + ts =3D (struct __kernel_timespec){ + .tv_sec =3D timeout / NANOSECONDS_PER_SECOND, + .tv_nsec =3D timeout % NANOSECONDS_PER_SECOND, + }; + + sqe =3D get_sqe(ctx); + io_uring_prep_timeout(sqe, &ts, 1, 0); + io_uring_sqe_set_data(sqe, NULL); } =20 fill_sq_ring(ctx); --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896806; cv=none; d=zohomail.com; s=zohoarc; b=UmDwJi45shsn096vwkuWuH1poINOBnYZsH1PSfq/6emGFAtiMd0PKuukCfzHeVl80LeySLQGheRl9TG21fvkBtX8u6kOa27RI4/2OBEAgsbBpW9m0rNfVS3QWtgxNAcx+JTmsAFoIdIEriL/BwUrdPKRGyGbWhD3K9QHLYipf4k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896806; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RhVA0YubTi5SGRuSN0fM7JVsTglFF5gQoqp/tSqn62I=; b=jfe6Kow0Q1cLJBLT8cY4qrGq+KjY1WscAjPvoN+Px6YNh7ngY+/1EdaTmhoqdLEANCus5S2UXgmuVOq2tUAx0uVRaqn7qLIgcGLLKOy+BdUxalxj/F3SniPFalL0LmeVz2FN6aZSFG0X2wLLJPpd44JiUblpLfENef+Y6yBbP5o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896806216227.09387573135177; Tue, 11 Nov 2025 13:33:26 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzF-0006m4-2m; Tue, 11 Nov 2025 16:33:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzD-0006ea-5j for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:11 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzB-0001ou-Kh for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:10 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-491-uHEK-ukvO42GKTPjuYnNuA-1; Tue, 11 Nov 2025 16:33:07 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 31B5719560B5; Tue, 11 Nov 2025 21:33:06 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7C10E30044E5; Tue, 11 Nov 2025 21:33:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896788; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RhVA0YubTi5SGRuSN0fM7JVsTglFF5gQoqp/tSqn62I=; b=R7pvdq2UvQaD/+t+VyVMWglNaNLo/pWE0wMc3Jp7Z3qKHB8cqJ8gGQfM2fTA6WxXZOGtU9 HJzojlQ2Sg7rJqlKHcsI2eBVBbwBXYtPG14gQPBRJwPFdNrmgZe8bFfnsM2hS4qKur8KzM bQSrlYc0s2pwo7a24aSvhkgK4rDYEzI= X-MC-Unique: uHEK-ukvO42GKTPjuYnNuA-1 X-Mimecast-MFC-AGG-ID: uHEK-ukvO42GKTPjuYnNuA_1762896786 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 03/28] aio-posix: fix spurious return from ->wait() due to signals Date: Tue, 11 Nov 2025 22:32:13 +0100 Message-ID: <20251111213238.181992-4-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896809471153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi io_uring_enter(2) only returns -EINTR in some cases when interrupted by a signal. Therefore the while loop in fdmon_io_uring_wait() is incomplete and can lead to a spurious early return. Handle the case when a signal interrupts io_uring_enter(2) but the syscall returns the number of SQEs submitted (that takes priority over -EINTR). This patch probably makes little difference for QEMU, but the test suite relies on the exact pattern of aio_poll() return values, so it's best to hide this io_uring syscall interface quirk. Here is the strace of test-aio receiving 3 SIGCONT signals after this fix has been applied. Notice how the io_uring_enter(2) return value is 1 the first time because an SQE was submitted, but -EINTR the other times: eventfd2(0, EFD_CLOEXEC|EFD_NONBLOCK) =3D 9 io_uring_enter(7, 1, 0, 0, NULL, 8) =3D 1 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=3D1, tv_nsec=3D0}, 0x7ffe38a46= 240) =3D 0 io_uring_enter(7, 1, 1, IORING_ENTER_GETEVENTS, NULL, 8) =3D 1 --- SIGCONT {si_signo=3DSIGCONT, si_code=3DSI_USER, si_pid=3D596096, si_u= id=3D1000} --- io_uring_enter(7, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8) =3D -1 EINTR (In= terrupted system call) --- SIGCONT {si_signo=3DSIGCONT, si_code=3DSI_USER, si_pid=3D596096, si_u= id=3D1000} --- io_uring_enter(7, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8 <... io_uring_enter resumed>) =3D -1 EINTR (Interrupted system call) --- SIGCONT {si_signo=3DSIGCONT, si_code=3DSI_USER, si_pid=3D596096, si_u= id=3D1000} --- io_uring_enter(7, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8 <... io_uring_enter resumed>) =3D 0 Reported-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-ID: <20251104022933.618123-4-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- util/fdmon-io_uring.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index b64ce42513..3d8638b0e5 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -299,9 +299,16 @@ static int fdmon_io_uring_wait(AioContext *ctx, AioHan= dlerList *ready_list, =20 fill_sq_ring(ctx); =20 + /* + * Loop to handle signals in both cases: + * 1. If no SQEs were submitted, then -EINTR is returned. + * 2. If SQEs were submitted then the number of SQEs submitted is retu= rned + * rather than -EINTR. + */ do { ret =3D io_uring_submit_and_wait(&ctx->fdmon_io_uring, wait_nr); - } while (ret =3D=3D -EINTR); + } while (ret =3D=3D -EINTR || + (ret >=3D 0 && wait_nr > io_uring_cq_ready(&ctx->fdmon_io_uri= ng))); =20 assert(ret >=3D 0); =20 --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896919; cv=none; d=zohomail.com; s=zohoarc; b=QE138+ZWbq0jmtuKy1XPqjM80EEzyR6N5L7Y+j7cFpQnlJ8JHrA43CCRJUKCdIOwQ/a4KBymkOPKKbg9VH+RnZqa0A4wL5NxP6WXCyT+KI/h/+3UFRyB8ov6qjBP1Eh/0frqzVtv3YY8QTcDnp0n9NCmxXqpTeQSQkMgqYIIIjs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896919; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AFueM27NLiZq+PeHyP0oGI+SClKRjJY3eSQT1wbYQdE=; b=dpe770FA1SXmR5L4S27vhY07ETwOn58rIRF7z262lY4uOPh1CbFX/DiUvjzWvQJFBE9uKg2YjdrWzAFdPbXL6+al3cgOn8LF/1LAs2THiHar48lqyUkzvY1WiaPUwWOI1qvkLgPCaVD/BWz7+4PJriySVF3eRIe96AIG/5Am7ZA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17628969198651002.0863271022565; Tue, 11 Nov 2025 13:35:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzN-0006sn-Hf; Tue, 11 Nov 2025 16:33:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzJ-0006qq-6w for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzH-0001pr-NA for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:16 -0500 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-553-CmDC6T5zMiCArtjO-Xqwpg-1; Tue, 11 Nov 2025 16:33:09 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 661DA1800451; Tue, 11 Nov 2025 21:33:08 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id ABB0530044E0; Tue, 11 Nov 2025 21:33:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AFueM27NLiZq+PeHyP0oGI+SClKRjJY3eSQT1wbYQdE=; b=A3tY9igYJEp2H2H46GpU18b8pL4T13qm23YOJ/Kz6RUobstH9OsfXgluppDqzPNLpz8Pew 08uUNiLwLl8P6YH6xFQnKkqLZwipU9I+lPKvEDUTvXT++itfhXw7PEElVGEVof1bapQ5He kk5LTDkqyoXuD1dqqqWW/29Qo+gI1fQ= X-MC-Unique: CmDC6T5zMiCArtjO-Xqwpg-1 X-Mimecast-MFC-AGG-ID: CmDC6T5zMiCArtjO-Xqwpg_1762896788 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 04/28] aio-posix: keep polling enabled with fdmon-io_uring.c Date: Tue, 11 Nov 2025 22:32:14 +0100 Message-ID: <20251111213238.181992-5-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896921399158500 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi Commit 816a430c517e ("util/aio: Defer disabling poll mode as long as possible") kept polling enabled when the event loop timeout is 0. Since there is no timeout the event loop will continue immediately and the overhead of disabling and re-enabling polling can be avoided. fdmon-io_uring.c is unable to take advantage of this optimization because its ->need_wait() function returns true whenever there are new io_uring SQEs to submit: if (timeout || ctx->fdmon_ops->need_wait(ctx)) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Polling will be disabled even when timeout =3D=3D 0. Extend the optimization to handle the case when need_wait() returns true and timeout =3D=3D 0. Cc: Chao Gao Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Message-ID: <20251104022933.618123-5-stefanha@redhat.com> Signed-off-by: Kevin Wolf --- util/aio-posix.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/util/aio-posix.c b/util/aio-posix.c index 2e0a5dadc4..824fdc34cc 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -559,7 +559,14 @@ static bool run_poll_handlers(AioContext *ctx, AioHand= lerList *ready_list, elapsed_time =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start_ti= me; max_ns =3D qemu_soonest_timeout(*timeout, max_ns); assert(!(max_ns && progress)); - } while (elapsed_time < max_ns && !ctx->fdmon_ops->need_wait(ctx)); + + if (ctx->fdmon_ops->need_wait(ctx)) { + if (fdmon_supports_polling(ctx)) { + *timeout =3D 0; /* stay in polling mode */ + } + break; + } + } while (elapsed_time < max_ns); =20 if (remove_idle_poll_handlers(ctx, ready_list, start_time + elapsed_time)) { @@ -722,7 +729,7 @@ bool aio_poll(AioContext *ctx, bool blocking) * up IO threads when some work becomes pending. It is essential to * avoid hangs or unnecessary latency. */ - if (poll_set_started(ctx, &ready_list, false)) { + if (timeout && poll_set_started(ctx, &ready_list, false)) { timeout =3D 0; progress =3D true; } --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896842; cv=none; d=zohomail.com; s=zohoarc; b=Fk0qi3A321ns3HFX36NAGLbF9GnQ/Ehkhx3FhQn9i1y1eWzH/XcOfbjbx6/2CnfmCuHiS6QBOWexEu3adMCaJn1vqFqXT7B7UBpH3Jru/leCHxMaqyTIkAypC0YSCf2Xb/g+yv90D2y3O6/4TFVsUtcPE2b6xwa66CBjfSpCyKo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896842; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tab4CtFFHM98XVix6uSAQ4RulcmwiYQiWx2dY6YSd5o=; b=Jchsq6WoWURSpUm2/16lM+M395LIoED/l6AFCcPplTkw5uGXGMcidNJ4FKaThT7rHKzKGezOFi1LgApxMavQBwI+5fKlHEmFFF4BVIfPx2UFxg8GPY7NlTjmzK/TABQx7bbNF6ssqOEYek4uiopkoZeig3t6aMyHAOTggMiGSNY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896842536385.11639239038095; Tue, 11 Nov 2025 13:34:02 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzK-0006rB-4b; Tue, 11 Nov 2025 16:33:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzI-0006qG-3r for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:16 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzG-0001pY-Ka for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:15 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-589-VAKwvjztNnubdZuUiTFtdQ-1; Tue, 11 Nov 2025 16:33:11 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D2CF819560B2; Tue, 11 Nov 2025 21:33:10 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 411B530044E0; Tue, 11 Nov 2025 21:33:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tab4CtFFHM98XVix6uSAQ4RulcmwiYQiWx2dY6YSd5o=; b=XLRv8fuqCY81Dlj+5dcV3P3yAfFeqiErNxHU7l08DT4QgVsBnzhqThC+HiffnQqY+Unmsg lP0PWilP5V2Lm477//PPBy4jrbfeTJ+AAVAQ6iRHFrtCyWhP1DOEitWiL06vdRp5AVr/sj NCAq9T//iv4G23CZYYCqudUXolkhr8E= X-MC-Unique: VAKwvjztNnubdZuUiTFtdQ-1 X-Mimecast-MFC-AGG-ID: VAKwvjztNnubdZuUiTFtdQ_1762896790 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 05/28] tests/unit: skip test-nested-aio-poll with io_uring Date: Tue, 11 Nov 2025 22:32:15 +0100 Message-ID: <20251111213238.181992-6-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896844651153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi test-nested-aio-poll relies on internal details of how fdmon-poll.c handles AioContext polling. Skip it when other fdmon implementations are in use. The reason why fdmon-io_uring.c behaves differently from fdmon-poll.c is that its fdmon_ops->need_wait() function returns true when io_uring_enter(2) must be called (e.g. to submit pending SQEs). AioContext polling is skipped when ->need_wait() returns true, so the test case will never enter AioContext polling mode with fdmon-io_uring.c. Restrict this test to fdmon-poll.c and drop the aio_context_use_g_source() call since it's no longer necessary. Note that this test is only built on POSIX systems so it is safe to include "util/aio-posix.h". Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Message-ID: <20251104022933.618123-6-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- tests/unit/test-nested-aio-poll.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/tests/unit/test-nested-aio-poll.c b/tests/unit/test-nested-aio= -poll.c index d8fd92c43b..d13ecccd8c 100644 --- a/tests/unit/test-nested-aio-poll.c +++ b/tests/unit/test-nested-aio-poll.c @@ -15,6 +15,7 @@ #include "qemu/osdep.h" #include "block/aio.h" #include "qapi/error.h" +#include "util/aio-posix.h" =20 typedef struct { AioContext *ctx; @@ -71,17 +72,17 @@ static void test(void) .ctx =3D aio_context_new(&error_abort), }; =20 + if (td.ctx->fdmon_ops !=3D &fdmon_poll_ops) { + /* This test is tied to fdmon-poll.c */ + g_test_skip("fdmon_poll_ops not in use"); + return; + } + qemu_set_current_aio_context(td.ctx); =20 /* Enable polling */ aio_context_set_poll_params(td.ctx, 1000000, 2, 2, &error_abort); =20 - /* - * The GSource is unused but this has the side-effect of changing the = fdmon - * that AioContext uses. - */ - aio_get_g_source(td.ctx); - /* Make the event notifier active (set) right away */ event_notifier_init(&td.poll_notifier, 1); aio_set_event_notifier(td.ctx, &td.poll_notifier, --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896921; cv=none; d=zohomail.com; s=zohoarc; b=DWz7QSnfURkKqgU3ToM6FtI12wLyOYukNxju7ugg7zB31Kr7Kb/27t3GzZ1zAsvpPYaVmjIO2ZoKD0aE4SuiGB1V1APNqYuw4geF5VtAPZUiNqcuTIAQvA+DlQjugjhTfpPue/oVbgkF/bhSE61e0lGEu428Nw7mJs2uSIm1xEE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896921; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zghK8OsAu9+QGUgj06ZPB7erCE71CI4Vp/mVFBGHgZA=; b=MTqGn1HzBsI9YQ6gCcQ43EefAV1qOgcVjROboINgaSEqQbcFHzJQ0vp3T4EosrsW0UhoQvBqOR0OyLFCOx9IZewIVK437P+GXEffnv7oHrwU5+HeUMVCna/h6/7lhyNQcNixkefTjVu+G3riZp3B2dMU0AQ0BmlpogW4Axgr0u0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896921052253.52235464447938; Tue, 11 Nov 2025 13:35:21 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzO-0006u3-IX; Tue, 11 Nov 2025 16:33:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzM-0006sl-Fj for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzJ-0001qQ-PQ for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:20 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-27-nJaoeHZZPe2mc9fUJw533Q-1; Tue, 11 Nov 2025 16:33:13 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 008251956089; Tue, 11 Nov 2025 21:33:13 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5846E30044E5; Tue, 11 Nov 2025 21:33:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896797; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zghK8OsAu9+QGUgj06ZPB7erCE71CI4Vp/mVFBGHgZA=; b=h2SXzcaR6HsaCci6swGnreHyNs1t80oy1tXheEE32qUwWEdsjm9lqNd2lAvv7aQvLp1AWY NbFZOG5oIpMEL6yZnqexlZ/tKVR6uQGTL+7QmT2jac6UbXKNbIrbaQ0o6nTSAgPoCjqtvu Jklvc8TNSPxwWIiZCrQr0uhSTjLyUq4= X-MC-Unique: nJaoeHZZPe2mc9fUJw533Q-1 X-Mimecast-MFC-AGG-ID: nJaoeHZZPe2mc9fUJw533Q_1762896793 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 06/28] aio-posix: integrate fdmon into glib event loop Date: Tue, 11 Nov 2025 22:32:16 +0100 Message-ID: <20251111213238.181992-7-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896923485158500 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi AioContext's glib integration only supports ppoll(2) file descriptor monitoring. epoll(7) and io_uring(7) disable themselves and switch back to ppoll(2) when the glib event loop is used. The main loop thread cannot use epoll(7) or io_uring(7) because it always uses the glib event loop. Future QEMU features may require io_uring(7). One example is uring_cmd support in FUSE exports. Each feature could create its own io_uring(7) context and integrate it into the event loop, but this is inefficient due to extra syscalls. It would be more efficient to reuse the AioContext's existing fdmon-io_uring.c io_uring(7) context because fdmon-io_uring.c will already be active on systems where Linux io_uring is available. In order to keep fdmon-io_uring.c's AioContext operational even when the glib event loop is used, extend FDMonOps with an API similar to GSourceFuncs so that file descriptor monitoring can integrate into the glib event loop. A quick summary of the GSourceFuncs API: - prepare() is called each event loop iteration before waiting for file descriptors and timers. - check() is called to determine whether events are ready to be dispatched after waiting. - dispatch() is called to process events. More details here: https://docs.gtk.org/glib/struct.SourceFuncs.html Move the ppoll(2)-specific code from aio-posix.c into fdmon-poll.c and also implement epoll(7)- and io_uring(7)-specific file descriptor monitoring code for glib event loops. Note that it's still faster to use aio_poll() rather than the glib event loop since glib waits for file descriptor activity with ppoll(2) and does not support adaptive polling. But at least epoll(7) and io_uring(7) now work in glib event loops. Splitting this into multiple commits without temporarily breaking AioContext proved difficult so this commit makes all the changes. The next commit will remove the aio_context_use_g_source() API because it is no longer needed. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Message-ID: <20251104022933.618123-7-stefanha@redhat.com> [kwolf: Build fixes; fix AioContext.list_lock use after destroy] Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- include/block/aio.h | 36 ++++++++++++++++++ util/aio-posix.h | 5 +++ tests/unit/test-aio.c | 7 +++- util/aio-posix.c | 69 ++++++++--------------------------- util/async.c | 3 +- util/fdmon-epoll.c | 34 ++++++++++++++--- util/fdmon-io_uring.c | 46 ++++++++++++++++++++++- util/fdmon-poll.c | 85 ++++++++++++++++++++++++++++++++++++++++++- meson.build | 2 + 9 files changed, 224 insertions(+), 63 deletions(-) diff --git a/include/block/aio.h b/include/block/aio.h index 99ff48420b..aef43f9c51 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -106,6 +106,38 @@ typedef struct { * Returns: true if ->wait() should be called, false otherwise. */ bool (*need_wait)(AioContext *ctx); + + /* + * gsource_prepare: + * @ctx: the AioContext + * + * Prepare for the glib event loop to wait for events instead of the u= sual + * ->wait() call. See glib's GSourceFuncs->prepare(). + */ + void (*gsource_prepare)(AioContext *ctx); + + /* + * gsource_check: + * @ctx: the AioContext + * + * Called by the glib event loop from glib's GSourceFuncs->check() aft= er + * waiting for events. + * + * Returns: true when ready to be dispatched. + */ + bool (*gsource_check)(AioContext *ctx); + + /* + * gsource_dispatch: + * @ctx: the AioContext + * @ready_list: list for handlers that become ready + * + * Place ready AioHandlers on ready_list. Called as part of the glib e= vent + * loop from glib's GSourceFuncs->dispatch(). + * + * Called with list_lock incremented. + */ + void (*gsource_dispatch)(AioContext *ctx, AioHandlerList *ready_list); } FDMonOps; =20 /* @@ -222,6 +254,7 @@ struct AioContext { /* State for file descriptor monitoring using Linux io_uring */ struct io_uring fdmon_io_uring; AioHandlerSList submit_list; + void *io_uring_fd_tag; #endif =20 /* TimerLists for calling timers - one per clock type. Has its own @@ -254,6 +287,9 @@ struct AioContext { /* epoll(7) state used when built with CONFIG_EPOLL */ int epollfd; =20 + /* The GSource unix fd tag for epollfd */ + void *epollfd_tag; + const FDMonOps *fdmon_ops; }; =20 diff --git a/util/aio-posix.h b/util/aio-posix.h index 82a0201ea4..f9994ed79e 100644 --- a/util/aio-posix.h +++ b/util/aio-posix.h @@ -47,9 +47,14 @@ void aio_add_ready_handler(AioHandlerList *ready_list, A= ioHandler *node, =20 extern const FDMonOps fdmon_poll_ops; =20 +/* Switch back to poll(2). list_lock must be held. */ +void fdmon_poll_downgrade(AioContext *ctx); + #ifdef CONFIG_EPOLL_CREATE1 bool fdmon_epoll_try_upgrade(AioContext *ctx, unsigned npfd); void fdmon_epoll_setup(AioContext *ctx); + +/* list_lock must be held */ void fdmon_epoll_disable(AioContext *ctx); #else static inline bool fdmon_epoll_try_upgrade(AioContext *ctx, unsigned npfd) diff --git a/tests/unit/test-aio.c b/tests/unit/test-aio.c index e77d86be87..010d65b79a 100644 --- a/tests/unit/test-aio.c +++ b/tests/unit/test-aio.c @@ -527,7 +527,12 @@ static void test_source_bh_delete_from_cb(void) g_assert_cmpint(data1.n, =3D=3D, data1.max); g_assert(data1.bh =3D=3D NULL); =20 - assert(g_main_context_iteration(NULL, false)); + /* + * There may be up to one more iteration due to the aio_notify + * EventNotifier. + */ + g_main_context_iteration(NULL, false); + assert(!g_main_context_iteration(NULL, false)); } =20 diff --git a/util/aio-posix.c b/util/aio-posix.c index 824fdc34cc..9de05ee7e8 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -70,15 +70,6 @@ static AioHandler *find_aio_handler(AioContext *ctx, int= fd) =20 static bool aio_remove_fd_handler(AioContext *ctx, AioHandler *node) { - /* If the GSource is in the process of being destroyed then - * g_source_remove_poll() causes an assertion failure. Skip - * removal in that case, because glib cleans up its state during - * destruction anyway. - */ - if (!g_source_is_destroyed(&ctx->source)) { - g_source_remove_poll(&ctx->source, &node->pfd); - } - node->pfd.revents =3D 0; node->poll_ready =3D false; =20 @@ -153,7 +144,6 @@ void aio_set_fd_handler(AioContext *ctx, } else { new_node->pfd =3D node->pfd; } - g_source_add_poll(&ctx->source, &new_node->pfd); =20 new_node->pfd.events =3D (io_read ? G_IO_IN | G_IO_HUP | G_IO_ERR = : 0); new_node->pfd.events |=3D (io_write ? G_IO_OUT | G_IO_ERR : 0); @@ -267,37 +257,13 @@ bool aio_prepare(AioContext *ctx) poll_set_started(ctx, &ready_list, false); /* TODO what to do with this list? */ =20 + ctx->fdmon_ops->gsource_prepare(ctx); return false; } =20 bool aio_pending(AioContext *ctx) { - AioHandler *node; - bool result =3D false; - - /* - * We have to walk very carefully in case aio_set_fd_handler is - * called while we're walking. - */ - qemu_lockcnt_inc(&ctx->list_lock); - - QLIST_FOREACH_RCU(node, &ctx->aio_handlers, node) { - int revents; - - /* TODO should this check poll ready? */ - revents =3D node->pfd.revents & node->pfd.events; - if (revents & (G_IO_IN | G_IO_HUP | G_IO_ERR) && node->io_read) { - result =3D true; - break; - } - if (revents & (G_IO_OUT | G_IO_ERR) && node->io_write) { - result =3D true; - break; - } - } - qemu_lockcnt_dec(&ctx->list_lock); - - return result; + return ctx->fdmon_ops->gsource_check(ctx); } =20 static void aio_free_deleted_handlers(AioContext *ctx) @@ -390,10 +356,6 @@ static bool aio_dispatch_handler(AioContext *ctx, AioH= andler *node) return progress; } =20 -/* - * If we have a list of ready handlers then this is more efficient than - * scanning all handlers with aio_dispatch_handlers(). - */ static bool aio_dispatch_ready_handlers(AioContext *ctx, AioHandlerList *ready_list, int64_t block_ns) @@ -417,24 +379,18 @@ static bool aio_dispatch_ready_handlers(AioContext *c= tx, return progress; } =20 -/* Slower than aio_dispatch_ready_handlers() but only used via glib */ -static bool aio_dispatch_handlers(AioContext *ctx) -{ - AioHandler *node, *tmp; - bool progress =3D false; - - QLIST_FOREACH_SAFE_RCU(node, &ctx->aio_handlers, node, tmp) { - progress =3D aio_dispatch_handler(ctx, node) || progress; - } - - return progress; -} - void aio_dispatch(AioContext *ctx) { + AioHandlerList ready_list =3D QLIST_HEAD_INITIALIZER(ready_list); + qemu_lockcnt_inc(&ctx->list_lock); aio_bh_poll(ctx); - aio_dispatch_handlers(ctx); + + ctx->fdmon_ops->gsource_dispatch(ctx, &ready_list); + + /* block_ns is 0 because polling is disabled in the glib event loop */ + aio_dispatch_ready_handlers(ctx, &ready_list, 0); + aio_free_deleted_handlers(ctx); qemu_lockcnt_dec(&ctx->list_lock); =20 @@ -766,6 +722,7 @@ void aio_context_setup(AioContext *ctx) { ctx->fdmon_ops =3D &fdmon_poll_ops; ctx->epollfd =3D -1; + ctx->epollfd_tag =3D NULL; =20 /* Use the fastest fd monitoring implementation if available */ if (fdmon_io_uring_setup(ctx)) { @@ -778,7 +735,11 @@ void aio_context_setup(AioContext *ctx) void aio_context_destroy(AioContext *ctx) { fdmon_io_uring_destroy(ctx); + + qemu_lockcnt_lock(&ctx->list_lock); fdmon_epoll_disable(ctx); + qemu_lockcnt_unlock(&ctx->list_lock); + aio_free_deleted_handlers(ctx); } =20 diff --git a/util/async.c b/util/async.c index a736d2cd0d..83f1e73e34 100644 --- a/util/async.c +++ b/util/async.c @@ -418,10 +418,11 @@ aio_ctx_finalize(GSource *source) aio_set_event_notifier(ctx, &ctx->notifier, NULL, NULL, NULL); event_notifier_cleanup(&ctx->notifier); qemu_rec_mutex_destroy(&ctx->lock); - qemu_lockcnt_destroy(&ctx->list_lock); timerlistgroup_deinit(&ctx->tlg); unregister_aiocontext(ctx); aio_context_destroy(ctx); + /* aio_context_destroy() still needs the lock */ + qemu_lockcnt_destroy(&ctx->list_lock); } =20 static GSourceFuncs aio_source_funcs =3D { diff --git a/util/fdmon-epoll.c b/util/fdmon-epoll.c index 9fb8800dde..61118e1ee6 100644 --- a/util/fdmon-epoll.c +++ b/util/fdmon-epoll.c @@ -19,8 +19,12 @@ void fdmon_epoll_disable(AioContext *ctx) ctx->epollfd =3D -1; } =20 - /* Switch back */ - ctx->fdmon_ops =3D &fdmon_poll_ops; + if (ctx->epollfd_tag) { + g_source_remove_unix_fd(&ctx->source, ctx->epollfd_tag); + ctx->epollfd_tag =3D NULL; + } + + fdmon_poll_downgrade(ctx); } =20 static inline int epoll_events_from_pfd(int pfd_events) @@ -93,10 +97,29 @@ out: return ret; } =20 +static void fdmon_epoll_gsource_prepare(AioContext *ctx) +{ + /* Do nothing */ +} + +static bool fdmon_epoll_gsource_check(AioContext *ctx) +{ + return g_source_query_unix_fd(&ctx->source, ctx->epollfd_tag) & G_IO_I= N; +} + +static void fdmon_epoll_gsource_dispatch(AioContext *ctx, + AioHandlerList *ready_list) +{ + fdmon_epoll_wait(ctx, ready_list, 0); +} + static const FDMonOps fdmon_epoll_ops =3D { .update =3D fdmon_epoll_update, .wait =3D fdmon_epoll_wait, .need_wait =3D aio_poll_disabled, + .gsource_prepare =3D fdmon_epoll_gsource_prepare, + .gsource_check =3D fdmon_epoll_gsource_check, + .gsource_dispatch =3D fdmon_epoll_gsource_dispatch, }; =20 static bool fdmon_epoll_try_enable(AioContext *ctx) @@ -118,6 +141,8 @@ static bool fdmon_epoll_try_enable(AioContext *ctx) } =20 ctx->fdmon_ops =3D &fdmon_epoll_ops; + ctx->epollfd_tag =3D g_source_add_unix_fd(&ctx->source, ctx->epollfd, + G_IO_IN); return true; } =20 @@ -139,12 +164,11 @@ bool fdmon_epoll_try_upgrade(AioContext *ctx, unsigne= d npfd) } =20 ok =3D fdmon_epoll_try_enable(ctx); - - qemu_lockcnt_inc_and_unlock(&ctx->list_lock); - if (!ok) { fdmon_epoll_disable(ctx); } + + qemu_lockcnt_inc_and_unlock(&ctx->list_lock); return ok; } =20 diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index 3d8638b0e5..2460ac1556 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -262,6 +262,13 @@ static int process_cq_ring(AioContext *ctx, AioHandler= List *ready_list) unsigned num_ready =3D 0; unsigned head; =20 +#ifdef HAVE_IO_URING_CQ_HAS_OVERFLOW + /* If the CQ overflowed then fetch CQEs with a syscall */ + if (io_uring_cq_has_overflow(ring)) { + io_uring_get_events(ring); + } +#endif + io_uring_for_each_cqe(ring, head, cqe) { if (process_cqe(ctx, ready_list, cqe)) { num_ready++; @@ -274,6 +281,30 @@ static int process_cq_ring(AioContext *ctx, AioHandler= List *ready_list) return num_ready; } =20 +/* This is where SQEs are submitted in the glib event loop */ +static void fdmon_io_uring_gsource_prepare(AioContext *ctx) +{ + fill_sq_ring(ctx); + if (io_uring_sq_ready(&ctx->fdmon_io_uring)) { + while (io_uring_submit(&ctx->fdmon_io_uring) =3D=3D -EINTR) { + /* Keep trying if syscall was interrupted */ + } + } +} + +static bool fdmon_io_uring_gsource_check(AioContext *ctx) +{ + gpointer tag =3D ctx->io_uring_fd_tag; + return g_source_query_unix_fd(&ctx->source, tag) & G_IO_IN; +} + +/* This is where CQEs are processed in the glib event loop */ +static void fdmon_io_uring_gsource_dispatch(AioContext *ctx, + AioHandlerList *ready_list) +{ + process_cq_ring(ctx, ready_list); +} + static int fdmon_io_uring_wait(AioContext *ctx, AioHandlerList *ready_list, int64_t timeout) { @@ -339,12 +370,17 @@ static const FDMonOps fdmon_io_uring_ops =3D { .update =3D fdmon_io_uring_update, .wait =3D fdmon_io_uring_wait, .need_wait =3D fdmon_io_uring_need_wait, + .gsource_prepare =3D fdmon_io_uring_gsource_prepare, + .gsource_check =3D fdmon_io_uring_gsource_check, + .gsource_dispatch =3D fdmon_io_uring_gsource_dispatch, }; =20 bool fdmon_io_uring_setup(AioContext *ctx) { int ret; =20 + ctx->io_uring_fd_tag =3D NULL; + ret =3D io_uring_queue_init(FDMON_IO_URING_ENTRIES, &ctx->fdmon_io_uri= ng, 0); if (ret !=3D 0) { return false; @@ -352,6 +388,9 @@ bool fdmon_io_uring_setup(AioContext *ctx) =20 QSLIST_INIT(&ctx->submit_list); ctx->fdmon_ops =3D &fdmon_io_uring_ops; + ctx->io_uring_fd_tag =3D g_source_add_unix_fd(&ctx->source, + ctx->fdmon_io_uring.ring_fd, G_IO_IN); + return true; } =20 @@ -379,6 +418,11 @@ void fdmon_io_uring_destroy(AioContext *ctx) QSLIST_REMOVE_HEAD_RCU(&ctx->submit_list, node_submitted); } =20 - ctx->fdmon_ops =3D &fdmon_poll_ops; + g_source_remove_unix_fd(&ctx->source, ctx->io_uring_fd_tag); + ctx->io_uring_fd_tag =3D NULL; + + qemu_lockcnt_lock(&ctx->list_lock); + fdmon_poll_downgrade(ctx); + qemu_lockcnt_unlock(&ctx->list_lock); } } diff --git a/util/fdmon-poll.c b/util/fdmon-poll.c index 17df917cf9..0ae755cc13 100644 --- a/util/fdmon-poll.c +++ b/util/fdmon-poll.c @@ -72,6 +72,11 @@ static int fdmon_poll_wait(AioContext *ctx, AioHandlerLi= st *ready_list, =20 /* epoll(7) is faster above a certain number of fds */ if (fdmon_epoll_try_upgrade(ctx, npfd)) { + QLIST_FOREACH_RCU(node, &ctx->aio_handlers, node) { + if (!QLIST_IS_INSERTED(node, node_deleted) && node->pfd.events= ) { + g_source_remove_poll(&ctx->source, &node->pfd); + } + } npfd =3D 0; /* we won't need pollfds[], reset npfd */ return ctx->fdmon_ops->wait(ctx, ready_list, timeout); } @@ -97,11 +102,89 @@ static void fdmon_poll_update(AioContext *ctx, AioHandler *old_node, AioHandler *new_node) { - /* Do nothing, AioHandler already contains the state we'll need */ + if (old_node) { + /* + * If the GSource is in the process of being destroyed then + * g_source_remove_poll() causes an assertion failure. Skip remov= al in + * that case, because glib cleans up its state during destruction + * anyway. + */ + if (!g_source_is_destroyed(&ctx->source)) { + g_source_remove_poll(&ctx->source, &old_node->pfd); + } + } + + if (new_node) { + g_source_add_poll(&ctx->source, &new_node->pfd); + } +} + +static void fdmon_poll_gsource_prepare(AioContext *ctx) +{ + /* Do nothing */ +} + +static bool fdmon_poll_gsource_check(AioContext *ctx) +{ + AioHandler *node; + bool result =3D false; + + /* + * We have to walk very carefully in case aio_set_fd_handler is + * called while we're walking. + */ + qemu_lockcnt_inc(&ctx->list_lock); + + QLIST_FOREACH_RCU(node, &ctx->aio_handlers, node) { + int revents =3D node->pfd.revents & node->pfd.events; + + if (revents & (G_IO_IN | G_IO_HUP | G_IO_ERR) && node->io_read) { + result =3D true; + break; + } + if (revents & (G_IO_OUT | G_IO_ERR) && node->io_write) { + result =3D true; + break; + } + } + + qemu_lockcnt_dec(&ctx->list_lock); + + return result; +} + +static void fdmon_poll_gsource_dispatch(AioContext *ctx, + AioHandlerList *ready_list) +{ + AioHandler *node; + + QLIST_FOREACH_RCU(node, &ctx->aio_handlers, node) { + int revents =3D node->pfd.revents; + + if (revents) { + aio_add_ready_handler(ready_list, node, revents); + } + } } =20 const FDMonOps fdmon_poll_ops =3D { .update =3D fdmon_poll_update, .wait =3D fdmon_poll_wait, .need_wait =3D aio_poll_disabled, + .gsource_prepare =3D fdmon_poll_gsource_prepare, + .gsource_check =3D fdmon_poll_gsource_check, + .gsource_dispatch =3D fdmon_poll_gsource_dispatch, }; + +void fdmon_poll_downgrade(AioContext *ctx) +{ + AioHandler *node; + + ctx->fdmon_ops =3D &fdmon_poll_ops; + + QLIST_FOREACH_RCU(node, &ctx->aio_handlers, node) { + if (!QLIST_IS_INSERTED(node, node_deleted) && node->pfd.events) { + g_source_add_poll(&ctx->source, &node->pfd); + } + } +} diff --git a/meson.build b/meson.build index 48c1795b0f..df4460035c 100644 --- a/meson.build +++ b/meson.build @@ -2745,6 +2745,8 @@ endif if linux_io_uring.found() config_host_data.set('HAVE_IO_URING_PREP_WRITEV2', cc.has_header_symbol('liburing.h', 'io_uring_prep_w= ritev2')) + config_host_data.set('HAVE_IO_URING_CQ_HAS_OVERFLOW', + cc.has_header_symbol('liburing.h', 'io_uring_cq_has= _overflow')) endif config_host_data.set('HAVE_TCP_KEEPCNT', cc.has_header_symbol('netinet/tcp.h', 'TCP_KEEPCNT') = or --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896916; cv=none; d=zohomail.com; s=zohoarc; b=KhR9x/KH65RJ8DnK6nD7b25YyMXXVES01N22+Du4Mom2XVsKvnJ6iximAoU0skUqlD9me+NyRwGC2YXt9tK4E4s4pwRNVZ2smaH+Mbtwa89BUc2FKHfvdQ8MccMiy9A6dQfIKP1ciz4uTR8aKkUPVD6lHt+3BfhkqoQ6AfS6A60= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896916; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tyHY+uxi7KNyMyWckcS9a4fpYauibRji/fQXyLOkJ+Y=; b=Q5K5RN4dB74VtW1medfB9i+Np//CBD15x3FGheOwcnP/827GuOkZZQBqUJR/choFoIg3+Bbc64forzFjr691SF68H1mCubGL6nxM/y5e8Dkh9F8KD83R7mRuppL3Oe4XHyOt9R3FGQCHiG/EsRCK5VD5nDbIL0Z2D+69IE4xY9Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896916558652.6987431269255; Tue, 11 Nov 2025 13:35:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzQ-0006uo-0e; Tue, 11 Nov 2025 16:33:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzN-0006tY-PS for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzL-0001qm-P7 for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:21 -0500 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-SKqt6HNkNLu7XCBC7UBXvA-1; Tue, 11 Nov 2025 16:33:16 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 162DC1800447; Tue, 11 Nov 2025 21:33:15 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7776430044E9; Tue, 11 Nov 2025 21:33:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tyHY+uxi7KNyMyWckcS9a4fpYauibRji/fQXyLOkJ+Y=; b=NG38YUKGqFttP66H/InbYeRrxLNV9Zyzt5fO9buqGwYqTT1H+copwW6s+38MQsciFlWYLE +9GLhvkPAfSLZAS6s3d7QTXimxYgB3d0lqXen8auEr/RXBccl5+HrEutGiczw/UayWQX0T HCeQtOZhLgMMXguUZqesCWEAg5Rj8k4= X-MC-Unique: SKqt6HNkNLu7XCBC7UBXvA-1 X-Mimecast-MFC-AGG-ID: SKqt6HNkNLu7XCBC7UBXvA_1762896795 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 07/28] aio: remove aio_context_use_g_source() Date: Tue, 11 Nov 2025 22:32:17 +0100 Message-ID: <20251111213238.181992-8-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896918983153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi There is no need for aio_context_use_g_source() now that epoll(7) and io_uring(7) file descriptor monitoring works with the glib event loop. AioContext doesn't need to be notified that GSource is being used. On hosts with io_uring support this now enables fdmon-io_uring.c by default, replacing fdmon-poll.c and fdmon-epoll.c. In other words, the event loop will use io_uring! Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Message-ID: <20251104022933.618123-8-stefanha@redhat.com> Signed-off-by: Kevin Wolf --- include/block/aio.h | 3 --- util/aio-posix.c | 12 ------------ util/aio-win32.c | 4 ---- util/async.c | 1 - 4 files changed, 20 deletions(-) diff --git a/include/block/aio.h b/include/block/aio.h index aef43f9c51..3ee6015424 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -728,9 +728,6 @@ void aio_context_setup(AioContext *ctx); */ void aio_context_destroy(AioContext *ctx); =20 -/* Used internally, do not call outside AioContext code */ -void aio_context_use_g_source(AioContext *ctx); - /** * aio_context_set_poll_params: * @ctx: the aio context diff --git a/util/aio-posix.c b/util/aio-posix.c index 9de05ee7e8..bebd9ce3a2 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -743,18 +743,6 @@ void aio_context_destroy(AioContext *ctx) aio_free_deleted_handlers(ctx); } =20 -void aio_context_use_g_source(AioContext *ctx) -{ - /* - * Disable io_uring when the glib main loop is used because it doesn't - * support mixed glib/aio_poll() usage. It relies on aio_poll() being - * called regularly so that changes to the monitored file descriptors = are - * submitted, otherwise a list of pending fd handlers builds up. - */ - fdmon_io_uring_destroy(ctx); - aio_free_deleted_handlers(ctx); -} - void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns, int64_t grow, int64_t shrink, Error **err= p) { diff --git a/util/aio-win32.c b/util/aio-win32.c index c6fbce64c2..18cc9fb7a9 100644 --- a/util/aio-win32.c +++ b/util/aio-win32.c @@ -427,10 +427,6 @@ void aio_context_destroy(AioContext *ctx) { } =20 -void aio_context_use_g_source(AioContext *ctx) -{ -} - void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns, int64_t grow, int64_t shrink, Error **err= p) { diff --git a/util/async.c b/util/async.c index 83f1e73e34..9d6f0c73ee 100644 --- a/util/async.c +++ b/util/async.c @@ -434,7 +434,6 @@ static GSourceFuncs aio_source_funcs =3D { =20 GSource *aio_get_g_source(AioContext *ctx) { - aio_context_use_g_source(ctx); g_source_ref(&ctx->source); return &ctx->source; } --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896856; cv=none; d=zohomail.com; s=zohoarc; b=XBD5tV6aUA6AKbCJKgkAmoU6GKjegQ7k1F5e+vZwJv2gZ62S1Gddu0mPIfJmgn9lalteqNnZRs+mbd+cglgQ77GCel9OVrjH4cTGKLQ3mLIW69Knbklr8cxXxs3JDAGs54LpvmQWjTC9lgbjJz9cDjiWfY0MZHYaZDg0bR/k8/U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896856; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ot2YRVb68vmbRabsegBRxrSHPHgWCylNtI18CWvtA6o=; b=lSLJAIkW+1PEZJG8F/LYVY+em1xBd0mzbYsBB53SQeuIzv06Z4A1ZrfKwHMzH8j2nrrIbxj2fVvSEmgOLY+QeyGlEJmY/Fh6qjBtD3gUkRVVZomBhc1GD+q0ifR7j83x25Vw/5ukkTLSaMvKiWxp+Y2MlCfxGHR0F472mVz7psU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896856540964.6637056156347; Tue, 11 Nov 2025 13:34:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzR-0006w3-Si; Tue, 11 Nov 2025 16:33:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzQ-0006vR-IU for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzP-0001rP-1U for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:24 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-284-He6cJ8AeOw-fw7jMVbLYGw-1; Tue, 11 Nov 2025 16:33:19 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 19647180048E; Tue, 11 Nov 2025 21:33:18 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0C8BF30044E0; Tue, 11 Nov 2025 21:33:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ot2YRVb68vmbRabsegBRxrSHPHgWCylNtI18CWvtA6o=; b=h3KxkCyoM2dWYUPnZxEWCUcsH9dX18laPf3zYXpmFlHwHERFs8vAwgZIzBaiKNL60yFGPW i0By0G10zone+hps9AO6cLBYN83Gx7i6Nrsyu3D8D6V0nDEVUBhl0QCO3paiJ2SS1jX99l b6yj/jotudvCW2zVV+PWzzmTGTSaEw8= X-MC-Unique: He6cJ8AeOw-fw7jMVbLYGw-1 X-Mimecast-MFC-AGG-ID: He6cJ8AeOw-fw7jMVbLYGw_1762896798 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 08/28] aio: free AioContext when aio_context_new() fails Date: Tue, 11 Nov 2025 22:32:18 +0100 Message-ID: <20251111213238.181992-9-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896858775153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi g_source_destroy() only removes the GSource from the GMainContext it's attached to, if any. It does not free it. Use g_source_unref() instead so that the AioContext (which embeds a GSource) is freed. There is no need to call g_source_destroy() in aio_context_new() because the GSource isn't attached to a GMainContext yet. aio_ctx_finalize() expects everything to be set up already, so introduce the new ctx->initialized boolean and do nothing when called with !initialized. This also requires moving aio_context_setup() down after event_notifier_init() since aio_ctx_finalize() won't release any resources that aio_context_setup() acquired. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Message-ID: <20251104022933.618123-9-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- include/block/aio.h | 3 +++ util/async.c | 31 ++++++++++++++++++++++++++++--- 2 files changed, 31 insertions(+), 3 deletions(-) diff --git a/include/block/aio.h b/include/block/aio.h index 3ee6015424..6cddf4d13b 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -291,6 +291,9 @@ struct AioContext { void *epollfd_tag; =20 const FDMonOps *fdmon_ops; + + /* Was aio_context_new() successful? */ + bool initialized; }; =20 /** diff --git a/util/async.c b/util/async.c index 9d6f0c73ee..95b1c6b610 100644 --- a/util/async.c +++ b/util/async.c @@ -366,12 +366,16 @@ aio_ctx_dispatch(GSource *source, } =20 static void -aio_ctx_finalize(GSource *source) +aio_ctx_finalize(GSource *source) { AioContext *ctx =3D (AioContext *) source; QEMUBH *bh; unsigned flags; =20 + if (!ctx->initialized) { + return; + } + thread_pool_free_aio(ctx->thread_pool); =20 #ifdef CONFIG_LINUX_AIO @@ -580,16 +584,35 @@ AioContext *aio_context_new(Error **errp) int ret; AioContext *ctx; =20 + /* + * ctx is freed by g_source_unref() (e.g. aio_context_unref()). ctx's + * resources are freed as follows: + * + * 1. By aio_ctx_finalize() after aio_context_new() has returned and s= et + * ->initialized =3D true. + * + * 2. By manual cleanup code in this function's error paths before goto + * fail. + * + * Be careful to free resources in both cases! + */ ctx =3D (AioContext *) g_source_new(&aio_source_funcs, sizeof(AioConte= xt)); QSLIST_INIT(&ctx->bh_list); QSIMPLEQ_INIT(&ctx->bh_slice_list); - aio_context_setup(ctx); =20 ret =3D event_notifier_init(&ctx->notifier, false); if (ret < 0) { error_setg_errno(errp, -ret, "Failed to initialize event notifier"= ); goto fail; } + + /* + * Resources cannot easily be freed manually after aio_context_setup()= . If + * you add any new resources to AioContext, it's probably best to acqu= ire + * them before aio_context_setup(). + */ + aio_context_setup(ctx); + g_source_set_can_recurse(&ctx->source, true); qemu_lockcnt_init(&ctx->list_lock); =20 @@ -623,9 +646,11 @@ AioContext *aio_context_new(Error **errp) =20 register_aiocontext(ctx); =20 + ctx->initialized =3D true; + return ctx; fail: - g_source_destroy(&ctx->source); + g_source_unref(&ctx->source); return NULL; } =20 --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896834; cv=none; d=zohomail.com; s=zohoarc; b=ZGlcDfthimiupCXs432vnGl4D8XFjd9TEoYQpeoTuS5/HrQ9xs8KYoGZNv0GEdGKK/cZawH2YyDr9x38tv55ePBCe4CEocOgtZXrXTFIxrMA07KxIEiYXfNZ5Lp6tg0u+yaMNLS8tudrC/jk+N9Ysw18Rnv2Jbe5NfOuVhxFms4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896834; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uWYRxSrvtjEffMo7XGHawDdlzSxJaz46itet0EPADNw=; b=fTAp0XWbAwUeBXgN7E9ISmQI2SIZ5abDn4wxbtdHksm0/NuTe9fx3vg8UNFXgUjK+/qOE+8CJI+GpW/svZjXWdZijC4V6N8Pw09OeppUXXO3hcLcYPIlIgNrhSh4f+q4hjkLWEu4H/Yt1+Ivj7uE8lt5RCjJzBEnQyqR5IoOu4A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896834784442.1511263202384; Tue, 11 Nov 2025 13:33:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzV-00070r-Dw; Tue, 11 Nov 2025 16:33:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzT-0006zv-Sj for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzR-0001rp-H8 for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:26 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-679-YOHW-GbzOSKo4H3XOvLZDQ-1; Tue, 11 Nov 2025 16:33:21 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B34D3195605C; Tue, 11 Nov 2025 21:33:20 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 93AF530044E5; Tue, 11 Nov 2025 21:33:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896804; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uWYRxSrvtjEffMo7XGHawDdlzSxJaz46itet0EPADNw=; b=YBVUpqhqmzjKLQP93vi5kv/Jhoyu1qMVkb2gHAWoBhBjf1pa6KOzDiO6wQ6lFrG9n9dO3j jS5HmUmRRA0edTxOoOsVb9MiINOjFPl4pJRO08EGM5mUkx3P/Ca87wj0C89/ntkN6su5P6 CnqvBq9BPsJQW59e2k6w1NL7uU9BJck= X-MC-Unique: YOHW-GbzOSKo4H3XOvLZDQ-1 X-Mimecast-MFC-AGG-ID: YOHW-GbzOSKo4H3XOvLZDQ_1762896800 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 09/28] aio: add errp argument to aio_context_setup() Date: Tue, 11 Nov 2025 22:32:19 +0100 Message-ID: <20251111213238.181992-10-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896838180158500 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi When aio_context_new() -> aio_context_setup() fails at startup it doesn't really matter whether errors are returned to the caller or the process terminates immediately. However, it is not acceptable to terminate when hotplugging --object iothread at runtime. Refactor aio_context_setup() so that errors can be propagated. The next commit will set errp when fdmon_io_uring_setup() fails. Suggested-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Message-ID: <20251104022933.618123-10-stefanha@redhat.com> Signed-off-by: Kevin Wolf --- include/block/aio.h | 5 ++++- util/aio-posix.c | 5 +++-- util/aio-win32.c | 3 ++- util/async.c | 6 +++++- 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/include/block/aio.h b/include/block/aio.h index 6cddf4d13b..13b6d34238 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -718,10 +718,13 @@ void qemu_set_current_aio_context(AioContext *ctx); /** * aio_context_setup: * @ctx: the aio context + * @errp: error pointer * * Initialize the aio context. + * + * Returns: true on success, false otherwise */ -void aio_context_setup(AioContext *ctx); +bool aio_context_setup(AioContext *ctx, Error **errp); =20 /** * aio_context_destroy: diff --git a/util/aio-posix.c b/util/aio-posix.c index bebd9ce3a2..9806a75c12 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -718,7 +718,7 @@ bool aio_poll(AioContext *ctx, bool blocking) return progress; } =20 -void aio_context_setup(AioContext *ctx) +bool aio_context_setup(AioContext *ctx, Error **errp) { ctx->fdmon_ops =3D &fdmon_poll_ops; ctx->epollfd =3D -1; @@ -726,10 +726,11 @@ void aio_context_setup(AioContext *ctx) =20 /* Use the fastest fd monitoring implementation if available */ if (fdmon_io_uring_setup(ctx)) { - return; + return true; } =20 fdmon_epoll_setup(ctx); + return true; } =20 void aio_context_destroy(AioContext *ctx) diff --git a/util/aio-win32.c b/util/aio-win32.c index 18cc9fb7a9..6e6f699e4b 100644 --- a/util/aio-win32.c +++ b/util/aio-win32.c @@ -419,8 +419,9 @@ bool aio_poll(AioContext *ctx, bool blocking) return progress; } =20 -void aio_context_setup(AioContext *ctx) +bool aio_context_setup(AioContext *ctx, Error **errp) { + return true; } =20 void aio_context_destroy(AioContext *ctx) diff --git a/util/async.c b/util/async.c index 95b1c6b610..7030047cfb 100644 --- a/util/async.c +++ b/util/async.c @@ -581,6 +581,7 @@ static void co_schedule_bh_cb(void *opaque) =20 AioContext *aio_context_new(Error **errp) { + ERRP_GUARD(); int ret; AioContext *ctx; =20 @@ -611,7 +612,10 @@ AioContext *aio_context_new(Error **errp) * you add any new resources to AioContext, it's probably best to acqu= ire * them before aio_context_setup(). */ - aio_context_setup(ctx); + if (!aio_context_setup(ctx, errp)) { + event_notifier_cleanup(&ctx->notifier); + goto fail; + } =20 g_source_set_can_recurse(&ctx->source, true); qemu_lockcnt_init(&ctx->list_lock); --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897031; cv=none; d=zohomail.com; s=zohoarc; b=OQhpiWdgbLBPUKfgFANfoK+NiwfU/CpMAb6AFS6vzr4GDGUyVRML2U53b1fGNy7ESA6cbuyPetAttMYEMWzX76lIqiYnwUW5g5GcxcLM1WHNawTqVXv6EdrhfRKRID4ztevxKLH6CtG21EgFClM2iPx7srYXR9HhqU7kOmeE4t8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897031; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3bNt+OO7opzAatiKioZ5eD/e3WK22clvl6Ox0lhDrrc=; b=W10DToqhUhwaeEDk5ONrk+oakfHV75JeECoFoWLE4a3SmQypg7KG1IH/RXOcR/X5YH9ykthXY3PvrDuWCKQWZmHiDIqU8q2BKn0Z/jbnEqO/pNG+tgrgsikHUpKNwtmeCl7HQHic2STpMB/tIXt2Gdu0Ep8Xw2s9AGZ6YzwgPj4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897031722903.9298379915346; Tue, 11 Nov 2025 13:37:11 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzX-00079N-5V; Tue, 11 Nov 2025 16:33:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzV-00070s-AK for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzT-0001s2-M7 for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:29 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-29-FDjPAWWaOcCx-nRyqY6VBg-1; Tue, 11 Nov 2025 16:33:23 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C950918007F2; Tue, 11 Nov 2025 21:33:22 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3A74D30044E0; Tue, 11 Nov 2025 21:33:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896807; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3bNt+OO7opzAatiKioZ5eD/e3WK22clvl6Ox0lhDrrc=; b=R/YLI3YpI1ckXe1ockQxfRi+ZPeBa8Il3r+MR5Czdqy3NzuFTY+3j7gzAGHsLXhimnrVB/ GI00XG4BgD9En0WipCXfci/2v5eUFuMIJyn/1ItHLPE9N/ch84DHW17pxDlDUxm5YVkEbE woAoPJTR0Bg2KHWoDDUvo0VoKgNCF4c= X-MC-Unique: FDjPAWWaOcCx-nRyqY6VBg-1 X-Mimecast-MFC-AGG-ID: FDjPAWWaOcCx-nRyqY6VBg_1762896802 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 10/28] aio-posix: gracefully handle io_uring_queue_init() failure Date: Tue, 11 Nov 2025 22:32:20 +0100 Message-ID: <20251111213238.181992-11-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897033265158500 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi io_uring may not be available at runtime due to system policies (e.g. the io_uring_disabled sysctl) or creation could fail due to file descriptor resource limits. Handle failure scenarios as follows: If another AioContext already has io_uring, then fail AioContext creation so that the aio_add_sqe() API is available uniformly from all QEMU threads. Otherwise fall back to epoll(7) if io_uring is unavailable. Notes: - Update the comment about selecting the fastest fdmon implementation. At this point it's not about speed anymore, it's about aio_add_sqe() API availability. - Uppercase the error message when converting from error_report() to error_setg_errno() for consistency (but there are instances of lowercase in the codebase). - It's easier to move the #ifdefs from aio-posix.h to aio-posix.c. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Message-ID: <20251104022933.618123-11-stefanha@redhat.com> Signed-off-by: Kevin Wolf --- util/aio-posix.h | 12 ++---------- util/aio-posix.c | 28 +++++++++++++++++++++++++--- util/fdmon-io_uring.c | 5 +++-- 3 files changed, 30 insertions(+), 15 deletions(-) diff --git a/util/aio-posix.h b/util/aio-posix.h index f9994ed79e..dfa1a51c0b 100644 --- a/util/aio-posix.h +++ b/util/aio-posix.h @@ -18,6 +18,7 @@ #define AIO_POSIX_H =20 #include "block/aio.h" +#include "qapi/error.h" =20 struct AioHandler { GPollFD pfd; @@ -72,17 +73,8 @@ static inline void fdmon_epoll_disable(AioContext *ctx) #endif /* !CONFIG_EPOLL_CREATE1 */ =20 #ifdef CONFIG_LINUX_IO_URING -bool fdmon_io_uring_setup(AioContext *ctx); +bool fdmon_io_uring_setup(AioContext *ctx, Error **errp); void fdmon_io_uring_destroy(AioContext *ctx); -#else -static inline bool fdmon_io_uring_setup(AioContext *ctx) -{ - return false; -} - -static inline void fdmon_io_uring_destroy(AioContext *ctx) -{ -} #endif /* !CONFIG_LINUX_IO_URING */ =20 #endif /* AIO_POSIX_H */ diff --git a/util/aio-posix.c b/util/aio-posix.c index 9806a75c12..c0285a26a3 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -16,6 +16,7 @@ #include "qemu/osdep.h" #include "block/block.h" #include "block/thread-pool.h" +#include "qapi/error.h" #include "qemu/main-loop.h" #include "qemu/lockcnt.h" #include "qemu/rcu.h" @@ -724,10 +725,29 @@ bool aio_context_setup(AioContext *ctx, Error **errp) ctx->epollfd =3D -1; ctx->epollfd_tag =3D NULL; =20 - /* Use the fastest fd monitoring implementation if available */ - if (fdmon_io_uring_setup(ctx)) { - return true; +#ifdef CONFIG_LINUX_IO_URING + { + static bool need_io_uring; + Error *local_err =3D NULL; /* ERRP_GUARD() doesn't handle error_ab= ort */ + + /* io_uring takes precedence because it provides aio_add_sqe() sup= port */ + if (fdmon_io_uring_setup(ctx, &local_err)) { + /* + * If one AioContext gets io_uring, then all AioContexts need = io_uring + * so that aio_add_sqe() support is available across all threa= ds. + */ + need_io_uring =3D true; + return true; + } + if (need_io_uring) { + error_propagate(errp, local_err); + return false; + } + + /* Silently fall back on systems where io_uring is unavailable */ + error_free(local_err); } +#endif /* CONFIG_LINUX_IO_URING */ =20 fdmon_epoll_setup(ctx); return true; @@ -735,7 +755,9 @@ bool aio_context_setup(AioContext *ctx, Error **errp) =20 void aio_context_destroy(AioContext *ctx) { +#ifdef CONFIG_LINUX_IO_URING fdmon_io_uring_destroy(ctx); +#endif =20 qemu_lockcnt_lock(&ctx->list_lock); fdmon_epoll_disable(ctx); diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index 2460ac1556..bcfdddf398 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -45,6 +45,7 @@ =20 #include "qemu/osdep.h" #include +#include "qapi/error.h" #include "qemu/rcu_queue.h" #include "aio-posix.h" =20 @@ -375,7 +376,7 @@ static const FDMonOps fdmon_io_uring_ops =3D { .gsource_dispatch =3D fdmon_io_uring_gsource_dispatch, }; =20 -bool fdmon_io_uring_setup(AioContext *ctx) +bool fdmon_io_uring_setup(AioContext *ctx, Error **errp) { int ret; =20 @@ -383,6 +384,7 @@ bool fdmon_io_uring_setup(AioContext *ctx) =20 ret =3D io_uring_queue_init(FDMON_IO_URING_ENTRIES, &ctx->fdmon_io_uri= ng, 0); if (ret !=3D 0) { + error_setg_errno(errp, -ret, "Failed to initialize io_uring"); return false; } =20 @@ -390,7 +392,6 @@ bool fdmon_io_uring_setup(AioContext *ctx) ctx->fdmon_ops =3D &fdmon_io_uring_ops; ctx->io_uring_fd_tag =3D g_source_add_unix_fd(&ctx->source, ctx->fdmon_io_uring.ring_fd, G_IO_IN); - return true; } =20 --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896907; cv=none; d=zohomail.com; s=zohoarc; b=nUaMqtH+2LFduozFqYjo3WvUjXem6oC1ORzXiMc+Zu3Tqc8ZT5Y1zn2PmXkEUWDka9lw/mvk1gOwvCYQEH82Rt/awnEtmVxjO6vVs+tHecyzo7xq0++Vu0jcqDjgkOIgG6qGF0Lv+wZBpFd4dbgYVhRbL7SyAQrwVxIRtyxvkdY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896907; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Hp2HhpVLeErR7H41RjTM9sLOi9rsqHpHrG7Qo1Z6S2w=; b=g8ZbCKv3P2tMgQ+qkBmKjvaxYajRNXT9d+hmYt807C7zYut2ZWFjruJAuYlQDMbyZkKT8GzyqJgX0lgD8EgUgxWVp1pkc42utGjXqkd/8p+9YqVVZL6Lwfns//qisKP5ETzI4AIn190Tj/fc9LT0fTpw6h4v9A2+afk2vJOXl7s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896907645832.0842782048485; Tue, 11 Nov 2025 13:35:07 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzZ-0007EM-N5; Tue, 11 Nov 2025 16:33:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzX-0007AN-PS for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzW-0001sZ-9m for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:31 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-593-17pEUVkkNvyEZz-n54qswA-1; Tue, 11 Nov 2025 16:33:26 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6F32918011EF; Tue, 11 Nov 2025 21:33:25 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4FE5330044E0; Tue, 11 Nov 2025 21:33:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hp2HhpVLeErR7H41RjTM9sLOi9rsqHpHrG7Qo1Z6S2w=; b=DhXv3s1MzHDnVFgWqh3vA4DJVIhofB5iDz9tCGotEOqyXnDxx5+plO2COOPL7/kVxa7NE5 hVtOsJ5/ixduPV2jZgftDxAxnGSWSjHfPByxXC48b5tOdAzQ8Cv2eD6KwM9MiajC+nGjWn tzQEsaMge7MBZjrV568pV1Mul0VKQ28= X-MC-Unique: 17pEUVkkNvyEZz-n54qswA-1 X-Mimecast-MFC-AGG-ID: 17pEUVkkNvyEZz-n54qswA_1762896805 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 11/28] aio-posix: unindent fdmon_io_uring_destroy() Date: Tue, 11 Nov 2025 22:32:21 +0100 Message-ID: <20251111213238.181992-12-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896910916153001 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi Reduce the level of indentation to make further code changes easier to read. Signed-off-by: Stefan Hajnoczi Message-ID: <20251104022933.618123-12-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- util/fdmon-io_uring.c | 54 ++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 26 deletions(-) diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index bcfdddf398..6ee6adbebb 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -397,33 +397,35 @@ bool fdmon_io_uring_setup(AioContext *ctx, Error **er= rp) =20 void fdmon_io_uring_destroy(AioContext *ctx) { - if (ctx->fdmon_ops =3D=3D &fdmon_io_uring_ops) { - AioHandler *node; - - io_uring_queue_exit(&ctx->fdmon_io_uring); - - /* Move handlers due to be removed onto the deleted list */ - while ((node =3D QSLIST_FIRST_RCU(&ctx->submit_list))) { - unsigned flags =3D qatomic_fetch_and(&node->flags, - ~(FDMON_IO_URING_PENDING | - FDMON_IO_URING_ADD | - FDMON_IO_URING_REMOVE | - FDMON_IO_URING_DELETE_AIO_HANDLER)); - - if ((flags & FDMON_IO_URING_REMOVE) || - (flags & FDMON_IO_URING_DELETE_AIO_HANDLER)) { - QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, - node, node_deleted); - } - - QSLIST_REMOVE_HEAD_RCU(&ctx->submit_list, node_submitted); - } + AioHandler *node; + + if (ctx->fdmon_ops !=3D &fdmon_io_uring_ops) { + return; + } + + io_uring_queue_exit(&ctx->fdmon_io_uring); =20 - g_source_remove_unix_fd(&ctx->source, ctx->io_uring_fd_tag); - ctx->io_uring_fd_tag =3D NULL; + /* Move handlers due to be removed onto the deleted list */ + while ((node =3D QSLIST_FIRST_RCU(&ctx->submit_list))) { + unsigned flags =3D qatomic_fetch_and(&node->flags, + ~(FDMON_IO_URING_PENDING | + FDMON_IO_URING_ADD | + FDMON_IO_URING_REMOVE | + FDMON_IO_URING_DELETE_AIO_HANDLER)); =20 - qemu_lockcnt_lock(&ctx->list_lock); - fdmon_poll_downgrade(ctx); - qemu_lockcnt_unlock(&ctx->list_lock); + if ((flags & FDMON_IO_URING_REMOVE) || + (flags & FDMON_IO_URING_DELETE_AIO_HANDLER)) { + QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, + node, node_deleted); + } + + QSLIST_REMOVE_HEAD_RCU(&ctx->submit_list, node_submitted); } + + g_source_remove_unix_fd(&ctx->source, ctx->io_uring_fd_tag); + ctx->io_uring_fd_tag =3D NULL; + + qemu_lockcnt_lock(&ctx->list_lock); + fdmon_poll_downgrade(ctx); + qemu_lockcnt_unlock(&ctx->list_lock); } --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897122; cv=none; d=zohomail.com; s=zohoarc; b=ME2o/gXPMTrZH8W4lT0TSOnA9hyHIQ0PWWAwwMkUuk3XA6WO9DGdDniJafxhZJOEolptYBlY4Jqg/qCEM7gfc+/GiuD4Snz5ES3YwoG+K/8pCMjSHs3EjlsnEOycs632JdBotdey36JFG2VW1wRqWeJdn4xSvnTdUJnBN58Lzn8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897122; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=I9ja/8Y+aWX0+58GLKoSYSCZiUYja82CJjZUvwp2kZM=; b=OYPjiMJyzLGkha/zCif2tXvhVlwzoNL1mQQkXU72NnoZ3aTfpMblxLaife2Yf3G0lo/kl/KPBlv+1TSjs6tR/5CF3HWwExAbp2hIlrCMZYY64FhhOEa691EwDya4yp6v1HsdVluSOAikQZAKuY1oQP98Db/stUEqpMm7cNTK6Co= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897122061435.94872190398576; Tue, 11 Nov 2025 13:38:42 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzd-0007Hs-B2; Tue, 11 Nov 2025 16:33:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzb-0007FE-B5 for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:35 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzY-0001sv-Ky for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:35 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-166-qu0UsUPgOcO-liKDS3c1MQ-1; Tue, 11 Nov 2025 16:33:28 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8339D195608A; Tue, 11 Nov 2025 21:33:27 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E9C7230044E5; Tue, 11 Nov 2025 21:33:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I9ja/8Y+aWX0+58GLKoSYSCZiUYja82CJjZUvwp2kZM=; b=I3AzwNKWAkke4JdSB+NfKdWfDQFsEkTFDDfStwdsP1jTRXTOJmiI7rVy6/LZQuUV3nUwCr gRKijGPV7v8gefJre6R4q8VaFw1dLTBS3/xf/mGNhc5mliBIvHGjDZRdGjkUlq/efgzhl5 n2ybsM3MRTRtgelM3+lhZexqBouv6Yc= X-MC-Unique: qu0UsUPgOcO-liKDS3c1MQ-1 X-Mimecast-MFC-AGG-ID: qu0UsUPgOcO-liKDS3c1MQ_1762896807 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 12/28] aio-posix: add fdmon_ops->dispatch() Date: Tue, 11 Nov 2025 22:32:22 +0100 Message-ID: <20251111213238.181992-13-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897123344153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi The ppoll and epoll file descriptor monitoring implementations rely on the event loop's generic file descriptor, timer, and BH dispatch code to invoke user callbacks. The io_uring file descriptor monitoring implementation will need io_uring-specific dispatch logic for CQE handlers for custom SQEs. Introduce a new FDMonOps ->dispatch() callback that allows file descriptor monitoring implementations to invoke user callbacks. The next patch will use this new callback. Signed-off-by: Stefan Hajnoczi Message-ID: <20251104022933.618123-13-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- include/block/aio.h | 19 +++++++++++++++++++ util/aio-posix.c | 9 +++++++++ 2 files changed, 28 insertions(+) diff --git a/include/block/aio.h b/include/block/aio.h index 13b6d34238..fc1a3c06e2 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -107,6 +107,25 @@ typedef struct { */ bool (*need_wait)(AioContext *ctx); =20 + /* + * dispatch: + * @ctx: the AioContext + * + * Dispatch any work that is specific to this file descriptor monitori= ng + * implementation. Usually the event loop's generic file descriptor + * monitoring, BH, and timer dispatching code is sufficient, but file + * descriptor monitoring implementations offering additional functiona= lity + * may need to implement this function for custom behavior. Called at a + * point in the event loop when it is safe to invoke user-defined + * callbacks. + * + * This function is optional and may be NULL. + * + * Returns: true if progress was made (see aio_poll()'s return value), + * false otherwise. + */ + bool (*dispatch)(AioContext *ctx); + /* * gsource_prepare: * @ctx: the AioContext diff --git a/util/aio-posix.c b/util/aio-posix.c index c0285a26a3..6ff36b6e51 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -385,10 +385,15 @@ void aio_dispatch(AioContext *ctx) AioHandlerList ready_list =3D QLIST_HEAD_INITIALIZER(ready_list); =20 qemu_lockcnt_inc(&ctx->list_lock); + aio_bh_poll(ctx); =20 ctx->fdmon_ops->gsource_dispatch(ctx, &ready_list); =20 + if (ctx->fdmon_ops->dispatch) { + ctx->fdmon_ops->dispatch(ctx); + } + /* block_ns is 0 because polling is disabled in the glib event loop */ aio_dispatch_ready_handlers(ctx, &ready_list, 0); =20 @@ -707,6 +712,10 @@ bool aio_poll(AioContext *ctx, bool blocking) block_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start; } =20 + if (ctx->fdmon_ops->dispatch) { + progress |=3D ctx->fdmon_ops->dispatch(ctx); + } + progress |=3D aio_bh_poll(ctx); progress |=3D aio_dispatch_ready_handlers(ctx, &ready_list, block_ns); =20 --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897010; cv=none; d=zohomail.com; s=zohoarc; b=IctTmnXC0/A3SXYPQ/PN6DEArljyf0ZbHWFACjtKQxwZeG6WYEBx13jMUk4ICo1+hDGdsjZKu3Wm0SCdgrWutdBp6ZRPiz1p3VjuOz1HhXo5x/POlkt0TJCWhMMHmv0NA5aMInQpJGby7O6KU6bvxnzDBqviWcrpqMvS2Rbu1VY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897010; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lwXgUXD7qlEUCdsZvzgaK7cFPmP4zuXzuJbnOA0x1us=; b=Jhp25Sc8KKqZN3TOltmMDtqJ5lRE1Xtm5RKdViBmUiKniIYUn0U7GwKjaWz/g1QA0Ky2DVkf6aPLKmB4Www16v7Wb5NjwiBbWSfbSORvuTO3pbmfpEWi+8BIZ+phgKOvWgHEG4FH7V3fWeZNfHdH3VSjYy3cEqei3cwhom6Wlgk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897010320236.45074377618835; Tue, 11 Nov 2025 13:36:50 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzh-0007QO-Jt; Tue, 11 Nov 2025 16:33:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvze-0007KW-OY for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzc-0001u6-GS for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:38 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-7-RpSachDGMPmZcE8h_PttqA-1; Tue, 11 Nov 2025 16:33:30 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 98BCF1956089; Tue, 11 Nov 2025 21:33:29 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 099D130044E0; Tue, 11 Nov 2025 21:33:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896815; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lwXgUXD7qlEUCdsZvzgaK7cFPmP4zuXzuJbnOA0x1us=; b=GIwHm4F0ZQyKWrWMmfWulekMqignOClGAd2ZlgupQ3nwnUY9+TDFiLOfR1txBj8ZFktwea Ht0G+GfDHQH4jCTlCgpNMrRP4OFTsTCwzEg7LlxLxW4HUaiuKw29vRJpkG/lofnoqEpf0B rDitF8wJoUCZQo82LMh3Dhz54lAfAM8= X-MC-Unique: RpSachDGMPmZcE8h_PttqA-1 X-Mimecast-MFC-AGG-ID: RpSachDGMPmZcE8h_PttqA_1762896809 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 13/28] aio-posix: add aio_add_sqe() API for user-defined io_uring requests Date: Tue, 11 Nov 2025 22:32:23 +0100 Message-ID: <20251111213238.181992-14-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897013185153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi Introduce the aio_add_sqe() API for submitting io_uring requests in the current AioContext. This allows other components in QEMU, like the block layer, to take advantage of io_uring features without creating their own io_uring context. This API supports nested event loops just like file descriptor monitoring and BHs do. This comes at a complexity cost: CQE callbacks must be placed on a list so that nested event loops can invoke pending CQE callbacks from parent event loops. If you're wondering why CqeHandler exists instead of just a callback function pointer, this is why. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Message-ID: <20251104022933.618123-14-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- include/block/aio.h | 83 +++++++++++++++++++++++++++++++- util/aio-posix.h | 1 + util/aio-posix.c | 9 ++++ util/fdmon-io_uring.c | 109 ++++++++++++++++++++++++++++++++++++------ util/trace-events | 4 ++ 5 files changed, 190 insertions(+), 16 deletions(-) diff --git a/include/block/aio.h b/include/block/aio.h index fc1a3c06e2..0f77ae2f64 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -61,6 +61,27 @@ typedef struct LuringState LuringState; /* Is polling disabled? */ bool aio_poll_disabled(AioContext *ctx); =20 +#ifdef CONFIG_LINUX_IO_URING +/* + * Each io_uring request must have a unique CqeHandler that processes the = cqe. + * The lifetime of a CqeHandler must be at least from aio_add_sqe() until + * ->cb() invocation. + */ +typedef struct CqeHandler CqeHandler; +struct CqeHandler { + /* Called by the AioContext when the request has completed */ + void (*cb)(CqeHandler *handler); + + /* Used internally, do not access this */ + QSIMPLEQ_ENTRY(CqeHandler) next; + + /* This field is filled in before ->cb() is called */ + struct io_uring_cqe cqe; +}; + +typedef QSIMPLEQ_HEAD(, CqeHandler) CqeHandlerSimpleQ; +#endif /* CONFIG_LINUX_IO_URING */ + /* Callbacks for file descriptor monitoring implementations */ typedef struct { /* @@ -157,6 +178,27 @@ typedef struct { * Called with list_lock incremented. */ void (*gsource_dispatch)(AioContext *ctx, AioHandlerList *ready_list); + +#ifdef CONFIG_LINUX_IO_URING + /** + * add_sqe: Add an io_uring sqe for submission. + * @prep_sqe: invoked with an sqe that should be prepared for submissi= on + * @opaque: user-defined argument to @prep_sqe() + * @cqe_handler: the unique cqe handler associated with this request + * + * The caller's @prep_sqe() function is invoked to fill in the details= of + * the sqe. Do not call io_uring_sqe_set_data() on this sqe. + * + * The kernel may see the sqe as soon as @prep_sqe() returns or it may= take + * until the next event loop iteration. + * + * This function is called from the current AioContext and is not + * thread-safe. + */ + void (*add_sqe)(AioContext *ctx, + void (*prep_sqe)(struct io_uring_sqe *sqe, void *opaqu= e), + void *opaque, CqeHandler *cqe_handler); +#endif /* CONFIG_LINUX_IO_URING */ } FDMonOps; =20 /* @@ -274,7 +316,10 @@ struct AioContext { struct io_uring fdmon_io_uring; AioHandlerSList submit_list; void *io_uring_fd_tag; -#endif + + /* Pending callback state for cqe handlers */ + CqeHandlerSimpleQ cqe_handler_ready_list; +#endif /* CONFIG_LINUX_IO_URING */ =20 /* TimerLists for calling timers - one per clock type. Has its own * locking. @@ -782,4 +827,40 @@ void aio_context_set_aio_params(AioContext *ctx, int64= _t max_batch); */ void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min, int64_t max, Error **errp); + +#ifdef CONFIG_LINUX_IO_URING +/** + * aio_has_io_uring: Return whether io_uring is available. + * + * io_uring is either available in all AioContexts or in none, so this only + * needs to be called once from within any thread's AioContext. + */ +static inline bool aio_has_io_uring(void) +{ + AioContext *ctx =3D qemu_get_current_aio_context(); + return ctx->fdmon_ops->add_sqe; +} + +/** + * aio_add_sqe: Add an io_uring sqe for submission. + * @prep_sqe: invoked with an sqe that should be prepared for submission + * @opaque: user-defined argument to @prep_sqe() + * @cqe_handler: the unique cqe handler associated with this request + * + * The caller's @prep_sqe() function is invoked to fill in the details of = the + * sqe. Do not call io_uring_sqe_set_data() on this sqe. + * + * The sqe is submitted by the current AioContext. The kernel may see the = sqe + * as soon as @prep_sqe() returns or it may take until the next event loop + * iteration. + * + * When the AioContext is destroyed, pending sqes are ignored and their + * CqeHandlers are not invoked. + * + * This function must be called only when aio_has_io_uring() returns true. + */ +void aio_add_sqe(void (*prep_sqe)(struct io_uring_sqe *sqe, void *opaque), + void *opaque, CqeHandler *cqe_handler); +#endif /* CONFIG_LINUX_IO_URING */ + #endif diff --git a/util/aio-posix.h b/util/aio-posix.h index dfa1a51c0b..babbfa8314 100644 --- a/util/aio-posix.h +++ b/util/aio-posix.h @@ -36,6 +36,7 @@ struct AioHandler { #ifdef CONFIG_LINUX_IO_URING QSLIST_ENTRY(AioHandler) node_submitted; unsigned flags; /* see fdmon-io_uring.c */ + CqeHandler internal_cqe_handler; /* used for POLL_ADD/POLL_REMOVE */ #endif int64_t poll_idle_timeout; /* when to stop userspace polling */ bool poll_ready; /* has polling detected an event? */ diff --git a/util/aio-posix.c b/util/aio-posix.c index 6ff36b6e51..e24b955fd9 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -806,3 +806,12 @@ void aio_context_set_aio_params(AioContext *ctx, int64= _t max_batch) =20 aio_notify(ctx); } + +#ifdef CONFIG_LINUX_IO_URING +void aio_add_sqe(void (*prep_sqe)(struct io_uring_sqe *sqe, void *opaque), + void *opaque, CqeHandler *cqe_handler) +{ + AioContext *ctx =3D qemu_get_current_aio_context(); + ctx->fdmon_ops->add_sqe(ctx, prep_sqe, opaque, cqe_handler); +} +#endif /* CONFIG_LINUX_IO_URING */ diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index 6ee6adbebb..d0b56127c6 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -46,8 +46,10 @@ #include "qemu/osdep.h" #include #include "qapi/error.h" +#include "qemu/defer-call.h" #include "qemu/rcu_queue.h" #include "aio-posix.h" +#include "trace.h" =20 enum { FDMON_IO_URING_ENTRIES =3D 128, /* sq/cq ring size */ @@ -76,8 +78,8 @@ static inline int pfd_events_from_poll(int poll_events) } =20 /* - * Returns an sqe for submitting a request. Only be called within - * fdmon_io_uring_wait(). + * Returns an sqe for submitting a request. Only called from the AioContext + * thread. */ static struct io_uring_sqe *get_sqe(AioContext *ctx) { @@ -168,23 +170,46 @@ static void fdmon_io_uring_update(AioContext *ctx, } } =20 +static void fdmon_io_uring_add_sqe(AioContext *ctx, + void (*prep_sqe)(struct io_uring_sqe *sqe, void *opaque), + void *opaque, CqeHandler *cqe_handler) +{ + struct io_uring_sqe *sqe =3D get_sqe(ctx); + + prep_sqe(sqe, opaque); + io_uring_sqe_set_data(sqe, cqe_handler); + + trace_fdmon_io_uring_add_sqe(ctx, opaque, sqe->opcode, sqe->fd, sqe->o= ff, + cqe_handler); +} + +static void fdmon_special_cqe_handler(CqeHandler *cqe_handler) +{ + /* + * This is an empty function that is never called. It is used as a fun= ction + * pointer to distinguish it from ordinary cqe handlers. + */ +} + static void add_poll_add_sqe(AioContext *ctx, AioHandler *node) { struct io_uring_sqe *sqe =3D get_sqe(ctx); int events =3D poll_events_from_pfd(node->pfd.events); =20 io_uring_prep_poll_add(sqe, node->pfd.fd, events); - io_uring_sqe_set_data(sqe, node); + node->internal_cqe_handler.cb =3D fdmon_special_cqe_handler; + io_uring_sqe_set_data(sqe, &node->internal_cqe_handler); } =20 static void add_poll_remove_sqe(AioContext *ctx, AioHandler *node) { struct io_uring_sqe *sqe =3D get_sqe(ctx); + CqeHandler *cqe_handler =3D &node->internal_cqe_handler; =20 #ifdef LIBURING_HAVE_DATA64 - io_uring_prep_poll_remove(sqe, (uintptr_t)node); + io_uring_prep_poll_remove(sqe, (uintptr_t)cqe_handler); #else - io_uring_prep_poll_remove(sqe, node); + io_uring_prep_poll_remove(sqe, cqe_handler); #endif io_uring_sqe_set_data(sqe, NULL); } @@ -219,19 +244,13 @@ static void fill_sq_ring(AioContext *ctx) } } =20 -/* Returns true if a handler became ready */ -static bool process_cqe(AioContext *ctx, - AioHandlerList *ready_list, - struct io_uring_cqe *cqe) +static bool process_cqe_aio_handler(AioContext *ctx, + AioHandlerList *ready_list, + AioHandler *node, + struct io_uring_cqe *cqe) { - AioHandler *node =3D io_uring_cqe_get_data(cqe); unsigned flags; =20 - /* poll_timeout and poll_remove have a zero user_data field */ - if (!node) { - return false; - } - /* * Deletion can only happen when IORING_OP_POLL_ADD completes. If we = race * with enqueue() here then we can safely clear the FDMON_IO_URING_REM= OVE @@ -255,6 +274,35 @@ static bool process_cqe(AioContext *ctx, return true; } =20 +/* Returns true if a handler became ready */ +static bool process_cqe(AioContext *ctx, + AioHandlerList *ready_list, + struct io_uring_cqe *cqe) +{ + CqeHandler *cqe_handler =3D io_uring_cqe_get_data(cqe); + + /* poll_timeout and poll_remove have a zero user_data field */ + if (!cqe_handler) { + return false; + } + + /* + * Special handling for AioHandler cqes. They need ready_list and have= a + * return value. + */ + if (cqe_handler->cb =3D=3D fdmon_special_cqe_handler) { + AioHandler *node =3D container_of(cqe_handler, AioHandler, + internal_cqe_handler); + return process_cqe_aio_handler(ctx, ready_list, node, cqe); + } + + cqe_handler->cqe =3D *cqe; + + /* Handlers are invoked later by fdmon_io_uring_dispatch() */ + QSIMPLEQ_INSERT_TAIL(&ctx->cqe_handler_ready_list, cqe_handler, next); + return false; +} + static int process_cq_ring(AioContext *ctx, AioHandlerList *ready_list) { struct io_uring *ring =3D &ctx->fdmon_io_uring; @@ -299,6 +347,32 @@ static bool fdmon_io_uring_gsource_check(AioContext *c= tx) return g_source_query_unix_fd(&ctx->source, tag) & G_IO_IN; } =20 +/* Dispatch CQE handlers that are ready */ +static bool fdmon_io_uring_dispatch(AioContext *ctx) +{ + CqeHandlerSimpleQ *ready_list =3D &ctx->cqe_handler_ready_list; + bool progress =3D false; + + /* Handlers may use defer_call() to coalesce frequent operations */ + defer_call_begin(); + + while (!QSIMPLEQ_EMPTY(ready_list)) { + CqeHandler *cqe_handler =3D QSIMPLEQ_FIRST(ready_list); + + QSIMPLEQ_REMOVE_HEAD(ready_list, next); + + trace_fdmon_io_uring_cqe_handler(ctx, cqe_handler, + cqe_handler->cqe.res); + cqe_handler->cb(cqe_handler); + progress =3D true; + } + + defer_call_end(); + + return progress; +} + + /* This is where CQEs are processed in the glib event loop */ static void fdmon_io_uring_gsource_dispatch(AioContext *ctx, AioHandlerList *ready_list) @@ -371,9 +445,11 @@ static const FDMonOps fdmon_io_uring_ops =3D { .update =3D fdmon_io_uring_update, .wait =3D fdmon_io_uring_wait, .need_wait =3D fdmon_io_uring_need_wait, + .dispatch =3D fdmon_io_uring_dispatch, .gsource_prepare =3D fdmon_io_uring_gsource_prepare, .gsource_check =3D fdmon_io_uring_gsource_check, .gsource_dispatch =3D fdmon_io_uring_gsource_dispatch, + .add_sqe =3D fdmon_io_uring_add_sqe, }; =20 bool fdmon_io_uring_setup(AioContext *ctx, Error **errp) @@ -389,6 +465,7 @@ bool fdmon_io_uring_setup(AioContext *ctx, Error **errp) } =20 QSLIST_INIT(&ctx->submit_list); + QSIMPLEQ_INIT(&ctx->cqe_handler_ready_list); ctx->fdmon_ops =3D &fdmon_io_uring_ops; ctx->io_uring_fd_tag =3D g_source_add_unix_fd(&ctx->source, ctx->fdmon_io_uring.ring_fd, G_IO_IN); @@ -425,6 +502,8 @@ void fdmon_io_uring_destroy(AioContext *ctx) g_source_remove_unix_fd(&ctx->source, ctx->io_uring_fd_tag); ctx->io_uring_fd_tag =3D NULL; =20 + assert(QSIMPLEQ_EMPTY(&ctx->cqe_handler_ready_list)); + qemu_lockcnt_lock(&ctx->list_lock); fdmon_poll_downgrade(ctx); qemu_lockcnt_unlock(&ctx->list_lock); diff --git a/util/trace-events b/util/trace-events index bd8f25fb59..540d662507 100644 --- a/util/trace-events +++ b/util/trace-events @@ -24,6 +24,10 @@ buffer_move_empty(const char *buf, size_t len, const cha= r *from) "%s: %zd bytes buffer_move(const char *buf, size_t len, const char *from) "%s: %zd bytes = from %s" buffer_free(const char *buf, size_t len) "%s: capacity %zd" =20 +# fdmon-io_uring.c +fdmon_io_uring_add_sqe(void *ctx, void *opaque, int opcode, int fd, uint64= _t off, void *cqe_handler) "ctx %p opaque %p opcode %d fd %d off %"PRId64" = cqe_handler %p" +fdmon_io_uring_cqe_handler(void *ctx, void *cqe_handler, int cqe_res) "ctx= %p cqe_handler %p cqe_res %d" + # filemonitor-inotify.c qemu_file_monitor_add_watch(void *mon, const char *dirpath, const char *fi= lename, void *cb, void *opaque, int64_t id) "File monitor %p add watch dir= =3D'%s' file=3D'%s' cb=3D%p opaque=3D%p id=3D%" PRId64 qemu_file_monitor_remove_watch(void *mon, const char *dirpath, int64_t id)= "File monitor %p remove watch dir=3D'%s' id=3D%" PRId64 --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896914; cv=none; d=zohomail.com; s=zohoarc; b=jjAH9HdvyNRkqknMxjsgbQWs4bcYOg8rcXAeeXNGtIUpeX363LBZOkvoIyDY87m3djq8whk8/ao9KDHn+gbGuc4emFYo81Fed9nn169rxoxHq4OgWutqEDXE8+uHJ3UkAd9SPgkVYVOb4kIhZacBipUqszDPRvBza77t42ytSsA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896914; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Tze90Duhftw0zO8K2Pzvu/Y1WMWidL+4DajWwxYd7Ps=; b=ccQwMNtopKVv88hhg46uKNynuVGbga+/Zh7U+IjgddteU6yxLURBVA3Yn799IC+Ywjlh9u4c0PVOfEwh9uTpZxPwTcqNDkFtTjylXR3C5WmkGHjCsGwJN4ovzYpuvqTJDzO9SKYenjeO3VebhSU/So3R2rOvn38QV7kTypEhE5o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896914329301.49174256100673; Tue, 11 Nov 2025 13:35:14 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzi-0007Rg-6D; Tue, 11 Nov 2025 16:33:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvze-0007KV-Nv for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzb-0001tb-7O for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:38 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-159-0HrPStrLOOGuJiksPVHN3w-1; Tue, 11 Nov 2025 16:33:32 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id ECEA21956095; Tue, 11 Nov 2025 21:33:31 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1DCE030044E0; Tue, 11 Nov 2025 21:33:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tze90Duhftw0zO8K2Pzvu/Y1WMWidL+4DajWwxYd7Ps=; b=f7Dd/mY5Ux7PupwWuWbGC6Wut+ctVIJkWIcs5KTogQI4DpmWW6tAwhUCDeGVb8UZwJmcJ1 pPkf5uRjhuE7k3IQM+suNyepplkiflraIgR4yIxG1JHoXlam1K3CKOle8f+4mfN3l3ZH/z 2vbyjKbd+Fq4te6y2kD/T9cVW+XI2Sc= X-MC-Unique: 0HrPStrLOOGuJiksPVHN3w-1 X-Mimecast-MFC-AGG-ID: 0HrPStrLOOGuJiksPVHN3w_1762896812 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 14/28] block/io_uring: use aio_add_sqe() Date: Tue, 11 Nov 2025 22:32:24 +0100 Message-ID: <20251111213238.181992-15-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896917187153000 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi AioContext has its own io_uring instance for file descriptor monitoring. The disk I/O io_uring code was developed separately. Originally I thought the characteristics of file descriptor monitoring and disk I/O were too different, requiring separate io_uring instances. Now it has become clear to me that it's feasible to share a single io_uring instance for file descriptor monitoring and disk I/O. We're not using io_uring's IOPOLL feature or anything else that would require a separate instance. Unify block/io_uring.c and util/fdmon-io_uring.c using the new aio_add_sqe() API that allows user-defined io_uring sqe submission. Now block/io_uring.c just needs to submit readv/writev/fsync and most of the io_uring-specific logic is handled by fdmon-io_uring.c. There are two immediate advantages: 1. Fewer system calls. There is no need to monitor the disk I/O io_uring ring fd from the file descriptor monitoring io_uring instance. Disk I/O completions are now picked up directly. Also, sqes are accumulated in the sq ring until the end of the event loop iteration and there are fewer io_uring_enter(2) syscalls. 2. Less code duplication. Note that error_setg() messages are not supposed to end with punctuation, so I removed a '.' for the non-io_uring build error message. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Message-ID: <20251104022933.618123-15-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- include/block/aio.h | 7 - include/block/raw-aio.h | 5 - block/file-posix.c | 40 ++-- block/io_uring.c | 489 ++++++++++------------------------------ stubs/io_uring.c | 32 --- util/async.c | 35 --- block/trace-events | 12 +- stubs/meson.build | 3 - 8 files changed, 130 insertions(+), 493 deletions(-) delete mode 100644 stubs/io_uring.c diff --git a/include/block/aio.h b/include/block/aio.h index 0f77ae2f64..f38b584ac7 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -310,8 +310,6 @@ struct AioContext { struct LinuxAioState *linux_aio; #endif #ifdef CONFIG_LINUX_IO_URING - LuringState *linux_io_uring; - /* State for file descriptor monitoring using Linux io_uring */ struct io_uring fdmon_io_uring; AioHandlerSList submit_list; @@ -615,11 +613,6 @@ struct LinuxAioState *aio_setup_linux_aio(AioContext *= ctx, Error **errp); /* Return the LinuxAioState bound to this AioContext */ struct LinuxAioState *aio_get_linux_aio(AioContext *ctx); =20 -/* Setup the LuringState bound to this AioContext */ -LuringState *aio_setup_linux_io_uring(AioContext *ctx, Error **errp); - -/* Return the LuringState bound to this AioContext */ -LuringState *aio_get_linux_io_uring(AioContext *ctx); /** * aio_timer_new_with_attrs: * @ctx: the aio context diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h index 6570244496..30e5fc9a9f 100644 --- a/include/block/raw-aio.h +++ b/include/block/raw-aio.h @@ -74,15 +74,10 @@ static inline bool laio_has_fua(void) #endif /* io_uring.c - Linux io_uring implementation */ #ifdef CONFIG_LINUX_IO_URING -LuringState *luring_init(Error **errp); -void luring_cleanup(LuringState *s); - /* luring_co_submit: submit I/O requests in the thread's current AioContex= t. */ int coroutine_fn luring_co_submit(BlockDriverState *bs, int fd, uint64_t o= ffset, QEMUIOVector *qiov, int type, BdrvRequestFlags flags); -void luring_detach_aio_context(LuringState *s, AioContext *old_context); -void luring_attach_aio_context(LuringState *s, AioContext *new_context); bool luring_has_fua(void); #else static inline bool luring_has_fua(void) diff --git a/block/file-posix.c b/block/file-posix.c index 8c738674ce..8b7c02d19a 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -755,14 +755,23 @@ static int raw_open_common(BlockDriverState *bs, QDic= t *options, } #endif /* !defined(CONFIG_LINUX_AIO) */ =20 -#ifndef CONFIG_LINUX_IO_URING if (s->use_linux_io_uring) { +#ifdef CONFIG_LINUX_IO_URING + if (!aio_has_io_uring()) { + error_setg(errp, "aio=3Dio_uring was specified, but is not " + "available (disabled via io_uring_disabled " + "sysctl or blocked by container runtime " + "seccomp policy?)"); + ret =3D -EINVAL; + goto fail; + } +#else error_setg(errp, "aio=3Dio_uring was specified, but is not support= ed " - "in this build."); + "in this build"); ret =3D -EINVAL; goto fail; - } #endif /* !defined(CONFIG_LINUX_IO_URING) */ + } =20 s->has_discard =3D true; s->has_write_zeroes =3D true; @@ -2522,27 +2531,6 @@ static bool bdrv_qiov_is_aligned(BlockDriverState *b= s, QEMUIOVector *qiov) return true; } =20 -#ifdef CONFIG_LINUX_IO_URING -static inline bool raw_check_linux_io_uring(BDRVRawState *s) -{ - Error *local_err =3D NULL; - AioContext *ctx; - - if (!s->use_linux_io_uring) { - return false; - } - - ctx =3D qemu_get_current_aio_context(); - if (unlikely(!aio_setup_linux_io_uring(ctx, &local_err))) { - error_reportf_err(local_err, "Unable to use linux io_uring, " - "falling back to thread pool: "); - s->use_linux_io_uring =3D false; - return false; - } - return true; -} -#endif - #ifdef CONFIG_LINUX_AIO static inline bool raw_check_linux_aio(BDRVRawState *s) { @@ -2595,7 +2583,7 @@ raw_co_prw(BlockDriverState *bs, int64_t *offset_ptr,= uint64_t bytes, if (s->needs_alignment && !bdrv_qiov_is_aligned(bs, qiov)) { type |=3D QEMU_AIO_MISALIGNED; #ifdef CONFIG_LINUX_IO_URING - } else if (raw_check_linux_io_uring(s)) { + } else if (s->use_linux_io_uring) { assert(qiov->size =3D=3D bytes); ret =3D luring_co_submit(bs, s->fd, offset, qiov, type, flags); goto out; @@ -2692,7 +2680,7 @@ static int coroutine_fn raw_co_flush_to_disk(BlockDri= verState *bs) }; =20 #ifdef CONFIG_LINUX_IO_URING - if (raw_check_linux_io_uring(s)) { + if (s->use_linux_io_uring) { return luring_co_submit(bs, s->fd, 0, NULL, QEMU_AIO_FLUSH, 0); } #endif diff --git a/block/io_uring.c b/block/io_uring.c index dd4f304910..dd930ee57e 100644 --- a/block/io_uring.c +++ b/block/io_uring.c @@ -11,28 +11,20 @@ #include "qemu/osdep.h" #include #include "block/aio.h" -#include "qemu/queue.h" #include "block/block.h" #include "block/raw-aio.h" #include "qemu/coroutine.h" -#include "qemu/defer-call.h" -#include "qapi/error.h" #include "system/block-backend.h" #include "trace.h" =20 -/* Only used for assertions. */ -#include "qemu/coroutine_int.h" - -/* io_uring ring size */ -#define MAX_ENTRIES 128 - -typedef struct LuringAIOCB { +typedef struct { Coroutine *co; - struct io_uring_sqe sqeq; - ssize_t ret; QEMUIOVector *qiov; - bool is_read; - QSIMPLEQ_ENTRY(LuringAIOCB) next; + uint64_t offset; + ssize_t ret; + int type; + int fd; + BdrvRequestFlags flags; =20 /* * Buffered reads may require resubmission, see @@ -40,36 +32,51 @@ typedef struct LuringAIOCB { */ int total_read; QEMUIOVector resubmit_qiov; -} LuringAIOCB; - -typedef struct LuringQueue { - unsigned int in_queue; - unsigned int in_flight; - bool blocked; - QSIMPLEQ_HEAD(, LuringAIOCB) submit_queue; -} LuringQueue; =20 -struct LuringState { - AioContext *aio_context; + CqeHandler cqe_handler; +} LuringRequest; =20 - struct io_uring ring; - - /* No locking required, only accessed from AioContext home thread */ - LuringQueue io_q; - - QEMUBH *completion_bh; -}; - -/** - * luring_resubmit: - * - * Resubmit a request by appending it to submit_queue. The caller must en= sure - * that ioq_submit() is called later so that submit_queue requests are sta= rted. - */ -static void luring_resubmit(LuringState *s, LuringAIOCB *luringcb) +static void luring_prep_sqe(struct io_uring_sqe *sqe, void *opaque) { - QSIMPLEQ_INSERT_TAIL(&s->io_q.submit_queue, luringcb, next); - s->io_q.in_queue++; + LuringRequest *req =3D opaque; + QEMUIOVector *qiov =3D req->qiov; + uint64_t offset =3D req->offset; + int fd =3D req->fd; + BdrvRequestFlags flags =3D req->flags; + + switch (req->type) { + case QEMU_AIO_WRITE: +#ifdef HAVE_IO_URING_PREP_WRITEV2 + { + int luring_flags =3D (flags & BDRV_REQ_FUA) ? RWF_DSYNC : 0; + io_uring_prep_writev2(sqe, fd, qiov->iov, + qiov->niov, offset, luring_flags); + } +#else + assert(flags =3D=3D 0); + io_uring_prep_writev(sqe, fd, qiov->iov, qiov->niov, offset); +#endif + break; + case QEMU_AIO_ZONE_APPEND: + io_uring_prep_writev(sqe, fd, qiov->iov, qiov->niov, offset); + break; + case QEMU_AIO_READ: + { + if (req->resubmit_qiov.iov !=3D NULL) { + qiov =3D &req->resubmit_qiov; + } + io_uring_prep_readv(sqe, fd, qiov->iov, qiov->niov, + offset + req->total_read); + break; + } + case QEMU_AIO_FLUSH: + io_uring_prep_fsync(sqe, fd, IORING_FSYNC_DATASYNC); + break; + default: + fprintf(stderr, "%s: invalid AIO request type, aborting 0x%x.\n", + __func__, req->type); + abort(); + } } =20 /** @@ -78,385 +85,115 @@ static void luring_resubmit(LuringState *s, LuringAIO= CB *luringcb) * Short reads are rare but may occur. The remaining read request needs to= be * resubmitted. */ -static void luring_resubmit_short_read(LuringState *s, LuringAIOCB *luring= cb, - int nread) +static void luring_resubmit_short_read(LuringRequest *req, int nread) { QEMUIOVector *resubmit_qiov; size_t remaining; =20 - trace_luring_resubmit_short_read(s, luringcb, nread); + trace_luring_resubmit_short_read(req, nread); =20 /* Update read position */ - luringcb->total_read +=3D nread; - remaining =3D luringcb->qiov->size - luringcb->total_read; + req->total_read +=3D nread; + remaining =3D req->qiov->size - req->total_read; =20 /* Shorten qiov */ - resubmit_qiov =3D &luringcb->resubmit_qiov; + resubmit_qiov =3D &req->resubmit_qiov; if (resubmit_qiov->iov =3D=3D NULL) { - qemu_iovec_init(resubmit_qiov, luringcb->qiov->niov); + qemu_iovec_init(resubmit_qiov, req->qiov->niov); } else { qemu_iovec_reset(resubmit_qiov); } - qemu_iovec_concat(resubmit_qiov, luringcb->qiov, luringcb->total_read, - remaining); + qemu_iovec_concat(resubmit_qiov, req->qiov, req->total_read, remaining= ); =20 - /* Update sqe */ - luringcb->sqeq.off +=3D nread; - luringcb->sqeq.addr =3D (uintptr_t)luringcb->resubmit_qiov.iov; - luringcb->sqeq.len =3D luringcb->resubmit_qiov.niov; - - luring_resubmit(s, luringcb); + aio_add_sqe(luring_prep_sqe, req, &req->cqe_handler); } =20 -/** - * luring_process_completions: - * @s: AIO state - * - * Fetches completed I/O requests, consumes cqes and invokes their callbac= ks - * The function is somewhat tricky because it supports nested event loops,= for - * example when a request callback invokes aio_poll(). - * - * Function schedules BH completion so it can be called again in a nested - * event loop. When there are no events left to complete the BH is being - * canceled. - * - */ -static void luring_process_completions(LuringState *s) +static void luring_cqe_handler(CqeHandler *cqe_handler) { - struct io_uring_cqe *cqes; - int total_bytes; - - defer_call_begin(); - - /* - * Request completion callbacks can run the nested event loop. - * Schedule ourselves so the nested event loop will "see" remaining - * completed requests and process them. Without this, completion - * callbacks that wait for other requests using a nested event loop - * would hang forever. - * - * This workaround is needed because io_uring uses poll_wait, which - * is woken up when new events are added to the uring, thus polling on - * the same uring fd will block unless more events are received. - * - * Other leaf block drivers (drivers that access the data themselves) - * are networking based, so they poll sockets for data and run the - * correct coroutine. - */ - qemu_bh_schedule(s->completion_bh); + LuringRequest *req =3D container_of(cqe_handler, LuringRequest, cqe_ha= ndler); + int ret =3D cqe_handler->cqe.res; =20 - while (io_uring_peek_cqe(&s->ring, &cqes) =3D=3D 0) { - LuringAIOCB *luringcb; - int ret; + trace_luring_cqe_handler(req, ret); =20 - if (!cqes) { - break; + if (ret < 0) { + /* + * Only writev/readv/fsync requests on regular files or host block + * devices are submitted. Therefore -EAGAIN is not expected but it= 's + * known to happen sometimes with Linux SCSI. Submit again and hope + * the request completes successfully. + * + * For more information, see: + * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@= kernel.dk/T/#u + * + * If the code is changed to submit other types of requests in the + * future, then this workaround may need to be extended to deal wi= th + * genuine -EAGAIN results that should not be resubmitted + * immediately. + */ + if (ret =3D=3D -EINTR || ret =3D=3D -EAGAIN) { + aio_add_sqe(luring_prep_sqe, req, &req->cqe_handler); + return; } - - luringcb =3D io_uring_cqe_get_data(cqes); - ret =3D cqes->res; - io_uring_cqe_seen(&s->ring, cqes); - cqes =3D NULL; - - /* Change counters one-by-one because we can be nested. */ - s->io_q.in_flight--; - trace_luring_process_completion(s, luringcb, ret); - + } else if (req->qiov) { /* total_read is non-zero only for resubmitted read requests */ - total_bytes =3D ret + luringcb->total_read; + int total_bytes =3D ret + req->total_read; =20 - if (ret < 0) { - /* - * Only writev/readv/fsync requests on regular files or host b= lock - * devices are submitted. Therefore -EAGAIN is not expected bu= t it's - * known to happen sometimes with Linux SCSI. Submit again and= hope - * the request completes successfully. - * - * For more information, see: - * https://lore.kernel.org/io-uring/20210727165811.284510-3-ax= boe@kernel.dk/T/#u - * - * If the code is changed to submit other types of requests in= the - * future, then this workaround may need to be extended to dea= l with - * genuine -EAGAIN results that should not be resubmitted - * immediately. - */ - if (ret =3D=3D -EINTR || ret =3D=3D -EAGAIN) { - luring_resubmit(s, luringcb); - continue; - } - } else if (!luringcb->qiov) { - goto end; - } else if (total_bytes =3D=3D luringcb->qiov->size) { + if (total_bytes =3D=3D req->qiov->size) { ret =3D 0; - /* Only read/write */ } else { /* Short Read/Write */ - if (luringcb->is_read) { + if (req->type =3D=3D QEMU_AIO_READ) { if (ret > 0) { - luring_resubmit_short_read(s, luringcb, ret); - continue; - } else { - /* Pad with zeroes */ - qemu_iovec_memset(luringcb->qiov, total_bytes, 0, - luringcb->qiov->size - total_bytes); - ret =3D 0; + luring_resubmit_short_read(req, ret); + return; } + + /* Pad with zeroes */ + qemu_iovec_memset(req->qiov, total_bytes, 0, + req->qiov->size - total_bytes); + ret =3D 0; } else { ret =3D -ENOSPC; } } -end: - luringcb->ret =3D ret; - qemu_iovec_destroy(&luringcb->resubmit_qiov); - - /* - * If the coroutine is already entered it must be in ioq_submit() - * and will notice luringcb->ret has been filled in when it - * eventually runs later. Coroutines cannot be entered recursively - * so avoid doing that! - */ - assert(luringcb->co->ctx =3D=3D s->aio_context); - if (!qemu_coroutine_entered(luringcb->co)) { - aio_co_wake(luringcb->co); - } - } - - qemu_bh_cancel(s->completion_bh); - - defer_call_end(); -} - -static int ioq_submit(LuringState *s) -{ - int ret =3D 0; - LuringAIOCB *luringcb, *luringcb_next; - - while (s->io_q.in_queue > 0) { - /* - * Try to fetch sqes from the ring for requests waiting in - * the overflow queue - */ - QSIMPLEQ_FOREACH_SAFE(luringcb, &s->io_q.submit_queue, next, - luringcb_next) { - struct io_uring_sqe *sqes =3D io_uring_get_sqe(&s->ring); - if (!sqes) { - break; - } - /* Prep sqe for submission */ - *sqes =3D luringcb->sqeq; - QSIMPLEQ_REMOVE_HEAD(&s->io_q.submit_queue, next); - } - ret =3D io_uring_submit(&s->ring); - trace_luring_io_uring_submit(s, ret); - /* Prevent infinite loop if submission is refused */ - if (ret <=3D 0) { - if (ret =3D=3D -EAGAIN || ret =3D=3D -EINTR) { - continue; - } - break; - } - s->io_q.in_flight +=3D ret; - s->io_q.in_queue -=3D ret; } - s->io_q.blocked =3D (s->io_q.in_queue > 0); =20 - if (s->io_q.in_flight) { - /* - * We can try to complete something just right away if there are - * still requests in-flight. - */ - luring_process_completions(s); - } - return ret; -} - -static void luring_process_completions_and_submit(LuringState *s) -{ - luring_process_completions(s); + req->ret =3D ret; + qemu_iovec_destroy(&req->resubmit_qiov); =20 - if (s->io_q.in_queue > 0) { - ioq_submit(s); - } -} - -static void qemu_luring_completion_bh(void *opaque) -{ - LuringState *s =3D opaque; - luring_process_completions_and_submit(s); -} - -static void qemu_luring_completion_cb(void *opaque) -{ - LuringState *s =3D opaque; - luring_process_completions_and_submit(s); -} - -static bool qemu_luring_poll_cb(void *opaque) -{ - LuringState *s =3D opaque; - - return io_uring_cq_ready(&s->ring); -} - -static void qemu_luring_poll_ready(void *opaque) -{ - LuringState *s =3D opaque; - - luring_process_completions_and_submit(s); -} - -static void ioq_init(LuringQueue *io_q) -{ - QSIMPLEQ_INIT(&io_q->submit_queue); - io_q->in_queue =3D 0; - io_q->in_flight =3D 0; - io_q->blocked =3D false; -} - -static void luring_deferred_fn(void *opaque) -{ - LuringState *s =3D opaque; - trace_luring_unplug_fn(s, s->io_q.blocked, s->io_q.in_queue, - s->io_q.in_flight); - if (!s->io_q.blocked && s->io_q.in_queue > 0) { - ioq_submit(s); - } -} - -/** - * luring_do_submit: - * @fd: file descriptor for I/O - * @luringcb: AIO control block - * @s: AIO state - * @offset: offset for request - * @type: type of request - * - * Fetches sqes from ring, adds to pending queue and preps them - * - */ -static int luring_do_submit(int fd, LuringAIOCB *luringcb, LuringState *s, - uint64_t offset, int type, BdrvRequestFlags fl= ags) -{ - int ret; - struct io_uring_sqe *sqes =3D &luringcb->sqeq; - - switch (type) { - case QEMU_AIO_WRITE: -#ifdef HAVE_IO_URING_PREP_WRITEV2 - { - int luring_flags =3D (flags & BDRV_REQ_FUA) ? RWF_DSYNC : 0; - io_uring_prep_writev2(sqes, fd, luringcb->qiov->iov, - luringcb->qiov->niov, offset, luring_flags); - } -#else - assert(flags =3D=3D 0); - io_uring_prep_writev(sqes, fd, luringcb->qiov->iov, - luringcb->qiov->niov, offset); -#endif - break; - case QEMU_AIO_ZONE_APPEND: - io_uring_prep_writev(sqes, fd, luringcb->qiov->iov, - luringcb->qiov->niov, offset); - break; - case QEMU_AIO_READ: - io_uring_prep_readv(sqes, fd, luringcb->qiov->iov, - luringcb->qiov->niov, offset); - break; - case QEMU_AIO_FLUSH: - io_uring_prep_fsync(sqes, fd, IORING_FSYNC_DATASYNC); - break; - default: - fprintf(stderr, "%s: invalid AIO request type, aborting 0x%x.\n", - __func__, type); - abort(); - } - io_uring_sqe_set_data(sqes, luringcb); - - QSIMPLEQ_INSERT_TAIL(&s->io_q.submit_queue, luringcb, next); - s->io_q.in_queue++; - trace_luring_do_submit(s, s->io_q.blocked, s->io_q.in_queue, - s->io_q.in_flight); - if (!s->io_q.blocked) { - if (s->io_q.in_flight + s->io_q.in_queue >=3D MAX_ENTRIES) { - ret =3D ioq_submit(s); - trace_luring_do_submit_done(s, ret); - return ret; - } - - defer_call(luring_deferred_fn, s); + /* + * If the coroutine is already entered it must be in luring_co_submit(= ) and + * will notice req->ret has been filled in when it eventually runs lat= er. + * Coroutines cannot be entered recursively so avoid doing that! + */ + if (!qemu_coroutine_entered(req->co)) { + aio_co_wake(req->co); } - return 0; } =20 -int coroutine_fn luring_co_submit(BlockDriverState *bs, int fd, uint64_t o= ffset, - QEMUIOVector *qiov, int type, - BdrvRequestFlags flags) +int coroutine_fn luring_co_submit(BlockDriverState *bs, int fd, + uint64_t offset, QEMUIOVector *qiov, + int type, BdrvRequestFlags flags) { - int ret; - AioContext *ctx =3D qemu_get_current_aio_context(); - LuringState *s =3D aio_get_linux_io_uring(ctx); - LuringAIOCB luringcb =3D { + LuringRequest req =3D { .co =3D qemu_coroutine_self(), - .ret =3D -EINPROGRESS, .qiov =3D qiov, - .is_read =3D (type =3D=3D QEMU_AIO_READ), + .ret =3D -EINPROGRESS, + .type =3D type, + .fd =3D fd, + .offset =3D offset, + .flags =3D flags, }; - trace_luring_co_submit(bs, s, &luringcb, fd, offset, qiov ? qiov->size= : 0, - type); - ret =3D luring_do_submit(fd, &luringcb, s, offset, type, flags); =20 - if (ret < 0) { - return ret; - } + req.cqe_handler.cb =3D luring_cqe_handler; =20 - if (luringcb.ret =3D=3D -EINPROGRESS) { - qemu_coroutine_yield(); - } - return luringcb.ret; -} - -void luring_detach_aio_context(LuringState *s, AioContext *old_context) -{ - aio_set_fd_handler(old_context, s->ring.ring_fd, - NULL, NULL, NULL, NULL, s); - qemu_bh_delete(s->completion_bh); - s->aio_context =3D NULL; -} - -void luring_attach_aio_context(LuringState *s, AioContext *new_context) -{ - s->aio_context =3D new_context; - s->completion_bh =3D aio_bh_new(new_context, qemu_luring_completion_bh= , s); - aio_set_fd_handler(s->aio_context, s->ring.ring_fd, - qemu_luring_completion_cb, NULL, - qemu_luring_poll_cb, qemu_luring_poll_ready, s); -} + trace_luring_co_submit(bs, &req, fd, offset, qiov ? qiov->size : 0, ty= pe); + aio_add_sqe(luring_prep_sqe, &req, &req.cqe_handler); =20 -LuringState *luring_init(Error **errp) -{ - int rc; - LuringState *s =3D g_new0(LuringState, 1); - struct io_uring *ring =3D &s->ring; - - trace_luring_init_state(s, sizeof(*s)); - - rc =3D io_uring_queue_init(MAX_ENTRIES, ring, 0); - if (rc < 0) { - error_setg_errno(errp, -rc, "failed to init linux io_uring ring"); - g_free(s); - return NULL; + if (req.ret =3D=3D -EINPROGRESS) { + qemu_coroutine_yield(); } - - ioq_init(&s->io_q); - return s; - -} - -void luring_cleanup(LuringState *s) -{ - io_uring_queue_exit(&s->ring); - trace_luring_cleanup_state(s); - g_free(s); + return req.ret; } =20 bool luring_has_fua(void) diff --git a/stubs/io_uring.c b/stubs/io_uring.c deleted file mode 100644 index 622d1e4648..0000000000 --- a/stubs/io_uring.c +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Linux io_uring support. - * - * Copyright (C) 2009 IBM, Corp. - * Copyright (C) 2009 Red Hat, Inc. - * - * This work is licensed under the terms of the GNU GPL, version 2 or late= r. - * See the COPYING file in the top-level directory. - */ -#include "qemu/osdep.h" -#include "block/aio.h" -#include "block/raw-aio.h" - -void luring_detach_aio_context(LuringState *s, AioContext *old_context) -{ - abort(); -} - -void luring_attach_aio_context(LuringState *s, AioContext *new_context) -{ - abort(); -} - -LuringState *luring_init(Error **errp) -{ - abort(); -} - -void luring_cleanup(LuringState *s) -{ - abort(); -} diff --git a/util/async.c b/util/async.c index 7030047cfb..48f8828636 100644 --- a/util/async.c +++ b/util/async.c @@ -386,14 +386,6 @@ aio_ctx_finalize(GSource *source) } #endif =20 -#ifdef CONFIG_LINUX_IO_URING - if (ctx->linux_io_uring) { - luring_detach_aio_context(ctx->linux_io_uring, ctx); - luring_cleanup(ctx->linux_io_uring); - ctx->linux_io_uring =3D NULL; - } -#endif - assert(QSLIST_EMPTY(&ctx->scheduled_coroutines)); qemu_bh_delete(ctx->co_schedule_bh); =20 @@ -469,29 +461,6 @@ LinuxAioState *aio_get_linux_aio(AioContext *ctx) } #endif =20 -#ifdef CONFIG_LINUX_IO_URING -LuringState *aio_setup_linux_io_uring(AioContext *ctx, Error **errp) -{ - if (ctx->linux_io_uring) { - return ctx->linux_io_uring; - } - - ctx->linux_io_uring =3D luring_init(errp); - if (!ctx->linux_io_uring) { - return NULL; - } - - luring_attach_aio_context(ctx->linux_io_uring, ctx); - return ctx->linux_io_uring; -} - -LuringState *aio_get_linux_io_uring(AioContext *ctx) -{ - assert(ctx->linux_io_uring); - return ctx->linux_io_uring; -} -#endif - void aio_notify(AioContext *ctx) { /* @@ -631,10 +600,6 @@ AioContext *aio_context_new(Error **errp) ctx->linux_aio =3D NULL; #endif =20 -#ifdef CONFIG_LINUX_IO_URING - ctx->linux_io_uring =3D NULL; -#endif - ctx->thread_pool =3D NULL; qemu_rec_mutex_init(&ctx->lock); timerlistgroup_init(&ctx->tlg, aio_timerlist_notify, ctx); diff --git a/block/trace-events b/block/trace-events index 8e789e1f12..c9b4736ff8 100644 --- a/block/trace-events +++ b/block/trace-events @@ -62,15 +62,9 @@ qmp_block_stream(void *bs) "bs %p" file_paio_submit(void *acb, void *opaque, int64_t offset, int count, int t= ype) "acb %p opaque %p offset %"PRId64" count %d type %d" =20 # io_uring.c -luring_init_state(void *s, size_t size) "s %p size %zu" -luring_cleanup_state(void *s) "%p freed" -luring_unplug_fn(void *s, int blocked, int queued, int inflight) "LuringSt= ate %p blocked %d queued %d inflight %d" -luring_do_submit(void *s, int blocked, int queued, int inflight) "LuringSt= ate %p blocked %d queued %d inflight %d" -luring_do_submit_done(void *s, int ret) "LuringState %p submitted to kerne= l %d" -luring_co_submit(void *bs, void *s, void *luringcb, int fd, uint64_t offse= t, size_t nbytes, int type) "bs %p s %p luringcb %p fd %d offset %" PRId64 = " nbytes %zd type %d" -luring_process_completion(void *s, void *aiocb, int ret) "LuringState %p l= uringcb %p ret %d" -luring_io_uring_submit(void *s, int ret) "LuringState %p ret %d" -luring_resubmit_short_read(void *s, void *luringcb, int nread) "LuringStat= e %p luringcb %p nread %d" +luring_cqe_handler(void *req, int ret) "req %p ret %d" +luring_co_submit(void *bs, void *req, int fd, uint64_t offset, size_t nbyt= es, int type) "bs %p req %p fd %d offset %" PRId64 " nbytes %zd type %d" +luring_resubmit_short_read(void *req, int nread) "req %p nread %d" =20 # qcow2.c qcow2_add_task(void *co, void *bs, void *pool, const char *action, int clu= ster_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, void *qio= v, size_t qiov_offset) "co %p bs %p pool %p: %s: cluster_type %d file_clust= er_offset %" PRIu64 " offset %" PRIu64 " bytes %" PRIu64 " qiov %p qiov_off= set %zu" diff --git a/stubs/meson.build b/stubs/meson.build index 27be2dec9f..0b2778c568 100644 --- a/stubs/meson.build +++ b/stubs/meson.build @@ -32,9 +32,6 @@ if have_block or have_ga stub_ss.add(files('cpus-virtual-clock.c')) stub_ss.add(files('icount.c')) stub_ss.add(files('graph-lock.c')) - if linux_io_uring.found() - stub_ss.add(files('io_uring.c')) - endif if libaio.found() stub_ss.add(files('linux-aio.c')) endif --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897089; cv=none; d=zohomail.com; s=zohoarc; b=aEKx1o7tFu2jIWv2fVlRWIySkvwUUppgafl/5FBJ03w0NBN+5k6yu6AagJeM445JVH/CyyGMyB8wi8uYo+okhmhGSarzl8k9UzDHp7sKEQxzwSSg+QN3+qdv/3undQbat4Hg4Bafh7eVpdDOD4MK4PcqDHOWKn5ycXaGm6V+Fw8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897089; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=VTY5t+mi42Og4AXvn+4mUT3pJyYT+KoDFmfS0kG2Pro=; b=UpLRvGZa3Mzcv8/FdAgwazUWvGznCZLbqxOIH43wcCD/OgkNdI0iMQMjgU20/Z264XnuPw5G7snkm4Koqx/K6IO4TMwxLcqg5l5crvE9nAD6gzuWHN+ZhsbSoTkx5pblJl1VByDDqE1P7Z/1j/5MXGRbCsVpdIVj7E0e2mPmG8I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897089906274.5159497906071; Tue, 11 Nov 2025 13:38:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzo-0007gJ-Dw; Tue, 11 Nov 2025 16:33:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzg-0007QX-Rs for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:41 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzf-0001vc-AH for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:40 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-516-I29Pz94mMCelgrQOSLk2eg-1; Tue, 11 Nov 2025 16:33:35 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 084BA1800358; Tue, 11 Nov 2025 21:33:34 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6F8B330044E0; Tue, 11 Nov 2025 21:33:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VTY5t+mi42Og4AXvn+4mUT3pJyYT+KoDFmfS0kG2Pro=; b=JcBE/Z5ZwzT9Ffc0AKUcex4gJT4qO5tetM2Yw66rmUqajCnwPrXIzHRNsCpeg2OgIHD6Xw 4lqERnIAy550lMME433/UaN/Q/bnx5clE7NvVQMb4N8GeeDiyxKSktv8WqVRz7h8+KR/TQ RbGJNPbzFNEyNCuFDFqAzmErCv9SgiQ= X-MC-Unique: I29Pz94mMCelgrQOSLk2eg-1 X-Mimecast-MFC-AGG-ID: I29Pz94mMCelgrQOSLk2eg_1762896814 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 15/28] block/io_uring: use non-vectored read/write when possible Date: Tue, 11 Nov 2025 22:32:25 +0100 Message-ID: <20251111213238.181992-16-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897090337158500 Content-Type: text/plain; charset="utf-8" From: Stefan Hajnoczi The io_uring_prep_readv2/writev2() man pages recommend using the non-vectored read/write operations when possible for performance reasons. I didn't measure a significant difference but it doesn't hurt to have this optimization in place. Suggested-by: Eric Blake Signed-off-by: Stefan Hajnoczi Message-ID: <20251104022933.618123-16-stefanha@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/io_uring.c | 34 ++++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/block/io_uring.c b/block/io_uring.c index dd930ee57e..f1514cf024 100644 --- a/block/io_uring.c +++ b/block/io_uring.c @@ -46,17 +46,28 @@ static void luring_prep_sqe(struct io_uring_sqe *sqe, v= oid *opaque) =20 switch (req->type) { case QEMU_AIO_WRITE: -#ifdef HAVE_IO_URING_PREP_WRITEV2 { int luring_flags =3D (flags & BDRV_REQ_FUA) ? RWF_DSYNC : 0; - io_uring_prep_writev2(sqe, fd, qiov->iov, - qiov->niov, offset, luring_flags); - } + if (luring_flags !=3D 0 || qiov->niov > 1) { +#ifdef HAVE_IO_URING_PREP_WRITEV2 + io_uring_prep_writev2(sqe, fd, qiov->iov, + qiov->niov, offset, luring_flags); #else - assert(flags =3D=3D 0); - io_uring_prep_writev(sqe, fd, qiov->iov, qiov->niov, offset); + /* + * FUA should only be enabled with HAVE_IO_URING_PREP_WRITEV2,= see + * luring_has_fua(). + */ + assert(luring_flags =3D=3D 0); + + io_uring_prep_writev(sqe, fd, qiov->iov, qiov->niov, offset); #endif + } else { + /* The man page says non-vectored is faster than vectored */ + struct iovec *iov =3D qiov->iov; + io_uring_prep_write(sqe, fd, iov->iov_base, iov->iov_len, offs= et); + } break; + } case QEMU_AIO_ZONE_APPEND: io_uring_prep_writev(sqe, fd, qiov->iov, qiov->niov, offset); break; @@ -65,8 +76,15 @@ static void luring_prep_sqe(struct io_uring_sqe *sqe, vo= id *opaque) if (req->resubmit_qiov.iov !=3D NULL) { qiov =3D &req->resubmit_qiov; } - io_uring_prep_readv(sqe, fd, qiov->iov, qiov->niov, - offset + req->total_read); + if (qiov->niov > 1) { + io_uring_prep_readv(sqe, fd, qiov->iov, qiov->niov, + offset + req->total_read); + } else { + /* The man page says non-vectored is faster than vectored */ + struct iovec *iov =3D qiov->iov; + io_uring_prep_read(sqe, fd, iov->iov_base, iov->iov_len, + offset + req->total_read); + } break; } case QEMU_AIO_FLUSH: --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897039; cv=none; d=zohomail.com; s=zohoarc; b=cJ0D1z12TqjZNWHt68eDnDPqI/0lQ29rFFWKsh9xroSJJAm9wfEQWQgGuxUKuvhdNy5StG4lMU5UvA1VYqVXVAc0F1sWVXp70Mkxcd3BecTuCrOrSkI0PyifkHZBL+nnavKO1wpUDSad9lKIvORXdtuRuJHKDIQLkmwOLCcDEe0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897039; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=e3Twk6+8ae4PwFeo7u1w75GEqQK8dI54LHe5+TXEM+Q=; b=H5+kxkQo2LJyZVl0xmXqg4RT9SxNlZJC9b4gCpwcE6QNIDSwaOF/0x11RoXXqkKgQI1bX4GcnWbPVRgJjwkhMql1l2lcN5XtMWIvIDsVR2UkaxGtWN4ddKIZsAFHGPtK3TW5SWt4DQHO+3oIDEtkfr196Sdl0wa6P4pDei1EZjw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 176289703964345.960860441387354; Tue, 11 Nov 2025 13:37:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzp-0007lt-5P; Tue, 11 Nov 2025 16:33:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzk-0007Z1-92 for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzh-0001yj-EE for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:43 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-368-gQEaC3j3MB2v3uNOniggfQ-1; Tue, 11 Nov 2025 16:33:37 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 269801956089; Tue, 11 Nov 2025 21:33:36 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8519130044E7; Tue, 11 Nov 2025 21:33:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896820; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e3Twk6+8ae4PwFeo7u1w75GEqQK8dI54LHe5+TXEM+Q=; b=BDwQWADUmdFWMdBHwO3jSC20VvZC3ckhDEnJKj5G5x4GSb+4kphg9gGUgSn+v9cYodM5/h CxJMvwdzU+sSIJPny5bes5gndc/ByDLdaLuZL8kAIbPVbrNZ2Os461olJXnTylI+Vuwffl /TtNxf5TLPd3xjqo8JsOPxGs5UWopx8= X-MC-Unique: gQEaC3j3MB2v3uNOniggfQ-1 X-Mimecast-MFC-AGG-ID: gQEaC3j3MB2v3uNOniggfQ_1762896816 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 16/28] block: replace TABs with space Date: Tue, 11 Nov 2025 22:32:26 +0100 Message-ID: <20251111213238.181992-17-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897041245153000 Content-Type: text/plain; charset="utf-8" From: Yeqi Fu Bring the block files in line with the QEMU coding style, with spaces for indentation. This patch partially resolves the issue 371. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/371 Signed-off-by: Yeqi Fu Message-ID: <20230325085224.23842-1-fufuyqqqqqq@gmail.com> [thuth: Rebased the patch to the current master branch] Signed-off-by: Thomas Huth Message-ID: <20251007163511.334178-1-thuth@redhat.com> [kwolf: Fixed up vertical alignemnt] Signed-off-by: Kevin Wolf --- include/block/nbd.h | 2 +- block/bochs.c | 14 +++++------ block/file-posix.c | 58 ++++++++++++++++++++++----------------------- block/file-win32.c | 38 ++++++++++++++--------------- block/qcow.c | 10 ++++---- 5 files changed, 61 insertions(+), 61 deletions(-) diff --git a/include/block/nbd.h b/include/block/nbd.h index 92987c76fd..ab40842da9 100644 --- a/include/block/nbd.h +++ b/include/block/nbd.h @@ -296,7 +296,7 @@ enum { NBD_CMD_BLOCK_STATUS =3D 7, }; =20 -#define NBD_DEFAULT_PORT 10809 +#define NBD_DEFAULT_PORT 10809 =20 /* Maximum size of a single READ/WRITE data buffer */ #define NBD_MAX_BUFFER_SIZE (32 * 1024 * 1024) diff --git a/block/bochs.c b/block/bochs.c index b099fb52fe..bfda88017d 100644 --- a/block/bochs.c +++ b/block/bochs.c @@ -300,15 +300,15 @@ static void bochs_close(BlockDriverState *bs) } =20 static BlockDriver bdrv_bochs =3D { - .format_name =3D "bochs", - .instance_size =3D sizeof(BDRVBochsState), - .bdrv_probe =3D bochs_probe, - .bdrv_open =3D bochs_open, + .format_name =3D "bochs", + .instance_size =3D sizeof(BDRVBochsState), + .bdrv_probe =3D bochs_probe, + .bdrv_open =3D bochs_open, .bdrv_child_perm =3D bdrv_default_perms, .bdrv_refresh_limits =3D bochs_refresh_limits, - .bdrv_co_preadv =3D bochs_co_preadv, - .bdrv_close =3D bochs_close, - .is_format =3D true, + .bdrv_co_preadv =3D bochs_co_preadv, + .bdrv_close =3D bochs_close, + .is_format =3D true, }; =20 static void bdrv_bochs_init(void) diff --git a/block/file-posix.c b/block/file-posix.c index 8b7c02d19a..12d12970fa 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -133,7 +133,7 @@ #define FTYPE_FILE 0 #define FTYPE_CD 1 =20 -#define MAX_BLOCKSIZE 4096 +#define MAX_BLOCKSIZE 4096 =20 /* Posix file locking bytes. Libvirt takes byte 0, we start from higher by= tes, * leaving a few more bytes for its future use. */ @@ -4562,20 +4562,20 @@ static void coroutine_fn cdrom_co_lock_medium(Block= DriverState *bs, bool locked) } =20 static BlockDriver bdrv_host_cdrom =3D { - .format_name =3D "host_cdrom", - .protocol_name =3D "host_cdrom", - .instance_size =3D sizeof(BDRVRawState), - .bdrv_needs_filename =3D true, - .bdrv_probe_device =3D cdrom_probe_device, - .bdrv_parse_filename =3D cdrom_parse_filename, - .bdrv_open =3D cdrom_open, - .bdrv_close =3D raw_close, - .bdrv_reopen_prepare =3D raw_reopen_prepare, - .bdrv_reopen_commit =3D raw_reopen_commit, - .bdrv_reopen_abort =3D raw_reopen_abort, - .bdrv_co_create_opts =3D bdrv_co_create_opts_simple, - .create_opts =3D &bdrv_create_opts_simple, - .mutable_opts =3D mutable_opts, + .format_name =3D "host_cdrom", + .protocol_name =3D "host_cdrom", + .instance_size =3D sizeof(BDRVRawState), + .bdrv_needs_filename =3D true, + .bdrv_probe_device =3D cdrom_probe_device, + .bdrv_parse_filename =3D cdrom_parse_filename, + .bdrv_open =3D cdrom_open, + .bdrv_close =3D raw_close, + .bdrv_reopen_prepare =3D raw_reopen_prepare, + .bdrv_reopen_commit =3D raw_reopen_commit, + .bdrv_reopen_abort =3D raw_reopen_abort, + .bdrv_co_create_opts =3D bdrv_co_create_opts_simple, + .create_opts =3D &bdrv_create_opts_simple, + .mutable_opts =3D mutable_opts, .bdrv_co_invalidate_cache =3D raw_co_invalidate_cache, =20 .bdrv_co_preadv =3D raw_co_preadv, @@ -4688,20 +4688,20 @@ static void coroutine_fn cdrom_co_lock_medium(Block= DriverState *bs, bool locked) } =20 static BlockDriver bdrv_host_cdrom =3D { - .format_name =3D "host_cdrom", - .protocol_name =3D "host_cdrom", - .instance_size =3D sizeof(BDRVRawState), - .bdrv_needs_filename =3D true, - .bdrv_probe_device =3D cdrom_probe_device, - .bdrv_parse_filename =3D cdrom_parse_filename, - .bdrv_open =3D cdrom_open, - .bdrv_close =3D raw_close, - .bdrv_reopen_prepare =3D raw_reopen_prepare, - .bdrv_reopen_commit =3D raw_reopen_commit, - .bdrv_reopen_abort =3D raw_reopen_abort, - .bdrv_co_create_opts =3D bdrv_co_create_opts_simple, - .create_opts =3D &bdrv_create_opts_simple, - .mutable_opts =3D mutable_opts, + .format_name =3D "host_cdrom", + .protocol_name =3D "host_cdrom", + .instance_size =3D sizeof(BDRVRawState), + .bdrv_needs_filename =3D true, + .bdrv_probe_device =3D cdrom_probe_device, + .bdrv_parse_filename =3D cdrom_parse_filename, + .bdrv_open =3D cdrom_open, + .bdrv_close =3D raw_close, + .bdrv_reopen_prepare =3D raw_reopen_prepare, + .bdrv_reopen_commit =3D raw_reopen_commit, + .bdrv_reopen_abort =3D raw_reopen_abort, + .bdrv_co_create_opts =3D bdrv_co_create_opts_simple, + .create_opts =3D &bdrv_create_opts_simple, + .mutable_opts =3D mutable_opts, =20 .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, diff --git a/block/file-win32.c b/block/file-win32.c index af9aea631c..0efb609e1d 100644 --- a/block/file-win32.c +++ b/block/file-win32.c @@ -741,16 +741,16 @@ static QemuOptsList raw_create_opts =3D { }; =20 BlockDriver bdrv_file =3D { - .format_name =3D "file", - .protocol_name =3D "file", - .instance_size =3D sizeof(BDRVRawState), - .bdrv_needs_filename =3D true, - .bdrv_parse_filename =3D raw_parse_filename, - .bdrv_open =3D raw_open, - .bdrv_refresh_limits =3D raw_probe_alignment, - .bdrv_close =3D raw_close, - .bdrv_co_create_opts =3D raw_co_create_opts, - .bdrv_has_zero_init =3D bdrv_has_zero_init_1, + .format_name =3D "file", + .protocol_name =3D "file", + .instance_size =3D sizeof(BDRVRawState), + .bdrv_needs_filename =3D true, + .bdrv_parse_filename =3D raw_parse_filename, + .bdrv_open =3D raw_open, + .bdrv_refresh_limits =3D raw_probe_alignment, + .bdrv_close =3D raw_close, + .bdrv_co_create_opts =3D raw_co_create_opts, + .bdrv_has_zero_init =3D bdrv_has_zero_init_1, =20 .bdrv_reopen_prepare =3D raw_reopen_prepare, .bdrv_reopen_commit =3D raw_reopen_commit, @@ -914,15 +914,15 @@ done: } =20 static BlockDriver bdrv_host_device =3D { - .format_name =3D "host_device", - .protocol_name =3D "host_device", - .instance_size =3D sizeof(BDRVRawState), - .bdrv_needs_filename =3D true, - .bdrv_parse_filename =3D hdev_parse_filename, - .bdrv_probe_device =3D hdev_probe_device, - .bdrv_open =3D hdev_open, - .bdrv_close =3D raw_close, - .bdrv_refresh_limits =3D hdev_refresh_limits, + .format_name =3D "host_device", + .protocol_name =3D "host_device", + .instance_size =3D sizeof(BDRVRawState), + .bdrv_needs_filename =3D true, + .bdrv_parse_filename =3D hdev_parse_filename, + .bdrv_probe_device =3D hdev_probe_device, + .bdrv_open =3D hdev_open, + .bdrv_close =3D raw_close, + .bdrv_refresh_limits =3D hdev_refresh_limits, =20 .bdrv_aio_preadv =3D raw_aio_preadv, .bdrv_aio_pwritev =3D raw_aio_pwritev, diff --git a/block/qcow.c b/block/qcow.c index 8a3e7591a9..b442bfe835 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -1184,11 +1184,11 @@ static const char *const qcow_strong_runtime_opts[]= =3D { }; =20 static BlockDriver bdrv_qcow =3D { - .format_name =3D "qcow", - .instance_size =3D sizeof(BDRVQcowState), - .bdrv_probe =3D qcow_probe, - .bdrv_open =3D qcow_open, - .bdrv_close =3D qcow_close, + .format_name =3D "qcow", + .instance_size =3D sizeof(BDRVQcowState), + .bdrv_probe =3D qcow_probe, + .bdrv_open =3D qcow_open, + .bdrv_close =3D qcow_close, .bdrv_child_perm =3D bdrv_default_perms, .bdrv_reopen_prepare =3D qcow_reopen_prepare, .bdrv_co_create =3D qcow_co_create, --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896935; cv=none; d=zohomail.com; s=zohoarc; b=RoHPKmtxhsqHQfFUy80Y0U4yzQhED2FWwEwL/MhAZSJ9LXdrtKZ/CaIN5mUiQQE16fqHfTlPgl0P8BfmheZ+F1BZOPzGp6kj5Hp+erq7wR9ZKA9p6DOUPbUl9xTSqFRxmUzdIoj0YAA3UKD2a8jYFo0VXVmgxYgnsElWpl60xEo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896935; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lKtvsZ/mQSLh7/SR+SHV5PkkdTjZ3ppItOA25V3o2fM=; b=VcLMjyRsQGnf66ENL9NHNCMVF9vzfw51Up4LZaqDJofI0fSk2pKthjZPALTwE9f/F5hg7J5E5z5oITEjArxuqPtTcC2r/LYqhKTYdZ9Mf2133B1gITzVBHb4Sstt5gJgDmJHZnA6lzkGQ8uXQQcWCnJjc0jY5FBJdFiTzaiCNg4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896935577199.65416403756467; Tue, 11 Nov 2025 13:35:35 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzt-000856-3G; Tue, 11 Nov 2025 16:33:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzm-0007dt-Np for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzk-000224-9W for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:46 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-7HEBf6mVNn6ItE_7jqGzGA-1; Tue, 11 Nov 2025 16:33:39 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 401C41956061; Tue, 11 Nov 2025 21:33:38 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9FBF730044E0; Tue, 11 Nov 2025 21:33:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lKtvsZ/mQSLh7/SR+SHV5PkkdTjZ3ppItOA25V3o2fM=; b=DgvotE4PLL8tLMe0nZXg+INobsqA2VI7IFKA8kejCTLhIvfC81Tf2kpXHhXuADH3kVa1tJ GJVOeHWvl/UCbaCCA9dm8//qEAWtJ45crghWCARMv8H1CGLahri1MDFhF2+OR/RK6257Hw InzTAZQs8qz//GPmhl4dAH3XKHQ+YhI= X-MC-Unique: 7HEBf6mVNn6ItE_7jqGzGA-1 X-Mimecast-MFC-AGG-ID: 7HEBf6mVNn6ItE_7jqGzGA_1762896818 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 17/28] block: Drop detach_subchain for bdrv_replace_node Date: Tue, 11 Nov 2025 22:32:27 +0100 Message-ID: <20251111213238.181992-18-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896937027153000 Content-Type: text/plain; charset="utf-8" From: Wesley Hershberger Detaching filters using detach_subchain=3Dtrue can cause segfaults as described in #3149. More specifically, this was observed when executing concurrent block-stream and query-named-block-nodes. block-stream adds a copy-on-read filter as the main BDS for the blockjob; that filter was dropped with detach_subchain=3Dtrue but not unref'd until the the blockjob was free'd. Because query-named-block-nodes assumes that a filter will always have exactly one child, it caused a segfault when it observed the detached filter. Stacktrace: 0 bdrv_refresh_filename (bs=3D0x5efed72f8350) at /usr/src/qemu-1:10.1.0+ds-5ubuntu2/b/qemu/block.c:8082 1 0x00005efea73cf9dc in bdrv_block_device_info (blk=3D0x0, bs=3D0x5efed72f8350, flat=3Dtrue, errp=3D0x7ffeb829ebd8) at block/qapi.c:62 2 0x00005efea7391ed3 in bdrv_named_nodes_list (flat=3D, errp=3D0x7ffeb829ebd8) at /usr/src/qemu-1:10.1.0+ds-5ubuntu2/b/qemu/block.c:6275 3 0x00005efea7471993 in qmp_query_named_block_nodes (has_flat=3D, flat=3D, errp=3D0x7ffeb829e= bd8) at /usr/src/qemu-1:10.1.0+ds-5ubuntu2/b/qemu/blockdev.c:2834 4 qmp_marshal_query_named_block_nodes (args=3D, ret=3D0x7f2b753beec0, errp=3D0x7f2b753beec8) at qapi/qapi-commands-block-core.c:553 5 0x00005efea74f03a5 in do_qmp_dispatch_bh (opaque=3D0x7f2b753beed0) at qapi/qmp-dispatch.c:128 6 0x00005efea75108e6 in aio_bh_poll (ctx=3D0x5efed6f3f430) at util/async.c:219 7 0x00005efea74ffdb2 in aio_dispatch (ctx=3D0x5efed6f3f430) at util/aio-posix.c:436 8 0x00005efea7512846 in aio_ctx_dispatch (source=3D, callback=3D,user_data=3D) at util/async.c:361 9 0x00007f2b77809bfb in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 10 0x00007f2b77809e70 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 11 0x00005efea7517228 in glib_pollfds_poll () at util/main-loop.c:287 12 os_host_main_loop_wait (timeout=3D0) at util/main-loop.c:310 13 main_loop_wait (nonblocking=3D) at util/main-loop.c:589 14 0x00005efea7140482 in qemu_main_loop () at system/runstate.c:905 15 0x00005efea744e4e8 in qemu_default_main (opaque=3Dopaque@entry=3D0x0) at system/main.c:50 16 0x00005efea6e76319 in main (argc=3D, argv=3D) at system/main.c:93 As discussed in 20251024-second-fix-3149-v1-1-d997fa3d5ce2@canonical.com, a filter should not exist without children in the first place; therefore, drop the parameter entirely as it is only used for filters. This is a partial revert of 3108a15cf09865456d499b08fe14e3dbec4ccbb3. After this change, a blockdev-backup job's copy-before-write filter will hold references to its children until the filter is unref'd. This causes an additional flush during bdrv_close, so also update iotest 257. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3149 Suggested-by: Kevin Wolf Signed-off-by: Wesley Hershberger Reviewed-by: Vladimir Sementsov-Ogievskiy Message-ID: <20251029-third-fix-3149-v2-1-94932bb404f4@canonical.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block.c | 38 ++++---------------------------------- tests/qemu-iotests/257 | 8 ++++++-- tests/qemu-iotests/257.out | 14 +++++++------- 3 files changed, 17 insertions(+), 43 deletions(-) diff --git a/block.c b/block.c index cf08e64add..0fe0152a7c 100644 --- a/block.c +++ b/block.c @@ -5398,17 +5398,13 @@ bdrv_replace_node_noperm(BlockDriverState *from, * * With auto_skip=3Dfalse the error is returned if from has a parent which= should * not be updated. - * - * With @detach_subchain=3Dtrue @to must be in a backing chain of @from. I= n this - * case backing link of the cow-parent of @to is removed. */ static int GRAPH_WRLOCK bdrv_replace_node_common(BlockDriverState *from, BlockDriverState *to, - bool auto_skip, bool detach_subchain, Error **err= p) + bool auto_skip, Error **errp) { Transaction *tran =3D tran_new(); g_autoptr(GSList) refresh_list =3D NULL; - BlockDriverState *to_cow_parent =3D NULL; int ret; =20 GLOBAL_STATE_CODE(); @@ -5417,17 +5413,6 @@ bdrv_replace_node_common(BlockDriverState *from, Blo= ckDriverState *to, assert(to->quiesce_counter); assert(bdrv_get_aio_context(from) =3D=3D bdrv_get_aio_context(to)); =20 - if (detach_subchain) { - assert(bdrv_chain_contains(from, to)); - assert(from !=3D to); - for (to_cow_parent =3D from; - bdrv_filter_or_cow_bs(to_cow_parent) !=3D to; - to_cow_parent =3D bdrv_filter_or_cow_bs(to_cow_parent)) - { - ; - } - } - /* * Do the replacement without permission update. * Replacement may influence the permissions, we should calculate new @@ -5439,11 +5424,6 @@ bdrv_replace_node_common(BlockDriverState *from, Blo= ckDriverState *to, goto out; } =20 - if (detach_subchain) { - /* to_cow_parent is already drained because from is drained */ - bdrv_remove_child(bdrv_filter_or_cow_child(to_cow_parent), tran); - } - refresh_list =3D g_slist_prepend(refresh_list, to); refresh_list =3D g_slist_prepend(refresh_list, from); =20 @@ -5462,7 +5442,7 @@ out: int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp) { - return bdrv_replace_node_common(from, to, true, false, errp); + return bdrv_replace_node_common(from, to, true, errp); } =20 int bdrv_drop_filter(BlockDriverState *bs, Error **errp) @@ -5478,7 +5458,7 @@ int bdrv_drop_filter(BlockDriverState *bs, Error **er= rp) =20 bdrv_drained_begin(child_bs); bdrv_graph_wrlock(); - ret =3D bdrv_replace_node_common(bs, child_bs, true, true, errp); + ret =3D bdrv_replace_node_common(bs, child_bs, true, errp); bdrv_graph_wrunlock(); bdrv_drained_end(child_bs); =20 @@ -5929,17 +5909,7 @@ int bdrv_drop_intermediate(BlockDriverState *top, Bl= ockDriverState *base, updated_children =3D g_slist_prepend(updated_children, c); } =20 - /* - * It seems correct to pass detach_subchain=3Dtrue here, but it trigge= rs - * one more yet not fixed bug, when due to nested aio_poll loop we swi= tch to - * another drained section, which modify the graph (for example, remov= ing - * the child, which we keep in updated_children list). So, it's a TODO. - * - * Note, bug triggered if pass detach_subchain=3Dtrue here and run - * test-bdrv-drain. test_drop_intermediate_poll() test-case will crash. - * That's a FIXME. - */ - bdrv_replace_node_common(top, base, false, false, &local_err); + bdrv_replace_node_common(top, base, false, &local_err); bdrv_graph_wrunlock(); =20 if (local_err) { diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257 index 7d3720b8e5..cd0468aaa1 100755 --- a/tests/qemu-iotests/257 +++ b/tests/qemu-iotests/257 @@ -310,14 +310,18 @@ def test_bitmap_sync(bsync_mode, msync_mode=3D'bitmap= ', failure=3DNone): 'state': 1, 'new_state': 2 }, { - 'event': 'read_aio', + 'event': 'flush_to_disk', 'state': 2, 'new_state': 3 + }, { + 'event': "read_aio", + 'state': 3, + 'new_state': 4 }], 'inject-error': [{ 'event': 'read_aio', 'errno': 5, - 'state': 3, + 'state': 4, 'immediately': False, 'once': True }] diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out index c33dd7f3a9..fb28333cb2 100644 --- a/tests/qemu-iotests/257.out +++ b/tests/qemu-iotests/257.out @@ -272,7 +272,7 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fback= up2" =3D=3D> Identical, OK! =20 --- Preparing image & VM --- =20 -{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "flush_to_disk", "new-state": 3, "state"= : 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "dr= ive0"}} {"return": {}} =20 --- Write #0 --- @@ -1017,7 +1017,7 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fba= ckup2" =3D=3D> Identical, OK! =20 --- Preparing image & VM --- =20 -{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "flush_to_disk", "new-state": 3, "state"= : 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "dr= ive0"}} {"return": {}} =20 --- Write #0 --- @@ -1762,7 +1762,7 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fba= ckup2" =3D=3D> Identical, OK! =20 --- Preparing image & VM --- =20 -{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "flush_to_disk", "new-state": 3, "state"= : 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "dr= ive0"}} {"return": {}} =20 --- Write #0 --- @@ -2507,7 +2507,7 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fba= ckup2" =3D=3D> Identical, OK! =20 --- Preparing image & VM --- =20 -{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "flush_to_disk", "new-state": 3, "state"= : 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "dr= ive0"}} {"return": {}} =20 --- Write #0 --- @@ -3252,7 +3252,7 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fba= ckup2" =3D=3D> Identical, OK! =20 --- Preparing image & VM --- =20 -{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "flush_to_disk", "new-state": 3, "state"= : 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "dr= ive0"}} {"return": {}} =20 --- Write #0 --- @@ -3997,7 +3997,7 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fba= ckup2" =3D=3D> Identical, OK! =20 --- Preparing image & VM --- =20 -{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "flush_to_disk", "new-state": 3, "state"= : 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "dr= ive0"}} {"return": {}} =20 --- Write #0 --- @@ -4742,7 +4742,7 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fba= ckup2" =3D=3D> Identical, OK! =20 --- Preparing image & VM --- =20 -{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "flush_to_disk", "new-state": 3, "state"= : 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "dr= ive0"}} {"return": {}} =20 --- Write #0 --- --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897016; cv=none; d=zohomail.com; s=zohoarc; b=Lu91f554qUu8Ggo+tEF54IsvNddUnHqmF1DFmZWVq9LRCYSM/nFCBy2EDt4NMzJ0D+vsofpVeSLF0eKE6mUxXxs3vPxoH6OVtALLjJv/Hq0mggyJB2m0S6OS/6W8AgF9x9yCzZaWm5vibWsJBF8f2MRNe0Zsxx2oVSCCP5CC9kU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897016; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2ZC45b+YLC8hAjPqvQ8hxvi6YolSPofBUAJKu4Vdz0c=; b=bFjlFBXNG3d5Y20vR1BNtoop6/OeOgxE3zj6S/yBwvFWfhTTIjaKVG/n5D9J3j8ACA4gt5570zw4xX2TYdD9FRLhrbVDQiItDmjik3gfP4eLJb+KDNLCY/7xHxHRHALM6MceIiuWTe3ycsSlMYSFqvSJlKn7qzzcNDwz/MoWCtE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897016765523.5569348809195; Tue, 11 Nov 2025 13:36:56 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzr-0007va-7W; Tue, 11 Nov 2025 16:33:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzm-0007bu-0V for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzk-000213-1J for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:45 -0500 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-70-2-yi3z9oOCm6qeMFQSha4A-1; Tue, 11 Nov 2025 16:33:41 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 723B91800451; Tue, 11 Nov 2025 21:33:40 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B804530044E0; Tue, 11 Nov 2025 21:33:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2ZC45b+YLC8hAjPqvQ8hxvi6YolSPofBUAJKu4Vdz0c=; b=T16kuN/7O5DelW9gXdjxYVaNzE+MaexxjfMiUn7HcRxXtUOcAusv9w5Qyf75HNUDs1R8nj FJrbNPDnbGHAbM1I6ces9sCOO3GxLAH4oCF7KYdj8+IZ27odLq9yZkETkPhLXzl1JTfmvF SUHt1s0u+A05JdPJWpR/Zksloxt2ZrQ= X-MC-Unique: 2-yi3z9oOCm6qeMFQSha4A-1 X-Mimecast-MFC-AGG-ID: 2-yi3z9oOCm6qeMFQSha4A_1762896820 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 18/28] iotests: Test resizing file node under raw with size/offset Date: Tue, 11 Nov 2025 22:32:28 +0100 Message-ID: <20251111213238.181992-19-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897019072158500 Content-Type: text/plain; charset="utf-8" This adds some more tests for using the 'size' and 'offset' options of raw to the recently added resize-below-raw test. Signed-off-by: Kevin Wolf Message-ID: <20251028094328.17919-1-kwolf@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/tests/resize-below-raw | 53 +++++++++++++++++-- tests/qemu-iotests/tests/resize-below-raw.out | 4 +- 2 files changed, 51 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/tests/resize-below-raw b/tests/qemu-iotests= /tests/resize-below-raw index 3c9241c918..ddf3f44742 100755 --- a/tests/qemu-iotests/tests/resize-below-raw +++ b/tests/qemu-iotests/tests/resize-below-raw @@ -8,16 +8,27 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 import os +from typing import Dict, Optional + import iotests from iotests import imgfmt, qemu_img_create, QMPTestCase =20 image_size =3D 1 * 1024 * 1024 image =3D os.path.join(iotests.test_dir, 'test.img') =20 -class TestResizeBelowRaw(QMPTestCase): +class BaseResizeBelowRaw(QMPTestCase): + raw_size: Optional[int] =3D None + raw_offset: Optional[int] =3D None + def setUp(self) -> None: qemu_img_create('-f', imgfmt, image, str(image_size)) =20 + extra_options: Dict[str, str] =3D {} + if self.raw_size is not None: + extra_options['size'] =3D str(self.raw_size) + if self.raw_offset is not None: + extra_options['offset'] =3D str(self.raw_offset) + self.vm =3D iotests.VM() self.vm.add_blockdev(self.vm.qmp_to_opts({ 'driver': imgfmt, @@ -26,7 +37,8 @@ class TestResizeBelowRaw(QMPTestCase): 'driver': 'file', 'filename': image, 'node-name': 'file0', - } + }, + **extra_options })) self.vm.launch() =20 @@ -34,14 +46,16 @@ class TestResizeBelowRaw(QMPTestCase): self.vm.shutdown() os.remove(image) =20 - def assert_size(self, size: int) -> None: + def assert_size(self, size: int, file_size: Optional[int] =3D None) ->= None: nodes =3D self.vm.qmp('query-named-block-nodes', flat=3DTrue)['ret= urn'] self.assertEqual(len(nodes), 2) for node in nodes: - if node['drv'] =3D=3D 'file': + if node['drv'] =3D=3D 'file' and file_size is not None: + self.assertEqual(node['image']['virtual-size'], file_size) continue self.assertEqual(node['image']['virtual-size'], size) =20 +class TestResizeBelowUnlimitedRaw(BaseResizeBelowRaw): def test_resize_below_raw(self) -> None: self.assert_size(image_size) self.vm.qmp('block_resize', node_name=3D'file0', size=3D2*image_si= ze) @@ -49,5 +63,36 @@ class TestResizeBelowRaw(QMPTestCase): self.vm.qmp('block_resize', node_name=3D'node0', size=3D3*image_si= ze) self.assert_size(3*image_size) =20 +# offset =3D 0 behaves the same as absent offset +class TestResizeBelowRawWithZeroOffset(TestResizeBelowUnlimitedRaw): + raw_offset =3D 0 + +class TestResizeBelowRawWithSize(BaseResizeBelowRaw): + raw_size =3D image_size // 2 + + def test_resize_below_raw_with_size(self) -> None: + self.assert_size(image_size // 2, image_size) + + # This QMP command fails because node0 unshares RESIZE + self.vm.qmp('block_resize', node_name=3D'file0', size=3D2*image_si= ze) + self.assert_size(image_size // 2, image_size) + + # This QMP command fails because node0 is a fixed-size disk + self.vm.qmp('block_resize', node_name=3D'node0', size=3D3*image_si= ze) + self.assert_size(image_size // 2, image_size) + +class TestResizeBelowRawWithOffset(BaseResizeBelowRaw): + raw_offset =3D image_size // 4 + + def test_resize_below_raw_with_offset(self) -> None: + self.assert_size(image_size * 3 // 4, image_size) + + # This QMP command fails because node0 unshares RESIZE + self.vm.qmp('block_resize', node_name=3D'file0', size=3D2*image_si= ze) + self.assert_size(image_size * 3 // 4, image_size) + + self.vm.qmp('block_resize', node_name=3D'node0', size=3D3*image_si= ze) + self.assert_size(3 * image_size, image_size * 13 // 4) + if __name__ =3D=3D '__main__': iotests.main(supported_fmts=3D['raw'], supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/tests/resize-below-raw.out b/tests/qemu-iot= ests/tests/resize-below-raw.out index ae1213e6f8..89968f35d7 100644 --- a/tests/qemu-iotests/tests/resize-below-raw.out +++ b/tests/qemu-iotests/tests/resize-below-raw.out @@ -1,5 +1,5 @@ -. +.... ---------------------------------------------------------------------- -Ran 1 tests +Ran 4 tests =20 OK --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896924; cv=none; d=zohomail.com; s=zohoarc; b=iHl0LDhkMPJ93bHyazZjHGCsJT2R+ug7SK71Fn9c/h6UA6FKXAOzKpjdaGHWbu2IG+fArgI0P8KC8lI/FYCh/ubAtpEpKhvaMS+ML1FRIgTP65LQjcumul87n12z1BfAXnvIRdzIUIKWLREMyWORfX4OXUAxc8zTiTki6MX+0DE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896924; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dhD6yXzsyjJ2UbHXYb+eLNCm6MRXpg+VIswlV1nXwjc=; b=DPVvmhW3fqgKPAbQJ8UTbk2tlS5rQgMBbo2h3Qo+200v+pId0mY9ZLckhylrTLHJbndZxm3rxge0irr7JnR05478FvojdvIdYdfmGi6oSSHxqNATbchS9U4fuVjM9IK5p0ameMbvoJyCi2it8gvCylsGZLjBH2wH9d3NCPUjDtE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17628969243761021.136284037457; Tue, 11 Nov 2025 13:35:24 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzs-00083n-KY; Tue, 11 Nov 2025 16:33:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzq-0007sA-Pf for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzp-00025l-6v for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:50 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-493-N5BMzMs3NkqYhoYCnrZ3Kg-1; Tue, 11 Nov 2025 16:33:43 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6C73D1956094; Tue, 11 Nov 2025 21:33:42 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D033C30044E7; Tue, 11 Nov 2025 21:33:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dhD6yXzsyjJ2UbHXYb+eLNCm6MRXpg+VIswlV1nXwjc=; b=RfCZcWyy5BlqECKI61txJblNf79U8RF82a1mJvSieMiUWjhBsNXsxHLNoUtdzPojiB00/M t8cFPWhSleo04sixN+Y7O5hkmBj3WyaiomRbbqwEZOdY3EiyR2ynd7JBWHRE9R5BbSv68+ cH5lpW4Wr+lfIW/3reXfnzmCFlxXElA= X-MC-Unique: N5BMzMs3NkqYhoYCnrZ3Kg-1 X-Mimecast-MFC-AGG-ID: N5BMzMs3NkqYhoYCnrZ3Kg_1762896822 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 19/28] qemu-img: Fix amend option parse error handling Date: Tue, 11 Nov 2025 22:32:29 +0100 Message-ID: <20251111213238.181992-20-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896925368158500 Content-Type: text/plain; charset="utf-8" From: Akihiko Odaki qemu_opts_del(opts) dereferences opts->list, which is the old amend_opts pointer that can be dangling after executing qemu_opts_append(amend_opts, bs->drv->create_opts) and cause use-after-free. Fix the potential use-after-free by moving the qemu_opts_del() call before the qemu_opts_append() call. Signed-off-by: Akihiko Odaki Message-ID: <20251023-iotests-v1-1-fab143ca4c2f@rsg.ci.i.u-tokyo.ac.jp> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- qemu-img.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qemu-img.c b/qemu-img.c index a7791896c1..7a32d2d16c 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -4597,9 +4597,9 @@ static int img_amend(const img_cmd_t *ccmd, int argc,= char **argv) amend_opts =3D qemu_opts_append(amend_opts, bs->drv->amend_opts); opts =3D qemu_opts_create(amend_opts, NULL, 0, &error_abort); if (!qemu_opts_do_parse(opts, options, NULL, &err)) { + qemu_opts_del(opts); /* Try to parse options using the create options */ amend_opts =3D qemu_opts_append(amend_opts, bs->drv->create_opts); - qemu_opts_del(opts); opts =3D qemu_opts_create(amend_opts, NULL, 0, &error_abort); if (qemu_opts_do_parse(opts, options, NULL, NULL)) { error_append_hint(&err, --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896924; cv=none; d=zohomail.com; s=zohoarc; b=EnfvtGsKYz+aURm8VSRL7DupbMp+UUZoKiGT806u0u9TqroC1+wSDcmNtQ2uy2nc2TgbyHGrS8bx9aePl44nbv3bfJ7SASPNd/8MLCeuGLYCRPLkwZJmLUyjDnRGCTfFMqYF4PD9NPQiZyhvzGjPktCSmJaUjSTvV4p6xcpBF3Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896924; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/JyZOAu8kJZG1l3piTbT+IR3lNXsnQ6tJXrxHnnA3CY=; b=Tr0smCnxh4NB6RzDLIMC/25yCSaZ2tvhz+jyyPfFCrF5ga63V68ocUwinnPaSQG4HhBRUS5W7KTX0sPmTKpGq6oEai2cyGQELUySoYtz53LpO1WZEVsljDvxw9VW2kE1Rt3LmvhvIJiY5LL6eE9aBtjjQdlMooaj8SuMyEewfRA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896924827957.0505324623718; Tue, 11 Nov 2025 13:35:24 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzu-0008Bx-Gd; Tue, 11 Nov 2025 16:33:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzr-0007ui-1j for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:51 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzp-00026F-CO for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:50 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-PISut6q3OleDLHpjIuA9Xw-1; Tue, 11 Nov 2025 16:33:45 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 834B419560B2; Tue, 11 Nov 2025 21:33:44 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E7F4B30044E0; Tue, 11 Nov 2025 21:33:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/JyZOAu8kJZG1l3piTbT+IR3lNXsnQ6tJXrxHnnA3CY=; b=GIBjvz/69KxSXxKlNKb3FnnytowsjRa4NoU6TfDU1rmc5CA0wcl1y8lyYB8YezLypDjEBY 3Qyy7HmzSJ2DaFMeejIYE4P8UrPvBPaVLHMsLd5BA5/+sgp4zuoog+p1qfN9g94TdG/Gpg lXjsMiKd0HMy645AmBGEaAnh0wGYQBY= X-MC-Unique: PISut6q3OleDLHpjIuA9Xw-1 X-Mimecast-MFC-AGG-ID: PISut6q3OleDLHpjIuA9Xw_1762896824 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 20/28] iotests: Run iotests with sanitizers Date: Tue, 11 Nov 2025 22:32:30 +0100 Message-ID: <20251111213238.181992-21-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896926930153000 From: Akihiko Odaki Commit 2cc4d1c5eab1 ("tests/check-block: Skip iotests when sanitizers are enabled") changed iotests to skip when sanitizers are enabled. The rationale is that AddressSanitizer emits warnings and reports leaks, which results in test breakage. Later, sanitizers that are enabled for production environments (safe-stack and cfi-icall) were exempted. However, this approach has a few problems. - It requires rebuild to disable sanitizers if the existing build has them enabled. - It disables other useful non-production sanitizers. - The exemption of safe-stack and cfi-icall is not correctly implemented, so qemu-iotests are incorrectly enabled whenever either safe-stack or cfi-icall is enabled *and*, even if there is another sanitizer like AddressSanitizer. To solve these problems, direct AddressSanitizer warnings to separate files to avoid changing the test results, and selectively disable leak detection at runtime instead of requiring to disable all sanitizers at buildtime. Signed-off-by: Akihiko Odaki Message-ID: <20251023-iotests-v1-2-fab143ca4c2f@rsg.ci.i.u-tokyo.ac.jp> Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- tests/qemu-iotests/testrunner.py | 12 ++++++++++++ tests/qemu-iotests/meson.build | 8 -------- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunn= er.py index 14cc8492f9..e2a3658994 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -263,10 +263,21 @@ def do_run_test(self, test: str) -> TestResult: Path(env[d]).mkdir(parents=3DTrue, exist_ok=3DTrue) =20 test_dir =3D env['TEST_DIR'] + f_asan =3D Path(test_dir, f_test.name + '.out.asan') f_bad =3D Path(test_dir, f_test.name + '.out.bad') f_notrun =3D Path(test_dir, f_test.name + '.notrun') f_casenotrun =3D Path(test_dir, f_test.name + '.casenotrun') =20 + env['ASAN_OPTIONS'] =3D f'detect_leaks=3D0:log_path=3D{f_asan}' + + def unlink_asan(): + with os.scandir(test_dir) as it: + for entry in it: + if entry.name.startswith(f_asan.name): + os.unlink(entry) + + unlink_asan() + for p in (f_notrun, f_casenotrun): silent_unlink(p) =20 @@ -312,6 +323,7 @@ def do_run_test(self, test: str) -> TestResult: description=3Df'output mismatch (see {f_bad}= )', diff=3Ddiff, casenotrun=3Dcasenotrun) else: + unlink_asan() f_bad.unlink() return TestResult(status=3D'pass', elapsed=3Delapsed, casenotrun=3Dcasenotrun) diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build index fad340ad59..56b0446827 100644 --- a/tests/qemu-iotests/meson.build +++ b/tests/qemu-iotests/meson.build @@ -2,14 +2,6 @@ if not have_tools or host_os =3D=3D 'windows' subdir_done() endif =20 -foreach cflag: qemu_ldflags - if cflag.startswith('-fsanitize') and \ - not cflag.contains('safe-stack') and not cflag.contains('cfi-icall') - message('Sanitizers are enabled =3D=3D> Disabled the qemu-iotests.') - subdir_done() - endif -endforeach - bash =3D find_program('bash', required: false, version: '>=3D 4.0') if not bash.found() message('bash >=3D v4.0 not available =3D=3D> Disabled the qemu-iotests.= ') --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896954; cv=none; d=zohomail.com; s=zohoarc; b=eT7J9iAN4557/pUkkl/pGWWMKUMghs4stqzdooF02A71J73RIqIpDg6/Xz7HLkqhhlzlzp1HoQL6bWlacUVM77pxswR33fKTAxlsdh6HBZLTO4ypqwg/JBW8yX7N7n1emyvQGK+5Xj8UXFpdB4ueCy+Iz3EuIPoNDemQ/kMV0A0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896954; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mmHhshZnNaiyS7BfkMJf8KcDdSyD7WnoCR26ukj/Z4w=; b=O07hZSPlMVHkv0ZNOg8XHOZAtU7jWs5PSsNE6YGWJQO+wnSPRijb/sazOZJkDBqUXxBc9tuden1nAkKK1Kg8q2CXprriT0Xwmt7CKaATsMedGGVLUIRBCfyg5CePfYGCT8S7N4erKsPIgQWDIamnjcy39zR3v0U/d05Z1HfRI6g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896954396916.2068896175213; Tue, 11 Nov 2025 13:35:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzw-0008MJ-9s; Tue, 11 Nov 2025 16:33:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzt-000868-5P for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzr-00027X-Pf for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:52 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-370-FXOHVokvMJG5Mu1OxRDNRw-1; Tue, 11 Nov 2025 16:33:48 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 92D721800358; Tue, 11 Nov 2025 21:33:46 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 07A9F30044E0; Tue, 11 Nov 2025 21:33:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mmHhshZnNaiyS7BfkMJf8KcDdSyD7WnoCR26ukj/Z4w=; b=IGBwjp6tVqpauHYxrWzWckb4z3nbmttkygqLmzEqRPKPbfCbjsHOh740VIYSqdrBMTrUaA WVBwJTun87YuMtUm3ixTVVE9HhUVUn41WX7Q/FTfIRMOj8pTR7zxPgIdfziXRN3aYpe+0J wfNLkl+N79dQguqdDN1xeCjXHpsWKVg= X-MC-Unique: FXOHVokvMJG5Mu1OxRDNRw-1 X-Mimecast-MFC-AGG-ID: FXOHVokvMJG5Mu1OxRDNRw_1762896826 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 21/28] qcow2: rename update_refcount_discard to queue_discard Date: Tue, 11 Nov 2025 22:32:31 +0100 Message-ID: <20251111213238.181992-22-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896955893158500 Content-Type: text/plain; charset="utf-8" From: Jean-Louis Dupond The function just queues discards, and doesn't do any refcount change. So let's change the function name to align with its function. Signed-off-by: Jean-Louis Dupond Message-ID: <20250513132628.1055549-2-jean-louis@dupond.be> Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/qcow2-refcount.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 0266542cee..8fb210501c 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -754,8 +754,8 @@ void qcow2_process_discards(BlockDriverState *bs, int r= et) } } =20 -static void update_refcount_discard(BlockDriverState *bs, - uint64_t offset, uint64_t length) +static void queue_discard(BlockDriverState *bs, + uint64_t offset, uint64_t length) { BDRVQcow2State *s =3D bs->opaque; Qcow2DiscardRegion *d, *p, *next; @@ -902,7 +902,7 @@ update_refcount(BlockDriverState *bs, int64_t offset, i= nt64_t length, } =20 if (s->discard_passthrough[type]) { - update_refcount_discard(bs, cluster_offset, s->cluster_siz= e); + queue_discard(bs, cluster_offset, s->cluster_size); } } } @@ -3619,7 +3619,7 @@ qcow2_discard_refcount_block(BlockDriverState *bs, ui= nt64_t discard_block_offs) /* discard refblock from the cache if refblock is cached */ qcow2_cache_discard(s->refcount_block_cache, refblock); } - update_refcount_discard(bs, discard_block_offs, s->cluster_size); + queue_discard(bs, discard_block_offs, s->cluster_size); =20 return 0; } --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897070; cv=none; d=zohomail.com; s=zohoarc; b=I58JaenGN+dDonVjQXYoObhQ0+kEa2XvtW+VM0DO/453+9+/xdrRB+lAQPjrlLXuckhxN3RuZjbo9JpKOcgBvou55SMgSQe5X00ACTtplq1lII9S2wTPBXrOgWMkXS0eOU8zlPj006NY3F4+R8HUQsjF3/IHXOqP0qFgJG2Qd9c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897070; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CmefoS00iD0l2Q3juNsgx+lS0LSZfmBCRBRzku/u+p0=; b=ZHh8rjZusyjwuZA4MSbIj259hVoZhZtvk0hs1cNb/BFAf0iJ9XGOs4XEZ0AQj2Sw9K7pdLoI+1ZTNcHMIrfxTrJ33BfSyZjq9ypHa8yI00O5kwVjPrfcbDXEv9CLp/kyAGX6NPeboFDUDNVJPcE7PtB2aQwfFeXtrvKIOvsCleo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897070088534.6698592107145; Tue, 11 Nov 2025 13:37:50 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIvzy-0008Rt-8b; Tue, 11 Nov 2025 16:33:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzw-0008MU-5X for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzu-0002Aq-HR for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:55 -0500 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-653-h0TmapxgM9G85OPTmEbNlw-1; Tue, 11 Nov 2025 16:33:51 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9995C180047F; Tue, 11 Nov 2025 21:33:50 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0143430044E0; Tue, 11 Nov 2025 21:33:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896833; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CmefoS00iD0l2Q3juNsgx+lS0LSZfmBCRBRzku/u+p0=; b=YnT0GcRc+pZBy/SxH2z5W8zvk+EobXX3pF/7Sphhkl2aWP1BJqymCKPpmsvav98GJ9y43f xWM8UNzBHZ2SV14s2rNDCVr6zByLmhiX69SirHqVTFYNARhVj5iSaS7LZHlI/Z3KxcpAQa takA4tQmD9S7AtaSHqQs5XsT2K+5Czk= X-MC-Unique: h0TmapxgM9G85OPTmEbNlw-1 X-Mimecast-MFC-AGG-ID: h0TmapxgM9G85OPTmEbNlw_1762896830 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 22/28] qcow2: put discards in discard queue when discard-no-unref is enabled Date: Tue, 11 Nov 2025 22:32:32 +0100 Message-ID: <20251111213238.181992-23-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897071240153000 Content-Type: text/plain; charset="utf-8" From: Jean-Louis Dupond When discard-no-unref is enabled, discards are not queued like it should. This was broken since discard-no-unref was added. Add a helper function qcow2_discard_cluster which handles some common checks and calls the queue_discards function if needed to add the discard request to the queue. Signed-off-by: Jean-Louis Dupond Message-ID: <20250513132628.1055549-3-jean-louis@dupond.be> Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/qcow2.h | 4 ++++ block/qcow2-cluster.c | 16 ++++++---------- block/qcow2-refcount.c | 17 +++++++++++++++++ 3 files changed, 27 insertions(+), 10 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index a9e3481c6e..547bb2b814 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -880,6 +880,10 @@ void GRAPH_RDLOCK qcow2_free_clusters(BlockDriverState= *bs, void GRAPH_RDLOCK qcow2_free_any_cluster(BlockDriverState *bs, uint64_t l2_entry, enum qcow2_discard_type type); +void GRAPH_RDLOCK +qcow2_discard_cluster(BlockDriverState *bs, uint64_t offset, + uint64_t length, QCow2ClusterType ctype, + enum qcow2_discard_type dtype); =20 int GRAPH_RDLOCK qcow2_update_snapshot_refcount(BlockDriverState *bs, int64_t l1_table_offs= et, diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index ce8c0076b3..c655bf6df4 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1978,12 +1978,10 @@ discard_in_l2_slice(BlockDriverState *bs, uint64_t = offset, uint64_t nb_clusters, if (!keep_reference) { /* Then decrease the refcount */ qcow2_free_any_cluster(bs, old_l2_entry, type); - } else if (s->discard_passthrough[type] && - (cluster_type =3D=3D QCOW2_CLUSTER_NORMAL || - cluster_type =3D=3D QCOW2_CLUSTER_ZERO_ALLOC)) { + } else { /* If we keep the reference, pass on the discard still */ - bdrv_pdiscard(s->data_file, old_l2_entry & L2E_OFFSET_MASK, - s->cluster_size); + qcow2_discard_cluster(bs, old_l2_entry & L2E_OFFSET_MASK, + s->cluster_size, cluster_type, type); } } =20 @@ -2092,12 +2090,10 @@ zero_in_l2_slice(BlockDriverState *bs, uint64_t off= set, if (!keep_reference) { /* Then decrease the refcount */ qcow2_free_any_cluster(bs, old_l2_entry, QCOW2_DISCARD_REQ= UEST); - } else if (s->discard_passthrough[QCOW2_DISCARD_REQUEST] && - (type =3D=3D QCOW2_CLUSTER_NORMAL || - type =3D=3D QCOW2_CLUSTER_ZERO_ALLOC)) { + } else { /* If we keep the reference, pass on the discard still */ - bdrv_pdiscard(s->data_file, old_l2_entry & L2E_OFFSET_MASK, - s->cluster_size); + qcow2_discard_cluster(bs, old_l2_entry & L2E_OFFSET_MASK, + s->cluster_size, type, QCOW2_DISCARD_REQUEST); } } } diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 8fb210501c..6512cda407 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1205,6 +1205,23 @@ void qcow2_free_any_cluster(BlockDriverState *bs, ui= nt64_t l2_entry, } } =20 +void qcow2_discard_cluster(BlockDriverState *bs, uint64_t offset, + uint64_t length, QCow2ClusterType ctype, + enum qcow2_discard_type dtype) +{ + BDRVQcow2State *s =3D bs->opaque; + + if (s->discard_passthrough[dtype] && + (ctype =3D=3D QCOW2_CLUSTER_NORMAL || + ctype =3D=3D QCOW2_CLUSTER_ZERO_ALLOC)) { + if (has_data_file(bs)) { + bdrv_pdiscard(s->data_file, offset, length); + } else { + queue_discard(bs, offset, length); + } + } +} + int qcow2_write_caches(BlockDriverState *bs) { BDRVQcow2State *s =3D bs->opaque; --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897087; cv=none; d=zohomail.com; s=zohoarc; b=aTUt5WCswQiG8Zwxl0565aYNpJGbEqvv+yJ/zaaOqtc1NZxgWaj1EABGEIl2glrTM/cClPKwY/mx4Uqu0dv1Utgv6D3MIz0elB8oe5zWghEWEnm+tFL1Gw5Wb37YuVZ0KUtPXplnafMmLjSTvQyqbWHrMq+n9z6R392pSqugeik= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897087; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lH5xtqAOdfESiQM+REAWAsG45Zyi+yUbjY66ZXmScSM=; b=PAyrUwgleZOvIc0hQS3qIf+JIwZCETnQMvem3cU5vwln82uSiZ/bt1n3/YscfkRngkkFmOha3Py7iB1Id6NZtm0KFarBtbDAlFaVqBdPS2omelSQVRcEgDdktG6ZNJyr5b6y2DSIQ2OQaSpzmT0zTCGWQfSDjaejILqrpO+BYps= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897087702375.1839183438792; Tue, 11 Nov 2025 13:38:07 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIw01-0000BJ-6s; Tue, 11 Nov 2025 16:34:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzz-0008WR-Aj for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:59 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzx-0002Fl-Uc for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:33:58 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-631-H1ONXqH3PGKQlrlJGpVGvg-1; Tue, 11 Nov 2025 16:33:53 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BAEAD1801233; Tue, 11 Nov 2025 21:33:52 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1DAAD30044E5; Tue, 11 Nov 2025 21:33:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896837; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lH5xtqAOdfESiQM+REAWAsG45Zyi+yUbjY66ZXmScSM=; b=SKjsmt05SYKs8eXVQ2hyql1jh2Gwsnj1jvkupEpKWD6cYwnUJndzCucrPfU0VbOrKsXo4a wnkoqDet1VRw7nHNEub8IMdGqvoRuTudIjjgb+xAliMdQHC5CWxYItyLvLUp0hplY337lG D8N6L7W+Hgt/RW2+0i0aVAUmTrZCoOM= X-MC-Unique: H1ONXqH3PGKQlrlJGpVGvg-1 X-Mimecast-MFC-AGG-ID: H1ONXqH3PGKQlrlJGpVGvg_1762896832 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 23/28] tests/qemu-iotests/184: Fix skip message for qemu-img without throttle Date: Tue, 11 Nov 2025 22:32:33 +0100 Message-ID: <20251111213238.181992-24-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897088299158500 Content-Type: text/plain; charset="utf-8" From: Thomas Huth If qemu-img does not support throttling, test 184 currently skips with the message: not suitable for this image format: raw But that's wrong, it's not about the image format, it's about the throttling not being available in qemu-img. Thus fix this by using _notrun with a proper message instead. Reviewed-by: Kevin Wolf Signed-off-by: Thomas Huth Message-ID: <20251014104142.1281028-2-thuth@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/184 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184 index 6d0afe9d38..9248b3265d 100755 --- a/tests/qemu-iotests/184 +++ b/tests/qemu-iotests/184 @@ -51,7 +51,7 @@ run_qemu() } =20 test_throttle=3D$($QEMU_IMG --help|grep throttle) -[ "$test_throttle" =3D "" ] && _supported_fmt throttle +[ "$test_throttle" =3D "" ] && _notrun "qemu-img does not support throttle" =20 echo echo "=3D=3D checking interface =3D=3D" --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896925; cv=none; d=zohomail.com; s=zohoarc; b=K9o4M9A1bEXDXbPqvbpJx7aV2tjvM4J4Of/VyTO/RGl2jmH6wWZHyK/WeV/UzkRo10uqgllzBcREhTyPj2Zf5J2mKwfBSmGYgpdAVMA6BkRfD07vz0k0dGv3b/HDSFF6b5GIfBQO0ZeXWre4uK58WcMSzw1I62o/yaT9goiHVoQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896925; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=p5/pNDPxmWXSW9ibWFFaeN5+HXMRwTQf4MumZt267PQ=; b=V8KZZb9bAZr8OCJQ2Aow6oHWGFjkiTMRJMwWpCEjnOAj6EgdQ363iNBAhizUqQQO24AuSC2Rau1ZM+jKnB0euvc36Gg1pS/dPJbL6UYh6K7Eny6sjbZb9Cb8xzMjR+YE+1Mu+IzFbbxT9vOqFAl+iTOC6tVecYcmK7DM7G2qI5E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896925230657.1582407394005; Tue, 11 Nov 2025 13:35:25 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIw17-0000zs-TD; Tue, 11 Nov 2025 16:35:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw01-0000IP-PZ for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzz-0002Gu-Mf for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:01 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-669-AC6jVBARNIqwtna1C3Wd_w-1; Tue, 11 Nov 2025 16:33:55 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D464919560A2; Tue, 11 Nov 2025 21:33:54 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 40FFF30044E8; Tue, 11 Nov 2025 21:33:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p5/pNDPxmWXSW9ibWFFaeN5+HXMRwTQf4MumZt267PQ=; b=esIzXNy75t4GqL5nkw0AUnV351JZFqJu22fMn3v/DAHPL2Vul6QMw40u9ZYgVoFw1AqcNf WH5ekKJlKy1Ii0VT4UTSbusynrqHrjYhLbtwLRCGAHI8z+lpScfqpRxgMQ4bR+2dH5ibQt fGlElHp+7GZ+X02m/Tw0c+qD0+twM9g= X-MC-Unique: AC6jVBARNIqwtna1C3Wd_w-1 X-Mimecast-MFC-AGG-ID: AC6jVBARNIqwtna1C3Wd_w_1762896834 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 24/28] tests/qemu-iotests: Improve the dry run list to speed up thorough testing Date: Tue, 11 Nov 2025 22:32:34 +0100 Message-ID: <20251111213238.181992-25-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896927485158500 Content-Type: text/plain; charset="utf-8" From: Thomas Huth When running the tests in thorough mode, e.g. with: make -j$(nproc) check SPEED=3Dthorough we currently always get a huge amount of total tests that the test runner tries to execute (2457 in my case), but a big bunch of them are only skipped (1099 in my case, meaning that only 1358 got executed). This happens because we try to run the whole set of iotests for multiple image formats while a lot of the tests can only run with one certain format only and thus are marked as SKIP during execution. This is quite a waste of time during each test run, and also unnecessarily blows up the displayed list of executed tests in the console output. Thus let's try to be a little bit smarter: If the "check" script is run with "-n" and an image format switch (like "-qed") at the same time (which is what we do for discovering the tests for the meson test runner already), only report the tests that likely support the given format instead of providing the whole list of all tests. We can determine whether a test supports a format or not by looking at the lines in the file that contain a "supported_fmt" or "unsupported_fmt" statement. This is only heuristics, of course, but it is good enough for running the iotests via "make check-block" - I double-checked that the list of executed tests does not get changed by this patch, it's only the tests that are skipped anyway that are now not run anymore. This way the amount of total tests drops from 2457 to 1432 for me, and the amount of skipped tests drops from 1099 to just 74 (meaning that we still properly run 1432 - 74 =3D 1358 tests as we did before). Signed-off-by: Thomas Huth Message-ID: <20251014104142.1281028-3-thuth@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 42 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index d9b7c1d598..3941eac8e2 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -17,6 +17,7 @@ # along with this program. If not, see . =20 import os +import re import sys import argparse import shutil @@ -82,7 +83,7 @@ def make_argparser() -> argparse.ArgumentParser: g_env.add_argument('-i', dest=3D'aiomode', default=3D'threads', help=3D'sets AIOMODE environment variable') =20 - p.set_defaults(imgfmt=3D'raw', imgproto=3D'file') + p.set_defaults(imgproto=3D'file') =20 format_list =3D ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2', 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg', 'vv= fat'] @@ -137,15 +138,50 @@ def make_argparser() -> argparse.ArgumentParser: return p =20 =20 +def dry_run_list(test_dir, imgfmt, testlist): + for t in testlist: + if not imgfmt: + print('\n'.join([os.path.basename(t)])) + continue + # If a format has been given, we look for the "supported_fmt" + # and the "unsupported_fmt" lines in the test and try to find out + # whether the format is supported or not. This is only heuristics + # (it can e.g. fail if the "unsupported_fmts" and "supported_fmts" + # statements are in the same line), but it should be good enough + # to get a proper list for "make check-block" + with open(os.path.join(test_dir, t), 'r', encoding=3D'utf-8') as f= h: + supported =3D True + check_next_line =3D False + sd =3D "[ \t'\"]" # Start delimiter + ed =3D "([ \t'\"]|$)" # End delimiter + for line in fh: + if 'unsupported_fmt' in line: + if re.search(sd + imgfmt + ed, line): + supported =3D False + break + elif 'supported_fmt' in line or check_next_line: + if re.search(sd + 'generic' + ed, line): + continue # Might be followed by "unsupported"= line + supported =3D re.search(sd + imgfmt + ed, line) + check_next_line =3D not ']' in line and \ + ('supported_fmts=3D[' in line or check_next_li= ne) + if supported or not check_next_line: + break + if supported: + print('\n'.join([os.path.basename(t)])) + + if __name__ =3D=3D '__main__': warnings.simplefilter("default") os.environ["PYTHONWARNINGS"] =3D "default" =20 args =3D make_argparser().parse_args() =20 + image_format =3D args.imgfmt or 'raw' + env =3D TestEnv(source_dir=3Dargs.source_dir, build_dir=3Dargs.build_dir, - imgfmt=3Dargs.imgfmt, imgproto=3Dargs.imgproto, + imgfmt=3Dimage_format, imgproto=3Dargs.imgproto, aiomode=3Dargs.aiomode, cachemode=3Dargs.cachemode, imgopts=3Dargs.imgopts, misalign=3Dargs.misalign, debug=3Dargs.debug, valgrind=3Dargs.valgrind, @@ -189,7 +225,7 @@ if __name__ =3D=3D '__main__': =20 if args.dry_run: with env: - print('\n'.join([os.path.basename(t) for t in tests])) + dry_run_list(env.source_iotests, args.imgfmt, tests) else: with TestRunner(env, tap=3Dargs.tap, color=3Dargs.color) as tr: --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896958; cv=none; d=zohomail.com; s=zohoarc; b=ksN69wJ8GJ4vMxmlCe2sh/4WfeuZSCI4P2dn/SxySxU9YsIS2gZ2divbvAhy5nHlS1MC33VKxV+xgTltS+UtXyKkyQ7SFeY8tzq0bc8G+qtR4Hbc2Z6lrwsRFagolBZT6Sk9plp1qBkcSGaCUnuvNtXRbof6TqHmgIgeMn+wZJg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896958; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CvYQhUDKgum6nqXTtCFuvgeYHiUsIJMbCsl7EwUhXcQ=; b=fekzXBeGTkVUhrEKPmtZUMQPJuzyQF6lQkyN5xxsGolP0jRt9ycQGUheGCtIlLgT/jkD9MY+Z95cDB1lwiOE8Z+1sKD9VLJD6xyRDF1zpGjdDYrvEmfsNBXNZNFzaYV9EZ7JdgyLb54UcCEl7kklmyjmhv4lhh+FkthFnMORZAM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17628969589071007.1700930021434; Tue, 11 Nov 2025 13:35:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIw1C-0001VX-AQ; Tue, 11 Nov 2025 16:35:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw0N-0000R2-AS for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIvzz-0002Gx-NS for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:02 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-164-lUvUH-bnO3mURzXWpW3OYw-1; Tue, 11 Nov 2025 16:33:58 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 00FC018007F2; Tue, 11 Nov 2025 21:33:57 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5BAF730044E0; Tue, 11 Nov 2025 21:33:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CvYQhUDKgum6nqXTtCFuvgeYHiUsIJMbCsl7EwUhXcQ=; b=SMoXZj0NLknMrs9MoM35wePsZn0NAEo0uqvBmSAFv0GslXw+5JVHGQ4Gn5C1V7njyiRvPx AC6PoQkEOwZ2+reK/5meBfAs2kN0hn5hJeMErsS0P36XB85eS8VykX6qWfCK6JPiWVzRkS 3HXtets+U83k7OTabEcD0ff+Q182LuQ= X-MC-Unique: lUvUH-bnO3mURzXWpW3OYw-1 X-Mimecast-MFC-AGG-ID: lUvUH-bnO3mURzXWpW3OYw_1762896837 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 25/28] tests/qemu-iotest: Add more image formats to the thorough testing Date: Tue, 11 Nov 2025 22:32:35 +0100 Message-ID: <20251111213238.181992-26-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896959971158500 Content-Type: text/plain; charset="utf-8" From: Thomas Huth Now that the "check" script is a little bit smarter with providing a list of tests that are supported for an image format, we can also add more image formats that can be used for generic block layer testing. (Note: qcow1 and luks are not added because some tests there currently fail, and other formats like bochs, cloop, dmg and vvfat do not work with the generic tests and thus would only get skipped if we'd tried to add them here) Reviewed-by: Kevin Wolf Signed-off-by: Thomas Huth Message-ID: <20251014104142.1281028-4-thuth@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/meson.build | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build index 56b0446827..d7bae71ced 100644 --- a/tests/qemu-iotests/meson.build +++ b/tests/qemu-iotests/meson.build @@ -13,7 +13,10 @@ qemu_iotests_env =3D {'PYTHON': python.full_path()} qemu_iotests_formats =3D { 'qcow2': 'quick', 'raw': 'slow', + 'parallels': 'thorough', 'qed': 'thorough', + 'vdi': 'thorough', + 'vhdx': 'thorough', 'vmdk': 'thorough', 'vpc': 'thorough' } --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897028; cv=none; d=zohomail.com; s=zohoarc; b=BRAXIZ3/HMlY+mnchCYsk/rNnMMVw3vHaU7vZ0AbKkWUoDgQ4iK2l+M5bbaLWqMSZMRl+3ThLgKhWb88SgXQSZIHML4QbcEObu/twjFCujBnc712+Ypa8q/ElRvE0yoetsfNWplXklcrwLx7vpXqF9ahOyMu9TNPCYLoQB8snqU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897028; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fDFRarAf5ZaB6K/52baQUM68FuJ2NxkUnuHxok7x0jE=; b=kXPpiGJhbRlhiYIo8z9qvtmXYpNPl/OJ8i4Q4YPkTNI/3wBpCDL0I4w8WCaZSnVtfOCCFl4tPTLHQIBNBbb/2cQ0qqBvDCeX/1oYVuBe1ceBr4BEg30uC4KZwohKhB18t8AaF0HGbxWFLM+PJindZonIeAAyEba3Z9Nv/SnAn4E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897028362421.0630808786211; Tue, 11 Nov 2025 13:37:08 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIw1B-0001P4-EF; Tue, 11 Nov 2025 16:35:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw0P-0000UL-Cz for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw0N-0002Hf-2e for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:24 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-439-oFAL9tGqNcyq_oNHDb19-Q-1; Tue, 11 Nov 2025 16:34:01 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8B59C195608F; Tue, 11 Nov 2025 21:33:59 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7CF4830044E0; Tue, 11 Nov 2025 21:33:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fDFRarAf5ZaB6K/52baQUM68FuJ2NxkUnuHxok7x0jE=; b=AS3uQXj1ATntRpw3C620Sw4X1dfS9+ReTiCVVD1dheVYNMoAJMEjj6liGaHAqxoAnlWI80 Cx9yCDsPH4SsAsQcz90mEy96W8AMBcKJOUeKpLt99ylwUDd8qrc4oZoUgJRX7DlsXhx+dH D+BT8XsNGE46snJZbrH3FgXW0D9lie4= X-MC-Unique: oFAL9tGqNcyq_oNHDb19-Q-1 X-Mimecast-MFC-AGG-ID: oFAL9tGqNcyq_oNHDb19-Q_1762896839 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 26/28] block: Allow drivers to control protocol prefix at creation Date: Tue, 11 Nov 2025 22:32:36 +0100 Message-ID: <20251111213238.181992-27-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897031168153000 Content-Type: text/plain; charset="utf-8" From: Eric Blake This patch is pure refactoring: instead of hard-coding permission to use a protocol prefix when creating an image, the drivers can now pass in a parameter, comparable to what they could already do for opening a pre-existing image. This patch is purely mechanical (all drivers pass in true for now), but it will enable the next patch to cater to drivers that want to differ in behavior for the primary image vs. any secondary images that are opened at the same time as creating the primary image. Signed-off-by: Eric Blake Message-ID: <20250915213919.3121401-5-eblake@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- include/block/block-global-state.h | 3 ++- block.c | 4 ++-- block/crypto.c | 2 +- block/parallels.c | 2 +- block/qcow.c | 2 +- block/qcow2.c | 4 ++-- block/qed.c | 2 +- block/raw-format.c | 2 +- block/vdi.c | 2 +- block/vhdx.c | 2 +- block/vmdk.c | 2 +- block/vpc.c | 2 +- 12 files changed, 15 insertions(+), 14 deletions(-) diff --git a/include/block/block-global-state.h b/include/block/block-globa= l-state.h index 62da83c616..479ca2858e 100644 --- a/include/block/block-global-state.h +++ b/include/block/block-global-state.h @@ -65,7 +65,8 @@ int co_wrapper bdrv_create(BlockDriver *drv, const char *= filename, QemuOpts *opts, Error **errp); =20 int coroutine_fn GRAPH_UNLOCKED -bdrv_co_create_file(const char *filename, QemuOpts *opts, Error **errp); +bdrv_co_create_file(const char *filename, QemuOpts *opts, + bool allow_protocol_prefix, Error **errp); =20 BlockDriverState *bdrv_new(void); int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, diff --git a/block.c b/block.c index 0fe0152a7c..4f1581cedf 100644 --- a/block.c +++ b/block.c @@ -693,7 +693,7 @@ out: } =20 int coroutine_fn bdrv_co_create_file(const char *filename, QemuOpts *opts, - Error **errp) + bool allow_protocol_prefix, Error **e= rrp) { QemuOpts *protocol_opts; BlockDriver *drv; @@ -702,7 +702,7 @@ int coroutine_fn bdrv_co_create_file(const char *filena= me, QemuOpts *opts, =20 GLOBAL_STATE_CODE(); =20 - drv =3D bdrv_find_protocol(filename, true, errp); + drv =3D bdrv_find_protocol(filename, allow_protocol_prefix, errp); if (drv =3D=3D NULL) { return -ENOENT; } diff --git a/block/crypto.c b/block/crypto.c index 7c37b23e36..b97d027444 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -835,7 +835,7 @@ block_crypto_co_create_opts_luks(BlockDriver *drv, cons= t char *filename, } =20 /* Create protocol layer */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto fail; } diff --git a/block/parallels.c b/block/parallels.c index 3a375e2a8a..7a90fb5220 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1117,7 +1117,7 @@ parallels_co_create_opts(BlockDriver *drv, const char= *filename, } =20 /* Create and open the file (protocol layer) */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto done; } diff --git a/block/qcow.c b/block/qcow.c index b442bfe835..3d37d26ee8 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -978,7 +978,7 @@ qcow_co_create_opts(BlockDriver *drv, const char *filen= ame, } =20 /* Create and open the file (protocol layer) */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto fail; } diff --git a/block/qcow2.c b/block/qcow2.c index 4aa9f9e068..ec72e27214 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3956,7 +3956,7 @@ qcow2_co_create_opts(BlockDriver *drv, const char *fi= lename, QemuOpts *opts, } =20 /* Create and open the file (protocol layer) */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto finish; } @@ -3971,7 +3971,7 @@ qcow2_co_create_opts(BlockDriver *drv, const char *fi= lename, QemuOpts *opts, /* Create and open an external data file (protocol layer) */ val =3D qdict_get_try_str(qdict, BLOCK_OPT_DATA_FILE); if (val) { - ret =3D bdrv_co_create_file(val, opts, errp); + ret =3D bdrv_co_create_file(val, opts, true, errp); if (ret < 0) { goto finish; } diff --git a/block/qed.c b/block/qed.c index 4a36fb3929..da23a83d62 100644 --- a/block/qed.c +++ b/block/qed.c @@ -788,7 +788,7 @@ bdrv_qed_co_create_opts(BlockDriver *drv, const char *f= ilename, } =20 /* Create and open the file (protocol layer) */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto fail; } diff --git a/block/raw-format.c b/block/raw-format.c index df16ac1ea2..a57c2922d5 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -463,7 +463,7 @@ static int coroutine_fn GRAPH_UNLOCKED raw_co_create_opts(BlockDriver *drv, const char *filename, QemuOpts *opts, Error **errp) { - return bdrv_co_create_file(filename, opts, errp); + return bdrv_co_create_file(filename, opts, true, errp); } =20 static int raw_open(BlockDriverState *bs, QDict *options, int flags, diff --git a/block/vdi.c b/block/vdi.c index 3ddc62a569..87b874a7ef 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -938,7 +938,7 @@ vdi_co_create_opts(BlockDriver *drv, const char *filena= me, qdict =3D qemu_opts_to_qdict_filtered(opts, NULL, &vdi_create_opts, tr= ue); =20 /* Create and open the file (protocol layer) */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto done; } diff --git a/block/vhdx.c b/block/vhdx.c index b2a4b813a0..c16e4a00c8 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -2096,7 +2096,7 @@ vhdx_co_create_opts(BlockDriver *drv, const char *fil= ename, } =20 /* Create and open the file (protocol layer) */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto fail; } diff --git a/block/vmdk.c b/block/vmdk.c index 7b98debc2b..eb3c174eca 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -2334,7 +2334,7 @@ vmdk_create_extent(const char *filename, int64_t file= size, bool flat, int ret; BlockBackend *blk =3D NULL; =20 - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto exit; } diff --git a/block/vpc.c b/block/vpc.c index 801ff5793f..07e8ae0309 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -1118,7 +1118,7 @@ vpc_co_create_opts(BlockDriver *drv, const char *file= name, } =20 /* Create and open the file (protocol layer) */ - ret =3D bdrv_co_create_file(filename, opts, errp); + ret =3D bdrv_co_create_file(filename, opts, true, errp); if (ret < 0) { goto fail; } --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762897090; cv=none; d=zohomail.com; s=zohoarc; b=h0Dsuj69jRJJGjTfw+wz7ccqj0aHo+77XQ1WEhsswPMveNPrwYAc6H4camiaWqS5Lw/mN+iWPtjVr0G/EHacsOKl+IcPPBVCFbrlUER742ddnFMxaxeEN2YZtCoyY8KjXJcSX4sUhLZZ3FLudUjkujHP81V81l0ZAWehj9mo5pg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762897090; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DpNqXUG11AGsY3sUGvyckw8XphYJ8cR1pOTzBckv6uc=; b=eXTy42aopBrQRetiLBeEB86W/QaM+EBriqDjJM2zCTmlQI3RLS1x1XiT3ee7BVGXDnfqJ6paAamPD3oxuslVnRE8vbvPjujGfKgMlvvfBmz2QzGs9cR3cfP0UPcv4l7tuHd9POlqUAU3RfeICpHYq9K5hpWXkEBdn37YJoOwGVY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762897090546462.5650072058852; Tue, 11 Nov 2025 13:38:10 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIw1D-0001hk-Pq; Tue, 11 Nov 2025 16:35:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw0R-0000XP-Vt for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw0N-0002IH-Nf for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:26 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-214-PEx8NJJaMvm8lAPihEfVng-1; Tue, 11 Nov 2025 16:34:02 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9D0AC1956095; Tue, 11 Nov 2025 21:34:01 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0EEDE30044E0; Tue, 11 Nov 2025 21:33:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896847; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DpNqXUG11AGsY3sUGvyckw8XphYJ8cR1pOTzBckv6uc=; b=MqDYCmQ5+A+/J1pRaFhuxy7ZBnVPYs4PR9R8/ufcMHgVyKl+iBh0UOKEjZNiAjHBkG5T/4 b5Jz9XiHIDvnix+vUfrpqXjLb7E4Qp30Tuy8ruTZJea8Ci0QFWlmDdQPte/ZNj2+Egeo09 3nQUCsvCp0vo4V/SLTv51Nm1phjaTOk= X-MC-Unique: PEx8NJJaMvm8lAPihEfVng-1 X-Mimecast-MFC-AGG-ID: PEx8NJJaMvm8lAPihEfVng_1762896841 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 27/28] qcow2, vmdk: Restrict creation with secondary file using protocol Date: Tue, 11 Nov 2025 22:32:37 +0100 Message-ID: <20251111213238.181992-28-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762897093304153000 Content-Type: text/plain; charset="utf-8" From: Eric Blake Ever since CVE-2024-4467 (see commit 7ead9469 in qemu v9.1.0), we have intentionally treated the opening of secondary files whose name is specified in the contents of the primary file, such as a qcow2 data_file, as something that must be a local file and not a protocol prefix (it is still possible to open a qcow2 file that wraps an NBD data image by using QMP commands, but that is from the explicit action of the QMP overriding any string encoded in the qcow2 file). At the time, we did not prevent the use of protocol prefixes on the secondary image while creating a qcow2 file, but it results in a qcow2 file that records an empty string for the data_file, rather than the protocol passed in during creation: $ qemu-img create -f raw datastore.raw 2G $ qemu-nbd -e 0 -t -f raw datastore.raw & $ qemu-img create -f qcow2 -o data_file=3Dnbd://localhost:10809/ \ datastore_nbd.qcow2 2G Formatting 'datastore_nbd.qcow2', fmt=3Dqcow2 cluster_size=3D65536 extended= _l2=3Doff compression_type=3Dzlib size=3D2147483648 data_file=3Dnbd://local= host:10809/ lazy_refcounts=3Doff refcount_bits=3D16 $ qemu-img info datastore_nbd.qcow2 | grep data $ qemu-img info datastore_nbd.qcow2 | grep data image: datastore_nbd.qcow2 data file: data file raw: false filename: datastore_nbd.qcow2 And since an empty string was recorded in the file, attempting to open the image without using QMP to supply the NBD data store fails, with a somewhat confusing error message: $ qemu-io -f qcow2 datastore_nbd.qcow2 qemu-io: can't open device datastore_nbd.qcow2: The 'file' block driver req= uires a file name Although the ability to create an image with a convenience reference to a protocol data file is not a security hole (unlike the case with open, the image is not untrusted if we are the ones creating it), the above demo shows that it is still inconsistent. Thus, it makes more sense if we also insist that image creation rejects a protocol prefix when using the same syntax. Now, the above attempt produces: $ qemu-img create -f qcow2 -o data_file=3Dnbd://localhost:10809/ \ datastore_nbd.qcow2 2G Formatting 'datastore_nbd.qcow2', fmt=3Dqcow2 cluster_size=3D65536 extended= _l2=3Doff compression_type=3Dzlib size=3D2147483648 data_file=3Dnbd://local= host:10809/ lazy_refcounts=3Doff refcount_bits=3D16 qemu-img: datastore_nbd.qcow2: Could not create 'nbd://localhost:10809/': N= o such file or directory with datastore_nbd.qcow2 no longer created. Signed-off-by: Eric Blake Message-ID: <20250915213919.3121401-6-eblake@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/qcow2.c | 2 +- block/vmdk.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index ec72e27214..cb0bdb32ec 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3971,7 +3971,7 @@ qcow2_co_create_opts(BlockDriver *drv, const char *fi= lename, QemuOpts *opts, /* Create and open an external data file (protocol layer) */ val =3D qdict_get_try_str(qdict, BLOCK_OPT_DATA_FILE); if (val) { - ret =3D bdrv_co_create_file(val, opts, true, errp); + ret =3D bdrv_co_create_file(val, opts, false, errp); if (ret < 0) { goto finish; } diff --git a/block/vmdk.c b/block/vmdk.c index eb3c174eca..3b35b63cb5 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -2334,7 +2334,7 @@ vmdk_create_extent(const char *filename, int64_t file= size, bool flat, int ret; BlockBackend *blk =3D NULL; =20 - ret =3D bdrv_co_create_file(filename, opts, true, errp); + ret =3D bdrv_co_create_file(filename, opts, false, errp); if (ret < 0) { goto exit; } --=20 2.51.1 From nobody Sun Dec 14 06:34:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1762896940; cv=none; d=zohomail.com; s=zohoarc; b=mfIb3TBPbJBAV+tAIuNw86DuiguNmPMVTwo0bsUsKPmd3JHvyJH7WMBj+zHG0n9R75rpdIZIJodriPOkD0MDbmLLIPKPukxch8BSe/ObLkaCkYnZdba71eBuwKi+VfQSRNofoHzXy0WyYZFa5gm1XtSRAyW5nlhQkG7Fbr4Bf1g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762896940; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dsWpJ1fT5oClZZ7jlO7mqvDN0tCOobR4rddQnL5QirM=; b=A2IWlq4sSqGeWPculKmEos1uoiW3q8KjsZ7dK2n/yGl0ZOij49NostIbs+OZIyrEDbH/TY3NAF9s9PahNu61n9Y7XWhE2MbmCTqxQ2B00Ej+16vugiSJ8Lc9RIBXi0O/uLu96r7+h4TutoWRFCMrC05fGMAzagu2fBRS7qfW5R8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1762896940612964.5596810828191; Tue, 11 Nov 2025 13:35:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vIw17-0000yB-TT; Tue, 11 Nov 2025 16:35:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw0R-0000XO-Un for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vIw0N-0002I2-3i for qemu-devel@nongnu.org; Tue, 11 Nov 2025 16:34:25 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-682-9mM8K_mGM1yu25r8vN8R1w-1; Tue, 11 Nov 2025 16:34:04 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BB7951956063; Tue, 11 Nov 2025 21:34:03 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.45.225.214]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2534230044E0; Tue, 11 Nov 2025 21:34:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762896846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dsWpJ1fT5oClZZ7jlO7mqvDN0tCOobR4rddQnL5QirM=; b=IgXdn83OgJssYqoJCXBpXSWC1titsWG1XPXtXVRr+YJU6g3H5lWZ4FfoCMqPkXEz4CSDZZ VvTuj/u9xJpGuGE0XfU5WT0NFysfP8wa4kqXxeok67Jtplyl2zZCuMZik+c6cLsflr+t+X g7ELarGAXCTVtnosz2Y4vBH7W7Oi0tQ= X-MC-Unique: 9mM8K_mGM1yu25r8vN8R1w-1 X-Mimecast-MFC-AGG-ID: 9mM8K_mGM1yu25r8vN8R1w_1762896843 From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org Subject: [PULL v2 28/28] qemu-img rebase: don't exceed IO_BUF_SIZE in one operation Date: Tue, 11 Nov 2025 22:32:38 +0100 Message-ID: <20251111213238.181992-29-kwolf@redhat.com> In-Reply-To: <20251111213238.181992-1-kwolf@redhat.com> References: <20251111213238.181992-1-kwolf@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1762896942986153000 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia During a rebase operation data is copied from the backing chain into the target image using a loop, and each iteration looks for a contiguous region of allocated data of at most IO_BUF_SIZE (2 MB). Once that region is found, and in order to avoid partial writes, its boundaries are extended so they are aligned to the (sub)clusters of the target image (see commit 12df580b). This operation can however result in a region that exceeds the maximum allowed IO_BUF_SIZE, crashing qemu-img. This can be easily reproduced when the source image has a smaller cluster size than the target image: base <- int <- active $ qemu-img create -f qcow2 base.qcow2 4M $ qemu-img create -f qcow2 -F qcow2 -b base.qcow2 -o cluster_size=3D1M int.= qcow2 $ qemu-img create -f qcow2 -F qcow2 -b int.qcow2 -o cluster_size=3D2M acti= ve.qcow2 $ qemu-io -c "write -P 0xff 1M 2M" int.qcow2 $ qemu-img rebase -F qcow2 -b base.qcow2 active.qcow2 qemu-img: qemu-img.c:4102: img_rebase: Assertion `written + pnum <=3D IO_BU= F_SIZE' failed. Aborted Cc: qemu-stable Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3174 Fixes: 12df580b3b7f ("qemu-img: rebase: avoid unnecessary COW operations") Signed-off-by: Alberto Garcia Message-ID: <20251107091834.383781-1-berto@igalia.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- qemu-img.c | 2 +- tests/qemu-iotests/024 | 46 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/024.out | 26 +++++++++++++++++++++ 3 files changed, 73 insertions(+), 1 deletion(-) diff --git a/qemu-img.c b/qemu-img.c index 7a32d2d16c..c42dd4e995 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -4081,7 +4081,7 @@ static int img_rebase(const img_cmd_t *ccmd, int argc= , char **argv) n +=3D offset - QEMU_ALIGN_DOWN(offset, write_align); offset =3D QEMU_ALIGN_DOWN(offset, write_align); n +=3D QEMU_ALIGN_UP(offset + n, write_align) - (offset + n); - n =3D MIN(n, size - offset); + n =3D MIN(n, MIN(size - offset, IO_BUF_SIZE)); assert(!bdrv_is_allocated(unfiltered_bs, offset, n, &n_alloc) = && n_alloc =3D=3D n); =20 diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024 index b29c76e161..021169b4a1 100755 --- a/tests/qemu-iotests/024 +++ b/tests/qemu-iotests/024 @@ -315,6 +315,52 @@ echo =20 $QEMU_IMG map "$OVERLAY" | _filter_qemu_img_map =20 +# Check that the region to copy to the overlay during a rebase +# operation does not exceed the I/O buffer size. +# +# backing_new <-- backing_old <-- overlay +# +# Backing (new): -- -- -- -- <-- Empty image, size 4MB +# Backing (old):|--|ff|ff|--| <-- 4 clusters, 1MB each +# Overlay: |-- --|-- --| <-- 2 clusters, 2MB each +# +# The data at [1MB, 3MB) must be copied from the old backing image to +# the overlay. However the rebase code will extend that region to the +# overlay's (sub)cluster boundaries to avoid CoW (see commit 12df580b). +# This test checks that IO_BUF_SIZE (2 MB) is taken into account. + +echo +echo "=3D=3D=3D Test that the region to copy does not exceed 2MB (IO_BUF_S= IZE) =3D=3D=3D" +echo + +echo "Creating backing chain" +echo + +TEST_IMG=3D$BASE_NEW _make_test_img 4M +TEST_IMG=3D$BASE_OLD CLUSTER_SIZE=3D1M _make_test_img -b "$BASE_NEW" -F $I= MGFMT +TEST_IMG=3D$OVERLAY CLUSTER_SIZE=3D2M _make_test_img -b "$BASE_OLD" -F $I= MGFMT + +echo +echo "Writing data to region [1MB, 3MB)" +echo + +$QEMU_IO "$BASE_OLD" -c "write -P 0xff 1M 2M" | _filter_qemu_io + +echo +echo "Rebasing" +echo + +$QEMU_IMG rebase -b "$BASE_NEW" -F $IMGFMT "$OVERLAY" + +echo "Verifying the data" +echo + +$QEMU_IO "$OVERLAY" -c "read -P 0x00 0 1M" | _filter_qemu_io +$QEMU_IO "$OVERLAY" -c "read -P 0xff 1M 2M" | _filter_qemu_io +$QEMU_IO "$OVERLAY" -c "read -P 0x00 3M 1M" | _filter_qemu_io + +$QEMU_IMG map "$OVERLAY" | _filter_qemu_img_map + echo =20 # success, all done diff --git a/tests/qemu-iotests/024.out b/tests/qemu-iotests/024.out index 3d1e31927a..1b7522ba71 100644 --- a/tests/qemu-iotests/024.out +++ b/tests/qemu-iotests/024.out @@ -243,4 +243,30 @@ Offset Length File 0 0x20000 TEST_DIR/subdir/t.IMGFMT 0x40000 0x20000 TEST_DIR/subdir/t.IMGFMT =20 +=3D=3D=3D Test that the region to copy does not exceed 2MB (IO_BUF_SIZE) = =3D=3D=3D + +Creating backing chain + +Formatting 'TEST_DIR/subdir/t.IMGFMT.base_new', fmt=3DIMGFMT size=3D4194304 +Formatting 'TEST_DIR/subdir/t.IMGFMT.base_old', fmt=3DIMGFMT size=3D419430= 4 backing_file=3DTEST_DIR/subdir/t.IMGFMT.base_new backing_fmt=3DIMGFMT +Formatting 'TEST_DIR/subdir/t.IMGFMT', fmt=3DIMGFMT size=3D4194304 backing= _file=3DTEST_DIR/subdir/t.IMGFMT.base_old backing_fmt=3DIMGFMT + +Writing data to region [1MB, 3MB) + +wrote 2097152/2097152 bytes at offset 1048576 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +Rebasing + +Verifying the data + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2097152/2097152 bytes at offset 1048576 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x400000 TEST_DIR/subdir/t.IMGFMT + *** done --=20 2.51.1