From nobody Sat May 30 17:44:04 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1779287154; cv=none; d=zohomail.com; s=zohoarc; b=IGefUQyj4KCRliS2OmMZO3g8o/VEXJ5T9MyNBQ5sAV9AG1W3LvDk63ShjUZcWxbbGNu8edurfZTIi3B2kTgIzawvjSw33nzIDKCLDfbqNRnCJG2ObJ60L05rGAsjHmBYRZSlzTT4ap9d+f5NwOuMB7C7kjiE8ZsUXhmb+f0OIW8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1779287154; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=ZnPrldXdjgOSqpNTXO9aPPp/nDRZZ2rExe0cZtTcxto=; b=kIjSrpp5mfRhCc+lEC+d+Y2BwH1cBbYiBdLgDjgrVRp1IVx5wq3vo5/eYi2iwfrTTuQ+PT7oZL8jn2R4f48wY+76mT6FzmkWaqaj0KRmj33HcJPlFN9athGdQDxLaHO/rG5eZ6fEZmpt+f8LZep/o3bUA5JW2csOgrmJBaDa2PA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1779287154654618.7293899252886; Wed, 20 May 2026 07:25:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wPhrp-00065J-Fw; Wed, 20 May 2026 10:25:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wPhrD-0005rw-5W; Wed, 20 May 2026 10:25:20 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wPhr9-0004D9-UZ; Wed, 20 May 2026 10:25:10 -0400 Received: from ch-demo-asa.virtuozzo.com ([130.117.225.8] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1wPhnl-009SLR-2l; Wed, 20 May 2026 16:25:03 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=ZnPrldXdjgOSqpNTXO9aPPp/nDRZZ2rExe0cZtTcxto=; b=kxFbNiiuZR5K 07yeylbSqUOs4WF136u4kdXBIbrlFcFfSWZrkHYbPYEJV3/H3suDi+2KqJHeRDlVEhVi5YP1Lw7ld KTnzzrQAbh21Ac433bK0Vjy6UTbhHJ9CZ9ktZnkwBXfgLwVuxp2h7w8i1PX4vlSEtgQdnNw8PafOA LZlWfR6nPYVcc3WsezpMT0T7f4t2CP+kh5n/p3apFuaEyVNyxW1IyceHOWH+BIIyJRMUWY+cPXk8n ze/PtZK/7pTIy3Q0O+yfHwA0A4jqjlCBZJX9tpEiVWF1BwMRNyVvnGN+JxSnmjravSwq7ileeN2Jc 6UKhIRjsoZITUEboxzWUGA==; To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, qemu-stable@nongnu.org, kwolf@redhat.com, hreitz@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, "Denis V. Lunev" Subject: [PATCH v3] block/linux-aio: bound ioq_submit() recursion depth Date: Wed, 20 May 2026 16:25:03 +0200 Message-ID: <20260520142503.251959-2-den@openvz.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520142503.251959-1-den@openvz.org> References: <20260520142503.251959-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: softfail client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: "Denis V. Lunev" From: "Denis V. Lunev" via qemu development Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1779287157539154100 Content-Type: text/plain; charset="utf-8" qemu_laio_process_completions() wraps its body in defer_call_begin / defer_call_end. Inside the section, completion callbacks wake coroutines that queue new aiocbs; laio_do_submit() defers laio_deferred_fn. At the bottom of qemu_laio_process_completions() the defer_call_end() fires laio_deferred_fn, which calls ioq_submit(), closing the cycle: ioq_submit -> io_submit(2) // some sync completions -> qemu_laio_process_completions // defer_call_begin -> aio_co_wake // resumes coroutine -> laio_do_submit -> defer_call(laio_deferred_fn, s) // enqueued -> defer_call_end // nesting drops to 0 -> laio_deferred_fn -> ioq_submit // +1 stack frame, loop When io_submit(2) returns asynchronously (O_DIRECT) the cycle terminates in one extra frame: the fresh aiocb is still in flight, no completion is drained, no coroutine wakes, no new submission queues. When submissions complete synchronously (non-O_DIRECT, or per-descriptor drivers such as vmdk) each level enqueues more work for the next defer_call_end() to drain, so recursion grows without bound and QEMU crashes with SIGSEGV on the thread guard page. The cycle was closed by two performance commits, each correct in isolation: 076682885d ("block/linux-aio: convert to blk_io_plug_call() API") -- introduced laio_deferred_fn and wired laio_do_submit -> defer_call(laio_deferred_fn, s). 84d61e5f36 ("virtio: use defer_call() in virtio_irqfd_notify()") -- added defer_call_begin/end around qemu_laio_process_completions so virtio-irqfd notifications batch across a completion pass. The supported aio=3Dnative + cache=3Dnone pairing keeps submissions asynchronous, so the cycle stays bounded; nothing in the code enforces that contract. Observed in production as a SIGSEGV during a backup job configured with --cached + aio=3Dnative; reproducible on upstream with qemu-io against vmdk. Cap ioq_submit() recursion with a counter on LaioQueue, which is only accessed from the AioContext home thread. On overflow, return without submitting. The pending work is drained by s->completion_bh, which qemu_laio_process_completions() has already scheduled on entry -- no work is lost; one event-loop round-trip of latency is paid only when the bound is hit, which cannot happen on a supported configuration. Signed-off-by: Denis V. Lunev CC: Kevin Wolf CC: Hanna Reitz CC: Stefan Hajnoczi CC: Paolo Bonzini --- block/linux-aio.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/block/linux-aio.c b/block/linux-aio.c index 0a7424fbb3..5aaf2e8514 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -36,6 +36,19 @@ /* Maximum number of requests in a batch. (default value) */ #define DEFAULT_MAX_BATCH 32 =20 +/* + * Bound on how deep ioq_submit() may recurse on a single LaioQueue via the + * ioq_submit -> qemu_laio_process_completions -> defer_call_end -> + * laio_deferred_fn -> ioq_submit cycle. The cycle terminates naturally + * when io_submit(2) returns asynchronously (O_DIRECT), but can grow + * without bound when submissions complete synchronously. On overflow + * the caller returns without submitting; the outermost + * qemu_laio_process_completions() has already scheduled s->completion_bh + * (via qemu_bh_schedule() at the top of that function), which resumes + * submission from the next event-loop dispatch. + */ +#define IOQ_SUBMIT_MAX_DEPTH 8 + struct qemu_laiocb { Coroutine *co; LinuxAioState *ctx; @@ -61,6 +74,7 @@ typedef struct { unsigned int in_queue; unsigned int in_flight; bool blocked; + unsigned int submit_depth; QSIMPLEQ_HEAD(, qemu_laiocb) pending; } LaioQueue; =20 @@ -331,6 +345,7 @@ static void ioq_init(LaioQueue *io_q) io_q->in_queue =3D 0; io_q->in_flight =3D 0; io_q->blocked =3D false; + io_q->submit_depth =3D 0; } =20 static void ioq_submit(LinuxAioState *s) @@ -340,6 +355,11 @@ static void ioq_submit(LinuxAioState *s) QEMU_UNINITIALIZED struct iocb *iocbs[MAX_EVENTS]; QSIMPLEQ_HEAD(, qemu_laiocb) completed; =20 + if (s->io_q.submit_depth >=3D IOQ_SUBMIT_MAX_DEPTH) { + return; + } + s->io_q.submit_depth++; + do { if (s->io_q.in_flight >=3D MAX_EVENTS) { break; @@ -385,6 +405,8 @@ static void ioq_submit(LinuxAioState *s) * pended requests will be submitted from there. */ } + + s->io_q.submit_depth--; } =20 static uint64_t laio_max_batch(LinuxAioState *s, uint64_t dev_max_batch) base-commit: ac0cc20ad2fe0b8df2e5d9458e90a095ac711ab1 --=20 2.51.0