From nobody Wed Aug 26 07:24:39 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1786530609; cv=none; d=zohomail.com; s=zohoarc; b=DFNtoVileO+kb1p0ueDsHh6hCg3WmgAP8YHXV2gZzzHlbfY1uV1tg3UbEjAMXmdccIwG+pdKk9RWeg9yydUDcuTwJkpBzyfrZvEeBtyTDx2NdwLI2Sn2hQkM8MCOObUl1d7L50pTR6m7hQJOvYE0jGn4Tke6PciijGHm9qoLAqo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786530609; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NdFZtNt0B5CspdIiAGT2omAyJ7es0F/gGiPbHDyfU2Y=; b=Nq/TMLyYPypJwbW8Ldc6MQ9CyPkGRYDlNtaV43TDCenhVfSH/7UBvJB5P5XZwoJzk7scYGwVByHIv/LqLzXy4gfqrFgNeY0Cyml7/RQ6x4ecGOYBKE5UwdyYfHarM3XgNR+2J2EBnCSZU3JI8TAwRB1fUn4BENedP4LmA7QHqJc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786530609247788.2831791960091; Wed, 12 Aug 2026 03:30:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wu6Cy-0005W3-CQ; Wed, 12 Aug 2026 06:29:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wu6Cw-0005UZ-N1 for qemu-devel@nongnu.org; Wed, 12 Aug 2026 06:29:14 -0400 Received: from mail-ej1-x62d.google.com ([2a00:1450:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wu6Cu-0002iv-1a for qemu-devel@nongnu.org; Wed, 12 Aug 2026 06:29:14 -0400 Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-c20e70a0962so86765166b.2 for ; Wed, 12 Aug 2026 03:29:11 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:89f2:ad10:d9bb:681e]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c20f0ad270bsm70748466b.46.2026.08.12.03.29.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Aug 2026 03:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1786530550; x=1787135350; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NdFZtNt0B5CspdIiAGT2omAyJ7es0F/gGiPbHDyfU2Y=; b=lsyVmgjOvBDL/VTniWHrA8gNUOpQHOREm5wh/gQxDgUQuzppTFlEKIT9j50Mgj+UxI LbDwDy+A+90qwfWeMseUv+VxQ0AgGBuNVf+jXcgTN4+jmOJMcoTd9G2EIvn81xsXcu6R lhIDe5wZRabLTKLE2ydx1jwlrC94HB+z80899B0bMZC1xorikE//yAKNPeDZkKT5YRY/ 13XdrGcXG+hDP52JBHyFea28QBU8ybWjvtSFc2wRmtoMsmVoqE+McW0kn7NMyp22ICns Y3KULEN386zd6KA1P9zSZvGhcBjkP9hmu3r58g9PTScdeJw4L3mS2D9BZybL/o3AWXaj V99g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786530550; x=1787135350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NdFZtNt0B5CspdIiAGT2omAyJ7es0F/gGiPbHDyfU2Y=; b=CnRuCMKxTODisNyn2ec1fmuSeZAwdPHcrUBzEYr/JOgfhf2lkMk9BiR2piw2DRjhZh ffHHl9TTvfm/s67epxJBzHRQ2tCdw1T16QjIoX1+Akxrx3Qodx5oldEC8zTx7J/ATqMX vrNvVT6UgpVhnPn6lk5rirV/aIZ6/SaYJY5kdgnGW4iF+GqgemLMUjzFHpYXPFJjrakv hl23iWoIdMWIHxXNPEzM6gXIk/EH5EWhGGs1hPQAgHG89rvRfF7qKwwYMG4ifZRfmhAz FzKClzZgfojYt/jYr+i2ocAcmUzpop5c1A+dPpGB87jdHkLSqtveXv+3sT5JOKkHxeoS U4Pg== X-Gm-Message-State: AOJu0YwT8IJEGBgtw2rZNuN+LVVEY0gLPa4xuoQMswGJfc1seG4Uwq+E 0lyODE42lDEhKVhUrKApWsvAFVveKWU1dwExBLmEiCQcsyu5rBCoG9K2egbzHYOOU3JeoH+kI97 tX7PN X-Gm-Gg: AR+sD13Iqr+JNXXzldBZv7p+zmWAOgmN4IruhnxrKWy4lWeJSXQHInWrcUFo2d+Ht6W XhNMiaIcfQqBfF9JlM8Vs4giT5RSVWPJLebs44i5KYZ4uW5mPyw2moZhjCqbmO/4WzarRDWmLTo YQnlL4jCLE52WSIwNAhnV6mHFjJY3fCKINQ7MprvAklK0G2+J5bGgSy01OcvOu9S+8eAMmlLYXg b0hT/ijtsA3Vy35pDDFlg9na6uix8sHuns5ohcfjyYXSj5sRHY3j/w03BJ9COM980hV7a6e7q1u 75Hnfj+InsvxavYEz7iUqVm+toa69LnoLExexuOrY+dnCfxmAm0YVRnllIEA/bN4Fp+jiKM8DIs kwZ4/736WiQ77BCqFT8POO0mXLG7DOpkotMDdzfsAnmXSYd63rYPqAS2RLkC5yBL9gXNUfVyW4Q W8Ih3Cmwdn5dBEquSzX2oRFdPvxjynDcyFNWN6NHA714g7pt6BkM1fOJU6KA== X-Received: by 2002:a17:907:1c0c:b0:c08:417e:3696 with SMTP id a640c23a62f3a-c20f2fa0de5mr179368466b.20.1786530550408; Wed, 12 Aug 2026 03:29:10 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Eric Blake , Vladimir Sementsov-Ogievskiy Subject: [PATCH 1/3] block/nbd: clear reply.cookie when the reply is rejected Date: Wed, 12 Aug 2026 12:29:04 +0200 Message-ID: <20260812102906.894063-2-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260812102906.894063-1-den@openvz.org> References: <20260812102906.894063-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::62d; envelope-from=den@openvz.org; helo=mail-ej1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1786530610830158501 Content-Type: text/plain; charset="utf-8" nbd_receive_replies() reads a reply header into s->reply and, when the header turns out to be unusable, reports a channel error and returns without touching it. The cookie stays there until the request which owns the reply clears it, and until then the waiters are explicitly allowed to look at a cookie which is not theirs: if (s->reply.cookie !=3D 0) { ind2 =3D COOKIE_TO_INDEX(s->reply.cookie); assert(!s->requests[ind2].receiving); Two of the error paths leave a value chosen by the server behind: one returns before the cookie is validated at all, the other returns because that validation has failed. A waiter which picks such a cookie up turns it into an index which is not in requests[] and accesses the array out of bounds, at an offset the server controls. The reply is of no use to anybody at this point, so clear the cookie before the mutex is released and keep the invariant that a non-zero s->reply.cookie is always an index of a live request. Observing the stale cookie takes a second thread, which a multiqueue configuration provides. Within one AioContext there is no yield point between the failed read and the clearing done by the owner in nbd_co_receive_one_chunk(), so nothing else of this node runs in between. The parked waiters cannot see it either, as they are woken only after the cookie has been cleared. What can get in is a request entering nbd_receive_replies() afresh, one just sent or one back for its next reply chunk, because that path takes the mutex without looking at the state. Signed-off-by: Denis V. Lunev Cc: Eric Blake Cc: Vladimir Sementsov-Ogievskiy Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 5d231d5c4e..d9b776283f 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -466,20 +466,19 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDSt= ate *s, uint64_t cookie, error_setg(errp, "server dropped connection"); } if (ret < 0) { - nbd_channel_error(s, ret); - return ret; + goto err; } if (nbd_reply_is_structured(&s->reply) && s->info.mode < NBD_MODE_STRUCTURED) { - nbd_channel_error(s, -EINVAL); + ret =3D -EINVAL; error_setg(errp, "unexpected structured reply"); - return -EINVAL; + goto err; } ind2 =3D COOKIE_TO_INDEX(s->reply.cookie); if (ind2 >=3D MAX_NBD_REQUESTS || !s->requests[ind2].coroutine) { - nbd_channel_error(s, -EINVAL); + ret =3D -EINVAL; error_setg(errp, "unexpected cookie value"); - return -EINVAL; + goto err; } if (s->reply.cookie =3D=3D cookie) { /* We are done */ @@ -487,6 +486,13 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDSta= te *s, uint64_t cookie, } nbd_recv_coroutine_wake_one(&s->requests[ind2]); } + +err: + /* Waiters look at this cookie, so do not leave a rejected one behind.= */ + s->reply.cookie =3D 0; + nbd_channel_error(s, ret); + + return ret; } =20 static int coroutine_fn GRAPH_RDLOCK --=20 2.53.0 From nobody Wed Aug 26 07:24:39 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1786530592; cv=none; d=zohomail.com; s=zohoarc; b=QGLT8/tCvpKhIyEZCYwAG1Vna/TI+uQc8c29KLAt75I3uuVrsogqZPPwlr2fGm3SyiAiMVU4kb5hF4cUCgI95lB+0lewOPxhBV8wOclVOQimTphhtE+32NztuMnSgnAPVtw4Z/NPswai/6mCvScpcnqE4z0/JbhLYCGZg5IH0jg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786530592; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NtTCK5Dii6LDoNzyyQ9y0Yku0qPmidmnjmWHD+vYK6o=; b=k6gLQbEauykKe3ka8E7ca1zuY4ZbGxUY16s0XwL4CwHaqL/gghKNFfgC9cuDi8/LJdi0+Ggyc5tAAb4hRDwJmE1FhfaAWjIsfB6dbUBunwQibrlfkVWrGxoq7Sael4S+c0V0ux750I37fzfXJ1l6aYBMKl8eeNa8EO+61GEOQYw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786530592701712.1249443380934; Wed, 12 Aug 2026 03:29:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wu6Cz-0005WK-9z; Wed, 12 Aug 2026 06:29:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wu6Cx-0005V3-EO for qemu-devel@nongnu.org; Wed, 12 Aug 2026 06:29:15 -0400 Received: from mail-ej1-x62b.google.com ([2a00:1450:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wu6Cv-0002j6-CZ for qemu-devel@nongnu.org; Wed, 12 Aug 2026 06:29:15 -0400 Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-c207cb16cf5so113887566b.1 for ; Wed, 12 Aug 2026 03:29:12 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:89f2:ad10:d9bb:681e]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c20f0ad270bsm70748466b.46.2026.08.12.03.29.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Aug 2026 03:29:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1786530551; x=1787135351; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NtTCK5Dii6LDoNzyyQ9y0Yku0qPmidmnjmWHD+vYK6o=; b=HKPRnZzFrrv0WvbXFB6GG6aF/Okq8uorqxifYW8vReQjjwgsCxHnNu8Kn/ZQFWbp01 FDqcwcFyV/6bTUdKLTlQbmdCwxQrQlzHhV2xWkUuO8seMnofEhWOpWWHoiXZCWFQBRx/ HlEDxEyhSWY08qfsvQzDb/FV6RYjxvnL5KZM1p8hRV0V/BmVugyAD/4Q+7bsMorNwM89 v84Fthte4ZuQf2Hqu9Lso8n4U84HUHvPEdk+MfKFkbRb2Mc/nxTnCt6j1hWh9ihT6hK+ B5BOX0ITsRD1a3eeIlgQ9avtMOn6N643DMegRLNCJ/G3/lNoOo4PAeEfJhsBLh3vHRwq 9GTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786530551; x=1787135351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NtTCK5Dii6LDoNzyyQ9y0Yku0qPmidmnjmWHD+vYK6o=; b=SHmDxQG7GA+AIKsJFKTKmQpg9hFbzf8B+M6MjUlYVovPdEjXepBYMPftGJvSRGytqK ZVDCoQsW5QMjin/XtIdP8LGEh27WK8fHUguRzzOpdzD/Hv6N771NmB6kKfn4BHmeX/xF bWCQJyqUJPwIjdyTYxvh63JPkVqfm2k0GhBdcuQgSLeRahdfaLvgNrNsKW6531H35F05 2nMsmOR1TEPLV8VTNaIiY++OATbdAK62knCreU7JNwNrLx4sFqotnUxcw/j721fIP+Gc zV/nQt+UACrtAo5+0EG3h950MwMkhjca8GozY6N64lqijY5XF04C2JxqMTZKrPExGJs9 O/WQ== X-Gm-Message-State: AOJu0YwPrY5yHPTseKgdTDnPrT2fywLP5UZ5+jzFJ8Q9euZgmzUM+o8T HnJUcXdYyeB+EO9fx7TxYa+rGbllB44l+iMf6laO2nLzpnCA+Qf+HMALLuIBt06G0QpqoXqkSkJ GO8bh X-Gm-Gg: AR+sD12l5piVyRzDvKPg1NwYqFwjT9fvc1DYi1AiuIDTlXXhXMU57ZzrKyIm6OntFV/ qgoGy9XM35O9Vq9ROXuNqyj3ov08x/nHDAnf8V+Bjlzrj06sbopP2IyjdfMEJCml9Xw/1vbk0+o ywQyiGjxjCTDl0ywoc+vTy961yXMIo4/pCFJLYvMmhfWRpo05W4FpYx+Ypn7pbMvWscGpbDJZu/ 6z/63erFNG1SNoEGqOSMZw/5L28g3Afvn6fEmXdwvTVR8IjTQchpQdO24mhjobxSiZNbz0WrQvD vhmkaweCgBaB6xP5xkinw6AkLD+elV+j0+MsFz2bo6YvYop5P4JSaYPeerjdSMnDLnT/+SsvsGK ss48/s4TyOkWz2bOcTdP3OQZFjFrGtd5xyWHfCFaTFdnRbSwcDvHk4NTSeWQHnyErWCmcT3bxhc IadLS5DPC5HETr6Ak+aCjo0KJHZZI/hXOGfNiIyy+TRoMZNK0hUAa6qq0XVg== X-Received: by 2002:a17:907:d1d:b0:c1f:97f3:7225 with SMTP id a640c23a62f3a-c20f2e3c984mr189342966b.6.1786530551485; Wed, 12 Aug 2026 03:29:11 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Eric Blake , Vladimir Sementsov-Ogievskiy Subject: [PATCH 2/3] block/nbd: never index requests[] with an unchecked cookie Date: Wed, 12 Aug 2026 12:29:05 +0200 Message-ID: <20260812102906.894063-3-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260812102906.894063-1-den@openvz.org> References: <20260812102906.894063-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::62b; envelope-from=den@openvz.org; helo=mail-ej1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1786530594911158500 Content-Type: text/plain; charset="utf-8" Cookies are converted into indices of s->requests[] in several places and the result is used right away, without any check: int i =3D COOKIE_TO_INDEX(cookie); ... return nbd_co_receive_offset_data_payload(s, s->requests[i].offset, COOKIE_TO_INDEX() subtracts one, so a zero cookie becomes an index of -1 and the access lands in front of the array. This is undefined and, depending on the type of the index and on what the compiler has put there, it can as well pass silently: at the site above i is signed, so even a plain i < MAX_NBD_REQUESTS check would happily let -1 through. The only cookie which is checked today is the one taken from the wire, in nbd_receive_replies(), where an invalid value is a protocol error rather than an internal inconsistency and thus has to stay a channel error. Route every other conversion through a helper which asserts the range before it returns the slot, so that the check can not be forgotten again. The cookie of the reply in flight goes through the helper as well. It is not an unchecked value: it has either passed the check above, or it has been cleared by the previous commit, so a stale index there is an internal inconsistency too. Signed-off-by: Denis V. Lunev Cc: Eric Blake Cc: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index d9b776283f..21f0f9d40d 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -136,6 +136,16 @@ static void nbd_clear_bdrvstate(BlockDriverState *bs) s->x_dirty_bitmap =3D NULL; } =20 +/* Not for cookies coming from the wire, those are checked separately. */ +static NBDClientRequest *nbd_request_by_cookie(BDRVNBDState *s, uint64_t c= ookie) +{ + uint64_t ind =3D COOKIE_TO_INDEX(cookie); + + assert(ind < MAX_NBD_REQUESTS); + + return &s->requests[ind]; +} + /* Called with s->receive_mutex taken. */ static bool coroutine_fn nbd_recv_coroutine_wake_one(NBDClientRequest *req) { @@ -422,7 +432,8 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDStat= e *s, uint64_t cookie, Error **errp) { int ret; - uint64_t ind =3D COOKIE_TO_INDEX(cookie), ind2; + NBDClientRequest *req =3D nbd_request_by_cookie(s, cookie); + uint64_t ind2; QEMU_LOCK_GUARD(&s->receive_mutex); =20 while (true) { @@ -437,10 +448,9 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDSta= te *s, uint64_t cookie, * woken by whoever set s->reply.cookie (or never wait in this * yield). So, we should not wake it here. */ - ind2 =3D COOKIE_TO_INDEX(s->reply.cookie); - assert(!s->requests[ind2].receiving); + assert(!nbd_request_by_cookie(s, s->reply.cookie)->receiving); =20 - s->requests[ind].receiving =3D true; + req->receiving =3D true; qemu_co_mutex_unlock(&s->receive_mutex); =20 qemu_coroutine_yield(); @@ -454,7 +464,7 @@ static coroutine_fn int nbd_receive_replies(BDRVNBDStat= e *s, uint64_t cookie, */ =20 qemu_co_mutex_lock(&s->receive_mutex); - assert(!s->requests[ind].receiving); + assert(!req->receiving); continue; } =20 @@ -861,7 +871,6 @@ static coroutine_fn int nbd_co_do_receive_one_chunk( { ERRP_GUARD(); int ret; - int i =3D COOKIE_TO_INDEX(cookie); void *local_payload =3D NULL; NBDStructuredReplyChunk *chunk; =20 @@ -919,8 +928,8 @@ static coroutine_fn int nbd_co_do_receive_one_chunk( return -EINVAL; } =20 - return nbd_co_receive_offset_data_payload(s, s->requests[i].offset, - qiov, errp); + return nbd_co_receive_offset_data_payload( + s, nbd_request_by_cookie(s, cookie)->offset, qiov, errp); } =20 if (nbd_reply_type_is_error(chunk->type)) { @@ -1067,7 +1076,7 @@ static bool coroutine_fn nbd_reply_chunk_iter_receive= (BDRVNBDState *s, =20 break_loop: qemu_mutex_lock(&s->requests_lock); - s->requests[COOKIE_TO_INDEX(cookie)].coroutine =3D NULL; + nbd_request_by_cookie(s, cookie)->coroutine =3D NULL; s->in_flight--; qemu_co_queue_next(&s->free_sema); qemu_mutex_unlock(&s->requests_lock); --=20 2.53.0 From nobody Wed Aug 26 07:24:39 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1786530570; cv=none; d=zohomail.com; s=zohoarc; b=TSTFGS/iAfLnKs9haJYBVXEd3M9898V5PchqbWRU4PipbjfXqwr4JcltINPr0E7gRAoVsbPGj2xXqWf64FG0wHqkzZB7eINfQY4oUkEQBNPQoXDfM3+FXbrSSRC8kT+jgp5nvNijCb5hqKzf0A6W+I2/+DEJv26XTIZEYqBFzGg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786530570; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fA9IQDUBY9wYqV4KUbqbPe1RIFw+DaKwtNPF7u/KkP4=; b=dalvvsO9duNTDaZBy4ttAKUG7RSYKPYXmb7tMkMJjPAuo5jsY1sRZBe0tGLZCCzjnrvmNNvSLGdh26kMhHqVCaZIRqf7TDKBiErIrdpEj1b76hM2TNwwroHxQOXgP5UHDE1UayJVRL31ypyX13TMCVS3yhTYh+2jXOpfJ11cBcM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786530570435978.9305413252152; Wed, 12 Aug 2026 03:29:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wu6D0-0005XU-S9; Wed, 12 Aug 2026 06:29:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wu6Cy-0005W6-P0 for qemu-devel@nongnu.org; Wed, 12 Aug 2026 06:29:16 -0400 Received: from mail-ej1-x635.google.com ([2a00:1450:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wu6Cw-0002jZ-Ov for qemu-devel@nongnu.org; Wed, 12 Aug 2026 06:29:16 -0400 Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-c15e2dab83eso131251266b.1 for ; Wed, 12 Aug 2026 03:29:14 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:89f2:ad10:d9bb:681e]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c20f0ad270bsm70748466b.46.2026.08.12.03.29.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Aug 2026 03:29:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1786530553; x=1787135353; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fA9IQDUBY9wYqV4KUbqbPe1RIFw+DaKwtNPF7u/KkP4=; b=nKRqyUEDWSY91GK2Jeclzj3pjTgURORQwQVyXBJHH52e+KPTxPE8SsxlK6Z0YtcahK nRQ5Appeqk8okAYDVAvlTN4oGhsN/Y3CKls20T8lzg+wGyQEWiLpAtGCUVpnSm5aSOP3 Gm8BWU/T2GHu+wHaUUIGOKB/OJgU+DN6T1CrUGRe+1ZdpPKQKcQARCZEajvNrW5f8KuL B4uC2g15UPC+SBsB253HXJlpP5M6X7uyEWXdWlJxPpLBhJdVDi0+u1lvQ2a/UDVc5sQG +1qPNs1LgeqaQ191iu2E3kYSXA/T7Kv0D/l9iyrxyNtaiek9mS6uU1fb1GfqVizGqPhZ 9cAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786530553; x=1787135353; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fA9IQDUBY9wYqV4KUbqbPe1RIFw+DaKwtNPF7u/KkP4=; b=MLbADiz8R04LR/xICjn1Eo/SBtACiqWfFpr2OiqIOjKElZy7amUfnYfhM2Yw9QjK0+ fT1NYsqH5xyPL1NmMZ7N8Zr0NSScjBQwTXYkJREsHkj2UMv4uKfAj/YXDUUVC6jMPJvu +ySGmkkMMAIbW9TPXQa8PAXHGM4dbqgsEqe71qmmVp5OJP03VlkR9LceYA88GtrDZMXb sHLUL7gHRXljEJxHA6FBHq/57ivdVNvl3wyfTT6vdikhenJHqjg0EOXbMDjFiZ8vmf7N +giMOEeW0N37XCidgYjISOnleoDHcS2G6akeoLu9gI5bOXmCs8sXKdnCDtDdlcDrWGtS NIfw== X-Gm-Message-State: AOJu0YwGwrH+r6rcgkc6o8cWcTNcYCpQ06g4PKJ70gY5vRfS+61Scab4 PFjk2DRA1vPnQLEjEZn3bOu/Uey/KG+sIm1vYMhjnJv+mtrRZhs4SzmX0wD7wcOyNLMsx5Mg7Hi 48x3o X-Gm-Gg: AR+sD13V7iIfWpnP29qLyhZqP2+2R4txA43LIINaUj98d6M1Q46m0PqraW1pxDVI1ba cgjq6LThYErUIBADFpYPXJWuSE13UYqHb6R8mc4U6qkoWXDlAgknoo5TEq/xRN4IshjXSQ99QNp UIQMLa6epZbQX3rEI+Jr0NvAvTb1YPUL4FBtlWq+thRZG4nyJeCMvx3x/Ke8xw+aW/TkICSWcbA 2J4lUpfVNGPiIa3J3NZYroqzF3AyZ1eWzlow0LCNrFwT94PIDZnLzm/pa0jJfrErGlppSS6OPIw 3l3011rDbUX3iPhU0GdDjjjDj0EBpuWSAZuN5yxXYal5GPkdQ8vP3fpCPHYdf7fP0aeLLFe7Ph7 8cvXQZ3Of9WqtS600TdptdUFc38Mn9pTNpZsx7d9PyZh9ETMasyfPSFyLfcvqKoKRFYNo2O0Myr 68tPXMrzaXoMrLJr+C+a4l4ILnPHmSDsk+Z5xIOROYakGKVd0HJoMgvs+Khw== X-Received: by 2002:a17:906:7954:b0:c1f:c061:569a with SMTP id a640c23a62f3a-c20f2e4f67amr154679866b.7.1786530552874; Wed, 12 Aug 2026 03:29:12 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Eric Blake , Vladimir Sementsov-Ogievskiy Subject: [PATCH 3/3] block/nbd: clear reply.cookie under receive_mutex Date: Wed, 12 Aug 2026 12:29:06 +0200 Message-ID: <20260812102906.894063-4-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260812102906.894063-1-den@openvz.org> References: <20260812102906.894063-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::635; envelope-from=den@openvz.org; helo=mail-ej1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1786530572592158500 Content-Type: text/plain; charset="utf-8" s->reply is documented as protected by s->receive_mutex, but the cookie is cleared without it once the owning request has consumed its chunk. A waiter in nbd_receive_replies() inspects the very same field under the mutex, and does so with two separate loads: if (s->reply.cookie !=3D 0) { ind2 =3D COOKIE_TO_INDEX(s->reply.cookie); assert(!s->requests[ind2].receiving); Nothing keeps those two loads consistent. If the owner clears the cookie in between, the second one reads 0, COOKIE_TO_INDEX() turns it into an index of -1, and s->requests[] is accessed out of bounds: Assertion `!s->requests[ind2].receiving' failed. (gdb) p cookie $1 =3D 8 (gdb) p s->reply.cookie $2 =3D 0 (gdb) p &((NBDClientRequest *)s->requests)[-1].receiving $3 =3D (_Bool *) 0x5555558416c0 (gdb) p &s->in_flight $4 =3D (unsigned int *) 0x5555558416c0 (gdb) p s->in_flight $5 =3D 8 The cookie we wait for is 8, yet reply.cookie reads 0 one line after it was found non-zero, so the index is -1. requests[-1].receiving lands on in_flight, which is non-zero while requests are outstanding, and that is what the assertion trips over. Hitting this requires two coroutines of one NBD node to run in different threads, as there is no yield point between the two loads for the owner to squeeze into. A multiqueue configuration provides exactly that, with the virtqueues of one disk spread over several iothreads. Note that a compiler is free to merge the two loads into one, in which case the race is invisible, so builds with reduced optimization are much more likely to trip over it. Accessing s->reply without the mutex is fine for the coroutine that owns the reply: a non-zero cookie makes the field private to it. Releasing that ownership is not, as it races with the waiters which are explicitly allowed to look at the cookie. Clear it under the mutex, in the same critical section as the wakeup, and make nbd_recv_coroutines_wake() caller-locked, as CoMutex is not recursive. It has a single caller. The added acquisition cannot block behind the header read in nbd_receive_replies(), because that path is only reachable with reply.cookie =3D=3D 0 while we still own a non-zero cookie. Merging the clear with the wakeup also keeps a newcomer from starting a header read in between, which would stall this already completed request for the duration of that read. There is no cookie to own when we get here after an error, and then a newcomer can indeed be inside that read. It does not hold us for long either, as the channel has been shut down before the error was reported, so the read it sits in returns right away. Fixes: 4ddb5d2fde ("block/nbd: drop connection_co") Signed-off-by: Denis V. Lunev Cc: Eric Blake Cc: Vladimir Sementsov-Ogievskiy Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 21f0f9d40d..1659008784 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -158,11 +158,11 @@ static bool coroutine_fn nbd_recv_coroutine_wake_one(= NBDClientRequest *req) return false; } =20 +/* Called with s->receive_mutex taken. */ static void coroutine_fn nbd_recv_coroutines_wake(BDRVNBDState *s) { int i; =20 - QEMU_LOCK_GUARD(&s->receive_mutex); for (i =3D 0; i < MAX_NBD_REQUESTS; i++) { if (nbd_recv_coroutine_wake_one(&s->requests[i])) { return; @@ -970,9 +970,11 @@ static coroutine_fn int nbd_co_receive_one_chunk( /* For assert at loop start in nbd_connection_entry */ *reply =3D s->reply; } - s->reply.cookie =3D 0; =20 - nbd_recv_coroutines_wake(s); + WITH_QEMU_LOCK_GUARD(&s->receive_mutex) { + s->reply.cookie =3D 0; + nbd_recv_coroutines_wake(s); + } =20 return ret; } --=20 2.53.0