From nobody Sun Oct 5 13:58:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1748556221; cv=none; d=zohomail.com; s=zohoarc; b=mKp9DR6b8K4Aaf2q3tgL8Wyg/9IKkpf8t+DDn+vGXRUCIVJ+mr1oyiBwyrHKVpr3fj/HweWfXFXot6b4kJkS+jJVTt0LHMhZSDjDaUGj/nnSs/jWypwnzLsWC+psaQKoZjwfJELkuciLnB0vnp7QIJq7JtIDZjG9p7KvNBZILGU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748556221; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mKXtOIjLZ2HpnXimfT4KSht7b6TlRbU7mhUuvrVt6xE=; b=MXe7TxRw4JsVb+lthyYjhb1o3pe1VXfJr40H59AViV2QRGIVc7TDNP6Dloq7u8cThwXAhD3K80DrY+H9LNjGabjApVTDr/2qrq9bS58BGi+NoJLLRS0xnEDsE8siPHu2BgTeY+eZ+n2LGs90cakl80HxTdDx5Vo11fywyUDRAmo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1748556221536279.49077225578765; Thu, 29 May 2025 15:03:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uKlKm-0005WS-Nk; Thu, 29 May 2025 18:02:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKj-0005VY-N5 for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKg-0002Yp-Fh for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:41 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-554-WLODX7WRMROAkqHDlA4iPg-1; Thu, 29 May 2025 18:02:34 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CD5701956046; Thu, 29 May 2025 22:02:33 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.48]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E2C3619560A7; Thu, 29 May 2025 22:02:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1748556156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mKXtOIjLZ2HpnXimfT4KSht7b6TlRbU7mhUuvrVt6xE=; b=iMQzfN/HHy4ID98qKAdg9yxoWiX95/6R/SloGaE0iLXOv13zpRa2iEvD3GqRHoWQgmHwy+ F0PhGZL8Rnc+e54lBP4ylkMVv+usCm31LZakXC8vSXRRmwkc4jNo4YeO2J+p7L2GPvDMAl 857JEV5t/5lmtcTRP/43r9rFBF7ihkk= X-MC-Unique: WLODX7WRMROAkqHDlA4iPg-1 X-Mimecast-MFC-AGG-ID: WLODX7WRMROAkqHDlA4iPg_1748556153 From: Eric Blake To: qemu-devel@nongnu.org Cc: Nir Soffer , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PULL 1/6] io: Add helper for setting socket send buffer size Date: Thu, 29 May 2025 17:02:08 -0500 Message-ID: <20250529220228.1187563-9-eblake@redhat.com> In-Reply-To: <20250529220228.1187563-8-eblake@redhat.com> References: <20250529220228.1187563-8-eblake@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -49 X-Spam_score: -5.0 X-Spam_bar: ----- X-Spam_report: (-5.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.902, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1748556223603116600 From: Nir Soffer Testing reading and writing from qemu-nbd using a unix domain socket shows that the platform default send buffer size is too low, leading to poor performance and hight cpu usage. Add a helper for setting socket send buffer size to be used in NBD code. It can also be used in other contexts. We don't need a helper for receive buffer size since it is not used with unix domain sockets. This is documented for Linux, and not documented for macOS. Failing to set the socket buffer size is not a fatal error, but the caller may want to warn about the failure. Signed-off-by: Nir Soffer Message-ID: <20250517201154.88456-2-nirsof@gmail.com> Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Eric Blake --- include/io/channel-socket.h | 13 +++++++++++++ io/channel-socket.c | 11 +++++++++++ 2 files changed, 24 insertions(+) diff --git a/include/io/channel-socket.h b/include/io/channel-socket.h index ab15577d384..a88cf8b3a9f 100644 --- a/include/io/channel-socket.h +++ b/include/io/channel-socket.h @@ -261,5 +261,18 @@ QIOChannelSocket * qio_channel_socket_accept(QIOChannelSocket *ioc, Error **errp); +/** + * qio_channel_socket_set_send_buffer: + * @ioc: the socket channel object + * @size: buffer size + * @errp: pointer to a NULL-initialized error object + * + * Set the underlying socket send buffer size. + * + * Retruns: 0 on success, or -1 on error. + */ +int qio_channel_socket_set_send_buffer(QIOChannelSocket *ioc, + size_t size, + Error **errp); #endif /* QIO_CHANNEL_SOCKET_H */ diff --git a/io/channel-socket.c b/io/channel-socket.c index 088b49ffdb0..3b7ca924ff3 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -78,6 +78,17 @@ qio_channel_socket_new(void) return sioc; } +int qio_channel_socket_set_send_buffer(QIOChannelSocket *ioc, + size_t size, + Error **errp) +{ + if (setsockopt(ioc->fd, SOL_SOCKET, SO_SNDBUF, &size, sizeof(size)) < = 0) { + error_setg_errno(errp, errno, "Unable to set socket send buffer si= ze"); + return -1; + } + + return 0; +} static int qio_channel_socket_set_fd(QIOChannelSocket *sioc, --=20 2.49.0 From nobody Sun Oct 5 13:58:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1748556204; cv=none; d=zohomail.com; s=zohoarc; b=WzifG55WgDG3jIspjnt+FQwrzUbAmZWm+J+AEFIbsuejk1q0+ZvYrm8PphKWhaXM/JsPHkYRYJahnyQxRRU0Lv9xi8XR6qd/0Prv/Nv+0jmVrzatmaCO1iMtCDn7xEtj94/If70usXfsqvCNW31Y5bB+ESDpDkzqSBmvnqLDx4A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748556204; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=U0vEHaYcZpF3/g7DHEkOZXpi9Y1ZKVx3ya/+92EcPpQ=; b=E//4Gi99esXC9k4VrQqx0vnayp0u7KAs/bdqm82h9eDOJYy6VMtEjv5wnuJfGL13Nx/9ycfm3MtHqppuQDNBrh1LhjECtN0nSQqP4kBEG/FdV1DolNSMspqB38TO8Z3EGJWyp9h4CP/83/doeMo7Ul4mtEYzS5kad7YvlAPiDS4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1748556204339134.21963052973456; Thu, 29 May 2025 15:03:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uKlKm-0005WA-3F; Thu, 29 May 2025 18:02:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKk-0005Vh-1g for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKi-0002Yx-1h for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:41 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-257-FLvnN5IGPmaijhFYwypKyw-1; Thu, 29 May 2025 18:02:37 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EE23E1956086; Thu, 29 May 2025 22:02:35 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.48]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6B1AA19560AE; Thu, 29 May 2025 22:02:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1748556159; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U0vEHaYcZpF3/g7DHEkOZXpi9Y1ZKVx3ya/+92EcPpQ=; b=G56Z1BzgPfSXZkzRR85m0dvzYHUdLrmQ5Xy45IXprlhBrWtVhOY7V9gD6ujYLvgc1/aPpF 7VA6rxcRDmtCB0SQ5SI0FtA73A580Ic+FC7n1utIO8DCta9eaFOvOCYXgawgCv6ShYQiM0 Hj87UaWH6pN6okUFFTpowq0cBZAXQO0= X-MC-Unique: FLvnN5IGPmaijhFYwypKyw-1 X-Mimecast-MFC-AGG-ID: FLvnN5IGPmaijhFYwypKyw_1748556156 From: Eric Blake To: qemu-devel@nongnu.org Cc: Nir Soffer , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org (open list:Network Block Dev...) Subject: [PULL 2/6] nbd: Set unix socket send buffer on macOS Date: Thu, 29 May 2025 17:02:09 -0500 Message-ID: <20250529220228.1187563-10-eblake@redhat.com> In-Reply-To: <20250529220228.1187563-8-eblake@redhat.com> References: <20250529220228.1187563-8-eblake@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -49 X-Spam_score: -5.0 X-Spam_bar: ----- X-Spam_report: (-5.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.902, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1748556205878116600 From: Nir Soffer On macOS we need to increase unix socket buffers size on the client and server to get good performance. We set socket buffers on macOS after connecting or accepting a client connection. Testing shows that setting socket receive buffer size (SO_RCVBUF) has no effect on performance, so we set only the send buffer size (SO_SNDBUF). It seems to work like Linux but not documented. Testing shows that optimal buffer size is 512k to 4 MiB, depending on the test case. The difference is very small, so I chose 2 MiB. I tested reading from qemu-nbd and writing to qemu-nbd with qemu-img and computing a blkhash with nbdcopy and blksum. To focus on NBD communication and get less noisy results, I tested reading and writing to null-co driver. I added a read-pattern option to the null-co driver to return data full of 0xff: NULL=3D"json:{'driver': 'raw', 'file': {'driver': 'null-co', 'size': '1= 0g', 'read-pattern': 255}}" For testing buffer size I added an environment variable for setting the socket buffer size. Read from qemu-nbd via qemu-img convert. In this test buffer size of 2m is optimal (12.6 times faster). qemu-nbd -r -t -e 0 -f raw -k /tmp/nbd.sock "$NULL" & qemu-img convert -f raw -O raw -W -n "nbd+unix:///?socket=3D/tmp/nbd.so= ck" "$NULL" | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 13.361 | 2.653 | 5.702 | | 65536 | 2.283 | 0.204 | 1.318 | | 131072 | 1.673 | 0.062 | 1.008 | | 262144 | 1.592 | 0.053 | 0.952 | | 524288 | 1.496 | 0.049 | 0.887 | | 1048576 | 1.234 | 0.047 | 0.738 | | 2097152 | 1.060 | 0.080 | 0.602 | | 4194304 | 1.061 | 0.076 | 0.604 | Write to qemu-nbd with qemu-img convert. In this test buffer size of 2m is optimal (9.2 times faster). qemu-nbd -t -e 0 -f raw -k /tmp/nbd.sock "$NULL" & qemu-img convert -f raw -O raw -W -n "$NULL" "nbd+unix:///?socket=3D/tm= p/nbd.sock" | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 8.063 | 2.522 | 4.184 | | 65536 | 1.472 | 0.430 | 0.867 | | 131072 | 1.071 | 0.297 | 0.654 | | 262144 | 1.012 | 0.239 | 0.587 | | 524288 | 0.970 | 0.201 | 0.514 | | 1048576 | 0.895 | 0.184 | 0.454 | | 2097152 | 0.877 | 0.174 | 0.440 | | 4194304 | 0.944 | 0.231 | 0.535 | Compute a blkhash with nbdcopy, using 4 NBD connections and 256k request size. In this test buffer size of 4m is optimal (5.1 times faster). qemu-nbd -r -t -e 0 -f raw -k /tmp/nbd.sock "$NULL" & nbdcopy --blkhash "nbd+unix:///?socket=3D/tmp/nbd.sock" null: | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 8.624 | 5.727 | 6.507 | | 65536 | 2.563 | 4.760 | 2.498 | | 131072 | 1.903 | 4.559 | 2.093 | | 262144 | 1.759 | 4.513 | 1.935 | | 524288 | 1.729 | 4.489 | 1.924 | | 1048576 | 1.696 | 4.479 | 1.884 | | 2097152 | 1.710 | 4.480 | 1.763 | | 4194304 | 1.687 | 4.479 | 1.712 | Compute a blkhash with blksum, using 1 NBD connection and 256k read size. In this test buffer size of 512k is optimal (10.3 times faster). qemu-nbd -r -t -e 0 -f raw -k /tmp/nbd.sock "$NULL" & blksum "nbd+unix:///?socket=3D/tmp/nbd.sock" | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 13.085 | 5.664 | 6.461 | | 65536 | 3.299 | 5.106 | 2.515 | | 131072 | 2.396 | 4.989 | 2.069 | | 262144 | 1.607 | 4.724 | 1.555 | | 524288 | 1.271 | 4.528 | 1.224 | | 1048576 | 1.294 | 4.565 | 1.333 | | 2097152 | 1.299 | 4.569 | 1.344 | | 4194304 | 1.291 | 4.559 | 1.327 | Signed-off-by: Nir Soffer Message-ID: <20250517201154.88456-3-nirsof@gmail.com> Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Eric Blake --- nbd/nbd-internal.h | 5 +++++ nbd/client-connection.c | 3 +++ nbd/common.c | 25 +++++++++++++++++++++++++ nbd/server.c | 2 ++ 4 files changed, 35 insertions(+) diff --git a/nbd/nbd-internal.h b/nbd/nbd-internal.h index 715d92d6efc..6bafeef5ddc 100644 --- a/nbd/nbd-internal.h +++ b/nbd/nbd-internal.h @@ -74,4 +74,9 @@ static inline int nbd_write(QIOChannel *ioc, const void *= buffer, size_t size, int nbd_drop(QIOChannel *ioc, size_t size, Error **errp); +/* nbd_set_socket_send_buffer + * Set the socket send buffer size for optimal performance. + */ +void nbd_set_socket_send_buffer(QIOChannelSocket *sioc); + #endif diff --git a/nbd/client-connection.c b/nbd/client-connection.c index b11e266807d..79ea97e4cc1 100644 --- a/nbd/client-connection.c +++ b/nbd/client-connection.c @@ -31,6 +31,8 @@ #include "qapi/clone-visitor.h" #include "qemu/coroutine.h" +#include "nbd/nbd-internal.h" + struct NBDClientConnection { /* Initialization constants, never change */ SocketAddress *saddr; /* address to connect to */ @@ -140,6 +142,7 @@ static int nbd_connect(QIOChannelSocket *sioc, SocketAd= dress *addr, return ret; } + nbd_set_socket_send_buffer(sioc); qio_channel_set_delay(QIO_CHANNEL(sioc), false); if (!info) { diff --git a/nbd/common.c b/nbd/common.c index 589a748cfe6..9436e9d1d14 100644 --- a/nbd/common.c +++ b/nbd/common.c @@ -18,6 +18,9 @@ #include "qemu/osdep.h" #include "trace.h" +#include "io/channel-socket.h" +#include "qapi/error.h" +#include "qemu/units.h" #include "nbd-internal.h" /* Discard length bytes from channel. Return -errno on failure and 0 on @@ -264,3 +267,25 @@ const char *nbd_mode_lookup(NBDMode mode) return ""; } } + +/* + * Testing shows that 2m send buffer is optimal. Changing the receive buff= er + * size has no effect on performance. + */ +#if defined(__APPLE__) +#define UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE (2 * MiB) +#endif + +void nbd_set_socket_send_buffer(QIOChannelSocket *sioc) +{ +#ifdef UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE + if (sioc->localAddr.ss_family =3D=3D AF_UNIX) { + size_t size =3D UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE; + Error *errp =3D NULL; + + if (qio_channel_socket_set_send_buffer(sioc, size, &errp) < 0) { + warn_report_err(errp); + } + } +#endif /* UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE */ +} diff --git a/nbd/server.c b/nbd/server.c index 2076fb2666b..d242be98115 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -3291,6 +3291,8 @@ void nbd_client_new(QIOChannelSocket *sioc, client->close_fn =3D close_fn; client->owner =3D owner; + nbd_set_socket_send_buffer(sioc); + co =3D qemu_coroutine_create(nbd_co_client_start, client); qemu_coroutine_enter(co); } --=20 2.49.0 From nobody Sun Oct 5 13:58:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1748556203; cv=none; d=zohomail.com; s=zohoarc; b=mZQuJqx6bgzvUwnx1/X42U5FQpIjhjqw3GO1pI6qb02vfhQyKEQONHodCXXG3Gtd/8YA1g68PYRtshvPL4jG1QRkfqbGVbQRK063rYM/55aiN9P/lUgIK80tzG72TR4tKUp9h+menTJf7f2RJhv3B1x+ECqBqnKB6gzN5lFDXY8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748556203; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tkU3lGy5x3O+d/0bLwArMdgLP1ASjGcnbZa8XHDrIqs=; b=DTcYAaFyNf+teiyMkuDW1DbjGhdK+7e1GFxMHEMZwR0fbXZbQ6sxn3cGGpMQXJFkB1SisR/ReVO65TiZem6MhAapj2l7MFDfSWWgcuDOm0yllk4D5LKKJ8tzf9C4PAJGLhebXu1iM0dzIUTvSSgYHf7EBm+T+SwJtGHxypprxDA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1748556203587366.47268696753315; Thu, 29 May 2025 15:03:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uKlKp-0005Xb-33; Thu, 29 May 2025 18:02:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKn-0005Wm-8t for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKl-0002ZX-KF for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:45 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-331-Bg7H7zHeM0-VQm_2uJt1hA-1; Thu, 29 May 2025 18:02:39 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 46544195609E; Thu, 29 May 2025 22:02:38 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.48]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 85F6C19560AE; Thu, 29 May 2025 22:02:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1748556163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tkU3lGy5x3O+d/0bLwArMdgLP1ASjGcnbZa8XHDrIqs=; b=NdJ08+TEk4biQRmpKvdUaKLxlu1OMIt+rUGOPIcUKDdkw1In+nAvpYQqp7Z14uSPgxlkmf T5bdA+Jzjryrgn1I3r+mE492Eib8r9JNt4l3mfmnn9tcODeSfnr87xuDFtohWP0DE1ouMn FWqTMgODB2gKKiQ/ykBeLIiPOwl881g= X-MC-Unique: Bg7H7zHeM0-VQm_2uJt1hA-1 X-Mimecast-MFC-AGG-ID: Bg7H7zHeM0-VQm_2uJt1hA_1748556158 From: Eric Blake To: qemu-devel@nongnu.org Cc: Nir Soffer , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org (open list:Network Block Dev...) Subject: [PULL 3/6] nbd: Set unix socket send buffer on Linux Date: Thu, 29 May 2025 17:02:10 -0500 Message-ID: <20250529220228.1187563-11-eblake@redhat.com> In-Reply-To: <20250529220228.1187563-8-eblake@redhat.com> References: <20250529220228.1187563-8-eblake@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -49 X-Spam_score: -5.0 X-Spam_bar: ----- X-Spam_report: (-5.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.902, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1748556205823116600 From: Nir Soffer Like macOS we have similar issue on Linux. For TCP socket the send buffer size is 2626560 bytes (~2.5 MiB) and we get good performance. However for unix socket the default and maximum buffer size is 212992 bytes (208 KiB) and we see poor performance when using one NBD connection, up to 4 times slower than macOS on the same machine. Tracing shows that for every 2 MiB payload (qemu uses 2 MiB io size), we do 1 recvmsg call with TCP socket, and 10 recvmsg calls with unix socket. Fixing this issue requires changing the maximum send buffer size (the receive buffer size is ignored). This can be done using: $ cat /etc/sysctl.d/net-mem-max.conf net.core.wmem_max =3D 2097152 $ sudo sysctl -p /etc/sysctl.d/net-mem-max.conf With this we can set the socket buffer size to 2 MiB. With the defaults the value requested by qemu is clipped to the maximum size and has no effect. I tested on 2 machines: - Fedora 42 VM on MacBook Pro M2 Max - Dell PowerEdge R640 (Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz) On the older Dell machine we see very little improvement, up to 1.03 higher throughput. On the M2 machine we see up to 2.67 times higher throughput. The following results are from the M2 machine. Reading from qemu-nbd with qemu-img convert. In this test buffer size of 4m is optimal (2.28 times faster). | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 4.292 | 0.243 | 1.604 | | 524288 | 2.167 | 0.058 | 1.288 | | 1048576 | 2.041 | 0.060 | 1.238 | | 2097152 | 1.884 | 0.060 | 1.191 | | 4194304 | 1.881 | 0.054 | 1.196 | Writing to qemu-nbd with qemu-img convert. In this test buffer size of 1m is optimal (2.67 times faster). | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 3.113 | 0.334 | 1.094 | | 524288 | 1.173 | 0.179 | 0.654 | | 1048576 | 1.164 | 0.164 | 0.670 | | 2097152 | 1.227 | 0.197 | 0.663 | | 4194304 | 1.227 | 0.198 | 0.666 | Computing a blkhash with nbdcopy. In this test buffer size of 512k is optimal (1.19 times faster). | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 2.140 | 4.483 | 2.681 | | 524288 | 1.794 | 4.467 | 2.572 | | 1048576 | 1.807 | 4.447 | 2.644 | | 2097152 | 1.822 | 4.461 | 2.698 | | 4194304 | 1.827 | 4.465 | 2.700 | Computing a blkhash with blksum. In this test buffer size of 4m is optimal (2.65 times faster). | buffer size | time | user | system | |-------------|---------|---------|---------| | default | 3.582 | 4.595 | 2.392 | | 524288 | 1.499 | 4.384 | 1.482 | | 1048576 | 1.377 | 4.381 | 1.345 | | 2097152 | 1.388 | 4.389 | 1.354 | | 4194304 | 1.352 | 4.395 | 1.302 | Signed-off-by: Nir Soffer Message-ID: <20250517201154.88456-4-nirsof@gmail.com> Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Eric Blake --- nbd/common.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/nbd/common.c b/nbd/common.c index 9436e9d1d14..2a133a66c39 100644 --- a/nbd/common.c +++ b/nbd/common.c @@ -271,8 +271,9 @@ const char *nbd_mode_lookup(NBDMode mode) /* * Testing shows that 2m send buffer is optimal. Changing the receive buff= er * size has no effect on performance. + * On Linux we need to increase net.core.wmem_max to make this effective. */ -#if defined(__APPLE__) +#if defined(__APPLE__) || defined(__linux__) #define UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE (2 * MiB) #endif --=20 2.49.0 From nobody Sun Oct 5 13:58:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1748556255; cv=none; d=zohomail.com; s=zohoarc; b=bvpk4A0PemkMbh8sKVwquzob1aRcWzTI3cVY414qdKUYlcYuASekTj4tZrvtUtEhc34QKJy2+U1d1GBwvjiDGKxfAagysCdPfE9k3Gltou8pj4N4D8Pp7t1NcdszvU3+jhHBGs6RD3NjJ9q6U7f4NNIUOz7sCEHzK+RJ19KBS40= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748556255; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=infoBmjAFnt00Pn9BDmXDmuyPlBkDoA5gGMWMip+CFA=; b=YOT2PerTUCXolqNCjJXvh+YCQ55zDD1R5lmp/7PXuz5E/z5V/lPCfPfHAEaTdAFj4GoEPDu8wipYHvGTII7t1RMxjzkSOdnHyQR6Xlv8sjzDwK2z2x5/offF6hrPTgqKlpVCqRNsfuBgoXynnXzMP8CiWq2vPp8sJNyjd9XaoLY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1748556255334139.2230203589413; Thu, 29 May 2025 15:04:15 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uKlKq-0005YI-NE; Thu, 29 May 2025 18:02:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKo-0005XG-H6 for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKm-0002Zf-Ac for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:46 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-504-OdhmHnFAPqy0NW1DxsURbw-1; Thu, 29 May 2025 18:02:42 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D02B61956094; Thu, 29 May 2025 22:02:40 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.48]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 49CD119560A7; Thu, 29 May 2025 22:02:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1748556163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=infoBmjAFnt00Pn9BDmXDmuyPlBkDoA5gGMWMip+CFA=; b=AhyAeQeM5jjmkf4KAVsu0+oTF/wz82SpNFO6dQmloFSy7tId2ym7eQ7Pe14aFY8zErbonm 8Oar4mRw3+0/QnQRVTg7c1MNdNFK5XCyquklT0UNVgVMcQir5esNzv1kyx+UEK0KWPhsW4 FuOCm++3y2QRM6PlWE7sw4evE8oUvDo= X-MC-Unique: OdhmHnFAPqy0NW1DxsURbw-1 X-Mimecast-MFC-AGG-ID: OdhmHnFAPqy0NW1DxsURbw_1748556161 From: Eric Blake To: qemu-devel@nongnu.org Cc: Fiona Ebner , Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org (open list:Block layer core) Subject: [PULL 4/6] iotests: Use disk_usage in more places Date: Thu, 29 May 2025 17:02:11 -0500 Message-ID: <20250529220228.1187563-12-eblake@redhat.com> In-Reply-To: <20250529220228.1187563-8-eblake@redhat.com> References: <20250529220228.1187563-8-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -49 X-Spam_score: -5.0 X-Spam_bar: ----- X-Spam_report: (-5.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.902, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1748556256384116600 Content-Type: text/plain; charset="utf-8" Commit be9bac07 added a utility disk_usage function, but there are a couple of other tests that could also use it. Signed-off-by: Eric Blake Message-ID: <20250523163041.2548675-6-eblake@redhat.com> Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner --- tests/qemu-iotests/125 | 2 +- tests/qemu-iotests/308 | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/125 b/tests/qemu-iotests/125 index 46279d6b382..708e7c5ba21 100755 --- a/tests/qemu-iotests/125 +++ b/tests/qemu-iotests/125 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 get_image_size_on_host() { - echo $(($(stat -c '%b * %B' "$TEST_IMG_FILE"))) + disk_usage "$TEST_IMG_FILE" } # get standard environment and filters diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 index ea81dc496a0..437a9014da5 100755 --- a/tests/qemu-iotests/308 +++ b/tests/qemu-iotests/308 @@ -290,7 +290,7 @@ echo '--- Try growing non-growable export ---' # Get the current size so we can write beyond the EOF orig_len=3D$(get_proto_len "$EXT_MP" "$TEST_IMG") -orig_disk_usage=3D$(stat -c '%b' "$TEST_IMG") +orig_disk_usage=3D$(disk_usage "$TEST_IMG") # Should fail (exports are non-growable by default) # (Note that qemu-io can never write beyond the EOF, so we have to use @@ -312,7 +312,7 @@ else echo 'OK: Post-truncate image size is as expected' fi -new_disk_usage=3D$(stat -c '%b' "$TEST_IMG") +new_disk_usage=3D$(disk_usage "$TEST_IMG") if [ "$new_disk_usage" -gt "$orig_disk_usage" ]; then echo 'OK: Disk usage grew with fallocate' else --=20 2.49.0 From nobody Sun Oct 5 13:58:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1748556221; cv=none; d=zohomail.com; s=zohoarc; b=bLB6wssloiz1gWwT+/lxbarhBv+Th2efe2GU147QVZoTtGT9MjbjJSXIFXoQ/klUFgqpeuCnUj91RAD26Bk68CIil8kmZD676BLf0Ab87CnjUV7T3u13e32URIOr2d0jT2KWCdHVokdHoricn0aK+bV2b7tTokStrCbdNCHeE/4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748556221; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LeD2BIoGTKUjh+aXkMdd6B4gaNYXyHooACaXJjli2rw=; b=KfwgS3NfaD1QpAMNLOB+l0jOOowOfsQIC5VCZ4EPLJQxmEcNvYxBIYL0tVwKlqVo79HAzV33pgdilCnEscrnbGqELy52ZzCsHHb69Ft6pT6sIu2gagkkqNP5dAInQNMzT/X3vDHenTvNUITaaHVP52Mz6IIgrsNiVf9k1E6aMK4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1748556221612738.9419170802644; Thu, 29 May 2025 15:03:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uKlKt-0005ai-Rb; Thu, 29 May 2025 18:02:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKs-0005a0-T6 for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKq-0002aH-La for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:50 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-94-9hb9Qq9zMdaGr_n5HbBU3A-1; Thu, 29 May 2025 18:02:44 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 66C691800446; Thu, 29 May 2025 22:02:43 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.48]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3CE0A19560AE; Thu, 29 May 2025 22:02:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1748556166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LeD2BIoGTKUjh+aXkMdd6B4gaNYXyHooACaXJjli2rw=; b=VYSSmqqMES9AeZNXdwGVdClfxZR08PevBVjm/QeOPQ50hpPTpxMLL4thKPaqP0TXW1mObe OR1QxtZpJNENK/62n+zq1smSiJOZOkP6SoSgp4+4bt/mG9y4gPyyR7kiHcaJQD7BNxLJdU Ls4rHpWrsghVo4kwWIiVeYtrsZJV/xw= X-MC-Unique: 9hb9Qq9zMdaGr_n5HbBU3A-1 X-Mimecast-MFC-AGG-ID: 9hb9Qq9zMdaGr_n5HbBU3A_1748556163 From: Eric Blake To: qemu-devel@nongnu.org Cc: Fiona Ebner , Markus Armbruster , Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org (open list:Block layer core) Subject: [PULL 5/6] iotests: Improve mirror-sparse on ext4 and xfs Date: Thu, 29 May 2025 17:02:12 -0500 Message-ID: <20250529220228.1187563-13-eblake@redhat.com> In-Reply-To: <20250529220228.1187563-8-eblake@redhat.com> References: <20250529220228.1187563-8-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -49 X-Spam_score: -5.0 X-Spam_bar: ----- X-Spam_report: (-5.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.902, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1748556223843116600 Content-Type: text/plain; charset="utf-8" Fiona reported that an ext4 filesystem on top of LVM can sometimes report over-allocation to du (based on the heuristics the filesystem is making while observing the contents being mirrored); even though the contents and actual size matched, about 50% of the time the size reported by disk_usage was too large by 4k, failing the test. In auditing other iotests, this is a common problem we've had to deal with. Meanwhile, Markus reported that an xfs filesystem reports disk usage at a default granularity of 1M (so the sparse file occupies 3M, since it has just over 2M data). Reported-by: Fiona Ebner Reported-by: Markus Armbruster Fixes: c0ddcb2c ("tests: Add iotest mirror-sparse for recent patches") Signed-off-by: Eric Blake Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner Message-ID: <20250523163041.2548675-7-eblake@redhat.com> [eblake: Also fix xfs issue] Signed-off-by: Eric Blake --- tests/qemu-iotests/tests/mirror-sparse | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/tests/mirror-sparse b/tests/qemu-iotests/te= sts/mirror-sparse index 8c52a4e2448..11418c08713 100755 --- a/tests/qemu-iotests/tests/mirror-sparse +++ b/tests/qemu-iotests/tests/mirror-sparse @@ -96,13 +96,15 @@ _send_qemu_cmd $h1 '{"execute": "blockdev-del", "argume= nts": {"node-name": "dst"}}' 'return' \ | _filter_block_job_offset | _filter_block_job_len $QEMU_IMG compare -U -f $IMGFMT -F raw $TEST_IMG.base $TEST_IMG +# Some filesystems can fudge allocations for various reasons; rather +# than expecting precise 2M and 20M images, it is better to allow for slop. result=3D$(disk_usage $TEST_IMG) -if test $result -lt $((3*1024*1024)); then +if test $result -lt $((4*1024*1024)); then actual=3Dsparse -elif test $result =3D $((20*1024*1024)); then +elif test $result -gt $((19*1024*1024)); then actual=3Dfull else - actual=3Dunknown + actual=3D"unexpected size ($result)" fi echo "Destination is $actual; expected $expected" } --=20 2.49.0 From nobody Sun Oct 5 13:58:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1748556276; cv=none; d=zohomail.com; s=zohoarc; b=e2B95U5dkkMERQfIQ2h8zVJsjy16S/ot1cFgASXu3R9YidNCvWpVMq+QCTIvGFRxhEV+dKa0q9WI3gu8rZXcmQ5XqvQQxJ0BB24kw3/oHkqmB4aK0NWyVW0RU1J3nYKR2HyeW0mJa887ysYn1aWP/8ZYvcPk3AkOZ/5CJcA4DHg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1748556276; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=trLlnZ7xN6wN3xsvXNehE4gJcK9/Aq7r+pfz0MLrc2g=; b=oAWveDlYrum1AYi5A593+CBLf2/T56Y8//FKopBqOpHQvT0HeBhyDZxVPT5OLAHlZA0CCOT4zNRdcFE1eyg2/kExF4cjhBFgO2dWT5qdZFFuumUjf4lXDYo794LHhFlBgIt4yhpBd8BAw0gZDoKVMSG5VIDuGo3tKvCB+RbuoOM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1748556276159998.3063125008866; Thu, 29 May 2025 15:04:36 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uKlKu-0005b5-T4; Thu, 29 May 2025 18:02:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKt-0005aF-6T for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uKlKr-0002aj-DZ for qemu-devel@nongnu.org; Thu, 29 May 2025 18:02:50 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-214-29m3TvenMRCflAEIbsfPVw-1; Thu, 29 May 2025 18:02:46 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2B88E1800447; Thu, 29 May 2025 22:02:45 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.48]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E0BD619560A7; Thu, 29 May 2025 22:02:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1748556168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=trLlnZ7xN6wN3xsvXNehE4gJcK9/Aq7r+pfz0MLrc2g=; b=NnlLouhr8k8hrDnBu/CHfQAxVGhz2uXP1pMLSLRIFrNpVjTYQQWV3FL/Owudszn9LQoc64 MYcoWvRo+crzbX1px7oqS/lHqHMybNchytE1BXNGZKsU26eAhiTc/h6UOCMsUYcsWeOrV7 2WYZp0eAAPDtiuVd2soQpoVq8MCkJ7w= X-MC-Unique: 29m3TvenMRCflAEIbsfPVw-1 X-Mimecast-MFC-AGG-ID: 29m3TvenMRCflAEIbsfPVw_1748556165 From: Eric Blake To: qemu-devel@nongnu.org Cc: Fiona Ebner , Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org (open list:Block layer core) Subject: [PULL 6/6] iotests: Filter out ZFS in several tests Date: Thu, 29 May 2025 17:02:13 -0500 Message-ID: <20250529220228.1187563-14-eblake@redhat.com> In-Reply-To: <20250529220228.1187563-8-eblake@redhat.com> References: <20250529220228.1187563-8-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -49 X-Spam_score: -5.0 X-Spam_bar: ----- X-Spam_report: (-5.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.902, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1748556278480116600 Content-Type: text/plain; charset="utf-8" Fiona reported that ZFS makes sparse file testing awkward, since: - it has asynchronous allocation (not even 'fsync $file' makes du see the desired size; it takes the slower 'fsync -f $file' which is not appropriate for the tests) - for tests of fully allocated files, ZFS with compression enabled still reports smaller disk usage Add a new _require_disk_usage that quickly probes whether an attempt to create a sparse 5M file shows as less than 1M usage, while the same file with -o preallocation=3Dfull shows as more than 4M usage without sync, which should filter out ZFS behavior. Then use it in various affected tests. This does not add the new filter on all tests that Fiona is seeing ZFS failures on, but only those where I could quickly spot that there is at least one place where the test depends on the output of 'du -b' or 'stat -c %b'. Reported-by: Fiona Ebner Signed-off-by: Eric Blake Message-ID: <20250523163041.2548675-8-eblake@redhat.com> Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner --- tests/qemu-iotests/common.rc | 30 +++++++++++++++++++++ tests/qemu-iotests/106 | 1 + tests/qemu-iotests/175 | 1 + tests/qemu-iotests/221 | 1 + tests/qemu-iotests/253 | 1 + tests/qemu-iotests/308 | 1 + tests/qemu-iotests/tests/mirror-sparse | 1 + tests/qemu-iotests/tests/write-zeroes-unmap | 1 + 8 files changed, 37 insertions(+) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 237f746af88..e977cb4eb61 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -996,6 +996,36 @@ _require_large_file() rm "$FILENAME" } +# Check whether disk_usage can be reliably used. +_require_disk_usage() +{ + local unusable=3Dfalse + # ZFS triggers known failures on this front; it does not immediately + # allocate files, and then aggressively compresses writes even when fu= ll + # allocation was requested. + if [ -z "$TEST_IMG_FILE" ]; then + FILENAME=3D"$TEST_IMG" + else + FILENAME=3D"$TEST_IMG_FILE" + fi + if [ -e "FILENAME" ]; then + echo "unwilling to overwrite existing file" + exit 1 + fi + $QEMU_IMG create -f raw "$FILENAME" 5M > /dev/null + if [ $(disk_usage "$FILENAME") -gt $((1024*1024)) ]; then + unusable=3Dtrue + fi + $QEMU_IMG create -f raw -o preallocation=3Dfull "$FILENAME" 5M > /dev/= null + if [ $(disk_usage "$FILENAME") -lt $((4*1024*1024)) ]; then + unusable=3Dtrue + fi + rm -f "$FILENAME" + if $unusable; then + _notrun "file system on $TEST_DIR does not handle sparse files nic= ely" + fi +} + # Check that a set of devices is available in the QEMU binary # _require_devices() diff --git a/tests/qemu-iotests/106 b/tests/qemu-iotests/106 index ae0fc466910..55548439aad 100755 --- a/tests/qemu-iotests/106 +++ b/tests/qemu-iotests/106 @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt raw _supported_proto file fuse _supported_os Linux +_require_disk_usage # in kB CREATION_SIZE=3D128 diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175 index f74f053b719..bbbf550a5af 100755 --- a/tests/qemu-iotests/175 +++ b/tests/qemu-iotests/175 @@ -77,6 +77,7 @@ _supported_os Linux _default_cache_mode none _supported_cache_modes none directsync +_require_disk_usage size=3D$((1 * 1024 * 1024)) diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 index c463fd4b113..eba00b80adb 100755 --- a/tests/qemu-iotests/221 +++ b/tests/qemu-iotests/221 @@ -41,6 +41,7 @@ _supported_os Linux _default_cache_mode writeback _supported_cache_modes writeback writethrough unsafe +_require_disk_usage echo echo "=3D=3D=3D Check mapping of unaligned raw image =3D=3D=3D" diff --git a/tests/qemu-iotests/253 b/tests/qemu-iotests/253 index 35039d20a89..6da85e6a113 100755 --- a/tests/qemu-iotests/253 +++ b/tests/qemu-iotests/253 @@ -41,6 +41,7 @@ _supported_os Linux _default_cache_mode none _supported_cache_modes none directsync +_require_disk_usage echo echo "=3D=3D=3D Check mapping of unaligned raw image =3D=3D=3D" diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 index 437a9014da5..6eced3aefb9 100755 --- a/tests/qemu-iotests/308 +++ b/tests/qemu-iotests/308 @@ -51,6 +51,7 @@ _unsupported_fmt vpc _supported_proto file # We create the FUSE export manually _supported_os Linux # We need /dev/urandom +_require_disk_usage # $1: Export ID # $2: Options (beyond the node-name and ID) diff --git a/tests/qemu-iotests/tests/mirror-sparse b/tests/qemu-iotests/te= sts/mirror-sparse index 11418c08713..cfcaa600ab4 100755 --- a/tests/qemu-iotests/tests/mirror-sparse +++ b/tests/qemu-iotests/tests/mirror-sparse @@ -40,6 +40,7 @@ cd .. _supported_fmt qcow2 raw # Format of the source. dst is always raw file _supported_proto file _supported_os Linux +_require_disk_usage echo echo "=3D=3D=3D Initial image setup =3D=3D=3D" diff --git a/tests/qemu-iotests/tests/write-zeroes-unmap b/tests/qemu-iotes= ts/tests/write-zeroes-unmap index 7cfeeaf8391..f90fb8e8d27 100755 --- a/tests/qemu-iotests/tests/write-zeroes-unmap +++ b/tests/qemu-iotests/tests/write-zeroes-unmap @@ -32,6 +32,7 @@ cd .. _supported_fmt raw _supported_proto file _supported_os Linux +_require_disk_usage create_test_image() { _make_test_img -f $IMGFMT 1m --=20 2.49.0