From nobody Wed Nov 5 00:35:24 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1500711796465256.83829670621344; Sat, 22 Jul 2017 01:23:16 -0700 (PDT) Received: from localhost ([::1]:46106 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYpgt-0002xc-7D for importer@patchew.org; Sat, 22 Jul 2017 04:23:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52832) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYpg4-0002bS-1C for qemu-devel@nongnu.org; Sat, 22 Jul 2017 04:22:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dYpg0-0005RC-R2 for qemu-devel@nongnu.org; Sat, 22 Jul 2017 04:22:20 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:26927) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dYpg0-0005Qz-Dc for qemu-devel@nongnu.org; Sat, 22 Jul 2017 04:22:16 -0400 Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v6M8MDJv027215 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 08:22:13 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id v6M8MDRc029736 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 08:22:13 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v6M8MCWG004984; Sat, 22 Jul 2017 08:22:12 GMT Received: from abi.no.oracle.com (/10.172.144.123) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sat, 22 Jul 2017 00:49:47 -0700 From: Knut Omang To: "Daniel P . Berrange" , Gerd Hoffmann , Paolo Bonzini Date: Sat, 22 Jul 2017 09:49:30 +0200 Message-Id: <0487fb49fe3e3f3bb0f74517607e0ccb74258753.1500709594.git-series.knut.omang@oracle.com> X-Mailer: git-send-email 2.9.4 In-Reply-To: References: X-Source-IP: userv0021.oracle.com [156.151.31.71] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 156.151.31.81 Subject: [Qemu-devel] [PATCH v5 1/4] tests: Add test-listen - a stress test for QEMU socket listen X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Knut Omang , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There's a potential race condition between multiple bind()'s attempting to bind to the same port, which occasionally allows more than one bind to succeed against the same port. When a subsequent listen() call is made with the same socket only one will succeed. The current QEMU code does however not take this situation into account and the listen will cause the code to break out and fail even when there are actually available ports to use. This test exposes two subtests: /socket/listen-serial /socket/listen-compete The "compete" subtest creates a number of threads and have them all trying = to bind to the same port with a large enough offset input to allow all threads to get it's own port. The "serial" subtest just does the same, except in series in a single thread. The serial version passes, probably in most versions of QEMU. The parallel version exposes the problem in a relatively reliable way, eg. it fails a majority of times, but not with a 100% rate, occasional passes can be seen. Nevertheless this is quite good given that the bug was tricky to reproduce and has been left undetected for a while. The problem seems to be present in all versions of QEMU. The original failure scenario occurred with VNC port allocation in a traditional Xen based build, in different code but with similar functionality. Reported-by: Bhavesh Davda Signed-off-by: Knut Omang Reviewed-by: Yuval Shaia Reviewed-by: Bhavesh Davda Reviewed-by: Girish Moodalbail --- tests/Makefile.include | 2 +- tests/test-listen.c | 253 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 255 insertions(+) create mode 100644 tests/test-listen.c diff --git a/tests/Makefile.include b/tests/Makefile.include index 7af278d..b37c0c8 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -128,6 +128,7 @@ check-unit-y +=3D tests/test-bufferiszero$(EXESUF) gcov-files-check-bufferiszero-y =3D util/bufferiszero.c check-unit-y +=3D tests/test-uuid$(EXESUF) check-unit-y +=3D tests/ptimer-test$(EXESUF) +#check-unit-y +=3D tests/test-listen$(EXESUF) gcov-files-ptimer-test-y =3D hw/core/ptimer.c check-unit-y +=3D tests/test-qapi-util$(EXESUF) gcov-files-test-qapi-util-y =3D qapi/qapi-util.c @@ -769,6 +770,7 @@ tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer= .o tests/test-qapi-util$(EXESUF): tests/test-qapi-util.o $(test-util-obj-y) tests/numa-test$(EXESUF): tests/numa-test.o tests/vmgenid-test$(EXESUF): tests/vmgenid-test.o tests/boot-sector.o test= s/acpi-utils.o +tests/test-listen$(EXESUF): tests/test-listen.o $(test-util-obj-y) =20 tests/migration/stress$(EXESUF): tests/migration/stress.o $(call quiet-command, $(LINKPROG) -static -O3 $(PTHREAD_LIB) -o $@ $< ,"L= INK","$(TARGET_DIR)$@") diff --git a/tests/test-listen.c b/tests/test-listen.c new file mode 100644 index 0000000..5c07537 --- /dev/null +++ b/tests/test-listen.c @@ -0,0 +1,253 @@ +/* + * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. + * Author: Knut Omang + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 + * as published by the Free Software Foundation. + * + * Test parallel port listen configuration with + * dynamic port allocation + */ + +#include "qemu/osdep.h" +#include "libqtest.h" +#include "qemu-common.h" +#include "qemu/thread.h" +#include "qemu/sockets.h" +#include "qapi/error.h" + +#define NAME_LEN 1024 +#define PORT_LEN 16 + +struct thr_info { + QemuThread thread; + int to_port; + bool ipv4; + bool ipv6; + int got_port; + int eno; + int fd; + const char *errstr; + char hostname[NAME_LEN + 1]; + char port[PORT_LEN + 1]; +}; + + +/* These two functions taken from test-io-channel-socket.c */ +static int check_bind(const char *hostname, bool *has_proto) +{ + int fd =3D -1; + struct addrinfo ai, *res =3D NULL; + int rc; + int ret =3D -1; + + memset(&ai, 0, sizeof(ai)); + ai.ai_flags =3D AI_CANONNAME | AI_ADDRCONFIG; + ai.ai_family =3D AF_UNSPEC; + ai.ai_socktype =3D SOCK_STREAM; + + /* lookup */ + rc =3D getaddrinfo(hostname, NULL, &ai, &res); + if (rc !=3D 0) { + if (rc =3D=3D EAI_ADDRFAMILY || + rc =3D=3D EAI_FAMILY) { + *has_proto =3D false; + goto done; + } + goto cleanup; + } + + fd =3D qemu_socket(res->ai_family, res->ai_socktype, res->ai_protocol); + if (fd < 0) { + goto cleanup; + } + + if (bind(fd, res->ai_addr, res->ai_addrlen) < 0) { + if (errno =3D=3D EADDRNOTAVAIL) { + *has_proto =3D false; + goto done; + } + goto cleanup; + } + + *has_proto =3D true; + done: + ret =3D 0; + + cleanup: + if (fd !=3D -1) { + close(fd); + } + if (res) { + freeaddrinfo(res); + } + return ret; +} + +static int check_protocol_support(bool *has_ipv4, bool *has_ipv6) +{ + if (check_bind("127.0.0.1", has_ipv4) < 0) { + return -1; + } + if (check_bind("::1", has_ipv6) < 0) { + return -1; + } + + return 0; +} + +static void *listener_thread(void *arg) +{ + struct thr_info *thr =3D (struct thr_info *)arg; + SocketAddress addr =3D { + .type =3D SOCKET_ADDRESS_TYPE_INET, + .u =3D { + .inet =3D { + .host =3D thr->hostname, + .port =3D thr->port, + .has_ipv4 =3D thr->ipv4, + .ipv4 =3D thr->ipv4, + .has_ipv6 =3D thr->ipv6, + .ipv6 =3D thr->ipv6, + .has_to =3D true, + .to =3D thr->to_port, + }, + }, + }; + Error *err =3D NULL; + int fd; + + fd =3D socket_listen(&addr, &err); + if (fd < 0) { + thr->eno =3D errno; + thr->errstr =3D error_get_pretty(err); + } else { + struct sockaddr_in a; + socklen_t a_len =3D sizeof(a); + g_assert_cmpint(getsockname(fd, (struct sockaddr *)&a, &a_len), = =3D=3D, 0); + thr->got_port =3D ntohs(a.sin_port); + thr->fd =3D fd; + } + return arg; +} + + +static void listen_compete_nthr(bool threaded, int nthreads, + int start_port, int max_offset, + bool ipv4, bool ipv6) +{ + int i; + int failed_listens =3D 0; + struct thr_info *thr =3D g_new0(struct thr_info, nthreads); + int used[max_offset + 1]; + + memset(used, 0, sizeof(used)); + for (i =3D 0; i < nthreads; i++) { + snprintf(thr[i].port, PORT_LEN, "%d", start_port); + strcpy(thr[i].hostname, "localhost"); + thr[i].to_port =3D start_port + max_offset; + thr[i].ipv4 =3D ipv4; + thr[i].ipv6 =3D ipv6; + } + + for (i =3D 0; i < nthreads; i++) { + if (threaded) { + qemu_thread_create(&thr[i].thread, "listener", + listener_thread, &thr[i], + QEMU_THREAD_JOINABLE); + } else { + listener_thread(&thr[i]); + } + } + + if (threaded) { + for (i =3D 0; i < nthreads; i++) { + qemu_thread_join(&thr[i].thread); + } + } + for (i =3D 0; i < nthreads; i++) { + if (thr[i].got_port) { + closesocket(thr[i].fd); + } + } + + for (i =3D 0; i < nthreads; i++) { + if (thr[i].eno !=3D 0) { + const char *m; + g_printerr("** Failed to assign a port to thread %d (errno =3D= %d)\n", + i, thr[i].eno); + /* This is what we are interested in capturing - + * catch and report details if something unexpected happens: + */ + m =3D strstr(thr[i].errstr, "Failed to listen on socket"); + if (m !=3D NULL) { + g_assert_cmpstr(thr[i].errstr, =3D=3D, + "Failed to listen on socket: Address already in use"); + } + failed_listens++; + } else { + int assigned_port =3D thr[i].got_port; + g_assert_cmpint(assigned_port, <=3D , thr[i].to_port); + g_assert_cmpint(used[assigned_port - start_port], =3D=3D , 0); + } + } + g_assert_cmpint(failed_listens, =3D=3D, 0); + g_free(thr); +} + + +static void listen_compete_ipv4(void) +{ + listen_compete_nthr(true, 200, 5920, 300, true, false); +} + +static void listen_serial_ipv4(void) +{ + listen_compete_nthr(false, 200, 6300, 300, true, false); +} + +static void listen_compete_ipv6(void) +{ + listen_compete_nthr(true, 200, 5920, 300, true, false); +} + +static void listen_serial_ipv6(void) +{ + listen_compete_nthr(false, 200, 6300, 300, false, true); +} + +static void listen_compete_gen(void) +{ + listen_compete_nthr(true, 200, 5920, 300, true, true); +} + +static void listen_serial_gen(void) +{ + listen_compete_nthr(false, 200, 6300, 300, true, true); +} + + +int main(int argc, char **argv) +{ + bool has_ipv4, has_ipv6; + g_test_init(&argc, &argv, NULL); + + if (check_protocol_support(&has_ipv4, &has_ipv6) < 0) { + return 1; + } + + if (has_ipv4) { + g_test_add_func("/socket/listen-serial/ipv4", listen_serial_ipv4); + g_test_add_func("/socket/listen-compete/ipv4", listen_compete_ipv4= ); + } + if (has_ipv6) { + g_test_add_func("/socket/listen-serial/ipv6", listen_serial_ipv6); + g_test_add_func("/socket/listen-compete/ipv6", listen_compete_ipv6= ); + } + if (has_ipv4 && has_ipv6) { + g_test_add_func("/socket/listen-serial/generic", listen_serial_gen= ); + g_test_add_func("/socket/listen-compete/generic", listen_compete_g= en); + } + return g_test_run(); +} --=20 git-series 0.9.1 From nobody Wed Nov 5 00:35:24 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1500709882550898.4185643042665; Sat, 22 Jul 2017 00:51:22 -0700 (PDT) Received: from localhost ([::1]:46036 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYpC2-0005ZX-TW for importer@patchew.org; Sat, 22 Jul 2017 03:51:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48791) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYpAh-0004we-DE for qemu-devel@nongnu.org; Sat, 22 Jul 2017 03:49:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dYpAe-0001oM-AG for qemu-devel@nongnu.org; Sat, 22 Jul 2017 03:49:55 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:17904) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dYpAe-0001o2-0L for qemu-devel@nongnu.org; Sat, 22 Jul 2017 03:49:52 -0400 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v6M7nouo004350 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 07:49:50 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v6M7nnF2015012 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 07:49:49 GMT Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id v6M7nnu2018405; Sat, 22 Jul 2017 07:49:49 GMT Received: from abi.no.oracle.com (/10.172.144.123) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sat, 22 Jul 2017 00:49:49 -0700 From: Knut Omang To: "Daniel P . Berrange" , Gerd Hoffmann , Paolo Bonzini Date: Sat, 22 Jul 2017 09:49:31 +0200 Message-Id: X-Mailer: git-send-email 2.9.4 In-Reply-To: References: X-Source-IP: aserv0022.oracle.com [141.146.126.234] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 156.151.31.81 Subject: [Qemu-devel] [PATCH v5 2/4] sockets: factor out create_fast_reuse_socket X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Knut Omang , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" First refactoring step to prepare for fixing the problem exposed with the test-listen test in the previous commit Signed-off-by: Knut Omang --- util/qemu-sockets.c | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c index 1358c81..578f25b 100644 --- a/util/qemu-sockets.c +++ b/util/qemu-sockets.c @@ -149,6 +149,20 @@ int inet_ai_family_from_address(InetSocketAddress *add= r, return PF_UNSPEC; } =20 +static int create_fast_reuse_socket(struct addrinfo *e, Error **errp) +{ + int slisten =3D qemu_socket(e->ai_family, e->ai_socktype, e->ai_protoc= ol); + if (slisten < 0) { + if (!e->ai_next) { + error_setg_errno(errp, errno, "Failed to create socket"); + } + return -1; + } + + socket_set_fast_reuse(slisten); + return slisten; +} + static int inet_listen_saddr(InetSocketAddress *saddr, int port_offset, bool update_addr, @@ -210,21 +224,17 @@ static int inet_listen_saddr(InetSocketAddress *saddr, return -1; } =20 - /* create socket + bind */ + /* create socket + bind/listen */ for (e =3D res; e !=3D NULL; e =3D e->ai_next) { getnameinfo((struct sockaddr*)e->ai_addr,e->ai_addrlen, uaddr,INET6_ADDRSTRLEN,uport,32, NI_NUMERICHOST | NI_NUMERICSERV); - slisten =3D qemu_socket(e->ai_family, e->ai_socktype, e->ai_protoc= ol); + + slisten =3D create_fast_reuse_socket(e, &err); if (slisten < 0) { - if (!e->ai_next) { - error_setg_errno(errp, errno, "Failed to create socket"); - } continue; } =20 - socket_set_fast_reuse(slisten); - port_min =3D inet_getport(e); port_max =3D saddr->has_to ? saddr->to + port_offset : port_min; for (p =3D port_min; p <=3D port_max; p++) { --=20 git-series 0.9.1 From nobody Wed Nov 5 00:35:24 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1500709974246635.2272221605953; Sat, 22 Jul 2017 00:52:54 -0700 (PDT) Received: from localhost ([::1]:46040 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYpDY-0006QZ-T1 for importer@patchew.org; Sat, 22 Jul 2017 03:52:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48803) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYpAj-0004ws-7h for qemu-devel@nongnu.org; Sat, 22 Jul 2017 03:49:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dYpAg-0001of-59 for qemu-devel@nongnu.org; Sat, 22 Jul 2017 03:49:57 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:39416) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dYpAf-0001oW-Qt for qemu-devel@nongnu.org; Sat, 22 Jul 2017 03:49:54 -0400 Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v6M7np7o026609 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 07:49:51 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v6M7npv6012711 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 07:49:51 GMT Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v6M7noq3021153; Sat, 22 Jul 2017 07:49:51 GMT Received: from abi.no.oracle.com (/10.172.144.123) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sat, 22 Jul 2017 00:49:50 -0700 From: Knut Omang To: "Daniel P . Berrange" , Gerd Hoffmann , Paolo Bonzini Date: Sat, 22 Jul 2017 09:49:32 +0200 Message-Id: X-Mailer: git-send-email 2.9.4 In-Reply-To: References: X-Source-IP: userv0022.oracle.com [156.151.31.74] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 141.146.126.69 Subject: [Qemu-devel] [PATCH v5 3/4] sockets: factor out a new try_bind() function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Knut Omang , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Another refactoring step to prepare for the problem exposed by the test-listen test. This time simplify and reorganize the IPv6 specific extra measures and move it out of the for loop to increase code readability. No semantic changes. Signed-off-by: Knut Omang Reviewed-by: Daniel P. Berrange --- util/qemu-sockets.c | 69 ++++++++++++++++++++++++++-------------------- 1 file changed, 39 insertions(+), 30 deletions(-) diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c index 578f25b..39f207c 100644 --- a/util/qemu-sockets.c +++ b/util/qemu-sockets.c @@ -163,6 +163,44 @@ static int create_fast_reuse_socket(struct addrinfo *e= , Error **errp) return slisten; } =20 +static int try_bind(int socket, InetSocketAddress *saddr, struct addrinfo = *e) +{ +#ifndef IPV6_V6ONLY + return bind(socket, e->ai_addr, e->ai_addrlen); +#else + /* + * Deals with first & last cases in matrix in comment + * for inet_ai_family_from_address(). + */ + int v6only =3D + ((!saddr->has_ipv4 && !saddr->has_ipv6) || + (saddr->has_ipv4 && saddr->ipv4 && + saddr->has_ipv6 && saddr->ipv6)) ? 0 : 1; + int stat; + + rebind: + if (e->ai_family =3D=3D PF_INET6) { + qemu_setsockopt(socket, IPPROTO_IPV6, IPV6_V6ONLY, &v6only, + sizeof(v6only)); + } + + stat =3D bind(socket, e->ai_addr, e->ai_addrlen); + if (!stat) { + return 0; + } + + /* If we got EADDRINUSE from an IPv6 bind & v6only is unset, + * it could be that the IPv4 port is already claimed, so retry + * with v6only set + */ + if (e->ai_family =3D=3D PF_INET6 && errno =3D=3D EADDRINUSE && !v6only= ) { + v6only =3D 1; + goto rebind; + } + return stat; +#endif +} + static int inet_listen_saddr(InetSocketAddress *saddr, int port_offset, bool update_addr, @@ -238,39 +276,10 @@ static int inet_listen_saddr(InetSocketAddress *saddr, port_min =3D inet_getport(e); port_max =3D saddr->has_to ? saddr->to + port_offset : port_min; for (p =3D port_min; p <=3D port_max; p++) { -#ifdef IPV6_V6ONLY - /* - * Deals with first & last cases in matrix in comment - * for inet_ai_family_from_address(). - */ - int v6only =3D - ((!saddr->has_ipv4 && !saddr->has_ipv6) || - (saddr->has_ipv4 && saddr->ipv4 && - saddr->has_ipv6 && saddr->ipv6)) ? 0 : 1; -#endif inet_setport(e, p); -#ifdef IPV6_V6ONLY - rebind: - if (e->ai_family =3D=3D PF_INET6) { - qemu_setsockopt(slisten, IPPROTO_IPV6, IPV6_V6ONLY, &v6onl= y, - sizeof(v6only)); - } -#endif - if (bind(slisten, e->ai_addr, e->ai_addrlen) =3D=3D 0) { + if (try_bind(slisten, saddr, e) >=3D 0) { goto listen; } - -#ifdef IPV6_V6ONLY - /* If we got EADDRINUSE from an IPv6 bind & V6ONLY is unset, - * it could be that the IPv4 port is already claimed, so retry - * with V6ONLY set - */ - if (e->ai_family =3D=3D PF_INET6 && errno =3D=3D EADDRINUSE &&= !v6only) { - v6only =3D 1; - goto rebind; - } -#endif - if (p =3D=3D port_max) { if (!e->ai_next) { error_setg_errno(errp, errno, "Failed to bind socket"); --=20 git-series 0.9.1 From nobody Wed Nov 5 00:35:24 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1500715999473696.6444691009133; Sat, 22 Jul 2017 02:33:19 -0700 (PDT) Received: from localhost ([::1]:46263 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYqmj-0008Mi-Ay for importer@patchew.org; Sat, 22 Jul 2017 05:33:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37127) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dYqlv-00083o-9q for qemu-devel@nongnu.org; Sat, 22 Jul 2017 05:32:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dYqls-0005Uk-2T for qemu-devel@nongnu.org; Sat, 22 Jul 2017 05:32:27 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:42807) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dYqlr-0005Ug-OY for qemu-devel@nongnu.org; Sat, 22 Jul 2017 05:32:24 -0400 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v6M9WLX0020899 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 09:32:21 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v6M9WL9l000988 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 22 Jul 2017 09:32:21 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id v6M9WKIG006935; Sat, 22 Jul 2017 09:32:20 GMT Received: from abi.no.oracle.com (/10.172.144.123) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sat, 22 Jul 2017 00:49:51 -0700 From: Knut Omang To: "Daniel P . Berrange" , Gerd Hoffmann , Paolo Bonzini Date: Sat, 22 Jul 2017 09:49:33 +0200 Message-Id: X-Mailer: git-send-email 2.9.4 In-Reply-To: References: X-Source-IP: aserv0022.oracle.com [141.146.126.234] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 141.146.126.69 Subject: [Qemu-devel] [PATCH v5 4/4] sockets: Handle race condition between binds to the same port X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Knut Omang , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If an offset of ports is specified to the inet_listen_saddr function(), and two or more processes tries to bind from these ports at the same time, occasionally more than one process may be able to bind to the same port. The condition is detected by listen() but too late to avoid a failure. This function is called by socket_listen() and used by all socket listening code in QEMU, so all cases where any form of dynamic port selection is used should be subject to this issue. Add code to close and re-establish the socket when this condition is observed, hiding the race condition from the user. Also clean up some issues with error handling to allow more accurate reporting of the cause of an error. This has been developed and tested by means of the test-listen unit test in the previous commit. Enable the test for make check now that it passes. Signed-off-by: Knut Omang Reviewed-by: Bhavesh Davda Reviewed-by: Yuval Shaia Reviewed-by: Girish Moodalbail Signed-off-by: Knut Omang --- tests/Makefile.include | 2 +- util/qemu-sockets.c | 51 ++++++++++++++++++++++++++++++++----------- 2 files changed, 39 insertions(+), 14 deletions(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index b37c0c8..9d2131d 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -128,7 +128,7 @@ check-unit-y +=3D tests/test-bufferiszero$(EXESUF) gcov-files-check-bufferiszero-y =3D util/bufferiszero.c check-unit-y +=3D tests/test-uuid$(EXESUF) check-unit-y +=3D tests/ptimer-test$(EXESUF) -#check-unit-y +=3D tests/test-listen$(EXESUF) +check-unit-y +=3D tests/test-listen$(EXESUF) gcov-files-ptimer-test-y =3D hw/core/ptimer.c check-unit-y +=3D tests/test-qapi-util$(EXESUF) gcov-files-test-qapi-util-y =3D qapi/qapi-util.c diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c index 39f207c..8ca3ba6 100644 --- a/util/qemu-sockets.c +++ b/util/qemu-sockets.c @@ -210,7 +210,9 @@ static int inet_listen_saddr(InetSocketAddress *saddr, char port[33]; char uaddr[INET6_ADDRSTRLEN+1]; char uport[33]; - int slisten, rc, port_min, port_max, p; + int rc, port_min, port_max, p; + int slisten =3D 0; + int saved_errno =3D 0; Error *err =3D NULL; =20 memset(&ai,0, sizeof(ai)); @@ -277,27 +279,50 @@ static int inet_listen_saddr(InetSocketAddress *saddr, port_max =3D saddr->has_to ? saddr->to + port_offset : port_min; for (p =3D port_min; p <=3D port_max; p++) { inet_setport(e, p); - if (try_bind(slisten, saddr, e) >=3D 0) { - goto listen; - } - if (p =3D=3D port_max) { - if (!e->ai_next) { + rc =3D try_bind(slisten, saddr, e); + if (rc) { + if (errno =3D=3D EADDRINUSE) { + continue; + } else { error_setg_errno(errp, errno, "Failed to bind socket"); + goto listen_failed; + } + } + rc =3D listen(slisten, 1); + if (!rc) { + goto listen_ok; + } else if (errno =3D=3D EADDRINUSE) { + /* Someone else managed to bind to the same port and beat = us + * to listen on it! Socket semantics does not allow us to + * recover from this situation, so we need to recreate the + * socket to allow bind attempts for subsequent ports: + */ + closesocket(slisten); + slisten =3D create_fast_reuse_socket(e, errp); + if (slisten >=3D 0) { + continue; } } + error_setg_errno(errp, errno, "Failed to listen on socket"); + goto listen_failed; } + } + if (err) { + error_propagate(errp, err); + } else { + error_setg_errno(errp, errno, "Failed to find an available port"); + } + +listen_failed: + saved_errno =3D errno; + if (slisten >=3D 0) { closesocket(slisten); } freeaddrinfo(res); + errno =3D saved_errno; return -1; =20 -listen: - if (listen(slisten,1) !=3D 0) { - error_setg_errno(errp, errno, "Failed to listen on socket"); - closesocket(slisten); - freeaddrinfo(res); - return -1; - } +listen_ok: if (update_addr) { g_free(saddr->host); saddr->host =3D g_strdup(uaddr); --=20 git-series 0.9.1