From nobody Tue Feb 10 10:54:08 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=suse.de Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1650655961414560.0874103242089; Fri, 22 Apr 2022 12:32:41 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-125-l6iys7d3OsS0Akt2Nivohg-1; Fri, 22 Apr 2022 15:32:36 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 04D9786B8A1; Fri, 22 Apr 2022 19:32:31 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4077565DD7; Fri, 22 Apr 2022 19:32:30 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id DF420194036F; Fri, 22 Apr 2022 19:32:29 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E74521940351 for ; Fri, 22 Apr 2022 19:32:28 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id CACC740F494C; Fri, 22 Apr 2022 19:32:28 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C695240F4948 for ; Fri, 22 Apr 2022 19:32:28 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AAD393C0FEA1 for ; Fri, 22 Apr 2022 19:32:28 +0000 (UTC) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-20-mYris8CgOgakwG1tXFd4-Q-1; Fri, 22 Apr 2022 15:32:27 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 3F0F921604; Fri, 22 Apr 2022 19:32:25 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id ECE8413AE1; Fri, 22 Apr 2022 19:32:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QBSXN8gCY2L8bAAAMHmgww (envelope-from ); Fri, 22 Apr 2022 19:32:24 +0000 X-MC-Unique: l6iys7d3OsS0Akt2Nivohg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: mYris8CgOgakwG1tXFd4-Q-1 From: Claudio Fontana To: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= Subject: [libvirt RFC v2 07/11] multifd-helper: new helper for parallel saves Date: Fri, 22 Apr 2022 21:32:13 +0200 Message-Id: <20220422193217.22513-8-cfontana@suse.de> In-Reply-To: <20220422193217.22513-1-cfontana@suse.de> References: <20220422193217.22513-1-cfontana@suse.de> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libvir-list@redhat.com, "Dr . David Alan Gilbert" , Claudio Fontana Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1650655961765100006 Content-Type: text/plain; charset="utf-8"; x-default="true" this helper listens on a unix socket which is provided to QEMU for multifd migration to/from files. The file descriptors are passed as command line parameters. Signed-off-by: Claudio Fontana --- src/libvirt_private.syms | 1 + src/util/meson.build | 13 +++ src/util/multifd-helper.c | 205 ++++++++++++++++++++++++++++++++++++++ src/util/virthread.c | 5 + src/util/virthread.h | 1 + 5 files changed, 225 insertions(+) create mode 100644 src/util/multifd-helper.c diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 8a3e5f7f7c..cd279b8713 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3427,6 +3427,7 @@ virThreadCreateFull; virThreadID; virThreadIsSelf; virThreadJoin; +virThreadJoinRet; virThreadMaxName; virThreadSelf; virThreadSelfID; diff --git a/src/util/meson.build b/src/util/meson.build index 58001a1699..8ea74ff9e8 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -179,6 +179,12 @@ io_helper_sources =3D [ 'runio.h', ] =20 +multifd_helper_sources =3D [ + 'multifd-helper.c', + 'runio.c', + 'runio.h', +] + virt_util_lib =3D static_library( 'virt_util', [ @@ -216,6 +222,13 @@ if conf.has('WITH_LIBVIRTD') dtrace_gen_headers, ], } + virt_helpers +=3D { + 'name': 'libvirt_multifd_helper', + 'sources': [ + files(multifd_helper_sources), + dtrace_gen_headers, + ], + } endif =20 util_inc_dir =3D include_directories('.') diff --git a/src/util/multifd-helper.c b/src/util/multifd-helper.c new file mode 100644 index 0000000000..fab28bdb79 --- /dev/null +++ b/src/util/multifd-helper.c @@ -0,0 +1,205 @@ +/* + * multifd-helper.c: listens on Unix socket to perform I/O to multiple fil= es + * + * Copyright (C) 2022 SUSE LLC + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * This has been written to support QEMU multifd migration to file, + * allowing better use of cpu resources to speed up the save/restore. + */ + +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "virthread.h" +#include "virfile.h" +#include "virerror.h" +#include "virstring.h" +#include "virgettext.h" +#include "runio.h" + +#define VIR_FROM_THIS VIR_FROM_STORAGE + +typedef struct _multiFdConnData multiFdConnData; +struct _multiFdConnData { + int clientfd; + int filefd; + int oflags; + const char *path; + virThread tid; + + off_t total; +}; + +typedef struct _multiFdThreadArgs multiFdThreadArgs; +struct _multiFdThreadArgs { + int nchannels; + multiFdConnData *conn; /* contains main fd + nchannels */ + const char *sun_path; /* unix socket name to use for the server */ + + off_t total; +}; + +static void clientThreadFunc(void *a) +{ + multiFdConnData *c =3D a; + c->total =3D runIO(c->path, c->filefd, c->oflags, c->clientfd, c->clie= ntfd); +} + +static void serverThreadFunc(void *a) +{ + struct sockaddr_un serv_addr; + multiFdThreadArgs *args =3D a; + int i; + int sockfd =3D socket(AF_UNIX, SOCK_STREAM, 0); + const char buf[1] =3D {'R'}; + + args->total =3D -1; + + if (sockfd < 0) { + virReportSystemError(errno, "%s", _("serverThread: socket() failed= ")); + return; + } + /* initialize socket structure */ + memset(&serv_addr, 0, sizeof(serv_addr)); + serv_addr.sun_family =3D AF_UNIX; + strncpy(serv_addr.sun_path, args->sun_path, sizeof(serv_addr.sun_path)= - 1); + unlink(args->sun_path); + if (bind(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0= ) { + virReportSystemError(errno, "%s", _("serverThread: bind() failed")= ); + goto cleanup; + } + if (listen(sockfd, args->nchannels + 1) < 0) { + virReportSystemError(errno, "%s", _("serverThread: listen() failed= ")); + goto cleanup; + } + + /* signal that the server is ready */ + if (safewrite(1, &buf, 1) !=3D 1) { + virReportSystemError(errno, "%s", _("serverThread: safewrite faile= d")); + goto cleanup; + } + + for (i =3D 0; i < args->nchannels + 1; i++) { + /* Wait for incoming connection. */ + multiFdConnData *c =3D &args->conn[i]; + if ((c->clientfd =3D accept(sockfd, NULL, NULL)) < 0) { + virReportSystemError(errno, "%s", _("serverThread: accept() fa= iled")); + goto cleanup; + } + if (virThreadCreate(&c->tid, true, &clientThreadFunc, c) < 0) { + virReportSystemError(errno, "%s", _("serverThread: client thre= ad creation failed")); + goto cleanup; + } + } + + args->total =3D 0; + for (i =3D 0; i < args->nchannels + 1; i++) { + multiFdConnData *c =3D &args->conn[i]; + if (virThreadJoinRet(&args->conn[i].tid) < 0) { + args->total =3D -1; + } else if (args->total >=3D 0) { + args->total +=3D c->total; + } + if (VIR_CLOSE(c->clientfd) < 0) { + args->total =3D -1; + } + } + + cleanup: + for (i =3D 0; i < args->nchannels + 1; i++) { + multiFdConnData *c =3D &args->conn[i]; + VIR_FORCE_CLOSE(c->clientfd); + } + if (VIR_CLOSE(sockfd) < 0) + args->total =3D -1; +} + +static const char *program_name; + +G_GNUC_NORETURN static void +usage(int status) +{ + if (status) { + fprintf(stderr, _("%s: try --help for more details"), program_name= ); + } else { + fprintf(stderr, _("Usage: %s UNIX_SOCNAME N MAINFD FD0 FD1 ... FDn= "), program_name); + } + exit(status); +} + +int +main(int argc, char **argv) +{ + virThread tid; + multiFdThreadArgs args =3D { 0 }; + int i; + + program_name =3D argv[0]; + + if (virGettextInitialize() < 0 || + virErrorInitialize() < 0) { + fprintf(stderr, _("%s: initialization failed"), program_name); + exit(EXIT_FAILURE); + } + + if (argc > 1 && STREQ(argv[1], "--help")) + usage(EXIT_SUCCESS); + if (argc < 4) + usage(EXIT_FAILURE); + + args.sun_path =3D argv[1]; + if (virStrToLong_i(argv[2], NULL, 10, &args.nchannels) < 0) + fprintf(stderr, _("%s: malformed number of channels N %s"), progra= m_name, argv[2]); + + if (argc < 4 + args.nchannels) + usage(EXIT_FAILURE); + + args.conn =3D g_new0(multiFdConnData, args.nchannels + 1); + + for (i =3D 3; i < 3 + args.nchannels + 1; i++) { + multiFdConnData *c =3D &args.conn[i - 3]; + + if (virStrToLong_i(argv[i], NULL, 10, &c->filefd) < 0) { + fprintf(stderr, _("%s: malformed FD %s"), program_name, argv[i= ]); + usage(EXIT_FAILURE); + } +#ifndef F_GETFL +#error "multifd-helper requires F_GETFL parameter of fcntl" +#endif + c->oflags =3D fcntl(c->filefd, F_GETFL); + } + + if (virThreadCreate(&tid, true, serverThreadFunc, &args) < 0) { + virReportSystemError(errno, _("%s: failed to create server thread"= ), program_name); + exit(EXIT_FAILURE); + } + + if (virThreadJoinRet(&tid) < 0) + exit(EXIT_FAILURE); + + if (args.total < 0) + exit(EXIT_FAILURE); + + exit(EXIT_SUCCESS); +} diff --git a/src/util/virthread.c b/src/util/virthread.c index 5422bb74fd..0f6c6a68fa 100644 --- a/src/util/virthread.c +++ b/src/util/virthread.c @@ -348,6 +348,11 @@ void virThreadJoin(virThread *thread) pthread_join(thread->thread, NULL); } =20 +int virThreadJoinRet(virThread *thread) +{ + return pthread_join(thread->thread, NULL); +} + void virThreadCancel(virThread *thread) { pthread_cancel(thread->thread); diff --git a/src/util/virthread.h b/src/util/virthread.h index 23abe0b6c9..5cecb9bd8a 100644 --- a/src/util/virthread.h +++ b/src/util/virthread.h @@ -89,6 +89,7 @@ int virThreadCreateFull(virThread *thread, void virThreadSelf(virThread *thread); bool virThreadIsSelf(virThread *thread); void virThreadJoin(virThread *thread); +int virThreadJoinRet(virThread *thread); =20 size_t virThreadMaxName(void); =20 --=20 2.34.1