From nobody Mon Feb 9 07:55:56 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1571916039; cv=none; d=zoho.com; s=zohoarc; b=kN9o5XkWJfIKBfTKHmHN61ndsEdFWg8/ObKGAWYjLC0fduoUWeLgYWrsi1qOaLW+MRpWP0wsk+4zXOdKyXnd/usUYX2YeHtfBjZJHOoq29EmmsU3bP7TT3qKS2GwG1nKjeN1jcoeLwza96N0h/tI/JaSxkIgFXO/wfj5CyHctw8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1571916039; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=AuHjnsJfixi1mdGb5rDbogFCok+K7oOPW4/ApU4ZAuc=; b=W2dE9O7vWRWeuUb7QCFlrTTTx+wsL3Xdc3k18wX26cbT2JWJrS00IBo21PE/33eHb31At7ZUyQJXIpCYJBbql24EsSfOSrHjFRy/i/gD/mAH8efM4kSlqm2ZU+ejw6FbU3ZObqO86cg+R2+0U6RnAjUw9r8KnIJugRuK/pDjHnE= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1571916039295386.7525361359143; Thu, 24 Oct 2019 04:20:39 -0700 (PDT) Received: from localhost ([::1]:36974 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iNZjm-0001GN-Eu for importer@patchew.org; Thu, 24 Oct 2019 05:48:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44163) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iNZ9t-0000da-Qi for qemu-devel@nongnu.org; Thu, 24 Oct 2019 05:11:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iNZ9r-0004x8-MG for qemu-devel@nongnu.org; Thu, 24 Oct 2019 05:11:53 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:46438) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iNZ9q-0004vN-Ph for qemu-devel@nongnu.org; Thu, 24 Oct 2019 05:11:51 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x9O94MeW116323; Thu, 24 Oct 2019 09:11:45 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2130.oracle.com with ESMTP id 2vqswtte3v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Oct 2019 09:11:44 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x9O982jm093151; Thu, 24 Oct 2019 09:11:44 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3030.oracle.com with ESMTP id 2vtm244kf2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Oct 2019 09:11:43 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x9O9Bef1009868; Thu, 24 Oct 2019 09:11:41 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 24 Oct 2019 02:11:40 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references; s=corp-2019-08-05; bh=AuHjnsJfixi1mdGb5rDbogFCok+K7oOPW4/ApU4ZAuc=; b=f+R//iSgIf0Uwjcz4UzFDrEfpuSMnQnvjZ8YDkKoiYdOuz1xFmYpVsaoFEH5rHzTHhyA S3LLO6+thIH6jPzSn9upuE9ZWVUUpEVmd2ipHseFdRaxdJ6UnY9sGRQ9+dqXsh0puZpj 1eJRwuN7xPk1AQXnx2Rf6S9H/ZDFmEcdb5Qwb36SLBSWvvW3O5Vj0MF5qsWrryaLZyCt 0hkvURjG2HjR1/8Ovmfj6uPIE0ny4voHxtk1nMZYjFXbc4uCuBqp4K07kHwZotBiAOpV OgxZT23Izx3xQbzN+tp2rBCqu+0rlwMM7PvthOW/5m/TjDptxArqVzrpnReMDJl8ruxr sQ== From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [RFC v4 PATCH 41/49] multi-process/mig: Enable VMSD save in the Proxy object Date: Thu, 24 Oct 2019 05:09:22 -0400 Message-Id: <8b15ab3d4fe51b792897ffc87e221bfb9317a836.1571905346.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9419 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910240089 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9419 signatures=668684 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1910240089 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 156.151.31.86 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, liran.alon@oracle.com, stefanha@redhat.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Collect the VMSD from remote process on the source and save it to the channel leading to the destination Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- New patch in v4 hw/proxy/qemu-proxy.c | 132 ++++++++++++++++++++++++++++++++++++++= ++++ include/hw/proxy/qemu-proxy.h | 2 + include/io/mpqemu-link.h | 1 + 3 files changed, 135 insertions(+) diff --git a/hw/proxy/qemu-proxy.c b/hw/proxy/qemu-proxy.c index 623a6c5..ce72e6a 100644 --- a/hw/proxy/qemu-proxy.c +++ b/hw/proxy/qemu-proxy.c @@ -52,6 +52,14 @@ #include "util/event_notifier-posix.c" #include "hw/boards.h" #include "include/qemu/log.h" +#include "io/channel.h" +#include "migration/qemu-file-types.h" +#include "qapi/error.h" +#include "io/channel-util.h" +#include "migration/qemu-file-channel.h" +#include "migration/qemu-file.h" +#include "migration/migration.h" +#include "migration/vmstate.h" =20 QEMUTimer *hb_timer; static void pci_proxy_dev_realize(PCIDevice *dev, Error **errp); @@ -62,6 +70,9 @@ static void stop_heartbeat_timer(void); static void childsig_handler(int sig, siginfo_t *siginfo, void *ctx); static void broadcast_msg(MPQemuMsg *msg, bool need_reply); =20 +#define PAGE_SIZE getpagesize() +uint8_t *mig_data; + static void childsig_handler(int sig, siginfo_t *siginfo, void *ctx) { /* TODO: Add proper handler. */ @@ -357,14 +368,135 @@ static void pci_proxy_dev_inst_init(Object *obj) dev->mem_init =3D false; } =20 +typedef struct { + QEMUFile *rem; + PCIProxyDev *dev; +} proxy_mig_data; + +static void *proxy_mig_out(void *opaque) +{ + proxy_mig_data *data =3D opaque; + PCIProxyDev *dev =3D data->dev; + uint8_t byte; + uint64_t data_size =3D PAGE_SIZE; + + mig_data =3D g_malloc(data_size); + + while (true) { + byte =3D qemu_get_byte(data->rem); + mig_data[dev->migsize++] =3D byte; + if (dev->migsize =3D=3D data_size) { + data_size +=3D PAGE_SIZE; + mig_data =3D g_realloc(mig_data, data_size); + } + } + + return NULL; +} + +static int proxy_pre_save(void *opaque) +{ + PCIProxyDev *pdev =3D opaque; + proxy_mig_data *mig_data; + QEMUFile *f_remote; + MPQemuMsg msg =3D {0}; + QemuThread thread; + Error *err =3D NULL; + QIOChannel *ioc; + uint64_t size; + int fd[2]; + + if (socketpair(AF_UNIX, SOCK_STREAM, 0, fd)) { + return -1; + } + + ioc =3D qio_channel_new_fd(fd[0], &err); + if (err) { + error_report_err(err); + return -1; + } + + qio_channel_set_name(QIO_CHANNEL(ioc), "PCIProxyDevice-mig"); + + f_remote =3D qemu_fopen_channel_input(ioc); + + pdev->migsize =3D 0; + + mig_data =3D g_malloc0(sizeof(proxy_mig_data)); + mig_data->rem =3D f_remote; + mig_data->dev =3D pdev; + + qemu_thread_create(&thread, "Proxy MIG_OUT", proxy_mig_out, mig_data, + QEMU_THREAD_DETACHED); + + msg.cmd =3D START_MIG_OUT; + msg.bytestream =3D 0; + msg.num_fds =3D 2; + msg.fds[0] =3D fd[1]; + msg.fds[1] =3D GET_REMOTE_WAIT; + + mpqemu_msg_send(pdev->mpqemu_link, &msg, pdev->mpqemu_link->com); + size =3D wait_for_remote(msg.fds[1]); + PUT_REMOTE_WAIT(msg.fds[1]); + + assert(size !=3D ULLONG_MAX); + + /* + * migsize is being update by a separate thread. Using volatile to + * instruct the compiler to fetch the value of this variable from + * memory during every read + */ + while (*((volatile uint64_t *)&pdev->migsize) < size) { + } + + qemu_thread_cancel(&thread); + + qemu_fclose(f_remote); + close(fd[1]); + + return 0; +} + +static int proxy_post_save(void *opaque) +{ + MigrationState *ms =3D migrate_get_current(); + PCIProxyDev *pdev =3D opaque; + uint64_t pos =3D 0; + + while (pos < pdev->migsize) { + qemu_put_byte(ms->to_dst_file, mig_data[pos]); + pos++; + } + + qemu_fflush(ms->to_dst_file); + + return 0; +} + +const VMStateDescription vmstate_pci_proxy_device =3D { + .name =3D "PCIProxyDevice", + .version_id =3D 2, + .minimum_version_id =3D 1, + .pre_save =3D proxy_pre_save, + .post_save =3D proxy_post_save, + .fields =3D (VMStateField[]) { + VMSTATE_PCI_DEVICE(parent_dev, PCIProxyDev), + VMSTATE_UINT64(migsize, PCIProxyDev), + VMSTATE_END_OF_LIST() + } +}; + static void pci_proxy_dev_class_init(ObjectClass *klass, void *data) { PCIDeviceClass *k =3D PCI_DEVICE_CLASS(klass); + DeviceClass *dc =3D DEVICE_CLASS(klass); =20 k->realize =3D pci_proxy_dev_realize; k->exit =3D pci_dev_exit; k->config_read =3D pci_proxy_read_config; k->config_write =3D pci_proxy_write_config; + + dc->vmsd =3D &vmstate_pci_proxy_device; } =20 static const TypeInfo pci_proxy_dev_type_info =3D { diff --git a/include/hw/proxy/qemu-proxy.h b/include/hw/proxy/qemu-proxy.h index 17e07ac..b122e6d 100644 --- a/include/hw/proxy/qemu-proxy.h +++ b/include/hw/proxy/qemu-proxy.h @@ -89,6 +89,8 @@ struct PCIProxyDev { void (*init_proxy) (PCIDevice *dev, char *command, bool need_spawn, Er= ror **errp); =20 ProxyMemoryRegion region[PCI_NUM_REGIONS]; + + uint64_t migsize; }; =20 typedef struct PCIProxyDevClass { diff --git a/include/io/mpqemu-link.h b/include/io/mpqemu-link.h index 6fcc6f5..0ed7750 100644 --- a/include/io/mpqemu-link.h +++ b/include/io/mpqemu-link.h @@ -75,6 +75,7 @@ typedef enum { PROXY_PING, MMIO_RETURN, DEVICE_RESET, + START_MIG_OUT, MAX, } mpqemu_cmd_t; =20 --=20 1.8.3.1