From nobody Sat Nov 15 03:14:59 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1755985798; cv=none; d=zohomail.com; s=zohoarc; b=EJnIfwLTE4GvyWeN1c68plQiiyc1sBdAlVgd7J1k4oTzriWGCBWPcJ7hgaUXHoDeb5mdHhxzOWEeQ32S5XWeIX6r4mzYXqlYWvpr1/QyfQyrA7R3pIgGNmxm95amQSNNt8FadnWlC9AHr9jjdbtHHN7rZRtT2J5NBW21krD5DD4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1755985798; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=acsqFSAj46G2LvtYQhzASMfYnVuSw7HwJqnKLXPYy+M=; b=BQ8U1IbBTkPY5rmy+qpHnTUFkwi05TJ0HoQV+uJ71GmZl1SdUKESGqB9nozyUST2Haccbhlw5KaQ38bvcWwfrbVnolXdUebvkyw/SFYKtxd2KV3r83AxFX8EgVaA9/muvRdKYibDH9CeqUv0rcRrDuCAljkCVRozdBIKYdkD1ls= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1755985798230454.49820483249323; Sat, 23 Aug 2025 14:49:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1upw6F-0004Va-0V; Sat, 23 Aug 2025 17:48:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6C-0004TW-9e for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from forwardcorp1b.mail.yandex.net ([2a02:6b8:c02:900:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw69-00053u-M2 for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net [IPv6:2a02:6b8:c21:2d8b:0:640:7d49:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id DB75D8073D; Sun, 24 Aug 2025 00:48:25 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6bf:8080:a5d::1:36]) by mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Nmd3dr0GuSw0-MUT1PbsE; Sun, 24 Aug 2025 00:48:25 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1755985705; bh=acsqFSAj46G2LvtYQhzASMfYnVuSw7HwJqnKLXPYy+M=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=I03aEz/5fno4SuFV1+JVIKWJQgkl1TGBjkMsrs9a6RN94upbB6l9oRnfzyGWktogF vzfkqxHseefAvve3EhWB4iqI6hz+K1gkc+ENhCHBq+6mFrDtxcL8I9fzWOcw0HrBIP saDdDF132rBSMi5zBX1lxKeEKs/tdxlhYUVbzBno= Authentication-Results: mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: jasowang@redhat.com Cc: qemu-devel@nongnu.org, vsementsov@yandex-team.ru, leiyang@redhat.com, steven.sistare@oracle.com, peterx@redhat.com, mst@redhat.com, farosas@suse.de, yc-core@yandex-team.ru Subject: [RFC 1/7] net/tap: add some trace points Date: Sun, 24 Aug 2025 00:48:15 +0300 Message-ID: <20250823214821.48342-2-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250823214821.48342-1-vsementsov@yandex-team.ru> References: <20250823214821.48342-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c02:900:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1755985800380116600 Content-Type: text/plain; charset="utf-8" Add trace points to simplify debugging migration. Signed-off-by: Vladimir Sementsov-Ogievskiy --- net/tap.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++ net/trace-events | 7 ++++++ 2 files changed, 65 insertions(+) diff --git a/net/tap.c b/net/tap.c index 45d1bcd326..44bd6eefd8 100644 --- a/net/tap.c +++ b/net/tap.c @@ -43,6 +43,7 @@ #include "qemu/main-loop.h" #include "qemu/sockets.h" #include "hw/virtio/vhost.h" +#include "trace.h" =20 #include "net/tap.h" =20 @@ -148,6 +149,9 @@ static ssize_t tap_receive_iov(NetClientState *nc, cons= t struct iovec *iov, g_autofree struct iovec *iov_copy =3D NULL; struct virtio_net_hdr hdr =3D { }; =20 + trace_tap_receive_iov(s->using_vnet_hdr, s->host_vnet_hdr_len, iovcnt, + iov->iov_len); + if (s->host_vnet_hdr_len && !s->using_vnet_hdr) { iov_copy =3D g_new(struct iovec, iovcnt + 1); iov_copy[0].iov_base =3D &hdr; @@ -183,6 +187,49 @@ static void tap_send_completed(NetClientState *nc, ssi= ze_t len) tap_read_poll(s, true); } =20 +static char *tap_dump_packet(const uint8_t *buf, int size) +{ + int i, j; + char hex_line[80]; /* Enough space for hex pairs + spaces */ + char ascii_line[17]; /* 16 + 1 for null terminator */ + GString *dump_str =3D g_string_new(NULL); + + g_string_append_printf(dump_str, "Packet dump (%d bytes):\n", size); + + for (i =3D 0; i < size; i +=3D 16) { + memset(hex_line, 0, sizeof(hex_line)); + memset(ascii_line, 0, sizeof(ascii_line)); + + /* Build hex line in groups of 2 bytes (4 hex chars) */ + int hex_pos =3D 0; + for (j =3D 0; j < 16 && (i + j) < size; j +=3D 2) { + if (i + j + 1 < size) { + /* Two bytes available */ + hex_pos +=3D snprintf(hex_line + hex_pos, + sizeof(hex_line) - hex_pos, + "%02x%02x ", buf[i + j], buf[i + j + 1= ]); + } else { + /* Only one byte left */ + hex_pos +=3D snprintf(hex_line + hex_pos, + sizeof(hex_line) - hex_pos, + "%02x ", buf[i + j]); + } + } + + /* Build ASCII line */ + for (j =3D 0; j < 16 && (i + j) < size; j++) { + uint8_t byte =3D buf[i + j]; + ascii_line[j] =3D (byte >=3D 32 && byte <=3D 126) ? byte : '.'; + } + + /* Add the line in tcpdump-like format */ + g_string_append_printf(dump_str, "\t0x%04x: %-40s %s\n", + i, hex_line, ascii_line); + } + + return g_string_free(dump_str, false); +} + static void tap_send(void *opaque) { TAPState *s =3D opaque; @@ -199,6 +246,13 @@ static void tap_send(void *opaque) break; } =20 + if (trace_event_get_state_backends(TRACE_TAP_PACKET_DUMP)) { + g_autofree char *dump =3D tap_dump_packet(s->buf, size); + trace_tap_packet_dump(dump); + } + + trace_tap_send_packet(s->using_vnet_hdr, s->host_vnet_hdr_len, siz= e); + if (s->host_vnet_hdr_len && size <=3D s->host_vnet_hdr_len) { /* Invalid packet */ break; @@ -992,6 +1046,8 @@ int tap_enable(NetClientState *nc) TAPState *s =3D DO_UPCAST(TAPState, nc, nc); int ret; =20 + trace_tap_enable(); + if (s->enabled) { return 0; } else { @@ -1009,6 +1065,8 @@ int tap_disable(NetClientState *nc) TAPState *s =3D DO_UPCAST(TAPState, nc, nc); int ret; =20 + trace_tap_disable(); + if (s->enabled =3D=3D 0) { return 0; } else { diff --git a/net/trace-events b/net/trace-events index cda960f42b..b51427f539 100644 --- a/net/trace-events +++ b/net/trace-events @@ -29,3 +29,10 @@ vhost_vdpa_set_address_space_id(void *v, unsigned vq_gro= up, unsigned asid_num) " vhost_vdpa_net_load_cmd(void *s, uint8_t class, uint8_t cmd, int data_num,= int data_size) "vdpa state: %p class: %u cmd: %u sg_num: %d size: %d" vhost_vdpa_net_load_cmd_retval(void *s, uint8_t class, uint8_t cmd, int r)= "vdpa state: %p class: %u cmd: %u retval: %d" vhost_vdpa_net_load_mq(void *s, int ncurqps) "vdpa state: %p current_qpair= s: %d" + +# tap.c +tap_receive_iov(bool using_vnet_hdr, uint32_t host_vnet_hdr_len, int iovcn= t, size_t iov_len) "using_vnet_hdr:%d host_vnet_hdr_len:%u iovcnt:%d iov_le= n:%zu" +tap_send_packet(bool using_vnet_hdr, uint32_t host_vnet_hdr_len, int size)= "using_vnet_hdr:%d host_vnet_hdr_len:%u size:%d" +tap_enable(void) "tap enabled" +tap_disable(void) "tap disabled" +tap_packet_dump(const char *dump_str) "%s" --=20 2.48.1 From nobody Sat Nov 15 03:14:59 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1755985784; cv=none; d=zohomail.com; s=zohoarc; b=RxX91tXjDamU08nyUUDcnhi+ZdggXXSBiiFXlIkAnc6tfmQ95+UFUEhQ/++YIxxtJnnWDR8PNS6dp5eSScR8A2GacyhctP6/rwwcEWj4uf6Efz8thLQdqwyy3tXJwa3+jrb9X0+62O7TGL/p3TAbveoYWYeT3GqdErIMh/9g8jM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1755985784; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=b8jGSS7yeLrqIs8AracVmSaO6BrmOsvR8NpSDhUH6NU=; b=Gmvw7UPB+f24CeUObMigK8U7Hf/OzOB+GQ4DN1Os8wfY+lFqJKZIowsuNBwXcjdDjyB9WWVZYQZLrZUmCE/s0DRbXEn66lSplRyl5bRyADWGyqsM5X7PouuO/5PeXsbrc7nVFab7IQmNIcb7TZw3+8fOiDlYkq2jmWgCyJNvyi0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1755985784500131.65056139224782; Sat, 23 Aug 2025 14:49:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1upw6E-0004Uw-7c; Sat, 23 Aug 2025 17:48:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6C-0004TV-7A for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from forwardcorp1b.mail.yandex.net ([2a02:6b8:c02:900:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw69-00053x-M3 for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:31 -0400 Received: from mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net [IPv6:2a02:6b8:c21:2d8b:0:640:7d49:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 57709807C9; Sun, 24 Aug 2025 00:48:26 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6bf:8080:a5d::1:36]) by mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Nmd3dr0GuSw0-kuy4qXyl; Sun, 24 Aug 2025 00:48:26 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1755985706; bh=b8jGSS7yeLrqIs8AracVmSaO6BrmOsvR8NpSDhUH6NU=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=dOmcr1VMxdSvy/JkO+erFGHuc8QX8GZu2X4RYiytE5FJa6kXZ2gdhUhT4XI2Db2t1 CvTy+tRHiTzy1xdv8YzElt5N5wI9QcVQ1w19yaQSH5cTyZLNLjMI+UELZTGMqPdxLs 0EPlA6H8GVITKgfKyusK5RLIv6jugHlxnnhqFcjI= Authentication-Results: mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: jasowang@redhat.com Cc: qemu-devel@nongnu.org, vsementsov@yandex-team.ru, leiyang@redhat.com, steven.sistare@oracle.com, peterx@redhat.com, mst@redhat.com, farosas@suse.de, yc-core@yandex-team.ru Subject: [RFC 2/7] net/tap: keep exit notifier only when downscript set Date: Sun, 24 Aug 2025 00:48:16 +0300 Message-ID: <20250823214821.48342-3-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250823214821.48342-1-vsementsov@yandex-team.ru> References: <20250823214821.48342-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c02:900:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1755985787271124100 Content-Type: text/plain; charset="utf-8" The notifier is used only to call the downscript. Avoid extra notifier for other cases. Signed-off-by: Vladimir Sementsov-Ogievskiy --- net/tap.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/net/tap.c b/net/tap.c index 44bd6eefd8..c07af762b0 100644 --- a/net/tap.c +++ b/net/tap.c @@ -384,8 +384,10 @@ static void tap_cleanup(NetClientState *nc) =20 qemu_purge_queued_packets(nc); =20 - tap_exit_notify(&s->exit, NULL); - qemu_remove_exit_notifier(&s->exit); + if (s->exit.notify) { + tap_exit_notify(&s->exit, NULL); + qemu_remove_exit_notifier(&s->exit); + } =20 tap_read_poll(s, false); tap_write_poll(s, false); @@ -787,9 +789,7 @@ static int net_tap_fd_init_common(const Netdev *netdev,= NetClientState *peer, } tap_read_poll(s, true); s->vhost_net =3D NULL; - - s->exit.notify =3D tap_exit_notify; - qemu_add_exit_notifier(&s->exit); + s->exit.notify =3D NULL; =20 if (netdev->type =3D=3D NET_CLIENT_DRIVER_BRIDGE) { const NetdevBridgeOptions *bridge =3D &netdev->u.bridge; @@ -817,6 +817,8 @@ static int net_tap_fd_init_common(const Netdev *netdev,= NetClientState *peer, snprintf(s->down_script, sizeof(s->down_script), "%s", downscr= ipt); snprintf(s->down_script_arg, sizeof(s->down_script_arg), "%s", ifname); + s->exit.notify =3D tap_exit_notify; + qemu_add_exit_notifier(&s->exit); } } =20 --=20 2.48.1 From nobody Sat Nov 15 03:14:59 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1755985808; cv=none; d=zohomail.com; s=zohoarc; b=jcwxsKenobRAKFhvBW+0ueGBp1q12p/xzMTYoiRmYvkhxPPgCVMlRUegozsmIKMjSLIUNhluftsDF2ea4LRQ9LVZ2uCuS60eG5iIlBrdkxOJHJCRnauYItZnUmoc31tJO2ocMQcXiSbBydzqpuAzuWDt662csKZ/2ZLkZsE0I2I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1755985808; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kxovEUtU6YaTA5oc//90u76Nkj25zCDmGOze5dgMzHk=; b=Zfo7Z6gXU42dVvI1nZCxSIheRyJa8zfHZYSWbl8jmb3Zrz4vUAEhNZA+faMNAQQo5g9i3kwuWCTVrqpSzOHdTla2vPUE5t8NqTh9iuqO++0ivaRzsCkHQAr2o4bUO/ouvqJTIj33G6ywJ4ApPxl8dYmZE30SfirC5ZQMpbW+LYo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1755985808128680.8457944582382; Sat, 23 Aug 2025 14:50:08 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1upw6F-0004VK-3l; Sat, 23 Aug 2025 17:48:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6C-0004Tf-Io for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from forwardcorp1b.mail.yandex.net ([178.154.239.136]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw69-000543-MC for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net [IPv6:2a02:6b8:c21:2d8b:0:640:7d49:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id D14F2807CA; Sun, 24 Aug 2025 00:48:26 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6bf:8080:a5d::1:36]) by mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Nmd3dr0GuSw0-J7Xc0DHy; Sun, 24 Aug 2025 00:48:26 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1755985706; bh=kxovEUtU6YaTA5oc//90u76Nkj25zCDmGOze5dgMzHk=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=vwswRc2y3/wcTulu+WvG4Mr85aKGJxpr9a5qUTe+bjgAILTkwwMocMDrCha3v1i9m M8SKLqzHoaKqVJPutirsrwbO/FmLas5l+U4nGcm3Yrxgg6cu5A4mp8NsBaxPTrGrCa Nh25rGcMK4vWsdoMA6/KlipU2ZfCXz4XOwMQf058= Authentication-Results: mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: jasowang@redhat.com Cc: qemu-devel@nongnu.org, vsementsov@yandex-team.ru, leiyang@redhat.com, steven.sistare@oracle.com, peterx@redhat.com, mst@redhat.com, farosas@suse.de, yc-core@yandex-team.ru Subject: [RFC 3/7] qapi: add interface for local TAP migration Date: Sun, 24 Aug 2025 00:48:17 +0300 Message-ID: <20250823214821.48342-4-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250823214821.48342-1-vsementsov@yandex-team.ru> References: <20250823214821.48342-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.136; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1755985809678124100 Content-Type: text/plain; charset="utf-8" To migrate TAP device (including open fds) locally, user should: 1. enable local-tap migration capability both on source and target 2. use additional local-incoming=3Dtrue option for tap on target Why capability is not enough? We need an option to modify early initialization of TAP, to avoid opening new fds. The capability may not be set at the moment of netdev initialization. Signed-off-by: Vladimir Sementsov-Ogievskiy --- migration/options.c | 7 +++++++ migration/options.h | 1 + qapi/migration.json | 9 ++++++++- qapi/net.json | 12 +++++++++++- 4 files changed, 27 insertions(+), 2 deletions(-) diff --git a/migration/options.c b/migration/options.c index 11b719c81b..fd0cca97b3 100644 --- a/migration/options.c +++ b/migration/options.c @@ -276,6 +276,13 @@ bool migrate_local_vhost_user_blk(void) return s->capabilities[MIGRATION_CAPABILITY_LOCAL_VHOST_USER_BLK]; } =20 +bool migrate_local_tap(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->capabilities[MIGRATION_CAPABILITY_LOCAL_TAP]; +} + bool migrate_ignore_shared(void) { MigrationState *s =3D migrate_get_current(); diff --git a/migration/options.h b/migration/options.h index 5a40ac073d..aa0ab6a61f 100644 --- a/migration/options.h +++ b/migration/options.h @@ -32,6 +32,7 @@ bool migrate_events(void); bool migrate_mapped_ram(void); bool migrate_local_char_socket(void); bool migrate_local_vhost_user_blk(void); +bool migrate_local_tap(void); bool migrate_ignore_shared(void); bool migrate_late_block_activate(void); bool migrate_multifd(void); diff --git a/qapi/migration.json b/qapi/migration.json index ead7f4d17c..bf554684f4 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -529,6 +529,12 @@ # For target device also @local-incoming option must be specified # (since 10.2) # +# @local-tap: Migrate TAPs locally, keeping backend alive. Open file +# descriptors and TAP-related state are migrated. Only may be +# used when migration channel is unix socket. For target device +# also @local-incoming option must be specified (since 10.2) +# (since 10.2) +# # Features: # # @unstable: Members @x-colo, @x-ignore-shared, @local-char-socket, @@ -551,7 +557,8 @@ 'zero-copy-send', 'postcopy-preempt', 'switchover-ack', 'dirty-limit', 'mapped-ram', { 'name': 'local-char-socket', 'features': [ 'unstable' ] }, - { 'name': 'local-vhost-user-blk', 'features': [ 'unstable' ] } = ] } + { 'name': 'local-vhost-user-blk', 'features': [ 'unstable' ] }, + { 'name': 'local-tap', 'features': [ 'unstable' ] } ] } =20 ## # @MigrationCapabilityStatus: diff --git a/qapi/net.json b/qapi/net.json index 78bcc9871e..8f53549d58 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -353,6 +353,15 @@ # @poll-us: maximum number of microseconds that could be spent on busy # polling for tap (since 2.7) # +# @local-incoming: Do load open file descriptor for that TAP +# on incoming migration. May be used only if QEMU is started +# for incoming migration. Will work only together with local-tap +# migration capability enabled (default: false) (Since: 10.2) +# +# Features: +# +# @unstable: Member @local-incoming is experimental +# # Since: 1.2 ## { 'struct': 'NetdevTapOptions', @@ -371,7 +380,8 @@ '*vhostfds': 'str', '*vhostforce': 'bool', '*queues': 'uint32', - '*poll-us': 'uint32'} } + '*poll-us': 'uint32', + '*local-incoming': { 'type': 'bool', 'features': [ 'unstable' ] } } } =20 ## # @NetdevSocketOptions: --=20 2.48.1 From nobody Sat Nov 15 03:14:59 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1755985805; cv=none; d=zohomail.com; s=zohoarc; b=TTtKOyrci/ZYgx6tk15EZakmJrCgt8owYkvQuU5sgEqcwZydvv40eXnMAhLH/nIgKQx4P+r5L8jz5QU7jkpFBRFyULpBU6lIurMzPk5mcCll5PLRaqYRmfrwjw/fRzEXra1liD+loRYboFjWzKvoXcJSgx16Qw/qdKxT/T3029w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1755985805; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qskqYXjap40rdYlZ1SzEptRV2plpL6riqRQlHCIbGiU=; b=YszZGdcRosOLpbsof56E1iHbfLMggFm3fM1fNBj/PjT3WaCmX5l15lUZ8TOwMdP3ClZegahy0uPAzHei6ORvBB0L9Uwml2KUHzTGNTfcnCypE9OmfjVT5pM2R5rqqRKiKTRXceFQFzgPsRxNUQHqyrboGvzv7H4tnHVGSfXIOGw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1755985805180680.0664477257801; Sat, 23 Aug 2025 14:50:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1upw6E-0004Um-6T; Sat, 23 Aug 2025 17:48:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6C-0004Td-F9 for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from forwardcorp1b.mail.yandex.net ([2a02:6b8:c02:900:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw69-000546-MC for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net [IPv6:2a02:6b8:c21:2d8b:0:640:7d49:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 69500807BA; Sun, 24 Aug 2025 00:48:27 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6bf:8080:a5d::1:36]) by mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Nmd3dr0GuSw0-9aPBcEQy; Sun, 24 Aug 2025 00:48:26 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1755985707; bh=qskqYXjap40rdYlZ1SzEptRV2plpL6riqRQlHCIbGiU=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=UAXOp5wLYIikFaL3N8x3CWkjURXAqpxNtDpiBMfqGNys80psO2wHV5X1p//bWjWZ3 nX7ATMU+Yl5CdGNnkFyXnQbzQIMDrpsQPFI35x+yoowK0m6k4YTB1i91UX2Y9Z/xdy lz1GaW/ndbpbL6ikMZmO9EMPfyAbUNZwZi4hQkrE= Authentication-Results: mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: jasowang@redhat.com Cc: qemu-devel@nongnu.org, vsementsov@yandex-team.ru, leiyang@redhat.com, steven.sistare@oracle.com, peterx@redhat.com, mst@redhat.com, farosas@suse.de, yc-core@yandex-team.ru Subject: [RFC 4/7] net/tap: implement interfaces for local migration Date: Sun, 24 Aug 2025 00:48:18 +0300 Message-ID: <20250823214821.48342-5-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250823214821.48342-1-vsementsov@yandex-team.ru> References: <20250823214821.48342-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c02:900:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1755985806498116600 Content-Type: text/plain; charset="utf-8" Handle local-incoming option: Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/net/tap.h | 4 ++ net/tap.c | 129 ++++++++++++++++++++++++++++++++++------------ 2 files changed, 99 insertions(+), 34 deletions(-) diff --git a/include/net/tap.h b/include/net/tap.h index 6f34f13eae..3ef2e2dbae 100644 --- a/include/net/tap.h +++ b/include/net/tap.h @@ -30,7 +30,11 @@ =20 int tap_enable(NetClientState *nc); int tap_disable(NetClientState *nc); +bool tap_local_incoming(NetClientState *nc); =20 int tap_get_fd(NetClientState *nc); =20 +int tap_load(NetClientState *nc, QEMUFile *f); +int tap_save(NetClientState *nc, QEMUFile *f); + #endif /* QEMU_NET_TAP_H */ diff --git a/net/tap.c b/net/tap.c index c07af762b0..4d3cdc0662 100644 --- a/net/tap.c +++ b/net/tap.c @@ -35,6 +35,8 @@ #include "net/eth.h" #include "net/net.h" #include "clients.h" +#include "migration/migration.h" +#include "migration/qemu-file.h" #include "monitor/monitor.h" #include "system/system.h" #include "qapi/error.h" @@ -81,6 +83,7 @@ typedef struct TAPState { VHostNetState *vhost_net; unsigned host_vnet_hdr_len; Notifier exit; + bool local_incoming; } TAPState; =20 static void launch_script(const char *setup_script, const char *ifname, @@ -727,21 +730,25 @@ static int net_tap_open_one(const Netdev *netdev, vnet_hdr_required =3D 0; } =20 - fd =3D RETRY_ON_EINTR(tap_open(ifname, ifname_sz, &vnet_hdr, - vnet_hdr_required, mq_required, errp)); - if (fd < 0) { - return -1; - } - - if (script && - script[0] !=3D '\0' && - strcmp(script, "no") !=3D 0) { - launch_script(script, ifname, fd, &err); - if (err) { - error_propagate(errp, err); - close(fd); + if (tap->local_incoming) { + fd =3D -1; + } else { + fd =3D RETRY_ON_EINTR(tap_open(ifname, ifname_sz, &vnet_hdr, + vnet_hdr_required, mq_required, errp)= ); + if (fd < 0) { return -1; } + + if (script && + script[0] !=3D '\0' && + strcmp(script, "no") !=3D 0) { + launch_script(script, ifname, fd, &err); + if (err) { + error_propagate(errp, err); + close(fd); + return -1; + } + } } =20 ret =3D net_tap_fd_init_common(netdev, peer, "tap", name, ifname, @@ -757,6 +764,42 @@ static int net_tap_open_one(const Netdev *netdev, =20 #define MAX_TAP_QUEUES 1024 =20 +int tap_save(NetClientState *nc, QEMUFile *f) +{ + TAPState *s =3D DO_UPCAST(TAPState, nc, nc); + + qemu_file_put_fd(f, s->fd); + qemu_put_byte(f, s->using_vnet_hdr); + qemu_put_byte(f, s->has_ufo); + qemu_put_byte(f, s->has_uso); + qemu_put_byte(f, s->enabled); + qemu_put_be32(f, s->host_vnet_hdr_len); + + return 0; +} + +int tap_load(NetClientState *nc, QEMUFile *f) +{ + TAPState *s =3D DO_UPCAST(TAPState, nc, nc); + + s->fd =3D qemu_file_get_fd(f); + if (s->fd < 0) { + return -1; + } + + s->using_vnet_hdr =3D qemu_get_byte(f); + s->has_ufo =3D qemu_get_byte(f); + s->has_uso =3D qemu_get_byte(f); + s->enabled =3D qemu_get_byte(f); + qemu_get_be32s(f, &s->host_vnet_hdr_len); + + tap_read_poll(s, true); + s->exit.notify =3D tap_exit_notify; + qemu_add_exit_notifier(&s->exit); + + return 0; +} + static int net_tap_fd_init_common(const Netdev *netdev, NetClientState *pe= er, const char *model, const char *name, const char *ifname, const char *script, @@ -764,30 +807,40 @@ static int net_tap_fd_init_common(const Netdev *netde= v, NetClientState *peer, const char *vhostfdname, int vnet_hdr, int fd, Error **errp) { - const NetdevTapOptions *tap; + const NetdevTapOptions *tap =3D NULL; int ret; NetClientState *nc; TAPState *s; + bool local_incoming =3D false; + + if (netdev->type =3D=3D NET_CLIENT_DRIVER_TAP) { + tap =3D &netdev->u.tap; + local_incoming =3D tap->local_incoming; + } =20 nc =3D qemu_new_net_client(&net_tap_info, peer, model, name); =20 s =3D DO_UPCAST(TAPState, nc, nc); - - s->fd =3D fd; - s->host_vnet_hdr_len =3D vnet_hdr ? sizeof(struct virtio_net_hdr) : 0; - s->using_vnet_hdr =3D false; - s->has_ufo =3D tap_probe_has_ufo(s->fd); - s->has_uso =3D tap_probe_has_uso(s->fd); - s->enabled =3D true; - tap_set_offload(&s->nc, 0, 0, 0, 0, 0, 0, 0); - /* - * Make sure host header length is set correctly in tap: - * it might have been modified by another instance of qemu. - */ - if (vnet_hdr) { - tap_fd_set_vnet_hdr_len(s->fd, s->host_vnet_hdr_len); + s->local_incoming =3D local_incoming; + + if (!local_incoming) { + s->fd =3D fd; + s->host_vnet_hdr_len =3D vnet_hdr ? sizeof(struct virtio_net_hdr) = : 0; + s->using_vnet_hdr =3D false; + s->has_ufo =3D tap_probe_has_ufo(s->fd); + s->has_uso =3D tap_probe_has_uso(s->fd); + s->enabled =3D true; + tap_set_offload(&s->nc, 0, 0, 0, 0, 0, 0, 0); + /* + * Make sure host header length is set correctly in tap: + * it might have been modified by another instance of qemu. + */ + if (vnet_hdr) { + tap_fd_set_vnet_hdr_len(s->fd, s->host_vnet_hdr_len); + } + tap_read_poll(s, true); } - tap_read_poll(s, true); + s->vhost_net =3D NULL; s->exit.notify =3D NULL; =20 @@ -799,9 +852,8 @@ static int net_tap_fd_init_common(const Netdev *netdev,= NetClientState *peer, } =20 assert(netdev->type =3D=3D NET_CLIENT_DRIVER_TAP); - tap =3D &netdev->u.tap; =20 - if (tap_set_sndbuf(s->fd, tap, errp) < 0) { + if (!local_incoming && tap_set_sndbuf(s->fd, tap, errp) < 0) { goto failed; } =20 @@ -822,9 +874,11 @@ static int net_tap_fd_init_common(const Netdev *netdev= , NetClientState *peer, } } =20 - ret =3D net_tap_setup_vhost(s, tap, vhostfdname, errp); - if (ret < 0) { - goto failed; + if (!local_incoming) { + ret =3D net_tap_setup_vhost(s, tap, vhostfdname, errp); + if (ret < 0) { + goto failed; + } } =20 return 0; @@ -1081,3 +1135,10 @@ int tap_disable(NetClientState *nc) return ret; } } + +bool tap_local_incoming(NetClientState *nc) +{ + TAPState *s =3D DO_UPCAST(TAPState, nc, nc); + + return s->local_incoming && runstate_check(RUN_STATE_INMIGRATE); +} --=20 2.48.1 From nobody Sat Nov 15 03:14:59 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1755985832; cv=none; d=zohomail.com; s=zohoarc; b=RXYt2DlqIYn1YGCOmvZvsVwHKqzc51fIFKv4AoFcN8bmgsaFmxPptFTeUlH+bnfXpc8HXqvCoW30rZnqvecvDTXgfazFANAl2It9bAYLw8s0XGKrs8KIjrd3i6QqxRSvY1V4POkkRpbFCtjaTHFPPu9xbsM775roHsj1OxcvX+8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1755985832; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GftTvKuCJ/20RJKBye2/TezLhjc0gyXvfMSFYxlvOO8=; b=ZOXeN6fYvEsoMs92Wua25obrj4QXH8pzbG5fHWk/l+XFpIR7tfxSLNKg6vykTFEP5IBVuYO1bt73f9pBCrZEotvxM8RknUp5nb7CTmiIbFeyFF4AjNfZyiDPVwYxSo2lbqJjS/pKhRWQ8w2gbbwfZZw9cV6EpbNa4VGzVrYrYuU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1755985831882530.3729839282207; Sat, 23 Aug 2025 14:50:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1upw6F-0004WV-QU; Sat, 23 Aug 2025 17:48:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6C-0004Te-GO for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from forwardcorp1b.mail.yandex.net ([2a02:6b8:c02:900:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw69-00054A-MK for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:32 -0400 Received: from mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net [IPv6:2a02:6b8:c21:2d8b:0:640:7d49:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 023F8807C8; Sun, 24 Aug 2025 00:48:28 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6bf:8080:a5d::1:36]) by mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Nmd3dr0GuSw0-GTo45y1a; Sun, 24 Aug 2025 00:48:27 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1755985707; bh=GftTvKuCJ/20RJKBye2/TezLhjc0gyXvfMSFYxlvOO8=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=g3YFHIXSZo+X5vcdph5+hatcE8swRJisd//UZvpUjdAd/WXLkxzgzUY0WdBQYhJ+k JDPGwaSHXNIi77Avmy2sCQpKtHl6g3J8AWnRuJpSGZBNywqyhqYPwJA1JtcUro0Gy+ 5phtS+PFSS6Sq/9xOnHOOgTgkMGFToRqhf2K/1Jk= Authentication-Results: mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: jasowang@redhat.com Cc: qemu-devel@nongnu.org, vsementsov@yandex-team.ru, leiyang@redhat.com, steven.sistare@oracle.com, peterx@redhat.com, mst@redhat.com, farosas@suse.de, yc-core@yandex-team.ru Subject: [RFC 5/7] virtio-net: support local tap migration Date: Sun, 24 Aug 2025 00:48:19 +0300 Message-ID: <20250823214821.48342-6-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250823214821.48342-1-vsementsov@yandex-team.ru> References: <20250823214821.48342-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c02:900:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1755985833570124100 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy --- hw/net/virtio-net.c | 100 ++++++++++++++++++++++++++++++++- include/hw/virtio/virtio-net.h | 2 + 2 files changed, 101 insertions(+), 1 deletion(-) diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 6b5b5dace3..874e349fee 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -38,6 +38,8 @@ #include "qapi/qapi-events-migration.h" #include "hw/virtio/virtio-access.h" #include "migration/misc.h" +#include "migration/migration.h" +#include "migration/options.h" #include "standard-headers/linux/ethtool.h" #include "system/system.h" #include "system/replay.h" @@ -2999,7 +3001,13 @@ static void virtio_net_set_multiqueue(VirtIONet *n, = int multiqueue) n->multiqueue =3D multiqueue; virtio_net_change_num_queues(n, max * 2 + 1); =20 - virtio_net_set_queue_pairs(n); + /* + * Called from set_features(0) on reset, when on target we + * doesn't have fds yet + */ + if (!n->tap_wait_incoming) { + virtio_net_set_queue_pairs(n); + } } =20 static int virtio_net_pre_load_queues(VirtIODevice *vdev, uint32_t n) @@ -3009,6 +3017,19 @@ static int virtio_net_pre_load_queues(VirtIODevice *= vdev, uint32_t n) return 0; } =20 +static int virtio_net_pre_save_device(void *opaque) +{ + VirtIONet *n =3D opaque; + int i, r; + + for (i =3D 0; i < n->curr_queue_pairs; i++) { + r =3D peer_detach(n, i); + assert(!r); + } + + return 0; +} + static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t featu= res, Error **errp) { @@ -3028,6 +3049,11 @@ static uint64_t virtio_net_get_features(VirtIODevice= *vdev, uint64_t features, =20 virtio_add_feature(&features, VIRTIO_NET_F_MAC); =20 + if (n->tap_wait_incoming) { + /* Excessive feature set is OK for early initialization */ + return features; + } + if (!peer_has_vnet_hdr(n)) { virtio_clear_feature(&features, VIRTIO_NET_F_CSUM); virtio_clear_feature(&features, VIRTIO_NET_F_HOST_TSO4); @@ -3494,11 +3520,69 @@ static const VMStateDescription vhost_user_net_back= end_state =3D { } }; =20 +static int virtio_net_tap_save(QEMUFile *f, void *pv, size_t size, + const VMStateField *field, + JSONWriter *vmdesc) +{ + VirtIONet *n =3D pv; + int i; + + for (i =3D 0; i < n->max_queue_pairs; i++) { + NetClientState *nc =3D qemu_get_subqueue(n->nic, i); + assert(nc->peer->info->type =3D=3D NET_CLIENT_DRIVER_TAP); + + tap_save(nc->peer, f); + } + + return 0; +} + +static int virtio_net_tap_load(QEMUFile *f, void *pv, size_t size, + const VMStateField *field) +{ + VirtIONet *n =3D pv; + VirtIODevice *vdev =3D VIRTIO_DEVICE(n); + VirtioDeviceClass *vdc =3D VIRTIO_DEVICE_GET_CLASS(vdev); + Error *local_err =3D NULL; + int i; + + for (i =3D 0; i < n->max_queue_pairs; i++) { + NetClientState *nc =3D qemu_get_subqueue(n->nic, i); + assert(nc->peer->info->type =3D=3D NET_CLIENT_DRIVER_TAP); + + tap_load(nc->peer, f); + } + + peer_test_vnet_hdr(n); + n->tap_wait_incoming =3D false; + + vdev->host_features =3D vdc->get_features(vdev, vdev->host_features, + &local_err); + if (local_err) { + error_report_err(local_err); + return -EINVAL; + } + + return 0; +} + +static bool virtio_net_is_tap_local(void *opaque, int version_id) +{ + VirtIONet *n =3D opaque; + NetClientState *nc; + + nc =3D qemu_get_queue(n->nic); + + return migrate_local_tap() && nc->peer && + nc->peer->info->type =3D=3D NET_CLIENT_DRIVER_TAP; +} + static const VMStateDescription vmstate_virtio_net_device =3D { .name =3D "virtio-net-device", .version_id =3D VIRTIO_NET_VM_VERSION, .minimum_version_id =3D VIRTIO_NET_VM_VERSION, .post_load =3D virtio_net_post_load_device, + .pre_save =3D virtio_net_pre_save_device, .fields =3D (const VMStateField[]) { VMSTATE_UINT8_ARRAY(mac, VirtIONet, ETH_ALEN), VMSTATE_STRUCT_POINTER(vqs, VirtIONet, @@ -3525,6 +3609,15 @@ static const VMStateDescription vmstate_virtio_net_d= evice =3D { * but based on the uint. */ VMSTATE_BUFFER_POINTER_UNSAFE(vlans, VirtIONet, 0, MAX_VLAN >> 3), + { + .name =3D "tap", + .info =3D &(const VMStateInfo) { + .name =3D "virtio-net vhost-user backend state", + .get =3D virtio_net_tap_load, + .put =3D virtio_net_tap_save, + }, + .field_exists =3D virtio_net_is_tap_local, + }, VMSTATE_WITH_TMP(VirtIONet, struct VirtIONetMigTmp, vmstate_virtio_net_has_vnet), VMSTATE_UINT8(mac_table.multi_overflow, VirtIONet), @@ -3954,6 +4047,11 @@ static void virtio_net_device_realize(DeviceState *d= ev, Error **errp) vhost_net_set_config(get_vhost_net(nc->peer), (uint8_t *)&netcfg, 0, ETH_ALEN, VHOST_SET_CONFIG_TYPE_FRONTEN= D); } + + if (nc->peer && nc->peer->info->type =3D=3D NET_CLIENT_DRIVER_TAP) { + n->tap_wait_incoming =3D tap_local_incoming(nc->peer); + } + QTAILQ_INIT(&n->rsc_chains); n->qdev =3D dev; =20 diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h index 73fdefc0dc..04ae0e4c06 100644 --- a/include/hw/virtio/virtio-net.h +++ b/include/hw/virtio/virtio-net.h @@ -231,6 +231,8 @@ struct VirtIONet { struct EBPFRSSContext ebpf_rss; uint32_t nr_ebpf_rss_fds; char **ebpf_rss_fds; + + bool tap_wait_incoming; }; =20 size_t virtio_net_handle_ctrl_iov(VirtIODevice *vdev, --=20 2.48.1 From nobody Sat Nov 15 03:14:59 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1755985757; cv=none; d=zohomail.com; s=zohoarc; b=Xax410GBZSLveUGFt+atuAHL8EyhXV689FjEP1A/hpfK2P8Z4agqEmHNj30rR7o7droQu+23yF/0UFcrBM0TYwZ5XuBJIiLPutqhPx2egLKg2jxW5oQf3962lgpUrLUyPJDt0a5fAaDsBJ/hjF4ASYfL9I+yvmDNfDFIBxAQvvM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1755985757; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zKfF0Jr3ZNXBdyUSBJ7GUqENZhLe0Y59SMSSlwzFldU=; b=jkOQ6I6Y7SFvuVMwy3Y5wj4yjLZIIX6dBK8J2gc78J0+6dN7af80f0famhkbxQy3/S5L7P3rhVDrW/lv6D6lVEOR6nieHxUBg3JOwWOvcB4cKr1zJrZ87w5R9CPWgGDtzE5sqpX3xNWDiSGdepQ//o2Ki/WSqyYAaxue9rVAipk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1755985756933500.8417011128307; Sat, 23 Aug 2025 14:49:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1upw6F-0004VY-0O; Sat, 23 Aug 2025 17:48:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6D-0004Uo-Sv for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:33 -0400 Received: from forwardcorp1b.mail.yandex.net ([178.154.239.136]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6A-00054Q-IA for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:33 -0400 Received: from mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net [IPv6:2a02:6b8:c21:2d8b:0:640:7d49:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 7CA2C80818; Sun, 24 Aug 2025 00:48:28 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6bf:8080:a5d::1:36]) by mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Nmd3dr0GuSw0-FxGAKwFt; Sun, 24 Aug 2025 00:48:28 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1755985708; bh=zKfF0Jr3ZNXBdyUSBJ7GUqENZhLe0Y59SMSSlwzFldU=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=vBluVYAEm0HRd3eXs6En1RH0DMWTRMwRzrW7pYAxQShntPNs2wB6Hl7UIFZkGD2el StC9W1J51ZhHdPiGqQsSCbeiSR+vYP+MMJt15m5AdBPP4yYxz6Zwr2eUdOckcaQB/5 PwdK9FenW0WfZIULajAYlXRyk78RP07xahhNba/8= Authentication-Results: mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: jasowang@redhat.com Cc: qemu-devel@nongnu.org, vsementsov@yandex-team.ru, leiyang@redhat.com, steven.sistare@oracle.com, peterx@redhat.com, mst@redhat.com, farosas@suse.de, yc-core@yandex-team.ru Subject: [RFC 6/7] net/tap: experimental support for vhost in local tap migration Date: Sun, 24 Aug 2025 00:48:20 +0300 Message-ID: <20250823214821.48342-7-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250823214821.48342-1-vsementsov@yandex-team.ru> References: <20250823214821.48342-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.136; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1755985760416116600 Content-Type: text/plain; charset="utf-8" Just reinitialize vhost after getting the fd on target. This significantly increase downtime (0.03s -> 0.06s in my simple testing). So next step is to rework it, completely migrating the vhost state to the new qemu without reinitialization. Signed-off-by: Vladimir Sementsov-Ogievskiy --- net/tap.c | 35 +++++++++++++++++------------------ 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/net/tap.c b/net/tap.c index 4d3cdc0662..7a4de07025 100644 --- a/net/tap.c +++ b/net/tap.c @@ -84,6 +84,9 @@ typedef struct TAPState { unsigned host_vnet_hdr_len; Notifier exit; bool local_incoming; + bool need_vhost; + char *vhost_fdname; + uint32_t vhost_busyloop_timeout; } TAPState; =20 static void launch_script(const char *setup_script, const char *ifname, @@ -98,8 +101,7 @@ static int net_tap_fd_init_common(const Netdev *netdev, = NetClientState *peer, const char *downscript, const char *vhostfdname, int vnet_hdr, int fd, Error **errp); -static int net_tap_setup_vhost(TAPState *s, const NetdevTapOptions *tap, - const char *vhostfdname, Error **errp); +static int net_tap_setup_vhost(TAPState *s, Error **errp); =20 static void tap_update_fd_handler(TAPState *s) { @@ -384,6 +386,7 @@ static void tap_cleanup(NetClientState *nc) g_free(s->vhost_net); s->vhost_net =3D NULL; } + g_free(s->vhost_fdname); =20 qemu_purge_queued_packets(nc); =20 @@ -794,10 +797,8 @@ int tap_load(NetClientState *nc, QEMUFile *f) qemu_get_be32s(f, &s->host_vnet_hdr_len); =20 tap_read_poll(s, true); - s->exit.notify =3D tap_exit_notify; - qemu_add_exit_notifier(&s->exit); =20 - return 0; + return net_tap_setup_vhost(s, NULL); } =20 static int net_tap_fd_init_common(const Netdev *netdev, NetClientState *pe= er, @@ -874,8 +875,12 @@ static int net_tap_fd_init_common(const Netdev *netdev= , NetClientState *peer, } } =20 + s->need_vhost =3D tap->has_vhost ? tap->vhost : + vhostfdname || (tap->has_vhostforce && tap->vhostforce); + s->vhost_fdname =3D g_strdup(vhostfdname); + s->vhost_busyloop_timeout =3D tap->has_poll_us ? tap->poll_us : 0; if (!local_incoming) { - ret =3D net_tap_setup_vhost(s, tap, vhostfdname, errp); + ret =3D net_tap_setup_vhost(s, errp); if (ret < 0) { goto failed; } @@ -888,28 +893,22 @@ failed: return -1; } =20 -static int net_tap_setup_vhost(TAPState *s, const NetdevTapOptions *tap, - const char *vhostfdname, Error **errp) +static int net_tap_setup_vhost(TAPState *s, Error **errp) { - if (tap->has_vhost ? tap->vhost : - vhostfdname || (tap->has_vhostforce && tap->vhostforce)) { + if (s->need_vhost) { VhostNetOptions options; int vhostfd; =20 options.backend_type =3D VHOST_BACKEND_TYPE_KERNEL; options.net_backend =3D &s->nc; - if (tap->has_poll_us) { - options.busyloop_timeout =3D tap->poll_us; - } else { - options.busyloop_timeout =3D 0; - } + options.busyloop_timeout =3D s->vhost_busyloop_timeout; =20 - if (vhostfdname) { - vhostfd =3D monitor_fd_param(monitor_cur(), vhostfdname, errp); + if (s->vhost_fdname) { + vhostfd =3D monitor_fd_param(monitor_cur(), s->vhost_fdname, e= rrp); if (vhostfd =3D=3D -1) { return -1; } - if (!set_fd_nonblocking(vhostfd, vhostfdname, errp)) { + if (!set_fd_nonblocking(vhostfd, s->vhost_fdname, errp)) { return -1; } } else { --=20 2.48.1 From nobody Sat Nov 15 03:14:59 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1755985757; cv=none; d=zohomail.com; s=zohoarc; b=jMazcRKtyUSPcpBTpRxvrD0sVV0d7AE87Xb6/D7eECxDkqxNhC9IVVlbOvej6VZPyx0vjeTzO/QJjupJxKb5RHlQJcz7uCiaF6sNzFA+7qCJj1tOCfKLZIgwjekDpyt3OxjA4SquKW0iCmHCWIdn7R7OzDM/HvvY1ib+bybcoi8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1755985757; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3LPYUtrlMzm8Kkb/TwFLjVqW2I4eWMHZK7XFbbirxyM=; b=XSifGTZ0pVFSMauf7qB9evhaSb0bd8DSWkiZ9lFGUN4jvEKu/kY97U24Flr4JjwBaW1e5FG7UP506mV3zJfFcu9MD5ufnQ1i/ioBDmCaGgpRyaNMw75JMYJGf9v49t4UdJ2rVGsq1CGFzJOvhiRhYofMo1dDbw3Y7aQEbMLqS4c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1755985756847757.214418460327; Sat, 23 Aug 2025 14:49:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1upw6G-0004XM-TM; Sat, 23 Aug 2025 17:48:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6E-0004V4-Ba for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:34 -0400 Received: from forwardcorp1b.mail.yandex.net ([2a02:6b8:c02:900:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1upw6A-00054T-D6 for qemu-devel@nongnu.org; Sat, 23 Aug 2025 17:48:34 -0400 Received: from mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net [IPv6:2a02:6b8:c21:2d8b:0:640:7d49:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 2B90480819; Sun, 24 Aug 2025 00:48:29 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6bf:8080:a5d::1:36]) by mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id Nmd3dr0GuSw0-nEOOXVl3; Sun, 24 Aug 2025 00:48:28 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1755985708; bh=3LPYUtrlMzm8Kkb/TwFLjVqW2I4eWMHZK7XFbbirxyM=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=sl5hhcSzDu0amuJrZrwoi5crJ/eohxVGMrLgGXO20mKeH3wgJhDyrwPXHGczN7vNy 6M3UERnD7TMEXrJsmgaFG4r06Rx9gF28ubXXVsvGlRnmtJjmgGxUiunV9QAc6bI8qJ bAmYzz8SaoK3woFLDLno9Klnh5NurYE5OGQk/xi0= Authentication-Results: mail-nwsmtp-smtp-corp-main-34.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: jasowang@redhat.com Cc: qemu-devel@nongnu.org, vsementsov@yandex-team.ru, leiyang@redhat.com, steven.sistare@oracle.com, peterx@redhat.com, mst@redhat.com, farosas@suse.de, yc-core@yandex-team.ru Subject: [RFC 7/7] RFC: add python/tap-migration-stand.py Date: Sun, 24 Aug 2025 00:48:21 +0300 Message-ID: <20250823214821.48342-8-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250823214821.48342-1-vsementsov@yandex-team.ru> References: <20250823214821.48342-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c02:900:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1755985760584116600 Content-Type: text/plain; charset="utf-8" How to use: 1. prepare test image (any distribution with working network), and modify IMAGE variable in script You'll need to setup static ip in the image, for example in Ubuntu: cat /etc/netplan/01-netcfg.yaml network: version: 2 renderer: networkd ethernets: ens1: dhcp4: no addresses: [10.0.1.2/24] gateway4: 10.0.1.1 nameservers: addresses: [8.8.8.8, 8.8.4.4] 2. run script with sudo (to manipulate with TAPs) cd python sudo ./tap-migration-stand.py --qemu-binary path/to/qemu-system-x86_64 \ --mode (new | cpr) mode=3Dnew means current series, mode=3Dcpr means "[RFC V2 0/8] Live update: tap and vhost" 3. run in separate terminal something like while true; do gvncviewer :0; gvncviewer :1; sleep 0.5; done to monitor our vms (on live update, vnc port will change every time) 4. run ping processes, for example in host: ping -f 10.0.1.2 -i 0.002 in vm: ping -f 10.0.1.1 -i 0 5. in running script, run command "u" (without quotes), which means live-update. 6. you may also pass a counter, like "u 1000", to update 1000 times, which helps to catch racy bugs. Signed-off-by: Vladimir Sementsov-Ogievskiy --- python/qemu/machine/machine.py | 37 ++-- python/tap-migration-stand.py | 348 +++++++++++++++++++++++++++++++++ 2 files changed, 369 insertions(+), 16 deletions(-) create mode 100755 python/tap-migration-stand.py diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index ebb58d5b68..fed54f768d 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -360,6 +360,7 @@ def _pre_launch(self) -> None: # _post_shutdown()! # pylint: disable=3Dconsider-using-with self._qemu_log_path =3D os.path.join(self.log_dir, self._name + ".= log") + print(f"Open log file: {self._qemu_log_path}") self._qemu_log_file =3D open(self._qemu_log_path, 'wb') =20 self._iolog =3D None @@ -435,17 +436,17 @@ def _post_shutdown(self) -> None: self._user_killed =3D False self._launched =3D False =20 - def launch(self) -> None: + def launch(self, do_launch=3DTrue, do_post_launch=3DTrue) -> None: """ Launch the VM and make sure we cleanup and expose the command line/output in case of exception """ =20 - if self._launched: + if self._launched and do_launch: raise QEMUMachineError('VM already launched') =20 try: - self._launch() + self._launch(do_launch, do_post_launch) except BaseException as exc: # We may have launched the process but it may # have exited before we could connect via QMP. @@ -468,23 +469,26 @@ def launch(self) -> None: # that exception. However, we still want to clean up. raise =20 - def _launch(self) -> None: + def _launch(self, do_launch=3DTrue, do_post_launch=3DTrue) -> None: """ Launch the VM and establish a QMP connection """ - self._pre_launch() - LOG.debug('VM launch command: %r', ' '.join(self._qemu_full_args)) + if do_launch: + self._pre_launch() + print('VM launch command: %r', ' '.join(self._qemu_full_args)) =20 - # Cleaning up of this subprocess is guaranteed by _do_shutdown. - # pylint: disable=3Dconsider-using-with - self._popen =3D subprocess.Popen(self._qemu_full_args, - stdin=3Dsubprocess.DEVNULL, - stdout=3Dself._qemu_log_file, - stderr=3Dsubprocess.STDOUT, - shell=3DFalse, - close_fds=3DFalse) - self._launched =3D True - self._post_launch() + # Cleaning up of this subprocess is guaranteed by _do_shutdown. + # pylint: disable=3Dconsider-using-with + self._popen =3D subprocess.Popen(self._qemu_full_args, + stdin=3Dsubprocess.DEVNULL, + stdout=3Dself._qemu_log_file, + stderr=3Dsubprocess.STDOUT, + shell=3DFalse, + close_fds=3DFalse) + self._launched =3D True + + if do_post_launch: + self._post_launch() =20 def _close_qmp_connection(self) -> None: """ @@ -732,6 +736,7 @@ def cmd(self, cmd: str, conv_keys =3D True =20 qmp_args =3D self._qmp_args(conv_keys, args) + print(cmd, qmp_args) ret =3D self._qmp.cmd(cmd, **qmp_args) if cmd =3D=3D 'quit': self._quit_issued =3D True diff --git a/python/tap-migration-stand.py b/python/tap-migration-stand.py new file mode 100755 index 0000000000..24e0e58e40 --- /dev/null +++ b/python/tap-migration-stand.py @@ -0,0 +1,348 @@ +#!/usr/bin/env python3 +import argparse +import subprocess +import time +from enum import Enum +from typing import Tuple + +from qemu.machine import QEMUMachine + + +IMAGE =3D "/home/vsementsov/work/vms/newfocal.raw" + + +def run(cmd: str, check: bool =3D True) -> None: + subprocess.run(cmd, check=3Dcheck, shell=3DTrue) + + +def del_tap(tap: str) -> None: + run(f"sudo ip tuntap del {tap} mode tap multi_queue", check=3DFalse) + + +def init_tap(tap: str) -> None: + run(f"sudo ip tuntap add dev {tap} mode tap multi_queue") + run(f"sudo ip link set dev {tap} address e6:1d:44:b5:03:5d") + run(f"sudo ip addr add 10.0.1.1/24 dev {tap}") + run(f"sudo ip link set {tap} up") + + +class MigrationFailed(Exception): + pass + + +class MyVM(QEMUMachine): + class Mode(Enum): + CPR =3D "cpr" + CPR_NO_TAP =3D "cpr-no-tap" + NO_TAP =3D "no-tap" + OPEN_SAME_TAP =3D "open-same-tap" + OPEN_NEW_TAP =3D "open-new-tap" + NEW =3D "new" + + def __init__( + self, + binary: str, + mode: Mode, + incoming: bool =3D False, + ind: int =3D 0, + vhost: bool =3D False, + ): + assert ind in (0, 1) + self.tap_name =3D f"tap{ind}" if mode =3D=3D MyVM.Mode.OPEN_NEW_TA= P else "tap0" + self.cpr =3D mode in (MyVM.Mode.CPR, MyVM.Mode.CPR_NO_TAP) + self.no_tap =3D mode in (MyVM.Mode.NO_TAP, MyVM.Mode.CPR_NO_TAP) + self.mode =3D mode + self.ind =3D ind + self.qemu_binary =3D binary + self.vhost =3D vhost + self.fds =3D None + auxshare_str =3D "-machine aux-ram-share=3Don" if self.cpr else "" + + additional_args =3D [] + if incoming: + additional_args =3D ["-incoming", "defer"] + if self.cpr: + additional_args +=3D [ + "-incoming", + '{"channel-type": "cpr","addr": ' + '{ "transport": "socket","type": "unix", "path": "/tmp= /cpr.sock"}}', + ] + + new_traces =3D "-trace tap_*" if mode =3D=3D MyVM.Mode.NEW else "" + + super().__init__( + binary=3Dbinary, + log_dir=3D"/tmp/logdir/", + name=3Df"mytest{ind}", + args=3Df""" + -device pxb-pcie,bus_nr=3D128,bus=3Dpcie.0,id=3Dpcie.1 + -device pcie-root-port,id=3Ds0,slot=3D0,bus=3Dpcie.1 + -device pcie-root-port,id=3Ds1,slot=3D1,bus=3Dpcie.1 + -device pcie-root-port,id=3Ds2,slot=3D2,bus=3Dpcie.1 + + -hda {IMAGE} + -m 4G -enable-kvm -M q35 -vnc :{ind} -nodefaults -vga std + -qmp stdio + -msg timestamp + -S + -trace migrate_* + -trace migration_cleanup + -trace migration_cancel + -trace handle_qmp_command + -trace monitor_qmp_respond + {new_traces} + -object memory-backend-file,id=3Dram0,size=3D4G,mem-path= =3D/dev/shm/ram0,share=3Don + -machine memory-backend=3Dram0 {auxshare_str} + """.split() + + additional_args, + ) + + def add_tap_netdev(self, tap, vhost: bool, local_incoming: bool =3D Fa= lse): + args =3D { + "id": "netdev.1", + "vhost": vhost, + "vhostforce": vhost, + "type": "tap", + "ifname": tap, + "script": "no", + "downscript": "no", + "queues": 4, + } + + if self.cpr: + args["cpr"] =3D True + elif local_incoming: + args["local-incoming"] =3D True + + self.cmd("netdev_add", args) + + self.cmd( + "device_add", + driver=3D"virtio-net-pci", + romfile=3D"", + id=3D"vnet.1", + netdev=3D"netdev.1", + mq=3DTrue, + vectors=3D18, + bus=3D"s1", + mac=3D"d6:0d:75:f8:0f:b7", + disable_legacy=3D"off", + ) + + def setup_network_first_time(self): + if self.no_tap: + return + + del_tap("tap0") + del_tap("tap1") + assert self.tap_name =3D=3D "tap0" + init_tap("tap0") + + self.add_tap_netdev(self.tap_name, self.vhost) + + def setup_network_incoming(self, fds=3DNone): + if self.no_tap: + return + + if self.mode =3D=3D MyVM.Mode.OPEN_NEW_TAP: + run(f"sudo ip tuntap add dev {self.tap_name} mode tap multi_qu= eue") + run(f"sudo ip link set {self.tap_name} up") + tap =3D self.tap_name + else: + tap =3D "tap0" + + self.add_tap_netdev( + tap, self.vhost, local_incoming=3D(self.mode =3D=3D MyVM.Mode.= NEW) + ) + + def pre_start_network_switch(self): + assert self.mode =3D=3D MyVM.Mode.OPEN_NEW_TAP + + a =3D time.time() + prev_tap =3D f"tap{1 - self.ind}" + run(f"sudo ip link set {self.tap_name} down") + run(f"sudo ip link set {prev_tap} down") + run(f"sudo ip addr delete 10.0.1.1/24 dev {prev_tap}") + run(f"sudo ip link set dev {self.tap_name} address e6:1d:44:b5:03:= 5d") + run(f"sudo ip addr add 10.0.1.1/24 dev {self.tap_name}") + run(f"sudo ip link set {self.tap_name} up") + b =3D time.time() + print("network switch:", b - a) + + def wait_migration_complete(self) -> bool: + while True: + event =3D self.event_wait("MIGRATION", timeout=3D1000) + print("source:", event) + assert event + if event["data"]["status"] =3D=3D "completed": + return True + if event["data"]["status"] =3D=3D "failed": + print("MIGRATION FAILED!") + print(self.cmd("query-migrate")) + return False + + def mig_cap(self): + if self.cpr: + self.cmd("migrate-set-parameters", {"mode": "cpr-transfer"}) + cap_list =3D ["events", "x-ignore-shared"] + if self.mode =3D=3D MyVM.Mode.NEW: + cap_list.append("local-tap") + caps =3D [{"capability": c, "state": True} for c in cap_list] + self.cmd("migrate-set-capabilities", {"capabilities": caps}) + + def migrate(self): + self.mig_cap() + if self.cpr: + self.cmd( + "migrate", + { + "channels": [ + { + "channel-type": "main", + "addr": { + "transport": "socket", + "type": "unix", + "path": "/tmp/migr.sock", + }, + }, + { + "channel-type": "cpr", + "addr": { + "transport": "socket", + "type": "unix", + "path": "/tmp/cpr.sock", + }, + }, + ] + }, + ) + else: + self.cmd("migrate", uri=3D"unix:/tmp/migr.sock") + + def live_update(self) -> Tuple["MyVM", float]: + ind =3D 1 - self.ind + target =3D MyVM( + binary=3Dself.qemu_binary, + ind=3Dind, + mode=3Dself.mode, + incoming=3DTrue, + vhost=3Dself.vhost, + ) + + if self.cpr: + print("launch target (cpr)") + target.launch(do_post_launch=3DFalse) + time.sleep(1) + + print("call migrate on source, will pass fds") + self.migrate() + + print("vm:", self.cmd("query-status"), self.cmd("query-migrate= ")) + print("post launch and qmp connect to target..") + target.launch(do_launch=3DFalse) + else: + print("launch target (usual)") + target.launch() + + target.setup_network_incoming(self.fds) + + target.mig_cap() + + if self.cpr: + freeze_start =3D time.time() + target.cmd("migrate-incoming", {"uri": "unix:/tmp/migr.sock"}) + else: + target.cmd("migrate-incoming", {"uri": "unix:/tmp/migr.sock"}) + freeze_start =3D time.time() + self.migrate() + + print("wait migration on source") + if not self.wait_migration_complete(): + target.shutdown() + raise MigrationFailed + + print("wait source STOP") + stop_event =3D self.event_wait("STOP", timeout=3D1000) + assert stop_event + print(stop_event) + + print("wait migration on target") + assert target.wait_migration_complete() + + result =3D self.qmp("query-status") + assert result["return"]["status"] =3D=3D "postmigrate" + + result =3D target.qmp("query-status") + assert result["return"]["status"] =3D=3D "paused" + + if self.mode =3D=3D MyVM.Mode.OPEN_NEW_TAP: + target.pre_start_network_switch() + + print("target CONT") + target.qmp("cont") + freeze_end =3D time.time() + + freeze_time =3D freeze_end - freeze_start + print("freeze-time: ", freeze_time) + + self.shutdown() + + if self.mode =3D=3D MyVM.Mode.OPEN_NEW_TAP: + del_tap(self.tap_name) + + print(target.cmd("query-version")) + print(target.cmd("query-status")) + return target, freeze_time + + +def main(): + # cleanup previous test runs + run("rm -rf /tmp/logdir", check=3DFalse) + run("mkdir /tmp/logdir", check=3DFalse) + run("killall qemu-system-x86_64", check=3DFalse) + + p =3D argparse.ArgumentParser() + p.add_argument("--qemu-binary", required=3DTrue) + p.add_argument("--vhost", action=3D"store_true") + p.add_argument("--mode", choices=3D[e.value for e in MyVM.Mode], requi= red=3DTrue) + args =3D p.parse_args() + + print("vhost:", args.vhost) + print("mode:", args.mode) + + vm =3D MyVM(binary=3Dargs.qemu_binary, mode=3DMyVM.Mode(args.mode), vh= ost=3Dargs.vhost) + vm.launch() + vm.setup_network_first_time() + vm.cmd("cont") + + while True: + cmd =3D input().strip() + if cmd =3D=3D "q": + break + + if cmd =3D=3D "s": + print(vm.cmd("query-status")) + vm.cmd("cont") + continue + + if cmd.startswith("u"): + spl =3D cmd.split() + assert len(spl) <=3D 2 + num =3D int(cmd.split(maxsplit=3D1)[1]) if len(spl) =3D=3D 2 e= lse 1 + total_freeze_time =3D 0 + try: + for i in range(num): + vm, freeze_time =3D vm.live_update() + print("DONE:", i) + total_freeze_time +=3D freeze_time + except MigrationFailed: + continue + + print(f"avg freeze-time: {total_freeze_time / num}") + continue + + vm.shutdown() + + +if __name__ =3D=3D "__main__": + main() --=20 2.48.1