From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405786; cv=none; d=zohomail.com; s=zohoarc; b=mGr5jv2lmAMjM2rIlRar1k9RwfjRtVb3oaNzQAsFgkrlOXMBIz232kOrXkCli2zBQ+y6q/gofhwcLwJLmOp12b9pJ2W/IOm91DK5aD6eU2Pllnc+6S47UXY/fanHhInPMhqtkVrOzGHGuJnbD0yiK4TpKcVaUXDPopq1lpUnJoY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405786; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AK8st6ZsSR9rdtKS3UXX81Tfh/+SrtFP6nIEbY2Os24=; b=Psb2YcPrR2GAGO+8xZZ3TK3aN9fsgB3htG49QBTs186j8FcLkagOi0px+CqsKkQ1QJW9eCG4ivT6osGhdqCSADrp+ZU+7Pa7cqJz3z2InVccMt0+FiHimLHNvSBTK50g7fuXnb2xFdokW7E+l3a1XbZjs79R3H9SAn05hJ4PiAs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178340578608337.890654536817465; Mon, 6 Jul 2026 23:29:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJG-00010O-K7; Tue, 07 Jul 2026 02:29:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJF-0000zo-0V for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJD-0006SU-71 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:32 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-395-qQ8K3XBANVyYacDrBNL9NQ-1; Tue, 07 Jul 2026 02:29:29 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 150031955DB7 for ; Tue, 7 Jul 2026 06:29:28 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D6A151800678; Tue, 7 Jul 2026 06:29:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405770; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AK8st6ZsSR9rdtKS3UXX81Tfh/+SrtFP6nIEbY2Os24=; b=ZlgyI7EaZcGqkqURKEFLwHD/qAeMr3TYO9Ag2vUIX2TnUj8UyovtPYx8E02toQIfUfSk3V CDaM26/BaWjWtYeYxiMvnfitIqfTMwte7hPvfQt8Cr6LCWhFckXlgCQK8sMlytblDwHcQr 0VgIETxkWZwxbOY2JY2Omf86iGU6j54= X-MC-Unique: qQ8K3XBANVyYacDrBNL9NQ-1 X-Mimecast-MFC-AGG-ID: qQ8K3XBANVyYacDrBNL9NQ_1783405768 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Mario Casquero , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 01/27] vfio/pci: Initialize rom_read_failed in vfio_pci_load_rom() Date: Tue, 7 Jul 2026 08:28:54 +0200 Message-ID: <20260707062920.317302-2-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405787863158500 From: Mario Casquero When vfio_device_get_region_info() fails in vfio_pci_load_rom(), the function returns without setting vdev->rom_read_failed to true, and without allocating vdev->rom. This leaves vdev->rom as NULL. Signed-off-by: Mario Casquero Reviewed-by: C=C3=A9dric Le Goater Link: https://lore.kernel.org/qemu-devel/20260602070857.356526-1-mcasquer@r= edhat.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/pci.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 4d822b96b530c614671551589ce3584e9598f386..68ee3ce7dc254dd045ed2c1b21b= ebc53a4b4e3d6 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1028,7 +1028,7 @@ static void vfio_update_msi(VFIOPCIDevice *vdev) } } =20 -static void vfio_pci_load_rom(VFIOPCIDevice *vdev) +static bool vfio_pci_load_rom(VFIOPCIDevice *vdev) { VFIODevice *vbasedev =3D &vdev->vbasedev; struct vfio_region_info *reg_info =3D NULL; @@ -1042,7 +1042,7 @@ static void vfio_pci_load_rom(VFIOPCIDevice *vdev) =20 if (ret !=3D 0) { error_report("vfio: Error getting ROM info: %s", strerror(-ret)); - return; + return false; } =20 trace_vfio_pci_load_rom(vbasedev->name, (unsigned long)reg_info->size, @@ -1053,12 +1053,11 @@ static void vfio_pci_load_rom(VFIOPCIDevice *vdev) vdev->rom_offset =3D reg_info->offset; =20 if (!vdev->rom_size) { - vdev->rom_read_failed =3D true; error_report("vfio-pci: Cannot read device rom at %s", vbasedev->n= ame); error_printf("Device option ROM contents are probably invalid " "(check dmesg).\nSkip option ROM probe with rombar=3D0= , " "or load from file with romfile=3D\n"); - return; + return false; } =20 vdev->rom =3D g_malloc(size); @@ -1114,6 +1113,8 @@ static void vfio_pci_load_rom(VFIOPCIDevice *vdev) data[6] =3D -csum; } } + + return true; } =20 /* "Raw" read of underlying config space. */ @@ -1147,7 +1148,7 @@ static uint64_t vfio_rom_read(void *opaque, hwaddr ad= dr, unsigned size) =20 /* Load the ROM lazily when the guest tries to read it */ if (unlikely(!vdev->rom && !vdev->rom_read_failed)) { - vfio_pci_load_rom(vdev); + vdev->rom_read_failed =3D !vfio_pci_load_rom(vdev); } =20 memcpy(&val, vdev->rom + addr, --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405901; cv=none; d=zohomail.com; s=zohoarc; b=AMM4Ftzsmf2LJYlleAVagGSvtrebPEJXlIfkP80yBc/mX8osE1D3EOmgzJHobduI89n+l1pAycv1b4f9LAuog4YR2n5hIPyELiT9wA1uyEDE2+1EmcWPEYcUaOtECH9XX+asBpqBbxbRwdUak3rCMIyYgSSHG6aC3fJIIXCsm2k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405901; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QIfe4KaSAWiFYGxBo0mP8MsVGGEQeTnzwnevk33Axp8=; b=bqYam4HZTMnAT8IO0H8v3ihVyCC47AvYscd7rLMWdz/dEErBpIq2rx02PNYp5ZQ/HbGsc60nN3AVdW1l9o+S9dyh25VbuaWPhxiMIHxN8XsxFH/mmQmD8B114JdAnuD9JN+1pPuTgZfmMPw8lcfMzYVHy4i5qZ9Y8DVpv1eFTvw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405901592849.0686102860612; Mon, 6 Jul 2026 23:31:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJN-00015d-3z; Tue, 07 Jul 2026 02:29:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJJ-00013P-Eb for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJH-0006a7-Vf for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:37 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-D4jYvKRxMImkg0TukPvtmw-1; Tue, 07 Jul 2026 02:29:31 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 58F001801311; Tue, 7 Jul 2026 06:29:30 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B0A57180065F; Tue, 7 Jul 2026 06:29:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QIfe4KaSAWiFYGxBo0mP8MsVGGEQeTnzwnevk33Axp8=; b=F4HHs3Lh+zsdvmcllGvO6nRMnoO2nWv5n+BTpe2td/Reh6ZVnxQHtODULP1qNfi+/371hk Wj9KYfuc+Jg5pSexkpGpPz9Mt9i43ZGzaUuVL7ea5huU9dAwtQeZWN0uY5kLwJpnyMvPtm NUb5CJbzywsYO4b4SdDGKnxTpxtRFx0= X-MC-Unique: D4jYvKRxMImkg0TukPvtmw-1 X-Mimecast-MFC-AGG-ID: D4jYvKRxMImkg0TukPvtmw_1783405770 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Mario Casquero Subject: [PULL 02/27] vfio/pci: Fix information leak in vfio_rom_read() Date: Tue, 7 Jul 2026 08:28:55 +0200 Message-ID: <20260707062920.317302-3-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405902681158500 vfio_rom_read() can leak uninitialized stack data to the guest when a read occurs at the end boundary of the ROM BAR. Fix this by initializing the val union to 0xff, matching the ROM initialization pattern used in vfio_pci_load_rom(). Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Mario Casquero Link: https://lore.kernel.org/qemu-devel/20260602074446.1206892-1-clg@redha= t.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 68ee3ce7dc254dd045ed2c1b21bebc53a4b4e3d6..6b04c6f94f9ade765e236d457e4= 828ac6199680a 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1143,7 +1143,7 @@ static uint64_t vfio_rom_read(void *opaque, hwaddr ad= dr, unsigned size) uint16_t word; uint32_t dword; uint64_t qword; - } val; + } val =3D { .qword =3D ~0ULL }; uint64_t data =3D 0; =20 /* Load the ROM lazily when the guest tries to read it */ --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405911; cv=none; d=zohomail.com; s=zohoarc; b=QCbM0DogZh4r8AY0CUeJ9gShM7rL2A+GqsdvkS+cLu/wgMzjJC5dsYgtpCaHa8zvr93dr3GG9zg+293kR/Weh0MZVgXiNUlGtnG6GZr84QXzgvcYU3beUugPlXmcGwB/YNNqSPWY95ftdIdELDNwiWlEP2ba88PRZxU9ge7eURU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405911; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tjbNF6RYtLwlGy7eCg9LgEfJgJ+JbZZyoXxcb8uVLPM=; b=YaGHYdmPhIylTKJVKuKxTYeUXX3PCUyeljo6EZObjjmmzZx5oS9hMEI2WD3h4CJqYZflw0j2qhNoHYie35Q0wUCymUoFQFmYsHDinc0xOLfaPfb6SrVAnvZjqWrGC3byaQklkrcjZsetQgVelQXfsLcU08V8BHzlZ2NCAxs3GbM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405911263789.3120663139678; Mon, 6 Jul 2026 23:31:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJO-00016K-Ou; Tue, 07 Jul 2026 02:29:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJL-000154-DE for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJJ-0006aO-IR for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:39 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-10-79i7G_h8Nym7ZGKDa6coPw-1; Tue, 07 Jul 2026 02:29:33 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 722821955BFE; Tue, 7 Jul 2026 06:29:32 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 058E2180065F; Tue, 7 Jul 2026 06:29:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405777; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tjbNF6RYtLwlGy7eCg9LgEfJgJ+JbZZyoXxcb8uVLPM=; b=XZcbaBoNsJOeB0Y+1+2tPsAd5xyQ4Jqrg8NmzqGMXA+Z2gEqGHGLaMwfRHYo4Tg3yh9D4V 1G0fxB09e1g8bvp++mU0a+bEVPOySP8SfDyYJc1Xm+HJC5QCo//rna7QGI85IQThv36kNd ZyRrW4QF7HCCSAW66a6V/E6EwyYzd44= X-MC-Unique: 79i7G_h8Nym7ZGKDa6coPw-1 X-Mimecast-MFC-AGG-ID: 79i7G_h8Nym7ZGKDa6coPw_1783405772 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Mattias Nissler , John Levon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 03/27] docs: Update vfio-user spec to describe DMA access mode bits Date: Tue, 7 Jul 2026 08:28:56 +0200 Message-ID: <20260707062920.317302-4-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, TVD_PH_SUBJ_META1=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405912671158500 From: Mattias Nissler This makes the intended access mode explicit when registering DMA regions with the server. A new "file I/O" access mode is defined, which can be used if the file descriptor provided by the client doesn't support `mmap()`. Signed-off-by: Mattias Nissler Reviewed-by: John Levon Link: https://lore.kernel.org/qemu-devel/20260602133829.305842-1-mnissler@m= eta.com Signed-off-by: C=C3=A9dric Le Goater --- docs/interop/vfio-user.rst | 82 ++++++++++++++++++++++++-------------- 1 file changed, 51 insertions(+), 31 deletions(-) diff --git a/docs/interop/vfio-user.rst b/docs/interop/vfio-user.rst index 12deb25102af217f7f292b4cb655dc4a6a92439a..1c92f442d232ce6ce880868bb70= 62e42481b69ae 100644 --- a/docs/interop/vfio-user.rst +++ b/docs/interop/vfio-user.rst @@ -604,27 +604,31 @@ Request =20 The request payload for this message is a structure of the following forma= t: =20 -+-------------+--------+-------------+ -| Name | Offset | Size | -+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ -| argsz | 0 | 4 | -+-------------+--------+-------------+ -| flags | 4 | 4 | -+-------------+--------+-------------+ -| | +-----+------------+ | -| | | Bit | Definition | | -| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | -| | | 0 | readable | | -| | +-----+------------+ | -| | | 1 | writeable | | -| | +-----+------------+ | -+-------------+--------+-------------+ -| offset | 8 | 8 | -+-------------+--------+-------------+ -| address | 16 | 8 | -+-------------+--------+-------------+ -| size | 24 | 8 | -+-------------+--------+-------------+ ++-------------+--------+------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------------------------+ +| flags | 4 | 4 | ++-------------+--------+------------------------+ +| | +-----+-----------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | readable | | +| | +-----+-----------------------+ | +| | | 1 | writeable | | +| | +-----+-----------------------+ | +| | | 2 | access mode: mmap | | +| | +-----+-----------------------+ | +| | | 3 | access mode: file I/O | | +| | +-----+-----------------------+ | ++-------------+--------+------------------------+ +| offset | 8 | 8 | ++-------------+--------+------------------------+ +| address | 16 | 8 | ++-------------+--------+------------------------+ +| size | 24 | 8 | ++-------------+--------+------------------------+ =20 * *argsz* is the size of the above structure. Note there is no reply paylo= ad, so this field differs from other message types. @@ -634,6 +638,8 @@ The request payload for this message is a structure of = the following format: =20 * *writeable* indicates that the region can be written to. =20 + * *access mode* bits indicate how the region is to be accessed by the se= rver. + * *offset* is the file offset of the region with respect to the associated= file descriptor, or zero if the region is not mappable * *address* is the base DMA address of the region. @@ -641,16 +647,30 @@ The request payload for this message is a structure o= f the following format: =20 This structure is 32 bytes in size, so the message size is 16 + 32 bytes. =20 -If the DMA region being added can be directly mapped by the server, a file -descriptor must be sent as part of the message meta-data. The region can be -mapped via the mmap() system call. On ``AF_UNIX`` sockets, the file descri= ptor -must be passed as ``SCM_RIGHTS`` type ancillary data. Otherwise, if the D= MA -region cannot be directly mapped by the server, no file descriptor must be= sent -as part of the message meta-data and the DMA region can be accessed by the -server using ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages, -explained in `Read and Write Operations`_. A command to map over an existi= ng -region must be failed by the server with ``EEXIST`` set in error field in = the -reply. +There are several alternative access modes for the server to use when acce= ssing +the region: + +* ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages, explained in + `Read and Write Operations`_. + +* ``mmap()`` a client-provided file descriptor, then perform direct access= es to + the underlying memory. + +* File I/O system calls (such as ``pread()`` / ``pwrite()``) against a + client-provided file descriptor. + +The access mode bits in the flags field indicate which access mode to use.= If +an access mode requiring a file descriptor is specified, but the client do= es +not provide a file descriptor, the server must fail the request with +``EINVAL``. If no access mode flag bit is set, the server should use ``mma= p()`` +based access if the client provided a file descriptor and message-based ac= cess +otherwise. + +On ``AF_UNIX`` sockets, the file descriptor must be passed as ``SCM_RIGHTS= `` +type ancillary data. + +A command to map over an existing region must be failed by the server with +``EEXIST`` set in the error field in the reply. =20 Reply ^^^^^ --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405913; cv=none; d=zohomail.com; s=zohoarc; b=Om5iypzJHVonvLjsINxjulEdALaCnJB6PcOsIE44sJFO2P4qHTEBFin1Spm22q+YLjVOAuTzCfKrtiX/7W0B/PCGP6faVL3mCh/lxkh77xejqDnVRIDwQ5l4hZdljZRuD81bPkw8+s8Ut+r1rxdU+amuUJli1hfmgw1FSkvYV1A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405913; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xvU+UT8RJH8CNRGV8KgYl6yHDkqxJ9spzV3Le1jKkow=; b=L3DVwBqQ+O9LJ6nU76t5chcC4WLuw03c6QnKDApiuDzVQKc1F/mhZWL2HttfxAgkYjMlBz3VxFneysXjzM7siPXUDqZQ631LW5X1GDhR1MmmtDmsHC0JiFV/x8aaTOOVcBE1auDZMPMPaCyuwPN8bOaTU0aQsPNw++5ReOVD/vY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405913497262.4935089808944; Mon, 6 Jul 2026 23:31:53 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJN-00015h-Bp; Tue, 07 Jul 2026 02:29:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJL-000150-D5 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJJ-0006aU-VD for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:39 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-690-OA8zarmwOn6BzsiEAaVYWA-1; Tue, 07 Jul 2026 02:29:35 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id ADDF81954B1F; Tue, 7 Jul 2026 06:29:34 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E1BED180065F; Tue, 7 Jul 2026 06:29:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405777; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xvU+UT8RJH8CNRGV8KgYl6yHDkqxJ9spzV3Le1jKkow=; b=UH7T2yzMzWikdTzlv2EH/YNCwuBuN8k6c12IsmV9/ZTNld94l7Gu4H7sQmjwRoQYksayPo utiL3U4+0HjCL8ktGb/GLpD8lxynNvNbuvzdtTSVghGndH51cwxUuF7HFFmeVVwYP8++Wn GYHRS1LGLZ+OACriPKVwXLYoVJbNnMc= X-MC-Unique: OA8zarmwOn6BzsiEAaVYWA-1 X-Mimecast-MFC-AGG-ID: OA8zarmwOn6BzsiEAaVYWA_1783405775 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: GuoHan Zhao , John Levon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 04/27] vfio-user: validate VERSION replies Date: Tue, 7 Jul 2026 08:28:57 +0200 Message-ID: <20260707062920.317302-5-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405914802158500 From: GuoHan Zhao The vfio-user protocol makes the VERSION payload optional, so a reply may legally stop after the major and minor fields. vfio_user_validate_version() currently assumes a capabilities string is always present and NUL-terminated. When the server replies without version data, QEMU ends up reusing the request-side capabilities buffer and the terminating-NUL check underflows. Replies shorter than the fixed VERSION header are also accessed before they are validated. Reject replies shorter than the fixed VERSION header and only parse capabilities when the reply actually carries version data. Fixes: 36227628d824 (vfio-user: implement message send infrastructure) Signed-off-by: GuoHan Zhao Reviewed-by: John Levon Link: https://lore.kernel.org/qemu-devel/20260603062138.4008583-1-zhaoguoha= n@kylinos.cn Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio-user/proxy.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 0f7d8425d61488f6af3747db4b9f5c774f3fc717..197aee07bf7a3e16b58f77b0ecb= 73bff46889ecb 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -1292,7 +1292,7 @@ bool vfio_user_validate_version(VFIOUserProxy *proxy,= Error **errp) { g_autofree VFIOUserVersion *msgp =3D NULL; GString *caps; - char *reply; + const char *reply =3D ""; int size, caplen; =20 caps =3D caps_json(); @@ -1322,17 +1322,24 @@ bool vfio_user_validate_version(VFIOUserProxy *prox= y, Error **errp) return false; } =20 - reply =3D msgp->capabilities; - if (reply[msgp->hdr.size - sizeof(*msgp) - 1] !=3D '\0') { - error_setg(errp, "corrupt version reply"); + if (msgp->hdr.size < sizeof(*msgp)) { + error_setg(errp, "short version reply"); return false; } =20 - if (!caps_check(proxy, msgp->minor, reply, errp)) { - return false; + if (msgp->hdr.size > sizeof(*msgp)) { + reply =3D msgp->capabilities; + if (reply[msgp->hdr.size - sizeof(*msgp) - 1] !=3D '\0') { + error_setg(errp, "corrupt version reply"); + return false; + } + + if (!caps_check(proxy, msgp->minor, reply, errp)) { + return false; + } } =20 - trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + trace_vfio_user_version(msgp->major, msgp->minor, reply); return true; } =20 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405816; cv=none; d=zohomail.com; s=zohoarc; b=FGOW07ztFuK46dryD+s1/Ecgy4LcdfRVODWNMYUWUwOcB/k4OSqPLau26iXn8Skug/KXWnTi1rTyZZgeaQTRz8eFLOGW9rZzqFnsAODWOAfeIjXJreAfHFJ+HJHh01bLLY12Og+Zgbud2HjHBZ8BOfemcyqfoNuZ+fyxMiLK+K0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405816; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=8k+sRBfo2hWcFjuBWsB2P9BAWhZ3y8I90k3sIy8zXso=; b=aAxJQBKOg5eyHxMD+oMdPFpZFVl/S59XWWeZvk1L6VowprohPtboWVyvhjtVOebFZ6dh7zrFhvhcIG/ajgp1WAxPILldPL73pcd68TaKMf3Cem8k9NOB9Naw+8QoAujyyX4m9DQWJbiTNcMw6Fryomp4IRRUfv2MS7w5Z3o7M6M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405816655567.5488575290061; Mon, 6 Jul 2026 23:30:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJQ-00016i-5u; Tue, 07 Jul 2026 02:29:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJO-00016I-LL for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJM-0006ae-PF for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:42 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-80-Thn8r-kgNri3zXzxo6d3Tg-1; Tue, 07 Jul 2026 02:29:37 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A42B418011D1; Tue, 7 Jul 2026 06:29:36 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 46FFA180065F; Tue, 7 Jul 2026 06:29:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405779; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8k+sRBfo2hWcFjuBWsB2P9BAWhZ3y8I90k3sIy8zXso=; b=XHhOZ2KVDnyoPEsc32Q2tnVMT8naVqMl60ixjfPIcLGcKMaGXu07crDkSAI6fPQrQMoBk3 BNQ61KAKX7LER1seVmmlJvTw/8g6xvvHsE6KB7pjWqFIIxy8zNAHHbUzE4btfi+jloByXw 5G3JNWbfbCgbL47Shr2Be2GkOe/h8BU= X-MC-Unique: Thn8r-kgNri3zXzxo6d3Tg-1 X-Mimecast-MFC-AGG-ID: Thn8r-kgNri3zXzxo6d3Tg_1783405776 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Alex Williamson Subject: [PULL 05/27] vfio/iommufd: Merge .dma_map_file() into .dma_map() Date: Tue, 7 Jul 2026 08:28:58 +0200 Message-ID: <20260707062920.317302-6-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405818361158500 Simplify the VFIOIOMMUClass interface by removing the dma_map_file handler. Move the logic to decide between the standard virtual and file-backed mapping into the IOMMUFD backend, utilizing the MemoryRegion already passed to the dma_map handler. This removes redundant dispatch logic from the generic container layer and let backends to manage their own mapping strategies. This is similar to the vfio-user implementation. Reviewed-by: Alex Williamson Link: https://lore.kernel.org/qemu-devel/20260608055758.359002-1-clg@redhat= .com Signed-off-by: C=C3=A9dric Le Goater --- include/hw/vfio/vfio-container.h | 15 ---------- hw/vfio/container.c | 38 ------------------------ hw/vfio/iommufd.c | 50 +++++++++++++++++++++++--------- 3 files changed, 37 insertions(+), 66 deletions(-) diff --git a/include/hw/vfio/vfio-container.h b/include/hw/vfio/vfio-contai= ner.h index b2e7f4312c31ed4b776ddb195b021e84ebed556f..a15ee2df2b9c82d7c5004685aa9= 1c3339587e06c 100644 --- a/include/hw/vfio/vfio-container.h +++ b/include/hw/vfio/vfio-container.h @@ -172,21 +172,6 @@ struct VFIOIOMMUClass { int (*dma_map)(const VFIOContainer *bcontainer, hwaddr iova, uint64_t size, void *vaddr, bool readonly, MemoryRegion *mr); - /** - * @dma_map_file - * - * Map a file range for the container. - * - * @bcontainer: #VFIOContainer to use for map - * @iova: start address to map - * @size: size of the range to map - * @fd: descriptor of the file to map - * @start: starting file offset of the range to map - * @readonly: map read only if true - */ - int (*dma_map_file)(const VFIOContainer *bcontainer, - hwaddr iova, uint64_t size, - int fd, unsigned long start, bool readonly); /** * @dma_unmap * diff --git a/hw/vfio/container.c b/hw/vfio/container.c index 56bd9ac0095df7b0f4da85f1d8dcb8d571fa9f2a..d09a6637324c6bf29b2fe223690= 82f87e280c98f 100644 --- a/hw/vfio/container.c +++ b/hw/vfio/container.c @@ -15,7 +15,6 @@ #include =20 #include "system/tcg.h" -#include "system/ramblock.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "hw/vfio/vfio-container.h" @@ -74,49 +73,12 @@ void vfio_address_space_insert(VFIOAddressSpace *space, bcontainer->space =3D space; } =20 -static bool vfio_container_can_dma_map_file(VFIOContainer *bcontainer, - MemoryRegion *mr, int *fd) -{ - VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); - RAMBlock *rb =3D mr->ram_block; - - if (!vioc->dma_map_file || !rb) { - return false; - } - - *fd =3D qemu_ram_get_fd(rb); - if (*fd < 0) { - return false; - } - - /* - * We can use IOMMU DMA mapping (IOMMU_IOAS_MAP_FILE) for : - * - * 1) Guest RAM blocks explicitly configured as shared (MAP_SHARED) - * 2) RAM device sub-regions (MMIO BARs) - * - * Private RAM mappings (MAP_PRIVATE) are strictly excluded. Because - * they are subject to copy-on-write (COW) anomalies, their underlying - * PFNs can permanently diverge from the backing file - */ - return qemu_ram_is_shared(rb) || memory_region_is_ram_device(mr); -} - int vfio_container_dma_map(VFIOContainer *bcontainer, hwaddr iova, uint64_t size, void *vaddr, bool readonly, MemoryRegion *mr) { VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); - int mfd; =20 - if (vfio_container_can_dma_map_file(bcontainer, mr, &mfd)) { - RAMBlock *rb =3D mr->ram_block; - unsigned long start =3D vaddr - qemu_ram_get_host_addr(rb); - unsigned long offset =3D qemu_ram_get_fd_offset(rb); - - return vioc->dma_map_file(bcontainer, iova, size, mfd, start + off= set, - readonly); - } g_assert(vioc->dma_map); return vioc->dma_map(bcontainer, iova, size, vaddr, readonly, mr); } diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c index 68f2ae6f9f667bb05c20e452e8c5b43da6558098..6ff668d2597e07c6953cf0f1997= 514aec199642c 100644 --- a/hw/vfio/iommufd.c +++ b/hw/vfio/iommufd.c @@ -20,6 +20,7 @@ #include "trace.h" #include "qapi/error.h" #include "system/iommufd.h" +#include "system/ramblock.h" #include "hw/core/iommu.h" #include "hw/core/qdev.h" #include "hw/vfio/vfio-cpr.h" @@ -35,26 +36,50 @@ #define TYPE_HOST_IOMMU_DEVICE_IOMMUFD_VFIO \ TYPE_HOST_IOMMU_DEVICE_IOMMUFD "-vfio" =20 +static bool iommufd_cdev_can_map_file_dma(MemoryRegion *mr, int *fd) +{ + RAMBlock *rb =3D mr ? mr->ram_block : NULL; + + if (!rb) { + return false; + } + + *fd =3D qemu_ram_get_fd(rb); + if (*fd < 0) { + return false; + } + + /* + * Use iommufd_backend_map_file_dma() (IOMMU_IOAS_MAP_FILE) for: + * 1) Guest RAM blocks explicitly configured as shared (MAP_SHARED) + * 2) RAM device sub-regions (MMIO BARs) + * + * Private RAM mappings (MAP_PRIVATE) are excluded: copy-on-write + * semantics can cause their underlying PFNs to permanently diverge + * from the backing file. + */ + return qemu_ram_is_shared(rb) || memory_region_is_ram_device(mr); +} + static int iommufd_cdev_map(const VFIOContainer *bcontainer, hwaddr iova, uint64_t size, void *vaddr, bool readonly, MemoryRegion *mr) { const VFIOIOMMUFDContainer *container =3D VFIO_IOMMU_IOMMUFD(bcontaine= r); + int fd; =20 - return iommufd_backend_map_dma(container->be, - container->ioas_id, - iova, size, vaddr, readonly); -} + if (iommufd_cdev_can_map_file_dma(mr, &fd)) { + RAMBlock *rb =3D mr->ram_block; + unsigned long start =3D vaddr - qemu_ram_get_host_addr(rb); + unsigned long offset =3D qemu_ram_get_fd_offset(rb); =20 -static int iommufd_cdev_map_file(const VFIOContainer *bcontainer, - hwaddr iova, uint64_t size, - int fd, unsigned long start, bool readonl= y) -{ - const VFIOIOMMUFDContainer *container =3D VFIO_IOMMU_IOMMUFD(bcontaine= r); + return iommufd_backend_map_file_dma(container->be, container->ioas= _id, + iova, size, fd, + start + offset, readonly); + } =20 - return iommufd_backend_map_file_dma(container->be, - container->ioas_id, - iova, size, fd, start, readonly); + return iommufd_backend_map_dma(container->be, container->ioas_id, + iova, size, vaddr, readonly); } =20 static int iommufd_cdev_unmap(const VFIOContainer *bcontainer, @@ -929,7 +954,6 @@ static void vfio_iommu_iommufd_class_init(ObjectClass *= klass, const void *data) VFIOIOMMUClass *vioc =3D VFIO_IOMMU_CLASS(klass); =20 vioc->dma_map =3D iommufd_cdev_map; - vioc->dma_map_file =3D iommufd_cdev_map_file; vioc->dma_unmap =3D iommufd_cdev_unmap; vioc->attach_device =3D iommufd_cdev_attach; vioc->detach_device =3D iommufd_cdev_detach; --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405907; cv=none; d=zohomail.com; s=zohoarc; b=B1UIqAc5bR+XFNu4SUjrECw3f+qSaUNa04VbVw9nbRW1Bkawo1uFR6b8hzCZSleMAVs7ADxEoc2pMaeRaria7x9M4WvmNoLG8n8VT9wDcVsPCbxXgBnqYWbyKmSzz5R9Jm0wFQt2lHeBQoDhAxa0oDRDYGKplGAsXHx7EgmDUKQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405907; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=npsqq4Bw+tGFasRkCLKdQlk2D7kcYHLMY3VKU8/dnj8=; b=GopUBf+zcR225LrURolgmHE9LhrAXR8ea6A/1mMhT0rxGDPjI9VuhKYKVvEQnmol0lVqkQc3XPzRom3K+bL18T089sgp7uj9q4/cj1kR3dt1rlrK6ieWarmJI8aVakv7f1/fOyZjk0AFeNb1XgjtuwxigIJidQrzosrB6RIOQTM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405907375399.06520437262134; Mon, 6 Jul 2026 23:31:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJi-0001Kw-5f; Tue, 07 Jul 2026 02:30:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJV-0001AL-O8 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJU-0006iY-2o for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:49 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-352-Mdi-DAOWNeiVnblXIOOELA-1; Tue, 07 Jul 2026 02:29:41 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3966C1954B07; Tue, 7 Jul 2026 06:29:39 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3D733180065F; Tue, 7 Jul 2026 06:29:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=npsqq4Bw+tGFasRkCLKdQlk2D7kcYHLMY3VKU8/dnj8=; b=Tv44swsW2y1be7NV22DLYBQZC1z8YHY4AQH3RYFMk/eqbRDW4nDt6XCatFJlpIZ+fw4H5s JLUsMvfxskgF7tbOX7EFRdvFnCEkkLRQpKC561dFWNExzgrgfC5l56MFioh5MqXjmkvZmP zRwzYov42nQtBeB3qiHyzi7pl1Ob0t0= X-MC-Unique: Mdi-DAOWNeiVnblXIOOELA-1 X-Mimecast-MFC-AGG-ID: Mdi-DAOWNeiVnblXIOOELA_1783405779 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 06/27] migration: Propagate errors in migration_completion_precopy() Date: Tue, 7 Jul 2026 08:28:59 +0200 Message-ID: <20260707062920.317302-7-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405908657158500 From: Avihai Horon migration_completion_precopy() doesn't propagate errors to migration core which leads to error information loss. Fix that. This prepares for a follow-up where migration_switchover_start() can fail on switchover-ack and still report a useful error. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-2-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- migration/savevm.h | 2 +- migration/migration.c | 13 ++++++++----- migration/savevm.c | 29 +++++++++++++++++------------ 3 files changed, 26 insertions(+), 18 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index 96fdf96d4ef6f7fd174f50e8c1402f0b6d6b9e1f..b6bb4fa977fae84ed121b0668cd= efdf49e3d9c67 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -44,7 +44,7 @@ void qemu_savevm_state_header(QEMUFile *f); int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); -int qemu_savevm_state_complete_precopy(MigrationState *s); +int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); void qemu_savevm_query_pending(MigPendingData *pending, bool exact); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); diff --git a/migration/migration.c b/migration/migration.c index 0f56432d903054883679a66ac5bd853577b5df66..7e0a0bea7ae823ee5ef6e7ba887= 6c9222f9ac834 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2820,7 +2820,7 @@ static bool migration_switchover_start(MigrationState= *s, Error **errp) return true; } =20 -static int migration_completion_precopy(MigrationState *s) +static int migration_completion_precopy(MigrationState *s, Error **errp) { int ret; =20 @@ -2829,16 +2829,17 @@ static int migration_completion_precopy(MigrationSt= ate *s) if (!migrate_mode_is_cpr()) { ret =3D migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to stop the VM"); goto out_unlock; } } =20 - if (!migration_switchover_start(s, NULL)) { + if (!migration_switchover_start(s, errp)) { ret =3D -EFAULT; goto out_unlock; } =20 - ret =3D qemu_savevm_state_complete_precopy(s); + ret =3D qemu_savevm_state_complete_precopy(s, errp); out_unlock: bql_unlock(); return ret; @@ -2875,7 +2876,7 @@ static void migration_completion(MigrationState *s) Error *local_err =3D NULL; =20 if (s->state =3D=3D MIGRATION_STATUS_ACTIVE) { - ret =3D migration_completion_precopy(s); + ret =3D migration_completion_precopy(s, &local_err); } else if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE) { migration_completion_postcopy(s); } else { @@ -2906,7 +2907,9 @@ static void migration_completion(MigrationState *s) return; =20 fail: - if (qemu_file_get_error_obj(s->to_dst_file, &local_err)) { + if (local_err) { + migrate_error_propagate(s, local_err); + } else if (qemu_file_get_error_obj(s->to_dst_file, &local_err)) { migrate_error_propagate(s, local_err); } else if (ret) { error_setg_errno(&local_err, -ret, "Error in migration completion"= ); diff --git a/migration/savevm.c b/migration/savevm.c index 23adaf9dd9263b8dafe808f0637724fe4a5482b2..7e2a0c2b4a7470d9c2159eac1d1= 8ae891740fe3e 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1771,28 +1771,34 @@ int qemu_savevm_state_non_iterable(QEMUFile *f, Err= or **errp) return 0; } =20 -int qemu_savevm_state_complete_precopy(MigrationState *s) +int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp) { + ERRP_GUARD(); QEMUFile *f =3D s->to_dst_file; - Error *local_err =3D NULL; int ret; =20 ret =3D qemu_savevm_state_complete_precopy_iterable(f, false); if (ret) { + qemu_file_get_error_obj(f, errp); + error_prepend(errp, "Failed to save iterable device state: "); return ret; } =20 - /* TODO: pass error upper */ - ret =3D qemu_savevm_state_non_iterable(f, &local_err); + ret =3D qemu_savevm_state_non_iterable(f, errp); if (ret) { - migrate_error_propagate(s, error_copy(local_err)); - error_report_err(local_err); return ret; } =20 qemu_savevm_state_end_precopy(s, f); =20 - return qemu_fflush(f); + ret =3D qemu_fflush(f); + if (ret) { + qemu_file_get_error_obj(f, errp); + error_prepend(errp, "Failed to flush QEMUFile: "); + return ret; + } + + return 0; } =20 void qemu_savevm_query_pending(MigPendingData *pending, bool exact) @@ -1874,13 +1880,12 @@ static int qemu_savevm_state(QEMUFile *f, Error **e= rrp) } =20 ret =3D qemu_file_get_error(f); - if (ret =3D=3D 0) { - qemu_savevm_state_complete_precopy(ms); - ret =3D qemu_file_get_error(f); - } - if (ret !=3D 0) { + if (ret) { error_setg_errno(errp, -ret, "Error while writing VM state"); + goto cleanup; } + + ret =3D qemu_savevm_state_complete_precopy(ms, errp); cleanup: qemu_savevm_state_cleanup(); =20 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405801; cv=none; d=zohomail.com; s=zohoarc; b=HvUMNRE3T7b8DJAA4sHi5X4YBOXjRjvnJ8OAc7zi9fEyqnVyXD28M/6YM9ShbVn4T5p5uCmbqoZMYRtDS0WtvTPaxGFLn3dZblDjAYoA2GkvLBPfzNZvjTDg+nrHMjOPjEVgbd4VaqdyeMxJ2jMDhlX4d59APtnzXxRaCUqTH5w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405801; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=K2GeVWVj42TRRgoCrVjZPsNMY1WIblmTifASA+IvYbY=; b=aDoaqy/pYe0RrwMNuJLxDfWJ2ZE9u4TaHPpO5wSJYluda9c1lpe0Ub1CNatTC/vL05TftQbzYH1frPUomEJNJLVMdJniuI45E4MfQ6hvKGmlWmvGjWvkscQwML6/JDElLVyobG95XpunVx82j+hcrSphbiZaG7RMZjWAIFOy9xE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17834058012421010.5869204873269; Mon, 6 Jul 2026 23:30:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJZ-0001AC-UC; Tue, 07 Jul 2026 02:29:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJT-00019v-8X for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJR-0006en-Q4 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:47 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-206-fYVcaIusMRyV-oE-yF5OMg-1; Tue, 07 Jul 2026 02:29:41 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0C14F1955D48; Tue, 7 Jul 2026 06:29:41 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 93A821800678; Tue, 7 Jul 2026 06:29:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405785; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K2GeVWVj42TRRgoCrVjZPsNMY1WIblmTifASA+IvYbY=; b=bireWkLw+y0vCmB3RV0S0mpuzDkiSqRcMoZHmiFhoPl4z9rkBhDVbm5Y2QAO7F2Wbj/Ar6 ySeKWPzKlXRDC3luFsUJhSWw99CgxqhjAr+TldNd+ZXUu0m3ZPlHs0uyqaA6TTALn7WXqs Cd05zSB7yY/KE/MYU+4psU/DPnl44X8= X-MC-Unique: fYVcaIusMRyV-oE-yF5OMg-1 X-Mimecast-MFC-AGG-ID: fYVcaIusMRyV-oE-yF5OMg_1783405781 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 07/27] migration/ram: Use migration_bitmap_sync_precopy() for postcopy discard Date: Tue, 7 Jul 2026 08:29:00 +0200 Message-ID: <20260707062920.317302-8-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405802027158500 From: Avihai Horon ram_postcopy_send_discard_bitmap() performs the final RAM dirty bitmap sync at postcopy switchover, before sending discard bitmap to the destination. Unlike the precopy switchover path, it currently calls the raw migration_bitmap_sync() helper and passes last_stage=3Dfalse. Postcopy switchover is also a stopped-VM final sync point, so use migration_bitmap_sync_precopy(true). This keeps RAM final bitmap sync handling consistent across precopy and postcopy switchover, including the precopy bitmap-sync notifier wrapper. Current notifier users are safe in postcopy: virtio-balloon free-page hinting already opts out when postcopy-ram is enabled [1]. This prepares for moving the final sync out of RAM completion/discard paths and into migration_switchover_start(), where the migration core can run a final save_query_pending pass for all modules uniformly. [1] fd51e54fa102 ("virtio-balloon: don't start free page hinting if postcop= y is possible") Reviewed-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-3-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index bd519f1931cb0d0cde27765a95730179aba03027..55aa9b47dc8ab8463fd1aeba197= 26a493b81b070 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2687,7 +2687,7 @@ void ram_postcopy_send_discard_bitmap(MigrationState = *ms) RCU_READ_LOCK_GUARD(); =20 /* This should be our last sync, the src is now paused */ - migration_bitmap_sync(rs, false); + migration_bitmap_sync_precopy(true); =20 /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block =3D NULL; --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405831; cv=none; d=zohomail.com; s=zohoarc; b=In+6izPPfxFyby/alWvdqEO6wwYo4UCtqF0TstKFN4JXrbdBlwgb+AsrileFkGCRpmWQPIeuMm7/37xXL9iz/Fvy6zrYgCEQ2gT5YHDTOy9xXfp9yJ2b86/fK9tNAmFValH5Rx3K9G/AgEZsPxQYomPyLMzSYp+1vTgmlEY4HFI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405831; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Q3jh81BNPvEa/25rPIyX5eJlJ4if2rSjEX2dWyFb+AE=; b=FOiayfZ8jDjo5FkRoNImCs99DuRwy+fIGLRg+tx7+WbP0FdTLQSoxGXq3j+IO+OW2XU2L636OzFcfgNfO3zVjoaikXfcc1rUFTs0N3J3GZRrQAXTRvvg/zz0hk2spR+2N8ufbhvgOdAN+1rhX2ZESLt1Q5zRVKyBDBn6mhBVDQ4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405831499363.5295873257894; Mon, 6 Jul 2026 23:30:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJl-0001X7-VY; Tue, 07 Jul 2026 02:30:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJX-0001Ad-E9 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJU-0006jS-4u for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:51 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-43-bkweqyIDNfeGnJ5x1X57_A-1; Tue, 07 Jul 2026 02:29:44 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2A8FB1954B17; Tue, 7 Jul 2026 06:29:43 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7B88D180065F; Tue, 7 Jul 2026 06:29:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q3jh81BNPvEa/25rPIyX5eJlJ4if2rSjEX2dWyFb+AE=; b=FhYGBOBzsjOf5rELtEokpyejT5rgtaZ94CLFYvRaINYE7iKaO6FsPyAacEKJJ2IUvC5nqy WU48VXePpQ8bCzuJMVDHyMHX0AptIT8i9X+ocXypNDrzhC57SgDSlQEgvlOZF+rqQHI5y8 /u4c8q6UqbK+NC/dKLFtkGcy2VaTokE= X-MC-Unique: bkweqyIDNfeGnJ5x1X57_A-1 X-Mimecast-MFC-AGG-ID: bkweqyIDNfeGnJ5x1X57_A_1783405783 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 08/27] migration: Run final save_query_pending at switchover Date: Tue, 7 Jul 2026 08:29:01 +0200 Message-ID: <20260707062920.317302-9-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405832213158501 From: Avihai Horon Before switchover, the source needs one last exact pending query so modules can flush dirty state. This is currently done ad hoc in modules handlers. For example, RAM syncs its dirty bitmap in its save_complete handler. This should be a general concept relevant for any module, so extract it to migration core instead by running a final save_query_pending before switchover. The final query requires special handling by modules (e.g., it's called with BQL locked, during VM stop), so extend save_query_pending SaveVMHandlers callback and qemu_savevm_query_pending() with a "final" flag so migration modules can tell the last pending query during switchover from periodic iteration queries. Call final pending query also in COLO checkpoint, which needs to flush dirty state before the checkpoint's live state is saved. Unlike a regular switchover, COLO reaches completion repeatedly for every checkpoint, so this must be done on each one. Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-4-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- include/migration/register.h | 41 ++++++++++++++++++---------------- migration/savevm.h | 3 ++- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c | 11 ++++++--- migration/block-dirty-bitmap.c | 11 ++++++--- migration/colo.c | 10 +++++++++ migration/migration.c | 14 ++++++++++-- migration/ram.c | 40 +++++++++++++++++++-------------- migration/savevm.c | 20 +++++++++++++---- hw/vfio/trace-events | 2 +- migration/trace-events | 2 +- 11 files changed, 104 insertions(+), 52 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index 5e5e0ee432e537781ce78e84b2ff2d81c33e8f37..6f632123f1d0710de0556e68ba5= 3e068a1e02640 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -171,6 +171,28 @@ typedef struct SaveVMHandlers { */ bool (*is_active_iterate)(void *opaque); =20 + /** + * @save_query_pending + * + * This estimates the remaining data to transfer on the source side. + * + * When @exact is true, a module must report accurate results. When + * @exact is false, a module may report estimates. + * + * It's highly recommended that modules implement a faster version of + * the query path (for example, by proper caching on the counters) if + * an accurate query will be time-consuming. + * + * @opaque: data pointer passed to register_savevm_live() + * @pending: pointer to a MigPendingData struct + * @exact: set to true for an accurate (slow) query + * @final: set to true for the final query during switchover. When fin= al is + * true, the query is called with BQL locked. Otherwise, it's called w= ith + * BQL unlocked. + */ + void (*save_query_pending)(void *opaque, MigPendingData *pending, + bool exact, bool final); + /* This runs outside the BQL in the migration case, and * within the lock in the savevm case. The callback had better only * use data that is local to the migration thread or protected @@ -210,25 +232,6 @@ typedef struct SaveVMHandlers { */ bool (*save_postcopy_prepare)(QEMUFile *f, void *opaque, Error **errp); =20 - /** - * @save_query_pending - * - * This estimates the remaining data to transfer on the source side. - * - * When @exact is true, a module must report accurate results. When - * @exact is false, a module may report estimates. - * - * It's highly recommended that modules implement a faster version of - * the query path (for example, by proper caching on the counters) if - * an accurate query will be time-consuming. - * - * @opaque: data pointer passed to register_savevm_live() - * @pending: pointer to a MigPendingData struct - * @exact: set to true for an accurate (slow) query - */ - void (*save_query_pending)(void *opaque, MigPendingData *pending, - bool exact); - /** * @load_state * diff --git a/migration/savevm.h b/migration/savevm.h index b6bb4fa977fae84ed121b0668cdefdf49e3d9c67..81abd96dda94c6c95cc40b3c188= df23d11c19463 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -45,7 +45,8 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); -void qemu_savevm_query_pending(MigPendingData *pending, bool exact); +void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact); +void qemu_savevm_query_pending_final(MigPendingData *pending); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index c334714b31ab4313e26c0cf57091a8cb21e55306..b0f04eb30c7f73e5dd91b1345d7= 1a14cb404a4eb 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -190,7 +190,7 @@ static int cmma_save_setup(QEMUFile *f, void *opaque, E= rror **errp) } =20 static void cmma_state_pending(void *opaque, MigPendingData *pending, - bool exact) + bool exact, bool final) { S390StAttribState *sas =3D S390_STATTRIB(opaque); S390StAttribClass *sac =3D S390_STATTRIB_GET_CLASS(sas); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index fb12b9717f773fdde657911517de9d74c1eb3931..195498845ed95d6a55cf68537a1= 00b4e83221fb0 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -622,13 +622,18 @@ static void vfio_state_pending_sync(VFIODevice *vbase= dev) } =20 static void vfio_state_pending(void *opaque, MigPendingData *pending, - bool exact) + bool exact, bool final) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; uint64_t precopy_size, stopcopy_size; =20 - if (exact) { + /* + * The final pending query runs during switchover downtime. VFIO does = not + * need a fresh device pending-data query then to get the latest dirty + * data, so avoid the extra work and report the cached counters below. + */ + if (exact && !final) { vfio_state_pending_sync(vbasedev); } =20 @@ -646,7 +651,7 @@ static void vfio_state_pending(void *opaque, MigPending= Data *pending, =20 trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, migration->precopy_init_size, - migration->precopy_dirty_size, exact); + migration->precopy_dirty_size, exact, final); } =20 static bool vfio_is_active_iterate(void *opaque) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 7ef3759e5348a6907c0fb3773e2c6cbb0850632e..cba54e25cd50da924e1958bdfc2= 3ce060784b0ac 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -767,13 +767,16 @@ static int dirty_bitmap_save_complete(QEMUFile *f, vo= id *opaque) } =20 static void dirty_bitmap_state_pending(void *opaque, MigPendingData *data, - bool exact) + bool exact, bool final) { DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms; uint64_t pending =3D 0; =20 - bql_lock(); + /* Final pending query is called with BQL locked */ + if (!final) { + bql_lock(); + } =20 QSIMPLEQ_FOREACH(dbms, &s->dbms_list, entry) { uint64_t gran =3D bdrv_dirty_bitmap_granularity(dbms->bitmap); @@ -783,7 +786,9 @@ static void dirty_bitmap_state_pending(void *opaque, Mi= gPendingData *data, pending +=3D DIV_ROUND_UP(sectors * BDRV_SECTOR_SIZE, gran); } =20 - bql_unlock(); + if (!final) { + bql_unlock(); + } =20 trace_dirty_bitmap_state_pending(pending); =20 diff --git a/migration/colo.c b/migration/colo.c index 2d36f933cf155c1084565162294a61d32e28fe86..08632d5e4ad0d05de386ecf0af4= 2f648fa24ecf5 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -409,6 +409,7 @@ static int colo_do_checkpoint_transaction(MigrationStat= e *s, QEMUFile *fb) { Error *local_err =3D NULL; + MigPendingData pending =3D {}; int ret =3D -1; =20 colo_send_message(s->to_dst_file, COLO_MESSAGE_CHECKPOINT_REQUEST, @@ -465,6 +466,15 @@ static int colo_do_checkpoint_transaction(MigrationSta= te *s, if (migrate_auto_converge()) { mig_throttle_counter_reset(); } + + /* + * Run the final pending query so migration modules can flush their di= rty + * state (e.g., RAM syncs its dirty bitmap) before this checkpoint's l= ive + * state is saved. Unlike a regular switchover, COLO reaches completion + * repeatedly for every checkpoint, so this must be done on each one. + */ + qemu_savevm_query_pending_final(&pending); + /* * Only save VM's live state, which not including device state. * TODO: We may need a timeout mechanism to prevent COLO process diff --git a/migration/migration.c b/migration/migration.c index 7e0a0bea7ae823ee5ef6e7ba8876c9222f9ac834..442f950c43cef47ed23856a16bd= da3af5549a5bf 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2793,12 +2793,22 @@ static bool migration_switchover_prepare(MigrationS= tate *s) static bool migration_switchover_start(MigrationState *s, Error **errp) { ERRP_GUARD(); + MigPendingData pending =3D {}; =20 if (!migration_switchover_prepare(s)) { error_setg(errp, "Switchover is interrupted"); return false; } =20 + /* + * The final query to the whole system on dirty data to make sure we + * collect the latest status of the VM. For precopy, source QEMU will + * dump all the dirty data during switchover. For postcopy, this will + * properly update all the dirty bitmaps to finally generate the + * correct discard bitmaps; see ram_postcopy_send_discard_bitmap(). + */ + qemu_savevm_query_pending_final(&pending); + /* Inactivate disks except in COLO */ if (!migrate_colo()) { /* @@ -3291,7 +3301,7 @@ static void migration_iteration_go_next(MigPendingDat= a *pending) /* * Do a slow sync first before boosting the iteration count. */ - qemu_savevm_query_pending(pending, true); + qemu_savevm_query_pending_iter(pending, true); =20 /* * Update the dirty information for the whole system for this @@ -3342,7 +3352,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ - qemu_savevm_query_pending(&pending, false); + qemu_savevm_query_pending_iter(&pending, false); =20 if (in_postcopy) { /* diff --git a/migration/ram.c b/migration/ram.c index 55aa9b47dc8ab8463fd1aeba19726a493b81b070..8918b2f03b6fbcc98532bb45ab9= 05f9bac59384c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2686,9 +2686,6 @@ void ram_postcopy_send_discard_bitmap(MigrationState = *ms) =20 RCU_READ_LOCK_GUARD(); =20 - /* This should be our last sync, the src is now paused */ - migration_bitmap_sync_precopy(true); - /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block =3D NULL; rs->last_seen_block =3D NULL; @@ -3376,10 +3373,6 @@ static int ram_save_complete(QEMUFile *f, void *opaq= ue) rs->last_stage =3D !migration_in_colo_state(); =20 WITH_RCU_READ_LOCK_GUARD() { - if (!migration_in_postcopy()) { - migration_bitmap_sync_precopy(true); - } - ret =3D rdma_registration_start(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); @@ -3442,25 +3435,38 @@ static int ram_save_complete(QEMUFile *f, void *opa= que) return qemu_fflush(f); } =20 -static void ram_state_pending(void *opaque, MigPendingData *pending, - bool exact) +static void ram_state_pending_sync(bool exact, bool final) { - RAMState **temp =3D opaque; - RAMState *rs =3D *temp; - uint64_t remaining_size; - /* * Sync is not needed either with: (1) a fast query, or (2) after * postcopy has started (no new dirty will generate anymore). */ - if (exact && !migration_in_postcopy()) { + if (!exact || migration_in_postcopy()) { + return; + } + + /* Final pending query is called with BQL locked */ + if (!final) { bql_lock(); - WITH_RCU_READ_LOCK_GUARD() { - migration_bitmap_sync_precopy(false); - } + } + + WITH_RCU_READ_LOCK_GUARD() { + migration_bitmap_sync_precopy(final); + } + + if (!final) { bql_unlock(); } +} + +static void ram_state_pending(void *opaque, MigPendingData *pending, + bool exact, bool final) +{ + RAMState **temp =3D opaque; + RAMState *rs =3D *temp; + uint64_t remaining_size; =20 + ram_state_pending_sync(exact, final); remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SIZE; =20 if (migrate_postcopy_ram()) { diff --git a/migration/savevm.c b/migration/savevm.c index 7e2a0c2b4a7470d9c2159eac1d18ae891740fe3e..74d622583e345bdc96b9189aad0= c8f6bad41964d 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1801,7 +1801,8 @@ int qemu_savevm_state_complete_precopy(MigrationState= *s, Error **errp) return 0; } =20 -void qemu_savevm_query_pending(MigPendingData *pending, bool exact) +static void qemu_savevm_query_pending(MigPendingData *pending, bool exact, + bool final) { SaveStateEntry *se; =20 @@ -1814,7 +1815,7 @@ void qemu_savevm_query_pending(MigPendingData *pendin= g, bool exact) if (!qemu_savevm_state_active(se)) { continue; } - se->ops->save_query_pending(se->opaque, pending, exact); + se->ops->save_query_pending(se->opaque, pending, exact, final); } =20 pending->total_bytes =3D pending->precopy_bytes + @@ -1826,13 +1827,24 @@ void qemu_savevm_query_pending(MigPendingData *pend= ing, bool exact) * close to reality when this got invoked frequently while iterating. */ mig_stats.dirty_bytes_total =3D pending->total_bytes; - - trace_qemu_savevm_query_pending(exact, pending->precopy_bytes, + trace_qemu_savevm_query_pending(exact, final, pending->precopy_bytes, pending->stopcopy_bytes, pending->postcopy_bytes, pending->total_bytes); } =20 +void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact) +{ + qemu_savevm_query_pending(pending, exact, false); +} + +void qemu_savevm_query_pending_final(MigPendingData *pending) +{ + g_assert(bql_locked()); + + qemu_savevm_query_pending(pending, true, true); +} + void qemu_savevm_state_cleanup(void) { SaveStateEntry *se; diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 4c28b3291cc687c06490bdd090f393b67c9c4d88..fa2204f002970c31eb987ae1311= 3a9a5edb3e172 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -176,7 +176,7 @@ vfio_save_device_config_state(const char *name) " (%s)" vfio_save_iterate(const char *name, uint64_t precopy_init_size, uint64_t p= recopy_dirty_size) " (%s) precopy initial size %"PRIu64" precopy dirty size= %"PRIu64 vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 -vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact) " (%s) stopcopy siz= e %"PRIu64" precopy initial size %"PRIu64" precopy dirty size %"PRIu64 " ex= act %d" +vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact, bool final) " (%s) = stopcopy size %"PRIu64", precopy initial size %"PRIu64", precopy dirty size= %"PRIu64", exact %d, final %d" vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" =20 diff --git a/migration/trace-events b/migration/trace-events index de99d976abc637e9361f8467daacf3cc8787976f..1c9212d3e20555902f008e4c999= f32642cc8fa48 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,7 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" -qemu_savevm_query_pending(bool exact, uint64_t precopy, uint64_t stopcopy,= uint64_t postcopy, uint64_t total) "exact=3D%d, precopy=3D%"PRIu64", stopc= opy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PRIu64 +qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total) "exact=3D%d, final=3D%d, pr= ecopy=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PR= Iu64 loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405838; cv=none; d=zohomail.com; s=zohoarc; b=ivjXEH+PsHlNnYjW4tcRLbfCazDPSuDU/aNTr3EQcwCG2XApBoIECjqegMo7VCu58XwY5DlVzb0yw3BDD0n7Fzo9P1uBB8R7ui4jLbLSCafiCrKzVQpGGck4lFr8vHmcKRCL5UOoftJ5WQgmXxP87Ict4KFjngaRJeGtxQRI/+k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405838; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=N7u/V06wBuEW+NjIDBIXIQOYPjyU39Z3GvXCDs8zL44=; b=LHc8gmfy6wtwG+a0Og0vNpjbsc9X1uK0vk7OoPYWpOor7Xh0Wh1uza4r/Z1IjpNYK1Nmpjoo2oZarRB06leWhf6MAZv5Tx1VOg8sLB0LJQmNRweSQhJTwSaY0HXoz7vjfb0JeFV/cHS8EQOvd9I9BFktuJ3H27MPu+ouUR6gZ24= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405838085525.9852199270506; Mon, 6 Jul 2026 23:30:38 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJe-0001JY-Bi; Tue, 07 Jul 2026 02:30:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJV-0001AI-9k for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJT-0006ie-Pv for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:49 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-18-iikCYM0eN0WMdncB1FdylA-1; Tue, 07 Jul 2026 02:29:46 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1332B1955D72; Tue, 7 Jul 2026 06:29:45 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9A9EF180067F; Tue, 7 Jul 2026 06:29:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N7u/V06wBuEW+NjIDBIXIQOYPjyU39Z3GvXCDs8zL44=; b=cNUyQjhkaF/6wnJ/Jk+ZESurj89PzQa0t03NaHRLKq1qFf2Y6i0Sgnf9ZvpGfvM1YciJ35 2IXbOAbTX7Kdg1x8C1yqyQ6WThIfj5MHXvzOnCD8+4ZFF66iX8OTqWq2DTnzhKtxwbx/Z7 MkQKtfZ5mHTbB1ihKqrV6gH9+PLi5Eo= X-MC-Unique: iikCYM0eN0WMdncB1FdylA-1 X-Mimecast-MFC-AGG-ID: iikCYM0eN0WMdncB1FdylA_1783405785 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 09/27] migration: Log the approver in qemu_loadvm_approve_switchover() Date: Tue, 7 Jul 2026 08:29:02 +0200 Message-ID: <20260707062920.317302-10-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405840183158500 From: Avihai Horon Pass the device name that approved switchover to qemu_loadvm_approve_switchover() and log it in the trace for debugging purposes. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-5-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- migration/savevm.h | 2 +- hw/vfio/migration.c | 2 +- migration/savevm.c | 4 ++-- migration/trace-events | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index 81abd96dda94c6c95cc40b3c188df23d11c19463..44424be347494c51e6976749d7c= 60d27ca7f0628 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -71,7 +71,7 @@ void qemu_loadvm_state_cleanup(MigrationIncomingState *mi= s); int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, Error **errp); int qemu_load_device_state(QEMUFile *f, Error **errp); -int qemu_loadvm_approve_switchover(void); +int qemu_loadvm_approve_switchover(const char *approver); int qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp); int qemu_savevm_state_non_iterable_early(QEMUFile *f, JSONWriter *vmdesc, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 195498845ed95d6a55cf68537a100b4e83221fb0..180b316baecb980fbd475b14267= 26697f340ccc8 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -847,7 +847,7 @@ static int vfio_load_state(QEMUFile *f, void *opaque, i= nt version_id) return -EINVAL; } =20 - ret =3D qemu_loadvm_approve_switchover(); + ret =3D qemu_loadvm_approve_switchover(vbasedev->name); if (ret) { error_report( "%s: qemu_loadvm_approve_switchover failed, err=3D%d (= %s)", diff --git a/migration/savevm.c b/migration/savevm.c index 74d622583e345bdc96b9189aad0c8f6bad41964d..be81de26160d195488e5fb09a88= dc37a7a06a3fe 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -3174,7 +3174,7 @@ int qemu_load_device_state(QEMUFile *f, Error **errp) return 0; } =20 -int qemu_loadvm_approve_switchover(void) +int qemu_loadvm_approve_switchover(const char *approver) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 @@ -3183,7 +3183,7 @@ int qemu_loadvm_approve_switchover(void) } =20 mis->switchover_ack_pending_num--; - trace_loadvm_approve_switchover(mis->switchover_ack_pending_num); + trace_loadvm_approve_switchover(approver, mis->switchover_ack_pending_= num); =20 if (mis->switchover_ack_pending_num) { return 0; diff --git a/migration/trace-events b/migration/trace-events index 1c9212d3e20555902f008e4c999f32642cc8fa48..c0c433744cd33d7e28496831bb8= ce93c65670bf5 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -24,7 +24,7 @@ loadvm_postcopy_ram_handle_discard_end(void) "" loadvm_postcopy_ram_handle_discard_header(const char *ramid, uint16_t len)= "%s: %ud" loadvm_process_command(const char *s, uint16_t len) "com=3D%s len=3D%d" loadvm_process_command_ping(uint32_t val) "0x%x" -loadvm_approve_switchover(unsigned int switchover_ack_pending_num) "Switch= over ack pending num=3D%u" +loadvm_approve_switchover(const char *approver, unsigned int switchover_ac= k_pending_num) "Approver %s, switchover_ack_pending_num %u" postcopy_ram_listen_thread_exit(void) "" postcopy_ram_listen_thread_start(void) "" qemu_savevm_send_postcopy_advise(void) "" --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405944; cv=none; d=zohomail.com; s=zohoarc; b=ZoID1BhtqCEGNvOIXB0G6lSmXWBxD1LuCJDX5kdlSwxzFQhjiYbSaiUp2nUXv6iwrSixYxnJBSgH33LJFiWSxzloemvjg1OLstfJ8tHqyiAZaAQZkqw+wVlBN6Q6yZ1ixiDLgJyHriXpFyYJQG3XHQoao/hAyYZ1eWZoaDktjQw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405944; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kxZUViNJbbsxiBwq4M6uL2MhQgeGVwJ6fjWdWl8xK5c=; b=HLhf/nqx8kv7BUDpezFJ9jJjFInXqXZgpYVHXKJqsTZ9n410PhLD3Kegh/kaBSIqRJMdwCjMp2oZf9PbbbTJM6CeNMQ6dmKz0Stoo/BxJCX80SDj64GIni5bOopOm4r3XnK5RWBSE7lmK2Gra57INGvz7spe0/K78WE6KPcebjI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405944785563.6320358533247; Mon, 6 Jul 2026 23:32:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJm-0001bd-Ut; Tue, 07 Jul 2026 02:30:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJZ-0001Al-BP for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJX-0006pw-Eh for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:53 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-464-bNbSYmeeMPOQjQ3ktrSaQA-1; Tue, 07 Jul 2026 02:29:47 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EE34B18011D1; Tue, 7 Jul 2026 06:29:46 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 83164180065F; Tue, 7 Jul 2026 06:29:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405790; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kxZUViNJbbsxiBwq4M6uL2MhQgeGVwJ6fjWdWl8xK5c=; b=CiMhnx7N4n6r1pcI4lH4g+u3gyTa33B7Xu6PZxlp0cZFOsjrC87JTLLz3zsv4+VQzvwVRM tYsa+neN5X+lt6pvsvHqL9VylHbGInMzl+nBbL6KLktVePw/xSqCtkW5Kpg6KopNWK+jwq fyf1O4Md2aq8O8eACILp/0fD2cXNsC8= X-MC-Unique: bNbSYmeeMPOQjQ3ktrSaQA-1 X-Mimecast-MFC-AGG-ID: bNbSYmeeMPOQjQ3ktrSaQA_1783405787 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 10/27] migration: Replace switchover_ack_needed SaveVMHandler Date: Tue, 7 Jul 2026 08:29:03 +0200 Message-ID: <20260707062920.317302-11-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405947873158500 From: Avihai Horon A new switchover-ack mechanism that will replace the existing one will be added in the following patches. The new mechanism will not use switchover_ack_needed SaveVMHandler, however, the old mechanism must still be kept for backward compatibility. To keep things clear and decrease API surface of old code, replace switchover_ack_needed SaveVMHandler with a regular function migration_request_switchover_ack(). No functional changes intended. Acked-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-6-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- docs/devel/migration/vfio.rst | 3 --- include/migration/misc.h | 2 ++ include/migration/register.h | 13 ------------- hw/vfio/migration.c | 18 ++++++++++-------- migration/migration.c | 15 +++++++++++++++ migration/savevm.c | 21 --------------------- migration/trace-events | 2 +- 7 files changed, 28 insertions(+), 46 deletions(-) diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 691061d1820453a09df479f020abf0aab5fbabfd..854277b11ce64672f998ce1728b= f8b5ddf1c129a 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -59,9 +59,6 @@ VFIO implements the device hooks for the iterative approa= ch as follows: * A ``save_live_iterate`` function that reads the VFIO device's data from = the vendor driver during iterative pre-copy phase. =20 -* A ``switchover_ack_needed`` function that checks if the VFIO device uses - "switchover-ack" migration capability when this capability is enabled. - * A ``switchover_start`` function that in the multifd mode starts a thread= that reassembles the multifd received data and loads it in-order into the dev= ice. In the non-multifd mode this function is a NOP. diff --git a/include/migration/misc.h b/include/migration/misc.h index d3d6e1f50d102d0f46f574423eb951232be9b86b..17b42949d4b68dc3ed9341d254e= 7ccc69e18735e 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -159,4 +159,6 @@ bool multifd_device_state_save_thread_should_exit(void); void multifd_abort_device_state_save_threads(void); bool multifd_join_device_state_save_threads(void); =20 +void migration_request_switchover_ack(const char *requester); + #endif diff --git a/include/migration/register.h b/include/migration/register.h index 6f632123f1d0710de0556e68ba53e068a1e02640..a61c4236d22423a8477876971f1= d73fe2e802f4a 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -302,19 +302,6 @@ typedef struct SaveVMHandlers { */ int (*resume_prepare)(MigrationState *s, void *opaque); =20 - /** - * @switchover_ack_needed - * - * Checks if switchover ack should be used. Called only on - * destination. - * - * @opaque: data pointer passed to register_savevm_live() - * - * Returns true if switchover ack should be used and false - * otherwise - */ - bool (*switchover_ack_needed)(void *opaque); - /** * @switchover_start * diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 180b316baecb980fbd475b1426726697f340ccc8..7055cfbd3e04bd2b1e95c9be45b= 879cb0bf06212 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -487,6 +487,14 @@ static bool vfio_precopy_supported(VFIODevice *vbasede= v) return migration->mig_flags & VFIO_MIGRATION_PRE_COPY; } =20 +static void vfio_request_switchover_ack(VFIODevice *vbasedev) +{ + if (vfio_precopy_supported(vbasedev)) { + /* Precopy support implies switchover-ack is needed */ + migration_request_switchover_ack(vbasedev->name); + } +} + /* ---------------------------------------------------------------------- = */ =20 static int vfio_save_prepare(void *opaque, Error **errp) @@ -776,6 +784,8 @@ static int vfio_load_setup(QEMUFile *f, void *opaque, E= rror **errp) return ret; } =20 + vfio_request_switchover_ack(vbasedev); + return 0; } =20 @@ -874,13 +884,6 @@ static int vfio_load_state(QEMUFile *f, void *opaque, = int version_id) return ret; } =20 -static bool vfio_switchover_ack_needed(void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - - return vfio_precopy_supported(vbasedev); -} - static int vfio_switchover_start(void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -904,7 +907,6 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { .load_setup =3D vfio_load_setup, .load_cleanup =3D vfio_load_cleanup, .load_state =3D vfio_load_state, - .switchover_ack_needed =3D vfio_switchover_ack_needed, /* * Multifd support */ diff --git a/migration/migration.c b/migration/migration.c index 442f950c43cef47ed23856a16bdda3af5549a5bf..9a8d47ba9637eb57ee9f3bc6cfe= 8f580161fb36c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2202,6 +2202,21 @@ void migration_rp_kick(MigrationState *s) qemu_sem_post(&s->rp_state.rp_sem); } =20 +/* This is called only on destination side */ +void migration_request_switchover_ack(const char *requester) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + if (!migrate_switchover_ack()) { + return; + } + + mis->switchover_ack_pending_num++; + + trace_migration_request_switchover_ack(requester, + mis->switchover_ack_pending_num= ); +} + static struct rp_cmd_args { ssize_t len; /* -1 =3D variable */ const char *name; diff --git a/migration/savevm.c b/migration/savevm.c index be81de26160d195488e5fb09a88dc37a7a06a3fe..98639630277b7543db7400ec97d= 6b009603a03bc 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2800,23 +2800,6 @@ static int qemu_loadvm_state_header(QEMUFile *f, Err= or **errp) return 0; } =20 -static void qemu_loadvm_state_switchover_ack_needed(MigrationIncomingState= *mis) -{ - SaveStateEntry *se; - - QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->switchover_ack_needed) { - continue; - } - - if (se->ops->switchover_ack_needed(se->opaque)) { - mis->switchover_ack_pending_num++; - } - } - - trace_loadvm_state_switchover_ack_needed(mis->switchover_ack_pending_n= um); -} - static int qemu_loadvm_state_setup(QEMUFile *f, Error **errp) { ERRP_GUARD(); @@ -3078,10 +3061,6 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp) return -EINVAL; } =20 - if (migrate_switchover_ack()) { - qemu_loadvm_state_switchover_ack_needed(mis); - } - cpu_synchronize_all_pre_loadvm(); =20 ret =3D qemu_loadvm_state_main(f, mis, errp); diff --git a/migration/trace-events b/migration/trace-events index c0c433744cd33d7e28496831bb8ce93c65670bf5..5955befcc66128b72dfdbc9711e= 5749f19b9faf8 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -8,7 +8,6 @@ qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total) "exact=3D%d, final=3D%d, pr= ecopy=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PR= Iu64 -loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" loadvm_handle_cmd_packaged(unsigned int length) "%u" @@ -199,6 +198,7 @@ process_incoming_migration_co_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" migration_precopy_complete(void) "" migration_call_notifiers(int type) "type=3D%d" +migration_request_switchover_ack(const char *requester, unsigned int switc= hover_ack_pending_num) "Requester %s, switchover_ack_pending_num %u" =20 # migration-stats migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd, uint64_t= rdma) "qemu_file %" PRIu64 " multifd %" PRIu64 " RDMA %" PRIu64 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405830; cv=none; d=zohomail.com; s=zohoarc; b=ZbIhqFJmf1Q4xmCtcNNkALPfFm2U0oLP3C3sH8eltGGfOJOTfRAZqBrJ/uswRKgwNz8qubnMpVSvMjqgtF/EdRQT77YtnFc0Z1c4pefuhRkoxZSfNWa3y82WThd8kXLCO3c1sFR8YiIREKAj3bf83len0JGEogduZ/vMf4a0I/k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405830; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jPyeZcZbBYm3wm6QQNUkMGhDrgOnK4aWwTChYaLdlBo=; b=YIewMv2KTuzurYJq+vYKnX8vhCCcfdTlT4TRsCtOzHLTWFxrOlvkqB7LQwy2FljHhtI+5GU3UaSO1V3n+46Ef9dbQUeg5cBK308rzhokrnmVQwugyXbzrq3fqdMrhzm0wSs0lkWjvyim9om3aRaeqTqCO2lChzESwCGbBxaNdVs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405830546458.65420419485554; Mon, 6 Jul 2026 23:30:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJk-0001QD-DK; Tue, 07 Jul 2026 02:30:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJa-0001BC-2G for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJX-0006qW-O8 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:53 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-331-g5lPDxkiNgyohj4C8v3bdg-1; Tue, 07 Jul 2026 02:29:49 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D51C31801305; Tue, 7 Jul 2026 06:29:48 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6A3421800678; Tue, 7 Jul 2026 06:29:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405791; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jPyeZcZbBYm3wm6QQNUkMGhDrgOnK4aWwTChYaLdlBo=; b=AHQW340iTfzDrr6mDp5pdfisqZ4dwxuAsyrrECQrpsXOHsNJWNkb8ody7KaG1FBCTASunu NxPBcuN6oNGkayWGPLVeMCoVhsyNG9M/EmNFO3QJh9tCI34NNd1lngUnKI1x3LYeVwQoPw jARSqpb/Y6QNcVQteo58fY+cVCehZy8= X-MC-Unique: g5lPDxkiNgyohj4C8v3bdg-1 X-Mimecast-MFC-AGG-ID: g5lPDxkiNgyohj4C8v3bdg_1783405788 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 11/27] migration: Rename switchover-ack code to legacy Date: Tue, 7 Jul 2026 08:29:04 +0200 Message-ID: <20260707062920.317302-12-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405832189158501 From: Avihai Horon A new switchover-ack mechanism will be added in the following patches. However, the old mechanism must still be kept for backward compatibility. Rename existing code that will be used only for old switchover-ack mechanism as legacy. This will help to distinguish legacy code from new code and make it more readable and easier for removal later when no longer needed. No functional change intended. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-7-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- include/migration/misc.h | 2 +- migration/migration.h | 2 +- hw/vfio/migration.c | 6 ++--- migration/migration.c | 8 +++---- migration/savevm.c | 49 +++++++++++++++++++++++++++------------- migration/trace-events | 4 ++-- 6 files changed, 44 insertions(+), 27 deletions(-) diff --git a/include/migration/misc.h b/include/migration/misc.h index 17b42949d4b68dc3ed9341d254e7ccc69e18735e..2b2fbb59a409a35163b4578cf0d= 386977d3e5f80 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -159,6 +159,6 @@ bool multifd_device_state_save_thread_should_exit(void); void multifd_abort_device_state_save_threads(void); bool multifd_join_device_state_save_threads(void); =20 -void migration_request_switchover_ack(const char *requester); +void migration_request_switchover_ack_legacy(const char *requester); =20 #endif diff --git a/migration/migration.h b/migration/migration.h index 293ad60e07bd4dd059ec0e0b5aa2ed2d2ba11d29..570349d8581354bda30f5a37ccd= df5476ec518ce 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -246,7 +246,7 @@ struct MigrationIncomingState { * zero an ACK that it's OK to do switchover is sent to the source. No= lock * is needed as this field is updated serially. */ - unsigned int switchover_ack_pending_num; + unsigned int switchover_ack_pending_num_legacy; =20 /* Do exit on incoming migration failure */ bool exit_on_error; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 7055cfbd3e04bd2b1e95c9be45b879cb0bf06212..6b7acb2fa19d3ee618e73365ca9= 95bbdb4676055 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -487,11 +487,11 @@ static bool vfio_precopy_supported(VFIODevice *vbased= ev) return migration->mig_flags & VFIO_MIGRATION_PRE_COPY; } =20 -static void vfio_request_switchover_ack(VFIODevice *vbasedev) +static void vfio_request_switchover_ack_legacy(VFIODevice *vbasedev) { if (vfio_precopy_supported(vbasedev)) { /* Precopy support implies switchover-ack is needed */ - migration_request_switchover_ack(vbasedev->name); + migration_request_switchover_ack_legacy(vbasedev->name); } } =20 @@ -784,7 +784,7 @@ static int vfio_load_setup(QEMUFile *f, void *opaque, E= rror **errp) return ret; } =20 - vfio_request_switchover_ack(vbasedev); + vfio_request_switchover_ack_legacy(vbasedev); =20 return 0; } diff --git a/migration/migration.c b/migration/migration.c index 9a8d47ba9637eb57ee9f3bc6cfe8f580161fb36c..1d3387da0f4d5c64d64e8afb25a= f38ea143dad32 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2203,7 +2203,7 @@ void migration_rp_kick(MigrationState *s) } =20 /* This is called only on destination side */ -void migration_request_switchover_ack(const char *requester) +void migration_request_switchover_ack_legacy(const char *requester) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 @@ -2211,10 +2211,10 @@ void migration_request_switchover_ack(const char *r= equester) return; } =20 - mis->switchover_ack_pending_num++; + mis->switchover_ack_pending_num_legacy++; =20 - trace_migration_request_switchover_ack(requester, - mis->switchover_ack_pending_num= ); + trace_migration_request_switchover_ack_legacy( + requester, mis->switchover_ack_pending_num_legacy); } =20 static struct rp_cmd_args { diff --git a/migration/savevm.c b/migration/savevm.c index 98639630277b7543db7400ec97d6b009603a03bc..804c76e3bd93d6b4433df12d524= a803491b0aa08 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2478,6 +2478,31 @@ static int loadvm_postcopy_handle_switchover_start(E= rror **errp) return 0; } =20 +/* + * If legacy switchover-ack is enabled but no device uses it, need to send= an + * ACK to source that it's OK to switchover. + */ +static int loadvm_switchover_ack_no_users_legacy(MigrationIncomingState *m= is, + Error **errp) +{ + int ret; + + if (!migrate_switchover_ack()) { + return 0; + } + + if (!mis->switchover_ack_pending_num_legacy) { + ret =3D migrate_send_rp_switchover_ack(mis); + if (ret) { + error_setg_errno(errp, -ret, + "Could not send switchover ack RP MSG"); + return ret; + } + } + + return 0; +} + /* * Process an incoming 'QEMU_VM_COMMAND' * 0 just a normal return @@ -2527,18 +2552,9 @@ static int loadvm_process_command(QEMUFile *f, Error= **errp) } mis->to_src_file =3D qemu_file_get_return_path(f); =20 - /* - * Switchover ack is enabled but no device uses it, so send an ACK= to - * source that it's OK to switchover. Do it here, after return pat= h has - * been created. - */ - if (migrate_switchover_ack() && !mis->switchover_ack_pending_num) { - ret =3D migrate_send_rp_switchover_ack(mis); - if (ret) { - error_setg_errno(errp, -ret, - "Could not send switchover ack RP MSG"); - return ret; - } + ret =3D loadvm_switchover_ack_no_users_legacy(mis, errp); + if (ret) { + return ret; } return 0; =20 @@ -3157,14 +3173,15 @@ int qemu_loadvm_approve_switchover(const char *appr= over) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 - if (!mis->switchover_ack_pending_num) { + if (!mis->switchover_ack_pending_num_legacy) { return -EINVAL; } =20 - mis->switchover_ack_pending_num--; - trace_loadvm_approve_switchover(approver, mis->switchover_ack_pending_= num); + mis->switchover_ack_pending_num_legacy--; + trace_loadvm_approve_switchover_legacy( + approver, mis->switchover_ack_pending_num_legacy); =20 - if (mis->switchover_ack_pending_num) { + if (mis->switchover_ack_pending_num_legacy) { return 0; } =20 diff --git a/migration/trace-events b/migration/trace-events index 5955befcc66128b72dfdbc9711e5749f19b9faf8..a6b8c31ee1d3eddc59798d7a9b9= 7d2718814e1cb 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -23,7 +23,7 @@ loadvm_postcopy_ram_handle_discard_end(void) "" loadvm_postcopy_ram_handle_discard_header(const char *ramid, uint16_t len)= "%s: %ud" loadvm_process_command(const char *s, uint16_t len) "com=3D%s len=3D%d" loadvm_process_command_ping(uint32_t val) "0x%x" -loadvm_approve_switchover(const char *approver, unsigned int switchover_ac= k_pending_num) "Approver %s, switchover_ack_pending_num %u" +loadvm_approve_switchover_legacy(const char *approver, unsigned int switch= over_ack_pending_num_legacy) "Approver %s, switchover_ack_pending_num_legac= y %u" postcopy_ram_listen_thread_exit(void) "" postcopy_ram_listen_thread_start(void) "" qemu_savevm_send_postcopy_advise(void) "" @@ -198,7 +198,7 @@ process_incoming_migration_co_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" migration_precopy_complete(void) "" migration_call_notifiers(int type) "type=3D%d" -migration_request_switchover_ack(const char *requester, unsigned int switc= hover_ack_pending_num) "Requester %s, switchover_ack_pending_num %u" +migration_request_switchover_ack_legacy(const char *requester, unsigned in= t switchover_ack_pending_num_legacy) "Requester %s, switchover_ack_pending_= num_legacy %u" =20 # migration-stats migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd, uint64_t= rdma) "qemu_file %" PRIu64 " multifd %" PRIu64 " RDMA %" PRIu64 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405896; cv=none; d=zohomail.com; s=zohoarc; b=UzCKBRG0kFzA/m07iwSmwZJKqWwFs+Kb/qvUWrXlVU8DLcGtoiTLQ/CetF1KtEAoceeQxcSRUbvhMxfELBefrkHjNdK2ULnYk324iKv6tU0q5yHlHYBaR0NLkBlCHvARp/RUj/lyu4ZcwxOBGYbR99tN/L1MEfEdRKbz49c0pYs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405896; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=x34HIOEKH6d6TycJrHe/MKsvH8yiBoVfcWEpa1MSU+4=; b=eDeeNQG4VI5+NE7/1mAOI7yW0Dm0XQCvm5amc0ir6mlK5eSDqhM6QqNn3nb/MjOAnhfdQLACto4I7AfLpLlM2UHLPZDc+qP9GQObySwXZtcwSxoQswPCu1BfGM0PU9JDzygTxyOwS2TvLa3598lgknnErLVDrlIWKcnztvcVC0g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405896114433.85099243337334; Mon, 6 Jul 2026 23:31:36 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJr-000219-JN; Tue, 07 Jul 2026 02:30:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJo-0001lY-Lq for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJl-00075t-6y for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:08 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-389-EUrNoZfNOEqkP6rKzD0cGg-1; Tue, 07 Jul 2026 02:29:54 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BD6221955DBB; Tue, 7 Jul 2026 06:29:50 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5163D180067F; Tue, 7 Jul 2026 06:29:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405803; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x34HIOEKH6d6TycJrHe/MKsvH8yiBoVfcWEpa1MSU+4=; b=GQ8TIQ28jZLcjc3eliyz1O67p+MmJF67KKkxpXRjSbY0SDBZXAFc4LnYfk26WjDvN/rrJW mL6RfQy2Ne86ePYig7AdAM9+gugB/2IbItbIZzMLi9lZDjTpIsLwwPXZeYof5KKUbHsc9T /YRdkA5b6AL3aX0llY1z3gkz9rn3Tq8= X-MC-Unique: EUrNoZfNOEqkP6rKzD0cGg-1 X-Mimecast-MFC-AGG-ID: EUrNoZfNOEqkP6rKzD0cGg_1783405790 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Markus Armbruster , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 12/27] migration: Make switchover-ack re-usable Date: Tue, 7 Jul 2026 08:29:05 +0200 Message-ID: <20260707062920.317302-13-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405896632158500 From: Avihai Horon Switchover-ack is a mechanism to synchronize between source and destination QEMU during migration to prevent the source from switching over prematurely. VFIO uses switchover-ack to ensure switchover happens only after destination side has loaded the precopy initial bytes. This is important for VFIO, as otherwise downtime could be impacted and be higher. In its current state, switchover-ack is a one-time mechanism, meaning that switchover is acked only once and past that another ACK cannot be requested again. This was sufficient until now, as VFIO precopy initial bytes was defined to be monotonically decreasing. Thus, when precopy initial bytes reached zero for all VFIO devices, a single ACK would be sent and its validity would hold. However, now the new VFIO_PRECOPY_INFO_REINIT feature allows precopy initial bytes to be re-initialized during precopy. Specifically, it means that initial bytes can grow after reaching zero, which would invalidate a previously sent switchover ACK. To solve this, make switchover-ack reusable and allow devices to request switchover ACKs when needed via the save_query_pending SaveVMHandler. Since now switchover ACK can be requested for a specific device and in different times, make switchover ACK per-device (instead of a single ACK for all devices) and let source side do the pending ACKs accounting. Keep the legacy switchover-ack mechanism for backward compatibility and turn it on by a compatibility property for older machines. Enable the property until VFIO implements the new switchover-ack. Acked-by: Markus Armbruster Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-8-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- qapi/migration.json | 14 ++++---- include/migration/client-options.h | 1 + include/migration/register.h | 2 ++ migration/migration.h | 32 ++++++++++++++++-- migration/savevm.h | 6 ++-- migration/colo.c | 2 +- migration/migration.c | 37 ++++++++++++++------- migration/options.c | 10 ++++++ migration/savevm.c | 53 +++++++++++++++++++++++------- migration/trace-events | 5 +-- 10 files changed, 124 insertions(+), 38 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index e8756308cf9610182f75997ab81946f3b03dea63..d2f42c4b0ffd0a9e6bb757b8d10= 5fed89b423537 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -508,14 +508,12 @@ # (since 7.1) # # @switchover-ack: If enabled, migration will not stop the source VM -# and complete the migration until an ACK is received from the -# destination that it's OK to do so. Exactly when this ACK is -# sent depends on the migrated devices that use this feature. For -# example, a device can use it to make sure some of its data is -# sent and loaded in the destination before doing switchover. -# This can reduce downtime if devices that support this capability -# are present. 'return-path' capability must be enabled to use -# it. (since 8.1) +# and complete the migration until the destination has +# acknowledged that it is OK to switchover. The acknowledgement +# may depend, for example, on some device's data being loaded in +# the destination before doing switchover. This can reduce +# downtime if devices that support this capability are present. +# Capability @return-path must be enabled to use it. (since 8.1) # # @dirty-limit: If enabled, migration will throttle vCPUs as needed to # keep their dirty page rate within @vcpu-dirty-limit. This can diff --git a/include/migration/client-options.h b/include/migration/client-= options.h index 289c9d776221ce3cf35d86f91ce3313fd79427d3..78b1daa1a615ceb601da5dc8914= f4fb5fe04f8a9 100644 --- a/include/migration/client-options.h +++ b/include/migration/client-options.h @@ -13,6 +13,7 @@ =20 /* properties */ bool migrate_send_switchover_start(void); +bool migrate_switchover_ack_legacy(void); =20 /* capabilities */ =20 diff --git a/include/migration/register.h b/include/migration/register.h index a61c4236d22423a8477876971f1d73fe2e802f4a..5825eb30cb33e1721bec399f8a1= c01df154d143b 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -23,6 +23,8 @@ typedef struct MigPendingData { uint64_t postcopy_bytes; /* Amount of pending bytes can be transferred only in stopcopy */ uint64_t stopcopy_bytes; + /* Number of new pending switchover ACKs */ + uint32_t switchover_ack_pending; /* * Total pending data, modules do not need to update this field, it * will be automatically calculated by migration core API. diff --git a/migration/migration.h b/migration/migration.h index 570349d8581354bda30f5a37ccddf5476ec518ce..44bce199cd8c3518998ad7204d5= 99566d1ef9ddd 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -487,6 +487,29 @@ struct MigrationState { */ uint8_t clear_bitmap_shift; =20 + /* + * This decides whether to use legacy switchover-ack or new switchover= -ack. + * The main difference between them is that the former allows acknowle= dging + * switchover only once while the latter multiple times. + * + * In legacy, the destination keeps track of a pending ACKs counter. As + * migration progresses, the devices on the destination acknowledge + * switchover, decreasing the counter. When the counter reaches zero, a + * single ACK message is sent to the source via the return path, indic= ating + * that it's OK to switchover. + * + * In new switchover-ack, the source is the one that keeps track of a + * pending ACKs counter. As migration progresses, the destination send= s ACK + * message per-device via the return path, which decrements the source + * counter. When the counter reaches zero, it's OK to switchover. Duri= ng + * precopy, source-side devices may request additional ACKs, which inc= rement + * the counter again. + * + * In both legacy and new schemes, we rely on per-device protocol to r= equest + * switchover ACK from the destination-side counterpart. + */ + bool switchover_ack_legacy; + /* * This save hostname when out-going migration starts */ @@ -496,10 +519,13 @@ struct MigrationState { JSONWriter *vmdesc; =20 /* - * Indicates whether an ACK from the destination that it's OK to do - * switchover has been received. + * Indicates the number of pending ACKs from the destination. The valu= e may + * increase or decrease during precopy as new ACKs are requested or + * received. When zero is reached, it's OK to switchover. In legacy + * switchover-ack, it's initialized to 1 and decreased to zero upon AC= K. */ - bool switchover_acked; + uint32_t switchover_ack_pending_num; + /* Is this a rdma migration */ bool rdma_migration; =20 diff --git a/migration/savevm.h b/migration/savevm.h index 44424be347494c51e6976749d7c60d27ca7f0628..fb92d3bc8586553c490eb03d68b= e3cb56c8d0554 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -45,8 +45,10 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy= ); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); -void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact); -void qemu_savevm_query_pending_final(MigPendingData *pending); +void qemu_savevm_query_pending_iter(MigrationState *s, MigPendingData *pen= ding, + bool exact); +void qemu_savevm_query_pending_final(MigrationState *s, + MigPendingData *pending); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/migration/colo.c b/migration/colo.c index 08632d5e4ad0d05de386ecf0af42f648fa24ecf5..0cef0f6b0e650500496f51f49b2= 149154166a90f 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -473,7 +473,7 @@ static int colo_do_checkpoint_transaction(MigrationStat= e *s, * state is saved. Unlike a regular switchover, COLO reaches completion * repeatedly for every checkpoint, so this must be done on each one. */ - qemu_savevm_query_pending_final(&pending); + qemu_savevm_query_pending_final(s, &pending); =20 /* * Only save VM's live state, which not including device state. diff --git a/migration/migration.c b/migration/migration.c index 1d3387da0f4d5c64d64e8afb25af38ea143dad32..2b625da114dbfbbc8f77a359d89= 6eb4b8c10378e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1713,7 +1713,9 @@ int migrate_init(MigrationState *s, Error **errp) s->vm_old_state =3D -1; s->iteration_initial_bytes =3D 0; s->threshold_size =3D 0; - s->switchover_acked =3D false; + /* Legacy switchover-ack sends a single ACK for all devices */ + qatomic_set(&s->switchover_ack_pending_num, + migrate_switchover_ack_legacy() ? 1 : 0); s->rdma_migration =3D false; =20 /* @@ -2207,7 +2209,7 @@ void migration_request_switchover_ack_legacy(const ch= ar *requester) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 - if (!migrate_switchover_ack()) { + if (!migrate_switchover_ack() || !migrate_switchover_ack_legacy()) { return; } =20 @@ -2463,9 +2465,18 @@ static void *source_return_path_thread(void *opaque) break; =20 case MIG_RP_MSG_SWITCHOVER_ACK: - ms->switchover_acked =3D true; - trace_source_return_path_thread_switchover_acked(); + { + uint32_t pending_num; + + pending_num =3D qatomic_dec_fetch(&ms->switchover_ack_pending_= num); + trace_source_return_path_thread_switchover_acked(pending_num); + if (pending_num =3D=3D UINT32_MAX) { + error_setg(&err, "Switchover ack pending num underflowed"); + goto out; + } + break; + } =20 default: break; @@ -2822,7 +2833,7 @@ static bool migration_switchover_start(MigrationState= *s, Error **errp) * properly update all the dirty bitmaps to finally generate the * correct discard bitmaps; see ram_postcopy_send_discard_bitmap(). */ - qemu_savevm_query_pending_final(&pending); + qemu_savevm_query_pending_final(s, &pending); =20 /* Inactivate disks except in COLO */ if (!migrate_colo()) { @@ -3272,7 +3283,7 @@ static bool migration_can_switchover(MigrationState *= s) return true; } =20 - return s->switchover_acked; + return qatomic_read(&s->switchover_ack_pending_num) =3D=3D 0; } =20 /* Migration thread iteration status */ @@ -3311,12 +3322,13 @@ static bool migration_iteration_next_ready(Migratio= nState *s, return false; } =20 -static void migration_iteration_go_next(MigPendingData *pending) +static void migration_iteration_go_next(MigrationState *s, + MigPendingData *pending) { /* * Do a slow sync first before boosting the iteration count. */ - qemu_savevm_query_pending_iter(pending, true); + qemu_savevm_query_pending_iter(s, pending, true); =20 /* * Update the dirty information for the whole system for this @@ -3362,12 +3374,12 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) Error *local_err =3D NULL; bool in_postcopy =3D (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE= || s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); - bool can_switchover =3D migration_can_switchover(s); + bool can_switchover; MigPendingData pending =3D { }; bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ - qemu_savevm_query_pending_iter(&pending, false); + qemu_savevm_query_pending_iter(s, &pending, false); =20 if (in_postcopy) { /* @@ -3408,9 +3420,12 @@ static MigIterateState migration_iteration_run(Migra= tionState *s) * during postcopy phase. */ if (migration_iteration_next_ready(s, &pending)) { - migration_iteration_go_next(&pending); + migration_iteration_go_next(s, &pending); } =20 + /* Check can switchover after qemu_savevm_query_pending() */ + can_switchover =3D migration_can_switchover(s); + /* Should we switch to postcopy now? */ if (can_switchover && postcopy_should_start(s, &pending)) { if (postcopy_start(s, &local_err)) { diff --git a/migration/options.c b/migration/options.c index 5cbfd29099ba69f8826cc57a0e9cb43ed68b3bf2..4c9b25372e03b55ee9127c027c5= d4eefba796b06 100644 --- a/migration/options.c +++ b/migration/options.c @@ -110,6 +110,9 @@ const Property migration_properties[] =3D { preempt_pre_7_2, false), DEFINE_PROP_BOOL("multifd-clean-tls-termination", MigrationState, multifd_clean_tls_termination, true), + /* Use legacy until VFIO implements new switchover-ack */ + DEFINE_PROP_BOOL("switchover-ack-legacy", MigrationState, + switchover_ack_legacy, true), =20 /* Migration parameters */ DEFINE_PROP_UINT8("x-throttle-trigger-threshold", MigrationState, @@ -467,6 +470,13 @@ bool migrate_rdma(void) return s->rdma_migration; } =20 +bool migrate_switchover_ack_legacy(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->switchover_ack_legacy; +} + typedef enum WriteTrackingSupport { WT_SUPPORT_UNKNOWN =3D 0, WT_SUPPORT_ABSENT, diff --git a/migration/savevm.c b/migration/savevm.c index 804c76e3bd93d6b4433df12d524a803491b0aa08..1f6e696c50d189370a311c25d8e= b092bc41c2b24 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1801,7 +1801,8 @@ int qemu_savevm_state_complete_precopy(MigrationState= *s, Error **errp) return 0; } =20 -static void qemu_savevm_query_pending(MigPendingData *pending, bool exact, +static void qemu_savevm_query_pending(MigrationState *s, + MigPendingData *pending, bool exact, bool final) { SaveStateEntry *se; @@ -1827,22 +1828,35 @@ static void qemu_savevm_query_pending(MigPendingDat= a *pending, bool exact, * close to reality when this got invoked frequently while iterating. */ mig_stats.dirty_bytes_total =3D pending->total_bytes; - trace_qemu_savevm_query_pending(exact, final, pending->precopy_bytes, - pending->stopcopy_bytes, - pending->postcopy_bytes, - pending->total_bytes); + + if (migrate_switchover_ack() && !migrate_switchover_ack_legacy() && + pending->switchover_ack_pending) { + /* + * NOTE: Currently we rely on per-device protocol to request switc= hover + * ACK from the device on the destination side. + */ + qatomic_add(&s->switchover_ack_pending_num, + pending->switchover_ack_pending); + } + + trace_qemu_savevm_query_pending( + exact, final, pending->precopy_bytes, pending->stopcopy_bytes, + pending->postcopy_bytes, pending->total_bytes, + pending->switchover_ack_pending, + qatomic_read(&s->switchover_ack_pending_num)); } =20 -void qemu_savevm_query_pending_iter(MigPendingData *pending, bool exact) +void qemu_savevm_query_pending_iter(MigrationState *s, MigPendingData *pen= ding, + bool exact) { - qemu_savevm_query_pending(pending, exact, false); + qemu_savevm_query_pending(s, pending, exact, false); } =20 -void qemu_savevm_query_pending_final(MigPendingData *pending) +void qemu_savevm_query_pending_final(MigrationState *s, MigPendingData *pe= nding) { g_assert(bql_locked()); =20 - qemu_savevm_query_pending(pending, true, true); + qemu_savevm_query_pending(s, pending, true, true); } =20 void qemu_savevm_state_cleanup(void) @@ -2487,7 +2501,7 @@ static int loadvm_switchover_ack_no_users_legacy(Migr= ationIncomingState *mis, { int ret; =20 - if (!migrate_switchover_ack()) { + if (!migrate_switchover_ack() || !migrate_switchover_ack_legacy()) { return 0; } =20 @@ -3169,7 +3183,7 @@ int qemu_load_device_state(QEMUFile *f, Error **errp) return 0; } =20 -int qemu_loadvm_approve_switchover(const char *approver) +static int qemu_loadvm_approve_switchover_legacy(const char *approver) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 @@ -3188,6 +3202,23 @@ int qemu_loadvm_approve_switchover(const char *appro= ver) return migrate_send_rp_switchover_ack(mis); } =20 +int qemu_loadvm_approve_switchover(const char *approver) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + if (!migrate_switchover_ack()) { + return 0; + } + + if (migrate_switchover_ack_legacy()) { + return qemu_loadvm_approve_switchover_legacy(approver); + } + + trace_loadvm_approve_switchover(approver); + + return migrate_send_rp_switchover_ack(mis); +} + bool qemu_loadvm_load_state_buffer(const char *idstr, uint32_t instance_id, char *buf, size_t len, Error **errp) { diff --git a/migration/trace-events b/migration/trace-events index a6b8c31ee1d3eddc59798d7a9b97d2718814e1cb..f5339f4193d20d2a15abf713eeb= 2869df7149b4d 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,7 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" -qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total) "exact=3D%d, final=3D%d, pr= ecopy=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PR= Iu64 +qemu_savevm_query_pending(bool exact, bool final, uint64_t precopy, uint64= _t stopcopy, uint64_t postcopy, uint64_t total, uint32_t switchover_ack_pen= ding, uint32_t total_switchover_ack_pending) "exact=3D%d, final=3D%d, preco= py=3D%"PRIu64", stopcopy=3D%"PRIu64", postcopy=3D%"PRIu64", total=3D%"PRIu6= 4", collected switchover ack pending=3D%"PRIu32", total switchover ack pend= ing=3D%"PRIu32 loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" loadvm_handle_cmd_packaged(unsigned int length) "%u" @@ -24,6 +24,7 @@ loadvm_postcopy_ram_handle_discard_header(const char *ram= id, uint16_t len) "%s: loadvm_process_command(const char *s, uint16_t len) "com=3D%s len=3D%d" loadvm_process_command_ping(uint32_t val) "0x%x" loadvm_approve_switchover_legacy(const char *approver, unsigned int switch= over_ack_pending_num_legacy) "Approver %s, switchover_ack_pending_num_legac= y %u" +loadvm_approve_switchover(const char *approver) "Approver %s" postcopy_ram_listen_thread_exit(void) "" postcopy_ram_listen_thread_start(void) "" qemu_savevm_send_postcopy_advise(void) "" @@ -189,7 +190,7 @@ source_return_path_thread_loop_top(void) "" source_return_path_thread_pong(uint32_t val) "0x%x" source_return_path_thread_shut(uint32_t val) "0x%x" source_return_path_thread_resume_ack(uint32_t v) "%"PRIu32 -source_return_path_thread_switchover_acked(void) "" +source_return_path_thread_switchover_acked(uint32_t pending_num) "switchov= er_ack_pending_num %" PRIu32 source_return_path_thread_postcopy_package_loaded(void) "" migration_thread_low_pending(uint64_t pending) "%" PRIu64 migrate_transferred(uint64_t transferred, uint64_t time_spent, uint64_t ba= ndwidth, uint64_t avail_bw, uint64_t size) "transferred %" PRIu64 " time_sp= ent %" PRIu64 " bandwidth %" PRIu64 " switchover_bw %" PRIu64 " max_size %"= PRId64 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405829; cv=none; d=zohomail.com; s=zohoarc; b=bpXNaH5l+Q4qPmEd7H/QH9ZnWaeFBf4V2bOPpoGjwnxyWfxxhZP0SmP9qIYPpj3s0Aehf77wZGtVWngRtKxActmH/MA3+za+dEr+QAR1AsqFj1owgIYlCkdBBC4jxg0EU9UIkoiALRu8UEdsTX848UaaMUSu7K1woYvd0stUwNw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405829; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5q2mnAdIQtPhNzdphqSInaXNkpfTLZku26D0w7YULzo=; b=G4YpJxTwxN374ExcgclaDir/obFIA9oOvI9hgi4GvCnaluSR/KSwjumBJyzPJgNzl1qiLKNELpICHNeX98r0sBNNU/ryEvTlMo2xUIwoIGdqTD+zSNAFGlV1YDBZNc2WS2Bf5pyn6BcjNvMwBDCTJce5Y4pPkOzIYiLUdNyUjXI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405829334972.0504776038131; Mon, 6 Jul 2026 23:30:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJo-0001iD-59; Tue, 07 Jul 2026 02:30:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJg-0001NX-JU for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJd-0006xq-AC for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:29:59 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-443-HZ0FF7qZMVmHht9brX44Aw-1; Tue, 07 Jul 2026 02:29:53 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 699B91805C1E; Tue, 7 Jul 2026 06:29:52 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 38AF11800678; Tue, 7 Jul 2026 06:29:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405796; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5q2mnAdIQtPhNzdphqSInaXNkpfTLZku26D0w7YULzo=; b=APcwu6bcvi0iz71tLBygyWGTF586joFrEtnNBEDlAQDJpsQMw8Qe47rQRqBJyTb0w/TBbb RZsTbcjEp1jdkfj3sFhANie64MESw8wf0m2SWSL9GmuKW8wkMaK1SLJZ+vpLrXxU/rVP8M x/RJ56fwwkfhQ0LKyNAc45mNhZkX6AY= X-MC-Unique: HZ0FF7qZMVmHht9brX44Aw-1 X-Mimecast-MFC-AGG-ID: HZ0FF7qZMVmHht9brX44Aw_1783405792 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 13/27] migration: Fail migration if switchover-ack is requested after switchover decision Date: Tue, 7 Jul 2026 08:29:06 +0200 Message-ID: <20260707062920.317302-14-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405830200158500 From: Avihai Horon Switchover ACK is checked only during precopy while the guest is still running. The last migration_can_switchover() decision and guest stop are not atomic, so a device may want to request another switchover ACK in the gap after switchover decision has been made but before the guest is stopped. Migration would then miss that request, which can increase downtime. Cover this case by failing the migration if a switchover-ack was requested during that time. Ideally, precopy iterations should be resumed in this case, however, VFIO doesn't support going back to precopy after being stopped, so implementing such logic would require non-trivial changes to the guest start/stop flow. Given the above and that this case should be rare, failing the migration seems reasonable. Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-9-avihaih@nvi= dia.com Signed-off-by: C=C3=A9dric Le Goater --- migration/savevm.h | 4 ++-- migration/colo.c | 6 +++++- migration/migration.c | 4 +++- migration/savevm.c | 19 ++++++++++++++++++- 4 files changed, 28 insertions(+), 5 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index fb92d3bc8586553c490eb03d68be3cb56c8d0554..415198423f578049c209a1e40b7= 96cfe63b410f0 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -47,8 +47,8 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); void qemu_savevm_query_pending_iter(MigrationState *s, MigPendingData *pen= ding, bool exact); -void qemu_savevm_query_pending_final(MigrationState *s, - MigPendingData *pending); +bool qemu_savevm_query_pending_final(MigrationState *s, + MigPendingData *pending, Error **errp= ); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/migration/colo.c b/migration/colo.c index 0cef0f6b0e650500496f51f49b2149154166a90f..0f30a5926fdd83c7dca10e89601= bced6e7af7124 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -473,7 +473,11 @@ static int colo_do_checkpoint_transaction(MigrationSta= te *s, * state is saved. Unlike a regular switchover, COLO reaches completion * repeatedly for every checkpoint, so this must be done on each one. */ - qemu_savevm_query_pending_final(s, &pending); + if (!qemu_savevm_query_pending_final(s, &pending, &local_err)) { + ret =3D -1; + bql_unlock(); + goto out; + } =20 /* * Only save VM's live state, which not including device state. diff --git a/migration/migration.c b/migration/migration.c index 2b625da114dbfbbc8f77a359d896eb4b8c10378e..98603ee19c34531d069400d1ab0= ed36b65bb228e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2833,7 +2833,9 @@ static bool migration_switchover_start(MigrationState= *s, Error **errp) * properly update all the dirty bitmaps to finally generate the * correct discard bitmaps; see ram_postcopy_send_discard_bitmap(). */ - qemu_savevm_query_pending_final(s, &pending); + if (!qemu_savevm_query_pending_final(s, &pending, errp)) { + return false; + } =20 /* Inactivate disks except in COLO */ if (!migrate_colo()) { diff --git a/migration/savevm.c b/migration/savevm.c index 1f6e696c50d189370a311c25d8eb092bc41c2b24..f654a1c563df40d546e42018a37= 06e92fb11cfa7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1852,11 +1852,28 @@ void qemu_savevm_query_pending_iter(MigrationState = *s, MigPendingData *pending, qemu_savevm_query_pending(s, pending, exact, false); } =20 -void qemu_savevm_query_pending_final(MigrationState *s, MigPendingData *pe= nding) +bool qemu_savevm_query_pending_final(MigrationState *s, MigPendingData *pe= nding, + Error **errp) { g_assert(bql_locked()); =20 qemu_savevm_query_pending(s, pending, true, true); + + /* + * Switchover-ack requests done after switchover decision are not allo= wed. + * Fail the migration in this case since we currently don't support go= ing + * back to precopy. + */ + if (migrate_switchover_ack() && !migrate_switchover_ack_legacy() && + pending->switchover_ack_pending > 0) { + error_setg(errp, + "Switchover ACK was requested by %" PRIu32 + " devices during switchover", + pending->switchover_ack_pending); + return false; + } + + return true; } =20 void qemu_savevm_state_cleanup(void) --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405841; cv=none; d=zohomail.com; s=zohoarc; b=Sh+D/yiryLrqz584uzeLjepJj8QLX5rpRlz4m4S2UJ2tmapM0yg+e7WT9ElrSvH+SZheFrm/2wCWG3+irQBxPIZFUZhd21g8Y47U26EeccRmf5FMVjAPSiY2HwsLuo5ehNQtVeb+SRzE79l4ZdYsf2OfmCPfso5T5frPMlGmWak= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405841; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=H7yS9ctWuCmArMzFElGqdVRK3eIDdE+8aeTp8HDoPlk=; b=eBXY6qeN39m8rjO411OwsVh/Nh6N6sHJOgqLNOnxSh0odVudjmT7K+RAgFpjJZ5hk9grADz8MWTfO9f6TwMt+ECZzrF9xk0nUH8jvGC15ZPMB+9phgi2iIUniVCyYhMQ8S9+LsSdP1PFouBhC0Psp37Hk/5oMqslM8d6tDLbA9E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178340584189188.31469008806664; Mon, 6 Jul 2026 23:30:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJl-0001Tt-Ao; Tue, 07 Jul 2026 02:30:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJg-0001NW-J0 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJe-0006xx-Qv for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:00 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-502-8399qBiVOj-xjis-cqxcNQ-1; Tue, 07 Jul 2026 02:29:55 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 69C7A1805C05; Tue, 7 Jul 2026 06:29:54 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DA327180065F; Tue, 7 Jul 2026 06:29:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H7yS9ctWuCmArMzFElGqdVRK3eIDdE+8aeTp8HDoPlk=; b=ROWLqHivlXzTXij2RiUNcCp9BhTVE3m443V2PDUlG57O99ymo2jcQAF/nYUrnoRRVWhou+ 7r6E/3Y9reNAYYDi3BwbSZQLv8KfnJILjZ+lr8hENjbBFSMytYcKaMWTQX3bf9mT4gaX5+ JylEa/u09CcSo6EpQDT6KWXv29fcg34= X-MC-Unique: 8399qBiVOj-xjis-cqxcNQ-1 X-Mimecast-MFC-AGG-ID: 8399qBiVOj-xjis-cqxcNQ_1783405794 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 14/27] vfio/migration: Extract VFIO_MIG_FLAG_DEV_INIT_DATA_SENT sending to helper Date: Tue, 7 Jul 2026 08:29:07 +0200 Message-ID: <20260707062920.317302-15-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405842176158500 From: Avihai Horon Extract the VFIO_MIG_FLAG_DEV_INIT_DATA_SENT flag sending logic from vfio_save_iterate() into vfio_send_init_data_flag() for clarity. Also add a trace while at it. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-10-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/migration.c | 26 +++++++++++++++++++++----- hw/vfio/trace-events | 1 + 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 6b7acb2fa19d3ee618e73365ca995bbdb4676055..45f8e346b4a69a798366d460ecc= f028f35e7a47f 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -480,6 +480,26 @@ static void vfio_update_estimated_pending_data(VFIOMig= ration *migration, data_size); } =20 +/* Returns true if the init data flag was sent, false otherwise */ +static bool vfio_send_init_data_flag(QEMUFile *f, VFIOMigration *migration) +{ + VFIODevice *vbasedev =3D migration->vbasedev; + + if (!migrate_switchover_ack()) { + return false; + } + + if (migration->precopy_init_size || migration->initial_data_sent) { + return false; + } + + qemu_put_be64(f, VFIO_MIG_FLAG_DEV_INIT_DATA_SENT); + migration->initial_data_sent =3D true; + trace_vfio_send_init_data_flag(vbasedev->name); + + return true; +} + static bool vfio_precopy_supported(VFIODevice *vbasedev) { VFIOMigration *migration =3D vbasedev->migration; @@ -693,11 +713,7 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) =20 vfio_update_estimated_pending_data(migration, data_size); =20 - if (migrate_switchover_ack() && !migration->precopy_init_size && - !migration->initial_data_sent) { - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_INIT_DATA_SENT); - migration->initial_data_sent =3D true; - } else { + if (!vfio_send_init_data_flag(f, migration)) { qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); } =20 diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index fa2204f002970c31eb987ae13113a9a5edb3e172..fe0d2cc06f946b3f0db28d13994= fbd0dfd854fbd 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -177,6 +177,7 @@ vfio_save_iterate(const char *name, uint64_t precopy_in= it_size, uint64_t precopy vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact, bool final) " (%s) = stopcopy size %"PRIu64", precopy initial size %"PRIu64", precopy dirty size= %"PRIu64", exact %d, final %d" +vfio_send_init_data_flag(const char *name) " (%s)" vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" =20 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405888; cv=none; d=zohomail.com; s=zohoarc; b=Nj2FFTohy87L/Wjk6ZjvkIE2mBXHR8P0g+rgHR0/kXleZOyLQ7dencUxQnCrAYkQ0WTH/aXFkDYe8p8VlNurxOctNGaOXXcwx2DrKPQ+WEfrUDGh/jEdeSRZOWor/CHZO6gvA4KrxIVQMAVBp1a5wGvyqVESReIdz12btGmSPx8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405888; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uez2a8uxaz6BujyZvQoXbH4tbHZREXlp4qTKW+DMPNk=; b=lsFvWssOgEy0V4h+w5CKYkIYZT66kV4U/LvXL99aYkbYX7rXsrSgXQ24ausYFBOq00FN7VjgE0/1L8OTa7tOElxX/acXmldnw9CNzAJuqdw1twO1QzGzw83CLhUycGfqkhIs3agDIPqjelR1NiHoKHUQoT9l/yfMKanNN35iZMw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405888916808.3281952735824; Mon, 6 Jul 2026 23:31:28 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJo-0001mY-Rr; Tue, 07 Jul 2026 02:30:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJh-0001No-2W for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJf-0006y1-3v for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:00 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-61-Wu__1f8EMWWqf9UOJZqLQw-1; Tue, 07 Jul 2026 02:29:57 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 203E31955D48; Tue, 7 Jul 2026 06:29:56 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C28A11800678; Tue, 7 Jul 2026 06:29:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uez2a8uxaz6BujyZvQoXbH4tbHZREXlp4qTKW+DMPNk=; b=hjN4wtvKOfpMnvQob1hCbijJBP5DhgtOrOp5PBK/em/OIzDjKCQikdIrNLldBimXFalDA5 d484m6geKvAH0e6pmiKJsLureATRunT9ITbsS7Pq46f8QvTl4TH98xDnAzc/3PN7/J56Cd Jroal7Ke0LQ53nBxuiwMA20BruhaCzI= X-MC-Unique: Wu__1f8EMWWqf9UOJZqLQw-1 X-Mimecast-MFC-AGG-ID: Wu__1f8EMWWqf9UOJZqLQw_1783405796 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 15/27] vfio/migration: Add Error ** parameter to vfio_migration_init() Date: Tue, 7 Jul 2026 08:29:08 +0200 Message-ID: <20260707062920.317302-16-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405890497158500 From: Avihai Horon vfio_migration_init() already has many failure points and a new one will be added in next patch. Add Error ** parameter to vfio_migration_init() to report a detailed error message through it. Refactor it to return bool as well. Reviewed-by: C=C3=A9dric Le Goater Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-11-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/migration.c | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 45f8e346b4a69a798366d460eccf028f35e7a47f..3ab6b7248fea34dec3b58e508f9= 7c8be4c962ce4 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -1056,7 +1056,7 @@ static bool vfio_dma_logging_supported(VFIODevice *vb= asedev) return !ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); } =20 -static int vfio_migration_init(VFIODevice *vbasedev) +static bool vfio_migration_init(VFIODevice *vbasedev, Error **errp) { int ret; Object *obj; @@ -1067,22 +1067,32 @@ static int vfio_migration_init(VFIODevice *vbasedev) VMChangeStateHandler *prepare_cb; =20 if (!vbasedev->ops->vfio_get_object) { - return -EINVAL; + error_setg(errp, "no vfio_get_object handler"); + return false; } =20 obj =3D vbasedev->ops->vfio_get_object(vbasedev); if (!obj) { - return -EINVAL; + error_setg(errp, "failed to get object"); + return false; } =20 ret =3D vfio_migration_query_flags(vbasedev, &mig_flags); if (ret) { - return ret; + if (ret =3D=3D -ENOTTY) { + error_setg_errno(errp, -ret, + "migration is not supported in kernel"); + } else { + error_setg_errno(errp, -ret, "failed to query migration flags"= ); + } + + return false; } =20 /* Basic migration functionality must be supported */ if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { - return -EOPNOTSUPP; + error_setg(errp, "VFIO_MIGRATION_STOP_COPY is not supported"); + return false; } =20 vbasedev->migration =3D g_new0(VFIOMigration, 1); @@ -1113,7 +1123,7 @@ static int vfio_migration_init(VFIODevice *vbasedev) migration_add_notifier(&migration->migration_state, vfio_migration_state_notifier); =20 - return 0; + return true; } =20 static Error *multiple_devices_migration_blocker; @@ -1279,18 +1289,8 @@ bool vfio_migration_realize(VFIODevice *vbasedev, Er= ror **errp) return !vfio_block_migration(vbasedev, err, errp); } =20 - ret =3D vfio_migration_init(vbasedev); - if (ret) { - if (ret =3D=3D -ENOTTY) { - error_setg(&err, "%s: VFIO migration is not supported in kerne= l", - vbasedev->name); - } else { - error_setg(&err, - "%s: Migration couldn't be initialized for VFIO dev= ice, " - "err: %d (%s)", - vbasedev->name, ret, strerror(-ret)); - } - + if (!vfio_migration_init(vbasedev, &err)) { + error_prepend(&err, "%s: VFIO migration init failed: ", vbasedev->= name); return !vfio_block_migration(vbasedev, err, errp); } =20 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405944; cv=none; d=zohomail.com; s=zohoarc; b=IGyODSBZug5AfsLVBbhOit2AyJGKoU9FXVRJrXaDYe4NMu7YtAOtkddMj+208+4kYxrddYPZ6FDo/+aBBIyimGl3RwzPgDhezVlEfy+l/oypRY6ls/pLMvhD206eMz82dIUNFG/v0xzYQiaJL52d5PPp+rQ3SPQQZHgLNMmFzw0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405944; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qndrN4sGwm0dPF7ABTKsJ6BQ2sV1oYGYaptfwPQjKQU=; b=I+quFh8PE+9JlGPyaiA+jVug241fZ/HB4dwqAHs/dgoLWUnsZGMxXx5BrFqZA2l4f7axg4Gp5ww+aRpL8I7jGA5TZqwc8kemQ1VvlfpSB2Nu6jxpriVo52+pKAd9w+ZESmJF/0+aSKuah8i2Amb4e4J/q50AKcrytsDf2TTKU4c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17834059447881003.6974865555587; Mon, 6 Jul 2026 23:32:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJn-0001cG-1r; Tue, 07 Jul 2026 02:30:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJi-0001Oa-W6 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJh-0006yK-A9 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:02 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-665-tH5B8fgNOIyu4EOf0iV3YA-1; Tue, 07 Jul 2026 02:29:59 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5B17F1954B1E; Tue, 7 Jul 2026 06:29:58 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 90ADB180065F; Tue, 7 Jul 2026 06:29:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qndrN4sGwm0dPF7ABTKsJ6BQ2sV1oYGYaptfwPQjKQU=; b=bwIspAXxHUNGNFanRfbxN574/kbXskJcr2tKjjFRZj1ar6zfplTEf4ebrSGwb66a2Bo3vj vBvmvHetBvn1sSphdHummS7kf70TTpR0LpeXo/yMgtWm6afmhbkhRJLGe4MZepxsz3qSyb V1G/UXoiOrrzmugvi6O4DsxnJazFJzQ= X-MC-Unique: tH5B8fgNOIyu4EOf0iV3YA-1 X-Mimecast-MFC-AGG-ID: tH5B8fgNOIyu4EOf0iV3YA_1783405798 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 16/27] vfio/migration: Add new switchover-ack mechanism Date: Tue, 7 Jul 2026 08:29:09 +0200 Message-ID: <20260707062920.317302-17-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405947823158500 From: Avihai Horon Add support for the new switchover-ack mechanism. This includes requesting a switchover ACK on the first save_query_pending call (with exact=3Dfalse) if VFIO precopy is supported. This achieves the same functionality of legacy switchover-ack but with the new switchover-ack mechanism. Keep legacy switchover-ack functionality for backward compatibility. Reviewed-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-12-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/vfio-migration-internal.h | 1 + hw/vfio/migration.c | 13 ++++++++++++- hw/vfio/trace-events | 2 +- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/hw/vfio/vfio-migration-internal.h b/hw/vfio/vfio-migration-int= ernal.h index a15fc74703998da1956a92c4fb8cfe9ff3a68867..dc741e51421eec6d4ad4d9ce0f9= c0c2f6199788d 100644 --- a/hw/vfio/vfio-migration-internal.h +++ b/hw/vfio/vfio-migration-internal.h @@ -58,6 +58,7 @@ typedef struct VFIOMigration { bool multifd_transfer; VFIOMultifd *multifd; bool initial_data_sent; + bool request_switchover_ack; =20 bool event_save_iterate_started; bool event_precopy_empty_hit; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 3ab6b7248fea34dec3b58e508f97c8be4c962ce4..ebe2eafded50cdc3c1518a2f1e7= 68fdfe6722c5a 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -582,6 +582,9 @@ static int vfio_save_setup(QEMUFile *f, void *opaque, E= rror **errp) } =20 vfio_query_precopy_size(migration); + if (migrate_switchover_ack() && !migrate_switchover_ack_legacy= ()) { + migration->request_switchover_ack =3D true; + } =20 break; case VFIO_DEVICE_STATE_STOP: @@ -634,6 +637,7 @@ static void vfio_save_cleanup(void *opaque) migration->precopy_init_size =3D 0; migration->precopy_dirty_size =3D 0; migration->initial_data_sent =3D false; + migration->request_switchover_ack =3D false; vfio_migration_cleanup(vbasedev); trace_vfio_save_cleanup(vbasedev->name); } @@ -655,6 +659,7 @@ static void vfio_state_pending(void *opaque, MigPending= Data *pending, VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; uint64_t precopy_size, stopcopy_size; + bool request_switchover_ack =3D false; =20 /* * The final pending query runs during switchover downtime. VFIO does = not @@ -676,10 +681,16 @@ static void vfio_state_pending(void *opaque, MigPendi= ngData *pending, =20 pending->precopy_bytes +=3D precopy_size; pending->stopcopy_bytes +=3D stopcopy_size; + if (migration->request_switchover_ack) { + pending->switchover_ack_pending++; + request_switchover_ack =3D true; + migration->request_switchover_ack =3D false; + } =20 trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, migration->precopy_init_size, - migration->precopy_dirty_size, exact, final); + migration->precopy_dirty_size, + request_switchover_ack, exact, final); } =20 static bool vfio_is_active_iterate(void *opaque) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index fe0d2cc06f946b3f0db28d13994fbd0dfd854fbd..c9d143826d38d77048ecf179cbd= a19a7184989a4 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -176,7 +176,7 @@ vfio_save_device_config_state(const char *name) " (%s)" vfio_save_iterate(const char *name, uint64_t precopy_init_size, uint64_t p= recopy_dirty_size) " (%s) precopy initial size %"PRIu64" precopy dirty size= %"PRIu64 vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 -vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool exact, bool final) " (%s) = stopcopy size %"PRIu64", precopy initial size %"PRIu64", precopy dirty size= %"PRIu64", exact %d, final %d" +vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size, bool request_switchover_ack, bo= ol exact, bool final) " (%s) stopcopy size %"PRIu64", precopy initial size = %"PRIu64", precopy dirty size %"PRIu64", request switchover ack %d, exact %= d, final %d" vfio_send_init_data_flag(const char *name) " (%s)" vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405910; cv=none; d=zohomail.com; s=zohoarc; b=RxkIZ+R3XtR0R/8EIG73KypymXOxDRzrfYwirAGCx8DMonvSgNUYpyQuE8cLu80iXhVJ/OcULh0GXSeSwB0DOHz68EjtQ4Hrlk0DfWzQ48px2Dhj4LaihMZZYTta8h3tLtsc4H1QcGfN9PNYS4o3EHWmGoaoAJpRjE4Gq9Pdzoc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405910; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=4VPifjRv5xHUPc53gqToCUkVHVOxByVu5PyMZz6oKZA=; b=m0kEC/r/3iO4/Q73sFkytXCfIyWXvQvXao/AAG0hMTfaCcrx67ufIDunF/ydb13kicc04WebIL7hRBohocBAPghPTDOCvn6R44ogOw6y8nMBFO8RHhyfON8wu8WhK26BnCZHZqqEn10Dv3Bn7x8jVtHEW5hKbqqp9Jv9o55ZpwU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405910637344.1529525107361; Mon, 6 Jul 2026 23:31:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJq-0001sA-8R; Tue, 07 Jul 2026 02:30:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJn-0001hl-Vr for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJl-0007Ag-9d for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:07 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-7oroddmMNgGJF_ZPijIwsA-1; Tue, 07 Jul 2026 02:30:01 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 71C6A1954B17; Tue, 7 Jul 2026 06:30:00 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CBD2B1800678; Tue, 7 Jul 2026 06:29:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405804; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4VPifjRv5xHUPc53gqToCUkVHVOxByVu5PyMZz6oKZA=; b=idjOqNnKd0Tb0/TYGXytKXN16DIfAKTZZ9ShuP9s0V81RaM2zxzj89HrIJVFR5YWZe2nXX ozC8dXHTJeDM+heHU4ru3+CJ5JY1I+zKNHn+u+jibBDUjLNNi3+JOtH4OM7vXz/PAkpk8t tKawfl90wsja+/FPGmFK/WRZPo0mWKU= X-MC-Unique: 7oroddmMNgGJF_ZPijIwsA-1 X-Mimecast-MFC-AGG-ID: 7oroddmMNgGJF_ZPijIwsA_1783405800 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 17/27] vfio/migration: Implement VFIO_PRECOPY_INFO_REINIT feature Date: Tue, 7 Jul 2026 08:29:10 +0200 Message-ID: <20260707062920.317302-18-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405912771158500 From: Avihai Horon According to VFIO uAPI, precopy initial_bytes is considered as critical data that should be transferred and loaded prior to moving to STOP_COPY state to ensure precopy phase would be effective. As currently defined, initial_bytes can only decrease as it's being read from the data fd. However, there are cases where a new chunk of initial_bytes should be transferred during precopy. The new VFIO_PRECOPY_INFO_REINIT feature addresses this and allows reporting a new value for initial_bytes regardless of any previously reported values. Implement VFIO_PRECOPY_INFO_REINIT feature: 1. Opt-in for VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2 to make VFIO_PRECOPY_INFO_REINIT available. 2. Request a new switchover ACK if initial_bytes increases post of a previous switchover ACK. This ensures the device is not moved to STOP_COPY before initial_bytes has reached zero again. Acked-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-13-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- docs/devel/migration/vfio.rst | 14 +++++++ hw/vfio/vfio-migration-internal.h | 1 + hw/vfio/migration.c | 68 ++++++++++++++++++++++++++++--- hw/vfio/trace-events | 4 +- 4 files changed, 80 insertions(+), 7 deletions(-) diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 854277b11ce64672f998ce1728bf8b5ddf1c129a..f235c2d4f9d1bae5aa8bdcc2701= 340780d558f89 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -23,6 +23,20 @@ and recommends that the initial bytes are sent and loade= d in the destination before stopping the source VM. Enabling this migration capability will guarantee that and thus, can potentially reduce downtime even further. =20 +For example, in mlx5 devices, the initial bytes hold metadata used for time +consuming pre-allocations of resources on the destination. Although init b= ytes +may be small in size and sending them may take little time, loading them i= n the +destination can take a significant amount of time. Switchover-ack guarante= es +that this pre-allocation doesn't happen during downtime. + +Initial bytes was originally defined to be monotonically decreasing, howev= er +there are cases where a new chunk of initial bytes should be transferred d= uring +precopy, e.g., due to a device reconfiguration, etc. The +VFIO_PRECOPY_INFO_REINIT feature addresses this and when supported, allows= to +report a new initial bytes value regardless of any previously reported val= ues. +In this case, a new switchover ACK will be requested to make sure the new +initial bytes are loaded in the destination before switching over. + To support migration of multiple devices that might do P2P transactions be= tween themselves, VFIO migration uAPI defines an intermediate P2P quiescent stat= e. While in the P2P quiescent state, P2P DMA transactions cannot be initiated= by diff --git a/hw/vfio/vfio-migration-internal.h b/hw/vfio/vfio-migration-int= ernal.h index dc741e51421eec6d4ad4d9ce0f9c0c2f6199788d..a1c58b112685d868c6f942d6225= b553d44bb6613 100644 --- a/hw/vfio/vfio-migration-internal.h +++ b/hw/vfio/vfio-migration-internal.h @@ -45,6 +45,7 @@ typedef struct VFIOMigration { void *data_buffer; size_t data_buffer_size; uint64_t mig_flags; + bool precopy_info_v2_used; /* * NOTE: all three sizes cached are reported from VFIO's uAPI, which * are defined as estimate only. QEMU should not trust these values diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index ebe2eafded50cdc3c1518a2f1e768fdfe6722c5a..1e172dd10bfc3561741aa90bf1e= 237cd6847eb63 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -373,9 +373,11 @@ static int vfio_query_stop_copy_size(VFIODevice *vbase= dev) =20 static int vfio_query_precopy_size(VFIOMigration *migration) { + VFIODevice *vbasedev =3D migration->vbasedev; struct vfio_precopy_info precopy =3D { .argsz =3D sizeof(precopy), }; + bool reinit =3D false; int ret =3D 0; =20 if (ioctl(migration->data_fd, VFIO_MIG_GET_PRECOPY_INFO, &precopy)) { @@ -383,25 +385,43 @@ static int vfio_query_precopy_size(VFIOMigration *mig= ration) migration->precopy_dirty_size =3D 0; ret =3D -errno; warn_report_once("VFIO device %s ioctl(VFIO_MIG_GET_PRECOPY_INFO) " - "failed (%d)", migration->vbasedev->name, ret); + "failed (%d)", vbasedev->name, ret); } else { bool overflow; =20 migration->precopy_init_size =3D precopy.initial_bytes; migration->precopy_dirty_size =3D precopy.dirty_bytes; + /* + * struct vfio_precopy_info.flags is valid only if + * VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2 is used. + */ + if (migration->precopy_info_v2_used) { + reinit =3D precopy.flags & VFIO_PRECOPY_INFO_REINIT; + } =20 - overflow =3D vfio_migration_check_overflow(migration->vbasedev, + overflow =3D vfio_migration_check_overflow(vbasedev, migration->precopy_init_size, "precopy init size= "); - overflow |=3D vfio_migration_check_overflow(migration->vbasedev, + overflow |=3D vfio_migration_check_overflow(vbasedev, migration->precopy_dirty_size, "precopy dirty siz= e"); if (overflow) { ret =3D -ERANGE; } } =20 - trace_vfio_query_precopy_size(migration->vbasedev->name, - migration->precopy_init_size, - migration->precopy_dirty_size, ret); + trace_vfio_query_precopy_size(vbasedev->name, migration->precopy_init_= size, + migration->precopy_dirty_size, reinit, r= et); + + /* + * If we got new initial_bytes after previous initial_bytes were + * transferred, request a new switchover ACK. Don't request if legacy + * switchover-ack is used. + */ + if (reinit && migration->initial_data_sent && + !migrate_switchover_ack_legacy()) { + migration->initial_data_sent =3D false; + migration->request_switchover_ack =3D true; + trace_vfio_query_precopy_size_request_switchover_ack(vbasedev->nam= e); + } =20 return ret; } @@ -1054,6 +1074,27 @@ static int vfio_migration_query_flags(VFIODevice *vb= asedev, uint64_t *mig_flags) return 0; } =20 +/* Returns 1 on success, 0 if not supported and negative errno on failure = */ +static int vfio_migration_set_precopy_info_v2(VFIODevice *vbasedev) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D + VFIO_DEVICE_FEATURE_SET | VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + if (errno =3D=3D ENOTTY) { + return 0; + } + + return -errno; + } + + return 1; +} + static bool vfio_dma_logging_supported(VFIODevice *vbasedev) { uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), @@ -1075,6 +1116,7 @@ static bool vfio_migration_init(VFIODevice *vbasedev,= Error **errp) char id[256] =3D ""; g_autofree char *path =3D NULL, *oid =3D NULL; uint64_t mig_flags =3D 0; + bool precopy_info_v2_used =3D false; VMChangeStateHandler *prepare_cb; =20 if (!vbasedev->ops->vfio_get_object) { @@ -1106,12 +1148,22 @@ static bool vfio_migration_init(VFIODevice *vbasede= v, Error **errp) return false; } =20 + if (mig_flags & VFIO_MIGRATION_PRE_COPY) { + ret =3D vfio_migration_set_precopy_info_v2(vbasedev); + if (ret < 0) { + error_setg_errno(errp, -ret, "failed to set precopy info v2"); + return false; + } + precopy_info_v2_used =3D ret; + } + vbasedev->migration =3D g_new0(VFIOMigration, 1); migration =3D vbasedev->migration; migration->vbasedev =3D vbasedev; migration->device_state =3D VFIO_DEVICE_STATE_RUNNING; migration->data_fd =3D -1; migration->mig_flags =3D mig_flags; + migration->precopy_info_v2_used =3D precopy_info_v2_used; =20 vbasedev->dirty_pages_supported =3D vfio_dma_logging_supported(vbasede= v); =20 @@ -1134,6 +1186,10 @@ static bool vfio_migration_init(VFIODevice *vbasedev= , Error **errp) migration_add_notifier(&migration->migration_state, vfio_migration_state_notifier); =20 + trace_vfio_migration_init(vbasedev->name, migration->mig_flags, + migration->precopy_info_v2_used, + vbasedev->dirty_pages_supported); + return true; } =20 diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index c9d143826d38d77048ecf179cbda19a7184989a4..f71d0bbc0a5440e4ccd374fac47= 33af08438e7be 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -159,11 +159,13 @@ vfio_load_state_device_buffer_starved(const char *nam= e, uint32_t idx) " (%s) idx vfio_load_state_device_buffer_load_start(const char *name, uint32_t idx) "= (%s) idx %"PRIu32 vfio_load_state_device_buffer_load_end(const char *name, uint32_t idx) " (= %s) idx %"PRIu32 vfio_load_state_device_buffer_end(const char *name) " (%s)" +vfio_migration_init(const char *name, uint64_t mig_flags, bool precopy_inf= o_v2_used, bool dirty_pages_supported) " (%s) mig_flags 0x%"PRIx64", precop= y_info_v2_used %d, dirty_pages_supported %d" vfio_migration_realize(const char *name) " (%s)" vfio_migration_set_device_state(const char *name, const char *state) " (%s= ) state %s" vfio_migration_set_state(const char *name, const char *new_state, const ch= ar *recover_state) " (%s) new state %s, recover state %s" vfio_migration_state_notifier(const char *name, int state) " (%s) state %d" -vfio_query_precopy_size(const char *name, uint64_t init_size, uint64_t dir= ty_size, int ret) " (%s) init %"PRIu64" dirty %"PRIu64" ret %d" +vfio_query_precopy_size(const char *name, uint64_t init_size, uint64_t dir= ty_size, bool reinit, int ret) " (%s) init %"PRIu64", dirty %"PRIu64", rein= it %d, ret %d" +vfio_query_precopy_size_request_switchover_ack(const char *name) " (%s)" vfio_query_stop_copy_size(const char *name, uint64_t size, int ret) " (%s)= stopcopy size %"PRIu64" ret %d" vfio_save_block(const char *name, int data_size) " (%s) data_size %d" vfio_save_block_precopy_empty_hit(const char *name) " (%s)" --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405837; cv=none; d=zohomail.com; s=zohoarc; b=folrHosvfdDh0ZtY85H3ST9HphIV9SswOOKCJbZthuQUgtlIkw5JjYaVBf6hjAPrNPd3Az7CVKrAsQ1MM2tIbEPyFjLWYnPnOytkYlakSBUPlKkdBBVo6qVJ5qvSebxZkreQBIYILWVplL300yrOorZNQ7mI2FaN2Ul7v5BXQIA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405837; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=llDNiwH7hZoMhlkvQSWhEOq7f9CVRFUm0UL7PJOc8Ro=; b=hu2nupUJnC3QSuBvmNPaOOYMSlw/I9PjCWEsVw3FfUCLlk5s6KrI8PxXSkWsoA436EFhT9PKssCVfBpEf0mZ1DdYzl3V02Y0x53qohrZ9kJidb3g7IrwtDWwAOmj8AcNJ7Mcx4R+fS7P2wBCtKlQ+IOKUKlw2nK1Zri1P6K/2pE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405837741765.8549417802694; Mon, 6 Jul 2026 23:30:37 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJu-0002Ay-4o; Tue, 07 Jul 2026 02:30:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJp-0001pc-NA for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJn-0007At-Dm for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:09 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-241-RiYWf6NIMia722pHe2S6pA-1; Tue, 07 Jul 2026 02:30:03 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 786621954B1D; Tue, 7 Jul 2026 06:30:02 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E14E8180067C; Tue, 7 Jul 2026 06:30:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405806; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=llDNiwH7hZoMhlkvQSWhEOq7f9CVRFUm0UL7PJOc8Ro=; b=Nyae5wJ3Xw+ziPbPi4jDO9IB+4Whebjed331Hh+0dFKZ0trnmXe46DUcJZPNm4/VuSNjmA Qr+1IRnYswIFUON+/bil+ATyfiyWQUaLymG/4AXcrBWQEs5LUD5l3oCpyxZTs7ZIn8o5UP s4lxQ3B+EmE+Maw+wll1V9U89ekUskE= X-MC-Unique: RiYWf6NIMia722pHe2S6pA-1 X-Mimecast-MFC-AGG-ID: RiYWf6NIMia722pHe2S6pA_1783405802 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , Peter Xu , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 18/27] vfio/migration: Check VFIO_PRECOPY_INFO_REINIT during switchover Date: Tue, 7 Jul 2026 08:29:11 +0200 Message-ID: <20260707062920.317302-19-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405840169158500 From: Avihai Horon VFIO_REPCOPY_INFO_REINIT is checked only during precopy, before the switchover decision. However, the switchover decision and guest stop are not atomic, so a VFIO device may want to set VFIO_PRECOPY_INFO_REINIT and request another switchover ACK in the gap after switchover decision has been made but before the guest is stopped. This would be missed and may increase downtime. Solve this by checking if VFIO_PRECOPY_INFO_REINIT was set during that gap, and request a new switchover-ack in the final save_state_pending call. Query precopy info after vCPUs are stopped but before transitioning from PRE_COPY state, when its valid to call the ioctl. Acked-by: Peter Xu Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-14-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/migration.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 1e172dd10bfc3561741aa90bf1e237cd6847eb63..73c49d8c24f21803de7c8615da6= 3abecfd76b882 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -685,6 +685,9 @@ static void vfio_state_pending(void *opaque, MigPending= Data *pending, * The final pending query runs during switchover downtime. VFIO does = not * need a fresh device pending-data query then to get the latest dirty * data, so avoid the extra work and report the cached counters below. + * On the other hand, precopy sync is needed to check if switchover AC= K was + * requested, but that's already done during guest stop when device is= in + * PRE_COPY state. */ if (exact && !final) { vfio_state_pending_sync(vbasedev); @@ -964,6 +967,26 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { =20 /* ---------------------------------------------------------------------- = */ =20 +static void vfio_final_precopy_reinit_check(VFIODevice *vbasedev) +{ + VFIOMigration *migration =3D vbasedev->migration; + int ret; + + if (!migration->precopy_info_v2_used || !migrate_switchover_ack() || + migrate_switchover_ack_legacy()) { + return; + } + + ret =3D vfio_query_precopy_size(migration); + if (ret) { + error_report("%s: Final precopy reinit check failed (err: %d)", + vbasedev->name, ret); + /* If query failed, assume reinit and request switchover-ack */ + migration->request_switchover_ack =3D true; + migration->initial_data_sent =3D false; + } +} + static void vfio_vmstate_change_prepare(void *opaque, bool running, RunState state) { @@ -977,6 +1000,15 @@ static void vfio_vmstate_change_prepare(void *opaque,= bool running, VFIO_DEVICE_STATE_PRE_COPY_P2P : VFIO_DEVICE_STATE_RUNNING_P2P; =20 + if (migration->device_state =3D=3D VFIO_DEVICE_STATE_PRE_COPY) { + /* + * Now that vCPUs are stopped, check if new init_bytes are availab= le + * since switchover decision, to be reported in the final + * save_query_pending. + */ + vfio_final_precopy_reinit_check(vbasedev); + } + ret =3D vfio_migration_set_state_or_reset(vbasedev, new_state, &local_= err); if (ret) { /* --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405912; cv=none; d=zohomail.com; s=zohoarc; b=ZOYPWtqKoym90T6+ZrWalLEqK/0PZfeGvhvGm/UN5/y/2pp7zkA28avoUYn/iXV8EkioYlARCAKjQaQhCwK43x3ka2UMT4p/D9PusRCkMW1iZHRdUZddofQYBSGPV+VXTVG4MEcw1jjm/iS4SDhwvwBJXXnuCrbs0RGbDxy7qVg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405912; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dpN74gRauG5t9btP42/ytdQm2tBLFJhOgrCQthQDpzQ=; b=P2KT6wqRSRWliR8JAmodbi5Hbhb1X1sKPLI73cc64YOSubONktAcyjH38p791CbI5fccF6w5IayJXvx/CC1OdZ2HQyGkmeM86nVd90OHCGwTI3MCBOniwhKncQJ8a53GJsBy7/+bxDLI8wB3WfAcR1BWzOs3YM+d2IigJMYL+nI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405912247799.1835196400857; Mon, 6 Jul 2026 23:31:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJs-00023R-D6; Tue, 07 Jul 2026 02:30:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJq-0001tQ-Ll for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJo-0007B9-Sx for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:10 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-711-kQCHw2tTNEaub131XZusNg-1; Tue, 07 Jul 2026 02:30:05 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 263311955DBA; Tue, 7 Jul 2026 06:30:04 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E777B180065F; Tue, 7 Jul 2026 06:30:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dpN74gRauG5t9btP42/ytdQm2tBLFJhOgrCQthQDpzQ=; b=idmG5cLIJn4WHIkbIm/+YmGDeikov0eFTBnyg3yaxSC9SWc+SrPu5Kgy2/YtFWGEcXnPi/ tZiStMORqFbJkjimpd05kBC4Blfk25dD7jLiRMGrdSzYd54uttI0QWzJxGwAo4SYJ/ANeQ UkykP++TZuuIFSuJjh9tP9/Mh95NWzs= X-MC-Unique: kQCHw2tTNEaub131XZusNg-1 X-Mimecast-MFC-AGG-ID: kQCHw2tTNEaub131XZusNg_1783405804 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 19/27] migration: Enable new switchover-ack Date: Tue, 7 Jul 2026 08:29:12 +0200 Message-ID: <20260707062920.317302-20-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405912611158500 From: Avihai Horon Now that VFIO has implemented new switchover-ack, enable it for new machines and keep legacy behavior for old machines. Reviewed-by: C=C3=A9dric Le Goater Signed-off-by: Avihai Horon Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-15-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- hw/core/machine.c | 1 + migration/options.c | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index 5b97df6db962476ed6fea5444d29e6cbaa810feb..15886a56b194239efba3db54e96= 1b612c70dbaa1 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -48,6 +48,7 @@ GlobalProperty hw_compat_11_0[] =3D { { TYPE_ARM_SMMUV3, "ril", "on" }, { TYPE_ARM_SMMUV3, "ssidsize", "0" }, { TYPE_ARM_SMMUV3, "oas", "44" }, + { "migration", "switchover-ack-legacy", "on" }, }; const size_t hw_compat_11_0_len =3D G_N_ELEMENTS(hw_compat_11_0); =20 diff --git a/migration/options.c b/migration/options.c index 4c9b25372e03b55ee9127c027c5d4eefba796b06..dfce19405d4583894c2d22578a8= e1d32da613d54 100644 --- a/migration/options.c +++ b/migration/options.c @@ -110,9 +110,8 @@ const Property migration_properties[] =3D { preempt_pre_7_2, false), DEFINE_PROP_BOOL("multifd-clean-tls-termination", MigrationState, multifd_clean_tls_termination, true), - /* Use legacy until VFIO implements new switchover-ack */ DEFINE_PROP_BOOL("switchover-ack-legacy", MigrationState, - switchover_ack_legacy, true), + switchover_ack_legacy, false), =20 /* Migration parameters */ DEFINE_PROP_UINT8("x-throttle-trigger-threshold", MigrationState, --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405897; cv=none; d=zohomail.com; s=zohoarc; b=QyZ3r7U94XLxxTMPs5WxCTj+hFX4RDt25YCSaSVm7gUdGtF2Q7NCF2bkKgu1dH71IHuxvu57U4TFnhV1xrvWg4MsrGhpKpdASEPQdYmAAVl2+oMubh1wp7ZnSM8zedzsmWh4e20BC0cJ2C5F5yqZ4Ha9l3fr0umOG6pWu8h6Z00= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405897; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ux4Zx6CrlDPxpVtI0bDiKKaSEwHrXHcs532LFmuX6oQ=; b=LMQuNs6dcOt6SL5XQUa1e48lfIR6lTVwEBW3pOxCxqtdZSkl8tWHnezDjkxVu/c1d1PX5ZTBLnfJvKF88UT5hjdN0suAHGsfm0xnKmuBRxJoimOLI9ZTes4q9o1CyUgVyvcpLaMhDUIm+3gdcEg4ktYlHuzEbC7cGaEXOl5cAoU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405897310848.6319870613714; Mon, 6 Jul 2026 23:31:37 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJw-0002NK-Q5; Tue, 07 Jul 2026 02:30:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJu-0002Bo-ET for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJr-0007Bk-Po for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:13 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-39-8nfOB6ioOUydLRei3r57GA-1; Tue, 07 Jul 2026 02:30:06 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C702C1954ADC; Tue, 7 Jul 2026 06:30:05 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 949BA180065F; Tue, 7 Jul 2026 06:30:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ux4Zx6CrlDPxpVtI0bDiKKaSEwHrXHcs532LFmuX6oQ=; b=MqV1Z24c2KXFJ60usmmbMTeAM+cLI2DJQg3s0nz7JCuZkvxcKcfLzxZu8vVjkdt0Rr8906 8asOGYZnFYodK8eGrvsDSeRBUDHqn30AGRW+ZZjomV6qAYHrn9scoOxSAKOdVZ+bZHoxJQ /eumGdOtlAoHqY/NNSmmdzbU7vcd0w4= X-MC-Unique: 8nfOB6ioOUydLRei3r57GA-1 X-Mimecast-MFC-AGG-ID: 8nfOB6ioOUydLRei3r57GA_1783405805 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 20/27] migration: Refactor migration_completion_precopy() to return bool Date: Tue, 7 Jul 2026 08:29:13 +0200 Message-ID: <20260707062920.317302-21-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405898645158500 From: Avihai Horon migration_completion_precopy() reports its error through the Error **errp argument, so its int return value carries no information beyond success/failure. Convert it to return a bool, matching the common convention. Convert its underlying helper qemu_savevm_state_complete_precopy() likewise, and in turn qemu_savevm_state_non_iterable(), which it calls. Adjust all callers accordingly. Refactor and clean migration_completion() code too, which no longer needs to track int return values. Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-16-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- migration/savevm.h | 4 ++-- migration/migration.c | 35 ++++++++++++++--------------------- migration/savevm.c | 32 +++++++++++++++----------------- 3 files changed, 31 insertions(+), 40 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index 415198423f578049c209a1e40b796cfe63b410f0..f7dcdd7d1b6b7b5397eb1b6aab5= cb5816a92f1a8 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -44,7 +44,7 @@ void qemu_savevm_state_header(QEMUFile *f); int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); -int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); +bool qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp); void qemu_savevm_query_pending_iter(MigrationState *s, MigPendingData *pen= ding, bool exact); bool qemu_savevm_query_pending_final(MigrationState *s, @@ -74,7 +74,7 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncoming= State *mis, Error **errp); int qemu_load_device_state(QEMUFile *f, Error **errp); int qemu_loadvm_approve_switchover(const char *approver); -int qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp); +bool qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp); int qemu_savevm_state_non_iterable_early(QEMUFile *f, JSONWriter *vmdesc, Error **errp); diff --git a/migration/migration.c b/migration/migration.c index 98603ee19c34531d069400d1ab0ed36b65bb228e..7d5acd93c32393c9611731ab153= b23f4bbc826a3 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2659,8 +2659,7 @@ static int postcopy_start(MigrationState *ms, Error *= *errp) */ qemu_savevm_send_postcopy_listen(fb); =20 - ret =3D qemu_savevm_state_non_iterable(fb, errp); - if (ret) { + if (!qemu_savevm_state_non_iterable(fb, errp)) { error_prepend(errp, "Postcopy save non-iterable states failed: "); goto fail_closefb; } @@ -2858,22 +2857,22 @@ static bool migration_switchover_start(MigrationSta= te *s, Error **errp) return true; } =20 -static int migration_completion_precopy(MigrationState *s, Error **errp) +static bool migration_completion_precopy(MigrationState *s, Error **errp) { - int ret; + bool ret =3D false; =20 bql_lock(); =20 if (!migrate_mode_is_cpr()) { - ret =3D migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE); - if (ret < 0) { - error_setg_errno(errp, -ret, "Failed to stop the VM"); + int r =3D migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE); + + if (r < 0) { + error_setg_errno(errp, -r, "Failed to stop the VM"); goto out_unlock; } } =20 if (!migration_switchover_start(s, errp)) { - ret =3D -EFAULT; goto out_unlock; } =20 @@ -2910,18 +2909,17 @@ static void migration_completion_postcopy(Migration= State *s) */ static void migration_completion(MigrationState *s) { - int ret =3D 0; Error *local_err =3D NULL; =20 if (s->state =3D=3D MIGRATION_STATUS_ACTIVE) { - ret =3D migration_completion_precopy(s, &local_err); + if (!migration_completion_precopy(s, &local_err)) { + goto fail; + } } else if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE) { migration_completion_postcopy(s); } else { - ret =3D -1; - } - - if (ret < 0) { + error_setg(&local_err, "Unexpected migration completion status %s", + MigrationStatus_str(s->state)); goto fail; } =20 @@ -2945,12 +2943,7 @@ static void migration_completion(MigrationState *s) return; =20 fail: - if (local_err) { - migrate_error_propagate(s, local_err); - } else if (qemu_file_get_error_obj(s->to_dst_file, &local_err)) { - migrate_error_propagate(s, local_err); - } else if (ret) { - error_setg_errno(&local_err, -ret, "Error in migration completion"= ); + if (local_err || qemu_file_get_error_obj(s->to_dst_file, &local_err)) { migrate_error_propagate(s, local_err); } =20 @@ -3863,7 +3856,7 @@ static void *bg_migration_thread(void *opaque) goto fail_with_bql; } =20 - if (qemu_savevm_state_non_iterable(fb, &local_err)) { + if (!qemu_savevm_state_non_iterable(fb, &local_err)) { error_prepend(&local_err, "Failed to save non-iterable devices "); goto fail_with_bql; } diff --git a/migration/savevm.c b/migration/savevm.c index f654a1c563df40d546e42018a3706e92fb11cfa7..bbb8d2b47c6452b228c67cbbdaa= cc7c333985be7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1737,13 +1737,12 @@ void qemu_savevm_state_end_precopy(MigrationState *= s, QEMUFile *f) qemu_savevm_state_vm_desc(s, f); } =20 -int qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp) +bool qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp) { MigrationState *ms =3D migrate_get_current(); int64_t start_ts_each, end_ts_each; JSONWriter *vmdesc =3D ms->vmdesc; SaveStateEntry *se; - int ret; =20 /* Making sure cpu states are synchronized before saving non-iterable = */ cpu_synchronize_all_states(); @@ -1756,9 +1755,8 @@ int qemu_savevm_state_non_iterable(QEMUFile *f, Error= **errp) =20 start_ts_each =3D qemu_clock_get_us(QEMU_CLOCK_REALTIME); =20 - ret =3D vmstate_save(f, se, vmdesc, errp); - if (ret) { - return ret; + if (vmstate_save(f, se, vmdesc, errp) < 0) { + return false; } =20 end_ts_each =3D qemu_clock_get_us(QEMU_CLOCK_REALTIME); @@ -1768,10 +1766,10 @@ int qemu_savevm_state_non_iterable(QEMUFile *f, Err= or **errp) =20 trace_vmstate_downtime_checkpoint("src-non-iterable-saved"); =20 - return 0; + return true; } =20 -int qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp) +bool qemu_savevm_state_complete_precopy(MigrationState *s, Error **errp) { ERRP_GUARD(); QEMUFile *f =3D s->to_dst_file; @@ -1781,12 +1779,11 @@ int qemu_savevm_state_complete_precopy(MigrationSta= te *s, Error **errp) if (ret) { qemu_file_get_error_obj(f, errp); error_prepend(errp, "Failed to save iterable device state: "); - return ret; + return false; } =20 - ret =3D qemu_savevm_state_non_iterable(f, errp); - if (ret) { - return ret; + if (!qemu_savevm_state_non_iterable(f, errp)) { + return false; } =20 qemu_savevm_state_end_precopy(s, f); @@ -1795,10 +1792,10 @@ int qemu_savevm_state_complete_precopy(MigrationSta= te *s, Error **errp) if (ret) { qemu_file_get_error_obj(f, errp); error_prepend(errp, "Failed to flush QEMUFile: "); - return ret; + return false; } =20 - return 0; + return true; } =20 static void qemu_savevm_query_pending(MigrationState *s, @@ -1928,7 +1925,9 @@ static int qemu_savevm_state(QEMUFile *f, Error **err= p) goto cleanup; } =20 - ret =3D qemu_savevm_state_complete_precopy(ms, errp); + if (!qemu_savevm_state_complete_precopy(ms, errp)) { + ret =3D -1; + } cleanup: qemu_savevm_state_cleanup(); =20 @@ -1962,9 +1961,8 @@ int qemu_save_device_state(QEMUFile *f, Error **errp) return ret; } =20 - ret =3D qemu_savevm_state_non_iterable(f, errp); - if (ret) { - return ret; + if (!qemu_savevm_state_non_iterable(f, errp)) { + return -1; } =20 qemu_savevm_state_end(f); --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405860; cv=none; d=zohomail.com; s=zohoarc; b=d+AQvfnlFH+fZvkl9MTmayvZijP9Q10bB3u3v5yBoJmOrd9AluzymSbTymkbK3jHqYmnpJA/LQJbf6ikZVLM4Wxx8ySXrIjq+Na0cYLExitj79DLnIqh3HGxZFH1uU+S54rf7lFA5qY0qyNdTNbXv2QzlDtG80PvsaqFmcLn68U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405860; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AFHGy4tBq00t0R5eWGT8REbeTwBgvPuvolIkuJG4p7g=; b=TOZawcFqUOrv7YehtY1loM0LA9Nz2vJshF9+VeKA7xt0q6Y3Wu6rCHobB7ELfCDW6y6XVy2CGgmFj+PCJsjJvKPlDXyTZCofvKWvh/mhocTruVK4FhtJWHG1aneg4WCUJFB6o5VabvpF+LPvRSarQyR+qSGOzRiCtXqU608rOJc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405860830789.5373761602851; Mon, 6 Jul 2026 23:31:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzJw-0002Mw-NB; Tue, 07 Jul 2026 02:30:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJu-0002Da-LD for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJs-0007Bn-9O for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:14 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-511-s3GkJHXAMM6IKGEyl61HAw-1; Tue, 07 Jul 2026 02:30:09 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 77C571806CA2; Tue, 7 Jul 2026 06:30:08 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 429A6180065F; Tue, 7 Jul 2026 06:30:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AFHGy4tBq00t0R5eWGT8REbeTwBgvPuvolIkuJG4p7g=; b=KbkAfrZiYPtuvr8+W/3i5DyeUTitdxJznLMPe+1P752ZTe4XJNIEOysYFyp5Nj8a+4b4P8 KDev/ly2NUU9zn2ROA5Z3Kd87Gr+YPcLlS+87an1pCRAjN5dKgUTC4ut6j4Mw9eX7KOpod eLzVKAffqvkCGMVsVCtNObCUyC0rVss= X-MC-Unique: s3GkJHXAMM6IKGEyl61HAw-1 X-Mimecast-MFC-AGG-ID: s3GkJHXAMM6IKGEyl61HAw_1783405808 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 21/27] migration: Fix "switchover" used as a verb in comments and docs Date: Tue, 7 Jul 2026 08:29:14 +0200 Message-ID: <20260707062920.317302-22-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405862338158500 From: Avihai Horon "Switchover" is a noun; the verb form is "switch over". Replace all instances where "switchover" was incorrectly used as a verb in comments and documentation. Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-17-avihaih@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- qapi/migration.json | 4 ++-- migration/migration.h | 6 +++--- migration/migration.c | 4 ++-- migration/savevm.c | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index d2f42c4b0ffd0a9e6bb757b8d105fed89b423537..66bd900465a2d757baf863baa09= 37a18b4585d2a 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -509,7 +509,7 @@ # # @switchover-ack: If enabled, migration will not stop the source VM # and complete the migration until the destination has -# acknowledged that it is OK to switchover. The acknowledgement +# acknowledged that it is OK to switch over. The acknowledgement # may depend, for example, on some device's data being loaded in # the destination before doing switchover. This can reduce # downtime if devices that support this capability are present. @@ -919,7 +919,7 @@ # migration can use during switchover phase, in bytes per # second. **Note:** this does not limit the bandwidth during # switchover, but only for calculations when making decisions to -# switchover. By default, this value is zero, which means QEMU +# switch over. By default, this value is zero, which means QEMU # will estimate the bandwidth automatically. This can be set # when the estimated value is not accurate, while the user is # able to guarantee such bandwidth is available when switching diff --git a/migration/migration.h b/migration/migration.h index 44bce199cd8c3518998ad7204d599566d1ef9ddd..631421c784bc1cee7073ff6ff00= 11188fc810592 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -496,12 +496,12 @@ struct MigrationState { * migration progresses, the devices on the destination acknowledge * switchover, decreasing the counter. When the counter reaches zero, a * single ACK message is sent to the source via the return path, indic= ating - * that it's OK to switchover. + * that it's OK to switch over. * * In new switchover-ack, the source is the one that keeps track of a * pending ACKs counter. As migration progresses, the destination send= s ACK * message per-device via the return path, which decrements the source - * counter. When the counter reaches zero, it's OK to switchover. Duri= ng + * counter. When the counter reaches zero, it's OK to switch over. Dur= ing * precopy, source-side devices may request additional ACKs, which inc= rement * the counter again. * @@ -521,7 +521,7 @@ struct MigrationState { /* * Indicates the number of pending ACKs from the destination. The valu= e may * increase or decrease during precopy as new ACKs are requested or - * received. When zero is reached, it's OK to switchover. In legacy + * received. When zero is reached, it's OK to switch over. In legacy * switchover-ack, it's initialized to 1 and decreased to zero upon AC= K. */ uint32_t switchover_ack_pending_num; diff --git a/migration/migration.c b/migration/migration.c index 7d5acd93c32393c9611731ab153b23f4bbc826a3..9736c14458b9eb87e14ac242e7a= ff0c6b5bf6900 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3293,7 +3293,7 @@ static bool migration_iteration_next_ready(MigrationS= tate *s, MigPendingData *pending) { /* - * If the estimated values already suggest us to switchover, mark this + * If the estimated values already suggest us to switch over, mark this * iteration finished, time to do a slow sync. */ if (pending->total_bytes <=3D s->threshold_size) { @@ -3418,7 +3418,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) migration_iteration_go_next(s, &pending); } =20 - /* Check can switchover after qemu_savevm_query_pending() */ + /* Check if we can switch over after qemu_savevm_query_pending() */ can_switchover =3D migration_can_switchover(s); =20 /* Should we switch to postcopy now? */ diff --git a/migration/savevm.c b/migration/savevm.c index bbb8d2b47c6452b228c67cbbdaacc7c333985be7..34dd06f9f732dee7d3c6d0458da= 8bb67f23d76b4 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2509,7 +2509,7 @@ static int loadvm_postcopy_handle_switchover_start(Er= ror **errp) =20 /* * If legacy switchover-ack is enabled but no device uses it, need to send= an - * ACK to source that it's OK to switchover. + * ACK to source that it's OK to switch over. */ static int loadvm_switchover_ack_no_users_legacy(MigrationIncomingState *m= is, Error **errp) --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405902; cv=none; d=zohomail.com; s=zohoarc; b=OKWnVln9z18MpRdvcc1uMFxMK8YKd4PGBCjK68EiJ5XEqhC9PEKfmTSNynfbRTaWBt6cvkYF1ab8BBPGK8MmHS6Oa9n3POIEjeIsonjY8cpUdNjXBHTDUFxucyxPa08ZWl6Te2IwoEpTnw1Zi8X6HUlIAFa3GH973dsUK0j0vkQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405902; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/BaSE9iED73RydcdKRYU3/Gp7j/oPk8CVubFRkdDiyk=; b=Kx26z7IoZvodRtlQEBzFSnnlkL7v9R46tiBWLHpuC40GU5qYIF4RHEKCJjnSzOORtDnalLQW+TRhh58hQ86ic1RXf1SXTQVqmAUp5jbxgjyWXnqDD2JI/RSJxSUCLwEFAQkC4BICZuzQvNKC7b9BZp+Rwd4VL26l5kmCF8nbdaQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405902465532.5793400750557; Mon, 6 Jul 2026 23:31:42 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzK2-0002dF-40; Tue, 07 Jul 2026 02:30:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzK0-0002Vl-1s for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJy-0007E1-7s for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:19 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-341-LuDD0-f2PFCGXmX7ooiG7Q-1; Tue, 07 Jul 2026 02:30:11 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 603141954B08; Tue, 7 Jul 2026 06:30:10 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E76C5180065F; Tue, 7 Jul 2026 06:30:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/BaSE9iED73RydcdKRYU3/Gp7j/oPk8CVubFRkdDiyk=; b=KxPBcv9jFmsNn1eUz0rATbRC9L1mMDV+Lkg25KhrQ/BYXNW0M6O/bSdXcMOaTbJ3N+c5al s7Hpr3QxG+xJe9G0ucKCpWXTwbq80EJnkZVRTNOt+fZ7qkWz/hBvJB0WyT1Lfdy1DeXcj4 WaXsSBpLT+JxPliv6IsiiY3AepZeuTM= X-MC-Unique: LuDD0-f2PFCGXmX7ooiG7Q-1 X-Mimecast-MFC-AGG-ID: LuDD0-f2PFCGXmX7ooiG7Q_1783405810 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Nathan Chen , Shameer Kolothum , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 22/27] iommufd: Introduce handler for device ATS support Date: Tue, 7 Jul 2026 08:29:15 +0200 Message-ID: <20260707062920.317302-23-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405904572158500 From: Nathan Chen Introduce a support_ats() handler that returns whether ATS is supported for a device associated with a host IOMMU device, checking if the IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED capability bit is set. Implement this handler in a new function for iommufd. Suggested-by: Shameer Kolothum Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Shameer Kolothum Signed-off-by: Nathan Chen Link: https://lore.kernel.org/qemu-devel/20260623204943.989903-2-nathanc@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- include/system/host_iommu_device.h | 9 +++++++++ backends/iommufd.c | 8 ++++++++ 2 files changed, 17 insertions(+) diff --git a/include/system/host_iommu_device.h b/include/system/host_iommu= _device.h index f0003015832855b7173475559ef411bee31d1c86..2cb53653406ceec30906120ffbf= d263a3d0810e8 100644 --- a/include/system/host_iommu_device.h +++ b/include/system/host_iommu_device.h @@ -133,6 +133,15 @@ struct HostIOMMUDeviceClass { * Returns: true on success, false on failure. */ bool (*get_pasid_info)(HostIOMMUDevice *hiod, PasidInfo *pasid_info); + /** + * @support_ats: Returns true if ATS can be used by the device, + * false if the host IOMMU reports it is unavailable. + * + * @hiod: handle to the host IOMMU device + * + * Returns: true if ATS is supported, false otherwise + */ + bool (*support_ats)(HostIOMMUDevice *hiod); }; =20 /* diff --git a/backends/iommufd.c b/backends/iommufd.c index 09624cd6526b4027b82b02dad3e6494231fedc2c..4b40aaebe6cbb9d27b0d0b15f09= 3a79fac9ed3b3 100644 --- a/backends/iommufd.c +++ b/backends/iommufd.c @@ -638,6 +638,13 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod,= int cap, Error **errp) } } =20 +static bool hiod_iommufd_support_ats(HostIOMMUDevice *hiod) +{ + HostIOMMUDeviceCaps *caps =3D &hiod->caps; + + return !(caps->hw_caps & IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED); +} + static bool hiod_iommufd_get_pasid_info(HostIOMMUDevice *hiod, PasidInfo *pasid_info) { @@ -660,6 +667,7 @@ static void hiod_iommufd_class_init(ObjectClass *oc, co= nst void *data) =20 hiodc->get_cap =3D hiod_iommufd_get_cap; hiodc->get_pasid_info =3D hiod_iommufd_get_pasid_info; + hiodc->support_ats =3D hiod_iommufd_support_ats; }; =20 static const TypeInfo types[] =3D { --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405912; cv=none; d=zohomail.com; s=zohoarc; b=k+nOwEsvOfFSmeUZgO7Ew+vj7yBTVTBKNbSd2n0ID2bk4wRFVWajzrJFPQPgIKzGixfb8ytq+InMLcY/DBIXQmKI5o7XskWpzbW7ucnjL499GMjYVA7N9emor6kKX0i1gYMUpCigG77rW0j2pwmcYqytfY7abzKfJThcXcKlkxs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405912; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZtAu414dVmGx7PUdTT13scNvsxilc3x2TkvJ9lmyISI=; b=TnS5nxTe8LqfJv3fq7caidU/q60iHxa0U7Vtpx6DcxOmW5+FsvQRNfTUH7roIr8BCThNVyTZ7+n9gGB73lXLXIGaFXP8AzHhD9ZU405eHTwGg6t3mklCiuhc+4KWximBtKaqnTBCep/7HQEuBXxDMhKa+yi3ZqMMRFiDFTq07fw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405912396877.4983398283923; Mon, 6 Jul 2026 23:31:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzK1-0002ak-Bo; Tue, 07 Jul 2026 02:30:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJz-0002V3-B1 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJx-0007Dy-Cq for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:19 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-583-wZ2o2QwlOjid5VvKfgGrSA-1; Tue, 07 Jul 2026 02:30:13 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4D7371800EED; Tue, 7 Jul 2026 06:30:12 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D0EA3180065F; Tue, 7 Jul 2026 06:30:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZtAu414dVmGx7PUdTT13scNvsxilc3x2TkvJ9lmyISI=; b=KkrUNbemBr7o2HP2tOnDN9FItPWmNPVd5u2tKoD39mkPDiIqIssMoNaB1c/xQXm8U8FX1p auAZdSlfln2bbJbpV4/JpX7skGjxFe+O+cBe/bi6wN+fGHKrbTimJMMca27EoNRrc60185 z0hvv4xx7GNF0RCxTSMaXg1UEy7T3j4= X-MC-Unique: wZ2o2QwlOjid5VvKfgGrSA-1 X-Mimecast-MFC-AGG-ID: wZ2o2QwlOjid5VvKfgGrSA_1783405812 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Nathan Chen , Shameer Kolothum , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 23/27] vfio/pci: Add ats property Date: Tue, 7 Jul 2026 08:29:16 +0200 Message-ID: <20260707062920.317302-24-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405914727158500 From: Nathan Chen Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=3Doff, or ats=3Dauto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it. If ATS is explicitly requested but not supported by the kernel, fail device realize. This aligns with the kernel's per-device effective ATS reporting and allows vfio-pci to mask ATS when the host kernel reports ATS as unsupported. Emit a warning when ats=3Don is requested but the physical device does not advertise ATS, since ATS cannot be exposed to the guest in this case. Emit a warning when ats=3Dauto, ats cap is present on the physical device, but kernel reports ATS as unsupported. Suggested-by: Shameer Kolothum Signed-off-by: Nathan Chen Reviewed-by: Shameer Kolothum Reviewed-by: C=C3=A9dric Le Goater Link: https://lore.kernel.org/qemu-devel/20260623204943.989903-3-nathanc@nv= idia.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/pci.h | 1 + hw/vfio/pci.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 85 insertions(+), 4 deletions(-) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index cf56711587058b8cf5197945755fff2456168c00..fe52e9df6e67707c9899d418b02= 61afeedf4aab2 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -188,6 +188,7 @@ struct VFIOPCIDevice { bool clear_parent_atomics_on_exit; bool skip_vsc_check; uint16_t vpasid_cap_offset; + OnOffAuto ats; VFIODisplay *dpy; Notifier irqchip_change_notifier; VFIOPCICPR cpr; diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 6b04c6f94f9ade765e236d457e4828ac6199680a..c7f163399f8fbcc4ff70819a0d6= 7879eb26980f9 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2548,10 +2548,53 @@ static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDe= vice *vdev, Error **errp) return true; } =20 -static void vfio_add_ext_cap(VFIOPCIDevice *vdev) +/* + * Determine whether ATS capability should be advertised for @vdev, based = on + * whether it was enabled on the command line and whether it is supported + * according to the kernel. + * + * Store whether ATS capability should be advertised in @ats_needed. + * + * Returns false only when ats=3Don is explicitly requested but the kernel + * reports it is not supported. Returns true in all other cases. + */ +static bool vfio_pci_ats_requested_and_supported(VFIOPCIDevice *vdev, + bool *ats_needed, Error *= *errp) +{ + HostIOMMUDevice *hiod =3D vdev->vbasedev.hiod; + HostIOMMUDeviceClass *hiodc; + bool ats_supported; + *ats_needed =3D false; + + if (vdev->ats =3D=3D ON_OFF_AUTO_OFF) { + return true; + } + + *ats_needed =3D true; + if (!hiod) { + return true; + } + hiodc =3D HOST_IOMMU_DEVICE_GET_CLASS(hiod); + if (!hiodc || !hiodc->support_ats) { + return true; + } + + ats_supported =3D hiodc->support_ats(hiod); + if (vdev->ats =3D=3D ON_OFF_AUTO_ON && !ats_supported) { + error_setg(errp, "vfio-pci: ATS requested but not supported by ker= nel"); + *ats_needed =3D false; + return false; + } + + *ats_needed =3D ats_supported; + return true; +} + +static void vfio_add_ext_cap(VFIOPCIDevice *vdev, bool ats_needed) { PCIDevice *pdev =3D PCI_DEVICE(vdev); bool pasid_cap_added =3D false; + bool ats_cap_present =3D false; Error *err =3D NULL; uint32_t header; uint16_t cap_id, next, size; @@ -2637,7 +2680,19 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev) */ case PCI_EXT_CAP_ID_PASID: pasid_cap_added =3D true; - /* fallthrough */ + pcie_add_capability(pdev, cap_id, cap_ver, next, size); + break; + case PCI_EXT_CAP_ID_ATS: + ats_cap_present =3D true; + /* + * If ATS is requested and supported according to the kernel, = add + * the ATS capability. If not supported according to the kerne= l or + * disabled on the qemu command line, omit the ATS cap. + */ + if (ats_needed) { + pcie_add_capability(pdev, cap_id, cap_ver, next, size); + } + break; default: pcie_add_capability(pdev, cap_id, cap_ver, next, size); } @@ -2648,6 +2703,16 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev) error_report_err(err); } =20 + if (vdev->ats =3D=3D ON_OFF_AUTO_ON && !ats_cap_present) { + warn_report("vfio-pci: ats=3Don requested, but host device has no " + "ATS extended capability"); + } + + if (vdev->ats =3D=3D ON_OFF_AUTO_AUTO && ats_cap_present && !ats_neede= d) { + warn_report("vfio-pci: host kernel reports ATS unsupported; " + "ATS capability will be masked"); + } + /* Cleanup chain head ID if necessary */ if (pci_get_word(pdev->config + PCI_CONFIG_SPACE_SIZE) =3D=3D 0xFFFF) { pci_set_word(pdev->config + PCI_CONFIG_SPACE_SIZE, 0); @@ -2659,6 +2724,7 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev) bool vfio_pci_add_capabilities(VFIOPCIDevice *vdev, Error **errp) { PCIDevice *pdev =3D PCI_DEVICE(vdev); + bool ats_needed =3D false; =20 if (!(pdev->config[PCI_STATUS] & PCI_STATUS_CAP_LIST) || !pdev->config[PCI_CAPABILITY_LIST]) { @@ -2669,7 +2735,11 @@ bool vfio_pci_add_capabilities(VFIOPCIDevice *vdev, = Error **errp) return false; } =20 - vfio_add_ext_cap(vdev); + if (!vfio_pci_ats_requested_and_supported(vdev, &ats_needed, errp)) { + return false; + } + + vfio_add_ext_cap(vdev, ats_needed); return true; } =20 @@ -3819,6 +3889,7 @@ static const Property vfio_pci_properties[] =3D { DEFINE_PROP_BOOL("skip-vsc-check", VFIOPCIDevice, skip_vsc_check, true= ), DEFINE_PROP_UINT16("x-vpasid-cap-offset", VFIOPCIDevice, vpasid_cap_offset, 0), + DEFINE_PROP_ON_OFF_AUTO("ats", VFIOPCIDevice, ats, ON_OFF_AUTO_AUTO), }; =20 static void vfio_pci_set_fd(Object *obj, const char *str, Error **errp) @@ -3970,13 +4041,22 @@ static void vfio_pci_class_init(ObjectClass *klass,= const void *data) "destination when doing live " "migration of device state via " "multifd channels"); - object_class_property_set_description(klass, /* 11.0 */ + object_class_property_set_description(klass, /* 11.0 */ "x-vpasid-cap-offset", "PCIe extended configuration spa= ce offset at which to place a " "synthetic PASID extended capabi= lity when PASID is enabled via " "a vIOMMU. A value of 0 (default= ) places the capability at the " "end of the extended configurati= on space. The offset must be " "4-byte aligned and within the P= CIe extended configuration space"); + object_class_property_set_description(klass, /* 11.1 */ + "ats", + "Control guest visibility of the= ATS PCIe extended capability. " + "Valid values are on, off, and a= uto (default). " + "'off' always masks ATS. " + "'on' requires ATS support for t= he device and fails realize if the " + "host kernel reports ATS as unav= ailable for this device. " + "'auto' masks ATS only when the = host kernel reports " + "ATS as unavailable"); } =20 static const TypeInfo vfio_pci_info =3D { --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405944; cv=none; d=zohomail.com; s=zohoarc; b=JOzqGKHyUdtTXOx3McWuYcf/EXYeHSMFdX31L7215HV6h7O33aFcExkzzFvKicSezp0cBAqfsmeo8gdBivVidgKOZhHuIVvHAUsp7E6LVevHPgJfhyTHo14PJUm+gEp1tkZzanZz/3gvZ8ODso6olyJxg5VGZ8s4lHRkLe8dsdk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405944; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jIx95q429t63sKVC1uPzadqlslaDv+W+0CTA/My+vYM=; b=gg5xpISUFwUQLc9BwGwleRkBrGQssPFk2K0AeWuByg42cBAXOKCcVfrkVqJ4A9Xvn9n47up12p0qUPRXwstt5REwjCjactoGfhCPWCdy7xBxML5D5KG/FzTACc38OBBcCfPzd4yGq+7Qg4HwxJriPo5Ig94PbWDd2ScfrRTjACU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405944878252.9148396111217; Mon, 6 Jul 2026 23:32:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzK3-0002km-DA; Tue, 07 Jul 2026 02:30:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzK1-0002bf-JS for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJz-0007EC-7l for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:21 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-558-jy4PCRTOOmW06A8TmwC-gQ-1; Tue, 07 Jul 2026 02:30:15 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3996D1954B22; Tue, 7 Jul 2026 06:30:14 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BEFE4180065F; Tue, 7 Jul 2026 06:30:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jIx95q429t63sKVC1uPzadqlslaDv+W+0CTA/My+vYM=; b=KI+guOpanwRyDPd9qA4ESbsZVZHRZz80H6PGXwTtoJXKmz9KctVM4omNtT00NMCdAGycxj XQPLL6I2uXyk7imWqk2ZCfxJsOft6caHr/1KJ9+6kyshsJyeEP4IYjTZi+xFQTl5w45bFz ++zo8AWcKxiwfk1YnVLdxXOxV45IVCk= X-MC-Unique: jy4PCRTOOmW06A8TmwC-gQ-1 X-Mimecast-MFC-AGG-ID: jy4PCRTOOmW06A8TmwC-gQ_1783405814 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Mario Casquero , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 24/27] vfio/pci: Propagate errors in vfio_pci_load_rom() using Error API Date: Tue, 7 Jul 2026 08:29:17 +0200 Message-ID: <20260707062920.317302-25-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405947801158500 From: Mario Casquero Updates vfio_pci_load_rom() to use Error API for error propagation instead of error_report(), improving error handling consistency. Signed-off-by: Mario Casquero Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Philippe Mathieu-Daud=C3=A9 Link: https://lore.kernel.org/qemu-devel/20260625094307.148542-1-mcasquer@r= edhat.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/pci.c | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index c7f163399f8fbcc4ff70819a0d67879eb26980f9..e6d1adfd3655ce3c1baf8977c82= 223d23af9b770 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1028,7 +1028,7 @@ static void vfio_update_msi(VFIOPCIDevice *vdev) } } =20 -static bool vfio_pci_load_rom(VFIOPCIDevice *vdev) +static bool vfio_pci_load_rom(VFIOPCIDevice *vdev, Error **errp) { VFIODevice *vbasedev =3D &vdev->vbasedev; struct vfio_region_info *reg_info =3D NULL; @@ -1041,7 +1041,7 @@ static bool vfio_pci_load_rom(VFIOPCIDevice *vdev) ®_info); =20 if (ret !=3D 0) { - error_report("vfio: Error getting ROM info: %s", strerror(-ret)); + error_setg_errno(errp, -ret, "vfio: Error getting ROM info"); return false; } =20 @@ -1053,10 +1053,13 @@ static bool vfio_pci_load_rom(VFIOPCIDevice *vdev) vdev->rom_offset =3D reg_info->offset; =20 if (!vdev->rom_size) { - error_report("vfio-pci: Cannot read device rom at %s", vbasedev->n= ame); - error_printf("Device option ROM contents are probably invalid " - "(check dmesg).\nSkip option ROM probe with rombar=3D0= , " - "or load from file with romfile=3D\n"); + vdev->rom_size =3D 0; + vdev->rom_offset =3D 0; + error_setg(errp, "vfio-pci: Device ROM size is zero at %s", + vbasedev->name); + error_append_hint(errp, "Device option ROM contents are probably " + "invalid (check dmesg).\nSkip option ROM probe " + "with rombar=3D0, or load from file with romfile= =3D\n"); return false; } =20 @@ -1077,10 +1080,12 @@ static bool vfio_pci_load_rom(VFIOPCIDevice *vdev) if (bytes =3D=3D -EINTR || bytes =3D=3D -EAGAIN) { continue; } - error_report("vfio: Error reading device ROM: %s", - strreaderror(bytes)); - - break; + error_setg_errno(errp, -bytes, "vfio: Error reading device ROM= "); + g_free(vdev->rom); + vdev->rom =3D NULL; + vdev->rom_size =3D 0; + vdev->rom_offset =3D 0; + return false; } } =20 @@ -1148,7 +1153,12 @@ static uint64_t vfio_rom_read(void *opaque, hwaddr a= ddr, unsigned size) =20 /* Load the ROM lazily when the guest tries to read it */ if (unlikely(!vdev->rom && !vdev->rom_read_failed)) { - vdev->rom_read_failed =3D !vfio_pci_load_rom(vdev); + Error *local_err =3D NULL; + + vdev->rom_read_failed =3D !vfio_pci_load_rom(vdev, &local_err); + if (vdev->rom_read_failed) { + error_report_err(local_err); + } } =20 memcpy(&val, vdev->rom + addr, --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405926; cv=none; d=zohomail.com; s=zohoarc; b=K1FDOzzOcpreTcBvfg/JMI46qCwBG5x5BAKBgiGEON1YXGkM/R+ceaVXMgv+Vfp8SzFfkXlGCNI5hBKFLW98QYGzdxgCnLpRXtdWZ4lwknCdAS2bZ3yFpsEdnN3Iw04er+K0W9HojSqizHDWph1rXJfXlc6E+O7m+eDWsC+5ZpI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405926; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MGVEUHYyzKvvMHZO41oX4EZXrdu6aa4irYk8hISV9p0=; b=O+3Urqm5L4/yTkGFv2nEYEVQyIOYysPc5nhClxpFwv57/w7+RqEZ3OI9RXkMI+uyOkybpTXX9+BbLiuNFKPdcLYMUJ5UYoMhZI3+kQ+GmK5Y9ZL6xaGWN2xBTJWApwYzmbUl7+lRM/IIf3kbYjej2SaW3K4eff6xVZ0CMmRC2rw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405926834472.75129735129747; Mon, 6 Jul 2026 23:32:06 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzK3-0002kD-6f; Tue, 07 Jul 2026 02:30:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzK0-0002Zs-Ra for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzJz-0007EA-7S for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:20 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-47-6rBHt0u-MXOX4UnmVQ9vVg-1; Tue, 07 Jul 2026 02:30:17 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2D57C1954B39; Tue, 7 Jul 2026 06:30:16 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AA1851800678; Tue, 7 Jul 2026 06:30:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MGVEUHYyzKvvMHZO41oX4EZXrdu6aa4irYk8hISV9p0=; b=D7zpgIPb5yhRxFLX/g6Se+ctTJWLwNemNHE1KcOnHRslDXSAdeYPafEBG2tiVuYbH6/sN1 E/5vB0BHSFYWsaOv7VvvgUPHxgEziD35h8Otyj6aLvCGJ5akx2aHRGb51xF1v1yIdmDWYv wVRRJXh5XHzHmablVV1PSeOwfR5bxws= X-MC-Unique: 6rBHt0u-MXOX4UnmVQ9vVg-1 X-Mimecast-MFC-AGG-ID: 6rBHt0u-MXOX4UnmVQ9vVg_1783405816 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Zhenzhong Duan , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 25/27] vfio/listener: Fix translated_addr for non-identity-mapped RAM sections Date: Tue, 7 Jul 2026 08:29:18 +0200 Message-ID: <20260707062920.317302-26-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405928721158500 In vfio_listener_region_del(), when dirty tracking is active and a writable RAM section is deleted, a synthetic IOMMUTLBEntry is built to flush dirty pages. Setting translated_addr to the IOVA (GPA) is only correct for identity-mapped regions where GPA =3D=3D ram_addr_t. For RAM sections with GPA far above main RAM (e.g., nested VT-d interrupt remapping table at 58 TB), translated_addr is too large, causing a crash in physical_memory_set_dirty_lebitmap() when indexing beyond the allocated dirty memory blocks array : bitmap_set_atomic(map=3DNULL, start=3D1, nr=3D1) physical_memory_set_dirty_range(start=3D0x380004040000, length=3D4096) physical_memory_set_dirty_lebitmap(start=3D0x380004040000, pages=3D3) vfio_container_query_dirty_bitmap(translated_addr=3D0x380004040000) vfio_legacy_dma_unmap_one(iova=3D0x380004040000, size=3D12288) vfio_listener_region_del() Fix this by setting translated_addr to the ram_addr_t of the section, which is consistent with other vfio dirty tracking code: translated_addr =3D memory_region_get_ram_addr(section->mr) + section->offset_within_region; Cc: Zhenzhong Duan Fixes: 6e360c06176c ("vfio/listener: Add missing dirty tracking in region_d= el") Reviewed-by: Zhenzhong Duan Tested-by: Zhenzhong Duan Reviewed-by: Philippe Mathieu-Daud=C3=A9 Link: https://lore.kernel.org/qemu-devel/20260625134352.3122572-1-clg@redha= t.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/listener.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/hw/vfio/listener.c b/hw/vfio/listener.c index 109b5d61af4a09cf22f0ebbb9f3fc4ca87295b38..c19600e980a8d02217b5d9df4ac= a8f879ab2c5d5 100644 --- a/hw/vfio/listener.c +++ b/hw/vfio/listener.c @@ -731,7 +731,7 @@ static void vfio_listener_region_del(MemoryListener *li= stener, } =20 /* - * Fake an IOTLB entry for writable identity mapping which is need= ed + * Fake an IOTLB entry for writable RAM sections which is needed * by dirty tracking when switch out of PT domain. In fact, in * unmap_bitmap, only translated_addr field is used to set dirty * bitmap. @@ -746,7 +746,8 @@ static void vfio_listener_region_del(MemoryListener *li= stener, if (global_dirty_tracking && memory_region_is_ram(section->mr) && !section->readonly) { entry.iova =3D iova; - entry.translated_addr =3D iova; + entry.translated_addr =3D memory_region_get_ram_addr(section->= mr) + + section->offset_within_region; iotlb =3D &entry; } =20 --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405911; cv=none; d=zohomail.com; s=zohoarc; b=LdZrF8XzlrsOX3bKFkfZ2lKWKmRaf3pEbKZAK7MZ2iz2f93F+EfS6eBCWYOGFYkfz92wyhi83WTkiFY7lJLwIpdKh38Ld0Znf8AKu6RZ5qRWz3xtKfYTopUZamHFSdWaN4WUoQENu1bPeXUQgUsa0hzHt9Al/q1YV7uI3PNV24Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405911; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+v9cWJV7DosWCGpuUiM+8fwDiTcaZT/z92D+GxJWmJs=; b=LLejC78iGQEgMIrO6ExhPdmDLmut++CWzlp+7Sym6Vc/E+h0MJrQWiLyP2VnPCevjQaw7DAPEliaEpfQ/JbFfG9p7Jbf+/Et6GJy3dk78cmNguS6SQq8Chd7H3j05weTplXXLyK42iBXj+V8GwiNNS9ksyR//7YsNvzCJcZDhDU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783405911595979.8611911178906; Mon, 6 Jul 2026 23:31:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzK3-0002mp-T8; Tue, 07 Jul 2026 02:30:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzK3-0002if-1q for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzK1-0007Ec-7g for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:22 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-124-cDPaDmtPMnWIuVd_qh6VoQ-1; Tue, 07 Jul 2026 02:30:19 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 445CA1805C05; Tue, 7 Jul 2026 06:30:18 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9CFE31800678; Tue, 7 Jul 2026 06:30:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405820; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+v9cWJV7DosWCGpuUiM+8fwDiTcaZT/z92D+GxJWmJs=; b=WBJntdp0TdFp1qErIWNlPcMNaGfNRhhNFci2JQ2+8n26K8s07/4Q4ebNycs9bixyiDGXta 3TFtIlNtK6D5SmC+TnK+x5yDObyai6hG7Kc3jJUWd/bi4S5CLevqoBbRxNK3XLH1jhKmde 82yNE/4hkh/xItB/jzNNxxAooqgWzlE= X-MC-Unique: cDPaDmtPMnWIuVd_qh6VoQ-1 X-Mimecast-MFC-AGG-ID: cDPaDmtPMnWIuVd_qh6VoQ_1783405818 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: Shameer Kolothum , Nicolin Chen , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PULL 26/27] backends/iommufd: Fix dev_id and type order in viommu trace Date: Tue, 7 Jul 2026 08:29:19 +0200 Message-ID: <20260707062920.317302-27-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405912585158500 From: Shameer Kolothum The trace event receives dev_id before type, but its format string prints them in the wrong order. Correct the order. Fixes: f2d31df0d925 ("backends/iommufd: Introduce iommufd_backend_alloc_vio= mmu") Reported-by: Nicolin Chen Signed-off-by: Shameer Kolothum Reviewed-by: C=C3=A9dric Le Goater Link: https://lore.kernel.org/qemu-devel/20260706103653.84243-1-skolothumth= o@nvidia.com Signed-off-by: C=C3=A9dric Le Goater --- backends/trace-events | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/backends/trace-events b/backends/trace-events index b63420b73ee7363518530d981590080b00d9bcbd..009a25b0be96a437af029b6de90= 782fc27a85669 100644 --- a/backends/trace-events +++ b/backends/trace-events @@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret= ) " iommufd=3D%d id=3D%d (% iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int r= et) " iommufd=3D%d hwpt=3D%u enable=3D%d (%d)" iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t i= ova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd= =3D%d hwpt=3D%u iova=3D0x%"PRIx64" size=3D0x%"PRIx64" flags=3D0x%"PRIx64" p= age_size=3D0x%"PRIx64" (%d)" iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_t= ype, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t da= ta_ptr, int ret) " iommufd=3D%d id=3D%u data_type=3D%u entry_len=3D%u entry= _num=3D%u done_num=3D%u data_ptr=3D0x%"PRIx64" (%d)" -iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, = uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id,= int ret) " iommufd=3D%d type=3D%u dev_id=3D%u hwpt_id=3D%u data_ptr=3D0x%"= PRIx64" data_len=3D0x%x viommu_id=3D%u (%d)" +iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, = uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id,= int ret) " iommufd=3D%d dev_id=3D%u type=3D%u hwpt_id=3D%u data_ptr=3D0x%"= PRIx64" data_len=3D0x%x viommu_id=3D%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_i= d, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=3D%d dev_id=3D%u = viommu_id=3D%u virt_id=3D0x%"PRIx64" vdev_id=3D%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type= , uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=3D%d viommu_= id=3D%u type=3D%u veventq_id=3D%u veventq_fd=3D%u (%d)" iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t q= ueue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id,= int ret) " iommufd=3D%d viommu_id=3D%u queue_type=3D%u index=3D%u addr=3D0= x%"PRIx64" size=3D0x%"PRIx64" queue_id=3D%u (%d)" --=20 2.54.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1783405856; cv=none; d=zohomail.com; s=zohoarc; b=byolIFWb9s/YUA0iW0lcNkWDYJXnpRQ9eDXLl/Eq7NwnFPnURyZI4C43aiyhd9+r+agvmXxZj5r9pRTWSqhlYBh2XAwy/1Zq/xHdFogIa9S4DFRfAm9pSGrF7ITQQkOIy6Iv1o3v0qk1c3d6rIbj7b5m7CaPLZQkuSyLWUBIgoI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783405856; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WECeFWui8FSkmR3NrFprq2XlEhAaMQcWIxEQ9Qc9tK8=; b=UDs0dl7d1ncRtua/XV5rq8PyPxQZLhG0HODEGATfOkewqsYOlw+r3QhXA3d/2jIeoFH0PYbYw8Sw+1yIIBuT5C9pSmwIKs81yvlqemdIP4K97QTwLiNC4sg5G9cYgeNBw7QkhBNK+OkVfqracmstNDGDB5Iez6ntRUMmgH10/rU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178340585660579.1767308119579; Mon, 6 Jul 2026 23:30:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgzKA-0003Mt-MK; Tue, 07 Jul 2026 02:30:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzK8-00036s-1l for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgzK5-0007F9-Q9 for qemu-devel@nongnu.org; Tue, 07 Jul 2026 02:30:27 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-553-W7rR6mYYN-ClMXx7e2z8Ag-1; Tue, 07 Jul 2026 02:30:21 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 828031954B1E; Tue, 7 Jul 2026 06:30:20 +0000 (UTC) Received: from corto.redhat.com (unknown [10.44.32.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B3D20180065F; Tue, 7 Jul 2026 06:30:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783405825; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WECeFWui8FSkmR3NrFprq2XlEhAaMQcWIxEQ9Qc9tK8=; b=EvJE5dNqD2k31tealdAKYCtwcyUCrmSoaQ9hjlugDj37XxSU62/F792IgVMlTNgAEDRhnL jH1+C/K9NrQE6X+92SC97o+/o4clHyJOXFJpcfH2eUe+Zofisqt5hE/t3rciORIAT80oz9 Tx5oA4T/GReyoi8XjXFn76e3X2mPy3I= X-MC-Unique: W7rR6mYYN-ClMXx7e2z8Ag-1 X-Mimecast-MFC-AGG-ID: W7rR6mYYN-ClMXx7e2z8Ag_1783405820 From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Feifan Qian , Alex Williamson Subject: [PULL 27/27] vfio/pci: Reject invalid MSI-X Table and PBA BIR values Date: Tue, 7 Jul 2026 08:29:20 +0200 Message-ID: <20260707062920.317302-28-clg@redhat.com> In-Reply-To: <20260707062920.317302-1-clg@redhat.com> References: <20260707062920.317302-1-clg@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1783405858339158500 The 3-bit MSI-X BIR fields permit values 0-7, but VFIOPCIDevice::bars[] only contains BAR0-BAR5 (6 entries). An invalid BIR value of 6 or 7 can cause an out-of-bounds array access (CWE-129) in vfio_msix_early_setup(= ). Add range checks immediately after extracting the BIR values. Reported-by: Feifan Qian Fixes: 65501a745dba ("vfio: vfio-pci device assignment driver") Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3878 Reviewed-by: Alex Williamson Link: https://lore.kernel.org/qemu-devel/20260706161334.2165482-1-clg@redha= t.com Signed-off-by: C=C3=A9dric Le Goater --- hw/vfio/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index e6d1adfd3655ce3c1baf8977c82223d23af9b770..c204706e63045c930bda7977adc= 8034f907b6890 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1791,6 +1791,14 @@ static bool vfio_msix_early_setup(VFIOPCIDevice *vde= v, Error **errp) msix->pba_offset =3D pba & ~PCI_MSIX_FLAGS_BIRMASK; msix->entries =3D (ctrl & PCI_MSIX_FLAGS_QSIZE) + 1; =20 + if (msix->table_bar >=3D ARRAY_SIZE(vdev->bars) || + msix->pba_bar >=3D ARRAY_SIZE(vdev->bars)) { + error_setg(errp, "invalid MSI-X BIR, table_bar=3D%d pba_bar=3D%d", + msix->table_bar, msix->pba_bar); + g_free(msix); + return false; + } + ret =3D vfio_device_get_irq_info(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_IN= DEX, &irq_info); if (ret < 0) { --=20 2.54.0