From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653933339; cv=pass; d=zohomail.com; s=zohoarc; b=bi40llvTcK1wumnrVWWNGnbNDepoLn2aUS03F/3AnyWtWRWN+aimhZ4Kxs7FCMe28u1DBZd0WzmYcLQ7+3Jc3tJjVfYL/Zk1LbsxqEgwi1+uEOd3SzEjRanxT4TV4x6u+ki3ismoNM+rGPtiisccDpcd8oK9068QPTS9nicWaeg= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653933339; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kO1Q0d7wykn/1VkH5MNBlO0UmPGgbg874YdG/5D9cHw=; b=YgetRPeYyMbQrJiZG2yImnaCZeoB5+tIH5JM/1Gfnpvmhvvj8aLhieinrPq8b43K0px6vEkek9Cv059UWT9ceeUs2mN1AkI9GCdgN7hhirnD2/q8zn+zCPtMztBcLPtlC/DdELUjWJgyxKuOgtOQGsNNRoJHGDKs4EaZfLjML9k= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653933339788760.9978013109888; Mon, 30 May 2022 10:55:39 -0700 (PDT) Received: from localhost ([::1]:48108 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjcA-00069q-P7 for importer@patchew.org; Mon, 30 May 2022 13:55:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvis1-0003GR-Dr for qemu-devel@nongnu.org; Mon, 30 May 2022 13:07:57 -0400 Received: from mail-bn8nam12on2061d.outbound.protection.outlook.com ([2a01:111:f400:fe5b::61d]:16096 helo=NAM12-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvirz-0008LP-9Z for qemu-devel@nongnu.org; Mon, 30 May 2022 13:07:56 -0400 Received: from MW4PR04CA0040.namprd04.prod.outlook.com (2603:10b6:303:6a::15) by MN2PR12MB4535.namprd12.prod.outlook.com (2603:10b6:208:267::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:07:48 +0000 Received: from CO1NAM11FT058.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6a:cafe::5a) by MW4PR04CA0040.outlook.office365.com (2603:10b6:303:6a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:48 +0000 Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT058.mail.protection.outlook.com (10.13.174.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:47 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:07:47 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:07:46 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:07:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OoaoAEgILqiPyShX8ux3Bt4r7d6wc6FGCtqIyPR5vbOKVoSeCV5DTHpdO+vPPai43nEGrYRUq6e0fsLt8ACDAK2MCX+rMKYx8SDmQPwqnHKIdKE8k8dfBZ77v4EboaU1H9hotR6+wpGU8axX9/LBeAbQnTN70Ss+EcCQakYzBPd3RtpqJtyGKnH1p60XHIvEBE2hyUjC8Tu2dSHPMkwgVpvjM7i9i6Irs/zTub/k1gmdwOKKVQrrHNrIhkm3uHNBc2EY3aw60tiRIBRhVVpYlz3V5lHXb9XJ+c+Za+B1DGugnMn6nHIfnZeO8gDIKTMI4gSaQwykIGf8H/NPxJ360A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kO1Q0d7wykn/1VkH5MNBlO0UmPGgbg874YdG/5D9cHw=; b=TrDmoCMYD/j+z/sazvdmXK4/VZj/HmjvSOuLHGT2RHpkyRpwL41Pqrdhn1obRciPBrjAF0mAhOFDmpISVeL8CQPS42xEFfXDrk9gwYFyGJipN+l4HcMVtG+Vw6CUTZtSyiESPCbHLXgUHG+9f4L9a8J+pISr315Yy2AwHBwuXi2RK6zXW/FrO5wbZ2Gk9abj2bvIVlCcQf03ZrSkE3nmB6xrH4F9Tg4tfSugpyrcuwO2cPSCATk3WWxrk3eevSo7AY6Vy4kDVw7bze/LFf7P5PDplPuNR9TdrwGcSPt1sWPlNMEMh9p0MpenoxoeuSwNs12NekE0xb3iAmAGrNSaUQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kO1Q0d7wykn/1VkH5MNBlO0UmPGgbg874YdG/5D9cHw=; b=PhggqPBKtKXIq9cWfkTZcrj3wNhjPeDkWE0Uf2z8yz6Zpcj/todUEQmXQp9TvoNowjr8a/IaZRRzWwZSSdSgWGvvN+BBykbW1O6XKqL7XUjGSKZIQUbfr5jTA+B9VGUDzEt2Lg6HmgcOuSPZLvhCOdb0Wn1KGGTuQPrduOAgLiFxpYDD9HxYZ2vqu+En/6j9fKlOuewoc9C2BhpQ93IW8OvbPF1BEiglTzRFIeGnXqmeXk85SyWFA3LEsdymfs82jKIl7c6mTPkr2jGNaPlXrgoK09M7S1ljRJcu+8h+IJvduAhcCV9fOqm7B7+I/c+A/Z1SOl/H4AyOYRmZQr1M7Q== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 01/11] vfio/migration: Fix NULL pointer dereference bug Date: Mon, 30 May 2022 20:07:29 +0300 Message-ID: <20220530170739.19072-2-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 759bf2b6-9c08-459c-5040-08da425eebf0 X-MS-TrafficTypeDiagnostic: MN2PR12MB4535:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4GsBO3Z6XPGkkcmnpjRudBR9IGO3hM0SdMtgqNZyl5Y1Pc8fT87IOv/yYwz8tNhNZEd8tUJWKvkfiCZ5ZAroH+7MGdcdwO01aL5y1ey8lNCD37ytdsCCwYnqSH6/6u/E54hkQnJLkTTj61u796m35ePhlIm49yD0Q5EGF07wdJmZT8P51g/Ag5CCDJv7mQFuBEjRR49sJ+RxIYj95NxY7u8L858g3JogJF1QKpC+hFO+kEMraz8xFJHFmJ5Q9j/y8bHEurmDPUxmYCgM3KBa+Ldj5+NG01jnU8q1hXr0KQydUGQW1lT8cEzJc79hDiVe+XmOqMkFQJIwKzOPX+ehjSbbE3tlrgTUFyIWOgILu6jIIEH8waER9qAqsTNES6EykplQUVXXcYdpa6+XbyfYbR4UYApv/hf3XRGIFS1s1C6szvChh9A/ZUE3mYGKQ9ZVY1f8E17lXFYFKJEpjQVl+fRTTpk1jPTm98WR5ryvv6w2nKonRZu7VVkJPpvtuLxLEGHM2cK05hJrVkOcNkz9Ig9k5MlINZhkXlGSIy5PfhV4HPhU1pklyMthugtzz1t/ArYZopLpbGsrssPb8wc3MldEgunHCDmt7bNNDmozGT45sXqZRAYFDUQSBROymz1xatIuu8Vw8USzWc8FwvJzHFTdEsE10BeqD586sHoA6nAtjsdX0x4eu9U3fGfvz3ZJp8hhMpWCFWJEnUMRX05log== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(336012)(186003)(40460700003)(6666004)(86362001)(81166007)(70206006)(107886003)(356005)(5660300002)(4326008)(70586007)(1076003)(36756003)(8936002)(426003)(83380400001)(7696005)(2616005)(8676002)(2906002)(110136005)(316002)(54906003)(26005)(508600001)(47076005)(36860700001)(82310400005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:07:47.8006 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 759bf2b6-9c08-459c-5040-08da425eebf0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT058.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4535 Received-SPF: softfail client-ip=2a01:111:f400:fe5b::61d; envelope-from=avihaih@nvidia.com; helo=NAM12-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653933341057100001 Content-Type: text/plain; charset="utf-8" As part of its error flow, vfio_vmstate_change() accesses MigrationState->to_dst_file without any checks. This can cause a NULL pointer dereference if the error flow is taken and MigrationState->to_dst_file is not set. For example, this can happen if VM is started or stopped not during migration and vfio_vmstate_change() error flow is taken, as MigrationState->to_dst_file is not set at that time. Fix it by checking that MigrationState->to_dst_file is set before using it. Fixes: 02a7e71b1e5b ("vfio: Add VM state change handler to know state of VM= ") Signed-off-by: Avihai Horon Reviewed-by: Juan Quintela --- hw/vfio/migration.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index a6ad1f8945..34f9f894ed 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -744,7 +744,9 @@ static void vfio_vmstate_change(void *opaque, bool runn= ing, RunState state) */ error_report("%s: Failed to set device state 0x%x", vbasedev->name, (migration->device_state & mask) | value); - qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + if (migrate_get_current()->to_dst_file) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + } } vbasedev->migration->vm_running =3D running; trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653932788; cv=pass; d=zohomail.com; s=zohoarc; b=oHRKYGTEdhA8aokBijNTYrLCX/gAGyhumr9/4iePdy4ktRJcyh2pTfqGdtRApDM6MQMASTv85WklMvFtUm2+KzMOQzBbs1RU6I0xj7jtApgTrijS1LykE5E9gY1sdoJEaq3qc5b/PjXMhB535Mu9mQHxe0xwjT5bStC6Tfvz3y8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653932788; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DWmmJp9DZzsJirVqwnDaGWW1CwYh74LsXTFYv6GpP9U=; b=m2DB+NtB6UejFQZ3kKHLrrui6LC2w9m6NrwNpchu1nH2e+YxjJkstCepjuBfUfEm2oGvtHq0Sjq/8sYXIII/7bIcGyTrYRvDgBQ56sl6k5p4nBWKyTIcrNJUf/RccN3E5ohGG6cHcAoj6lOVp0V2GDEMN2I1hUJsCqqDCxk7oWM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653932788500895.7749820289287; Mon, 30 May 2022 10:46:28 -0700 (PDT) Received: from localhost ([::1]:59992 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjTH-00038x-8D for importer@patchew.org; Mon, 30 May 2022 13:46:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvis1-0003G3-Di for qemu-devel@nongnu.org; Mon, 30 May 2022 13:07:57 -0400 Received: from mail-bn8nam08on20620.outbound.protection.outlook.com ([2a01:111:f400:7e8d::620]:49761 helo=NAM04-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nviry-0008LQ-KA for qemu-devel@nongnu.org; Mon, 30 May 2022 13:07:56 -0400 Received: from MWHPR02CA0015.namprd02.prod.outlook.com (2603:10b6:300:4b::25) by DM5PR12MB1337.namprd12.prod.outlook.com (2603:10b6:3:6e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.17; Mon, 30 May 2022 17:07:51 +0000 Received: from CO1NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:300:4b:cafe::e2) by MWHPR02CA0015.outlook.office365.com (2603:10b6:300:4b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:51 +0000 Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT039.mail.protection.outlook.com (10.13.174.110) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:51 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:07:50 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:07:50 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:07:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ynt4aRVr/xS5Uv+RCqcZ3C37AEs2ZJG9kGoKuYHEJFi46SFaQfZjYojjsDp1bZrtb49ARIDMD9cxnxkCl3GS+2yi/9UBCHmF1KLTGzHB2PuVG9pQ50ybkp9l9GX8oqJ1bhSpil7NRfbSUEoJn+we4ObEJ6A++sqpcDLpMIvXT4q3b1P6aTb7M/MXE9NIkA8pdU5EVDt3AXrhrcd1cFr0rU+YHXgsx61FAsXAo2NY9YhA3hlSKekT4F38KjJ0KvrIFFep6IIxJU+/TV+8i/1VqjAZmOiprZobTKmyB5bBjWQ3lroWrJi3KQXLWb3v3qPZbzBFtrcHoONdNbjfUzANQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DWmmJp9DZzsJirVqwnDaGWW1CwYh74LsXTFYv6GpP9U=; b=OQ7jeucBrTH0VWn/9InGZLQNHJ4SKGMDleH0Q8s54Zmzhy7iW55lbvLHAawKTfjl7SbIDQHm8v0IPBqWepnuleoPF4IKnc1uiPytt4YlGM4mMsxd2RcOxT1KW7N2KoeOa+D3HrzKr/ZM7y2qEDvnLSH/ExmUWuI8UTlPocKdilC4RjpzUhabpLYZCuK9QXIbimJbBJoNIo48COoyu2zTX+vCkHmtK3417Jb72uH8N9QDZOjkYTKggpWrazdSPsn3HvdAGIsJW2Nl4s8QMhj3pOnw81U9jQ+cDmCoIfVFX2pBBp5JHWPByQKvvA37Cq/n40sEaaOSW0kYwRUAPg4A0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DWmmJp9DZzsJirVqwnDaGWW1CwYh74LsXTFYv6GpP9U=; b=LR1EPCoQP8sSWAVDwbF0BlzWZCet3R3+G5SVPaviLq8B9iBCmlZk57RA7Xn7vqUi6hnBHNOwO5ZGehuKj6D9eWEQni530O6gpblwfejT3TnLtKidmxHOeNvuigvJ9E6NK+muFpbJw2resrEbDwUWKSOn9hvPMCqMKKV/o34QS7ozJDomlLB7ETf2LvGZd6JRYEeH7O3tuimFdOqSJCOtNN/4vqDD3SJj5jzTABh3THGYw3kEZnndN7vq6n9fnmlhJlDNAC8oivZkxd1sJD/NMmB1Nlx4m5y2dzjHrPwY3LFEO3iDZitGZRlCh6RCo8KJnwX3ST4QuZvPP85NGBEGiw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 02/11] vfio/migration: Skip pre-copy if dirty page tracking is not supported Date: Mon, 30 May 2022 20:07:30 +0300 Message-ID: <20220530170739.19072-3-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 97bca9f4-2cbc-4084-1c00-08da425eee0c X-MS-TrafficTypeDiagnostic: DM5PR12MB1337:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LSZtB1UMo80T2t1wKR/9u4W+9wKtMZsRxl1k4hPp4l7WSAUG+Tf3c6OualPqzawS9zslUv9uhMxHE4Q9hGR/z3As8zkx/20t8001aUPuu/GECLPFqn9Keqx6u8d8nb7URLtjhyLuCKIVfLzdT1dldqQF5G2WDEBxVC/nXAYhHt42a1sBluQsf9rS6Vkz9qOBGt+GUt/xHbCwPPETk9FiWZNuJIFrUxk9X4irUUu56QHcBSktWcGr/dhBSBXMmWZZQEQa8Hz+Qh4CIE3ExrjQfXxEGkG/CaYIo0SHmcC8Sgtwm1Ahkv93X2a0fUR8CGDBmQtZIBdJKUl+amVJn0NjeYN/b/wMZkK1gppNV9RbviCg7/biF7HMmEd4su7bEpX6l3liqYD/wbR67M/X+PcbfeG33gQvImf7G20XcVeECKnbMNJ2PnSblW5wqkL3TU1TE7JkEGDikhsdLrZWKX7SgigHGAcEIlQaYZG3Bx/bArgK6iWAp53U28+fs63q8AI3pGHju2/q3Sjy0/Rij07owX20wm2yJrFQJ1Sb41KodQwP0VVU4iDgHvUBbfpzD672mEG5sfNGNgDxJgSkjt+AWm5XRedBBlhBR+14+4iuPb4E9Y0McP4iWHOOoAoPRYXFp9TJq5ZyrtDq7UuUVjX/SoTLzkm0FgZvqKZx96qQhqAknE+OaO8sGqn3AQBiaOSvccIV/biZAHMVrWlwPwwob9CUkvBJnC2owUl5Db+5dU0= X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(316002)(356005)(47076005)(508600001)(36756003)(186003)(107886003)(81166007)(1076003)(2616005)(426003)(336012)(86362001)(110136005)(40460700003)(54906003)(36860700001)(8676002)(82310400005)(26005)(8936002)(83380400001)(5660300002)(4326008)(6666004)(7696005)(70586007)(2906002)(70206006)(14143004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:07:51.3382 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 97bca9f4-2cbc-4084-1c00-08da425eee0c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1337 Received-SPF: softfail client-ip=2a01:111:f400:7e8d::620; envelope-from=avihaih@nvidia.com; helo=NAM04-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653932790121100001 Content-Type: text/plain; charset="utf-8" Currently, if IOMMU of a VFIO container doesn't support dirty page tracking, migration is blocked completely. This is because a DMA-able VFIO device can dirty RAM pages without updating QEMU about it, thus breaking the migration. However, this doesn't mean that migration can't be done at all. If migration pre-copy phase is skipped, the VFIO device doesn't have a chance to dirty RAM pages that have been migrated already, thus eliminating the problem previously mentioned. Hence, in such case allow migration but skip pre-copy phase. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 9 ++++++++- migration/migration.c | 5 +++++ migration/migration.h | 3 +++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 34f9f894ed..d8f9b086c2 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -863,10 +863,17 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error = **errp) struct vfio_region_info *info =3D NULL; int ret =3D -ENOTSUP; =20 - if (!vbasedev->enable_migration || !container->dirty_pages_supported) { + if (!vbasedev->enable_migration) { goto add_blocker; } =20 + if (!container->dirty_pages_supported) { + warn_report_once( + "%s: IOMMU of the device's VFIO container doesn't support dirt= y page tracking, migration pre-copy phase will be skipped", + vbasedev->name); + migrate_get_current()->skip_precopy =3D true; + } + ret =3D vfio_get_dev_region_info(vbasedev, VFIO_REGION_TYPE_MIGRATION_DEPRECATED, VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATE= D, diff --git a/migration/migration.c b/migration/migration.c index 31739b2af9..217f0e3e94 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3636,6 +3636,11 @@ static MigIterateState migration_iteration_run(Migra= tionState *s) uint64_t pending_size, pend_pre, pend_compat, pend_post; bool in_postcopy =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE; =20 + if (s->skip_precopy) { + migration_completion(s); + return MIG_ITERATE_BREAK; + } + qemu_savevm_state_pending(s->to_dst_file, s->threshold_size, &pend_pre, &pend_compat, &pend_post); pending_size =3D pend_pre + pend_compat + pend_post; diff --git a/migration/migration.h b/migration/migration.h index 485d58b95f..0920a0950e 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -332,6 +332,9 @@ struct MigrationState { * This save hostname when out-going migration starts */ char *hostname; + + /* Whether to skip pre-copy phase of migration or not */ + bool skip_precopy; }; =20 void migrate_set_state(int *state, int old_state, int new_state); --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653934338; cv=pass; d=zohomail.com; s=zohoarc; b=XR8qdCxRrwiiTK042pi/0iHjDKloGiHJKk6zWSWLaVGhG6Jrno65CxKeLy3JDbYOiRUMvOoS0eEIyc8D7Q3mEv6TyHulIkao+Y+/vZDZA8WybtG+lWRPEPM/1jKH0c4LzSZXKcbNclCaXTt9tQLs1UI3ec2Ze2ljbIxtxNC0Igo= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653934338; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=33dWet8F7c64nY1h8u8FSdJegji/Xrkz5DD1ILOqYGM=; b=LrPzTfxZ3AXPZyXwEX1JJH33tdgbWUXfX32ztXsvd5RuqX5wCxR8bCSAHL/B22ay3jLfV3bdRDUc5qNIeraFdBG1i/0LNvkNA6WFUiOc08nY31Yn+LcLn84mzIvLouVMKN6kMyzyVZbiLtcj/WTUysrl+G23KFIB63UIZdH1E4I= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653934338435776.9474925093288; Mon, 30 May 2022 11:12:18 -0700 (PDT) Received: from localhost ([::1]:57612 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjsH-0007ih-Cb for importer@patchew.org; Mon, 30 May 2022 14:12:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54246) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvis4-0003TP-TW for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:00 -0400 Received: from mail-co1nam11on2060e.outbound.protection.outlook.com ([2a01:111:f400:7eab::60e]:26112 helo=NAM11-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvis2-0008Ll-Hr for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:00 -0400 Received: from MW4PR04CA0215.namprd04.prod.outlook.com (2603:10b6:303:87::10) by MWHPR12MB1742.namprd12.prod.outlook.com (2603:10b6:300:112::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:07:55 +0000 Received: from CO1NAM11FT026.eop-nam11.prod.protection.outlook.com (2603:10b6:303:87:cafe::bb) by MW4PR04CA0215.outlook.office365.com (2603:10b6:303:87::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:55 +0000 Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT026.mail.protection.outlook.com (10.13.175.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:54 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:07:54 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:07:53 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:07:50 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mtmTED1a37dDf1Wo498cCsqKQ2ZzOPu5mvcvVa+MBPJIGfkFM3UgVe1UaCj5MN4mA+9ao74VGw177PMQVNqZudPzIhNcH7FSBbjTGmjcSTWMsYj668/+G5XgbOI8HwshaERtKyLJNsNSOLOMEKqEKkh+fpBP6ALYAhGkCmI2+gyIZr7jf2dI9ZNSqOH8X8Sd9jTVTxtbSwBjcFIDncwW0cD0+/Q5p4eIiBPIHyHs6mtqw5yWdLJvAq0nsG3UqaCJZ3CWMO0pgZW3yXlncD16+r8/LDXwMQj5soStX0DN4dotDl4rCie4X2FWqXD8ptnugK++eCzZTTbaBJhyMkhUSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=33dWet8F7c64nY1h8u8FSdJegji/Xrkz5DD1ILOqYGM=; b=D7xGCX1bUFpEt8rnaYK0FxwTIcWcwwedJRsRGGbn+pwg1JfhP8GQ6/SwY7ZxLthIZ4dD5NR1VIIABWjz/3EuhqcAjlxvvdvXI2zDDzifr1fzYeO3xMqvIeWgtu2bUw4wUQg1CX0o4+89guLIFlUyJ0jJUruzdzWTrVuwuILrJ+2OY9peDy+rdnrbcpZf1fZMnN+QGnCPsUKgZJmjfb0B1nKp+4PpUaFQYn0LBAuBR3WBfL9Tbtef+JCONNN8hnKjnUxvG1DzvyabGoppQApD1gMW6szYudjt95qoukLDtvaWTNHl6Bkn65GCBAlMiqSiqDqtX5XKJmLRX7kDxTjTIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=33dWet8F7c64nY1h8u8FSdJegji/Xrkz5DD1ILOqYGM=; b=JTUfXZMet43l9BWhcRqDnQ46KGC/Bse92skmdQzSkZWCpV4xkcLdJgF30oBgFYqR9Lql8mSUXJStQ0DIprpLIWd4aXi8Ca+S5p3KKEJwu/3WOFWdkWnjb66uksakLvnS4or/0yVvdc//9jlDXKKZkb1DKWi5R4znLOh3M4Oqy0oLPWyeKrmcMlBBbM69dUEK/JTym2eCfGDigrgDimAaMoFk1EgXSYZTKaWdm499tCeBS4CBxooiwhHqDQU/sEwxdz/fT85XQKsFx8za6IJNjxQ212CaKNMUU+QcEu6gWY+00h0y6EtnaoozkPHDnfJELfmjpXyjwWmKVuBy194mvw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 03/11] migration/qemu-file: Add qemu_file_get_to_fd() Date: Mon, 30 May 2022 20:07:31 +0300 Message-ID: <20220530170739.19072-4-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 953c0b38-f026-42da-6957-08da425ef032 X-MS-TrafficTypeDiagnostic: MWHPR12MB1742:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MvUQii6WdxXcn8PwmNz9IFWor48bWhqaTkzqadQRKrq09brsx5Pj8yrhm9NhBulo7ZkFEdbKM65ddmCHeZIbfpuArwcbwepGXi3OEZ8XSQyOUbb9aWg57RPNMk2LDJgq9ppGFIt11wCtaE4VX/JG0kNHCMOS0xXNxgKDJ5r5V3SXDfxgrJMoiI/Bw87Wsxa6ZNNdGAOioIobVavmc5BDTo1LPqPqACGWBkMEYT970JSGFaGiduNtG/caKO2cbHisHNavz9DbKkkw3bjtwSdgr+u0s3smatqFIsJrXJ8kZ9Qig1Ju4pR77wWN557CL7DaidF9Lyul62DUg9CZ5sUjgy+foz1bVRxl3CexqaPzOClVvwBt5fG32gmA1I9ZyhGpQvXtwqYX29BHPRdC0pQj0nKHXXnArzwFTjopLopW/6qNLk75/N5pR07slQ2zyAvZRo0ALLOSnujUjY/Vn9IOTe61tclQ8aruNeG8VbPCtSQ3wp1KCQRCfF7dYGaP3R/GNnMcvQsjL/3qwdDjI8o5As9vTFOIAdHkpInc82Oc4+vyzjap432guOsj/UoPi4zlRkI48OWrT+B8cJK0tbvHe5eHaH1y8PRd10CIr81pxoGcIGcESKwiHtzViCuIF52ehyuuv0wY8oE8JG+XPo51WppLUkn5QTRRJqtDMTsDBuJlcZkOdc7ofU9joKOg/fFS7n6lQxp5QfsXCqtTjzzyNA== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(83380400001)(36860700001)(508600001)(86362001)(2906002)(40460700003)(81166007)(82310400005)(4326008)(8676002)(356005)(26005)(47076005)(6666004)(54906003)(70586007)(316002)(110136005)(70206006)(36756003)(5660300002)(2616005)(107886003)(7696005)(8936002)(1076003)(336012)(186003)(426003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:07:54.9447 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 953c0b38-f026-42da-6957-08da425ef032 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT026.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1742 Received-SPF: softfail client-ip=2a01:111:f400:7eab::60e; envelope-from=avihaih@nvidia.com; helo=NAM11-CO1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653934340557100001 Content-Type: text/plain; charset="utf-8" Add new function qemu_file_get_to_fd() that allows reading data from QEMUFile and writing it straight into a given fd. This will be used later in VFIO migration code. Signed-off-by: Avihai Horon --- migration/qemu-file.c | 34 ++++++++++++++++++++++++++++++++++ migration/qemu-file.h | 1 + 2 files changed, 35 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 1479cddad9..cad3d32eb3 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -867,3 +867,37 @@ QIOChannel *qemu_file_get_ioc(QEMUFile *file) { return file->has_ioc ? QIO_CHANNEL(file->opaque) : NULL; } + +/* + * Read size bytes from QEMUFile f and write them to fd. + */ +int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size) +{ + while (size) { + size_t pending =3D f->buf_size - f->buf_index; + ssize_t rc; + + if (!pending) { + rc =3D qemu_fill_buffer(f); + if (rc < 0) { + return rc; + } + if (rc =3D=3D 0) { + return -1; + } + continue; + } + + rc =3D write(fd, f->buf + f->buf_index, MIN(pending, size)); + if (rc < 0) { + return rc; + } + if (rc =3D=3D 0) { + return -1; + } + f->buf_index +=3D rc; + size -=3D rc; + } + + return 0; +} diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 3f36d4dc8c..dd26037450 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -162,6 +162,7 @@ int qemu_file_shutdown(QEMUFile *f); QEMUFile *qemu_file_get_return_path(QEMUFile *f); void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); +int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); =20 void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653933087; cv=pass; d=zohomail.com; s=zohoarc; b=HpUGz6YRWFynlSko6ayRr3YUvb3QkeT9cZHX9EXpq6+wROTrt05qPFmA4FmQyLOtZ2LrmtaBUwk6S70qmkMI8m1yLwodXDEr+wEUx+IC4KKfGMBknmh0WHvTmo7I6s1wicVAqZHLd4jrCHFVMQCurfe/DzHP9xu8hk7br65E34U= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653933087; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jypF+XZ5TJO5MGBaBe6ueZYxzrFZm4hXeUTs3lXLfRs=; b=fddgRctwAXJRMRtmASLtwQb/j4Vr4N9jSAQ9tw77gj6dGnS+RNfcirYyQnz5Bb7ctQ0MPbXRIDbRIJsHElflMTbcfGOEnMZoyPJFR1Ett4OZ81CVPQrbG0koOFD3+XCOJ1Zr2mpi8bScAboqIBnz8f8TyC0SN5H5rlz/Y+WB9tY= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653933087047856.3156304894192; Mon, 30 May 2022 10:51:27 -0700 (PDT) Received: from localhost ([::1]:40428 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjY4-0000g0-M8 for importer@patchew.org; Mon, 30 May 2022 13:51:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54262) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvis7-0003gO-Uq for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:03 -0400 Received: from mail-dm6nam10on20630.outbound.protection.outlook.com ([2a01:111:f400:7e88::630]:18414 helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvis5-0008M4-OI for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:03 -0400 Received: from MWHPR08CA0044.namprd08.prod.outlook.com (2603:10b6:300:c0::18) by BN8PR12MB2994.namprd12.prod.outlook.com (2603:10b6:408:42::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.18; Mon, 30 May 2022 17:07:59 +0000 Received: from CO1NAM11FT030.eop-nam11.prod.protection.outlook.com (2603:10b6:300:c0:cafe::fc) by MWHPR08CA0044.outlook.office365.com (2603:10b6:300:c0::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:58 +0000 Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT030.mail.protection.outlook.com (10.13.174.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:07:58 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:07:58 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:07:57 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:07:54 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qza9g1Jg51/dJcohJyhKUS1hG1Itssup65hsV/xWCkRlC+hBE/+vWVmzsyKU6uZSmURGRNsD1U1yk2QrbEOramLNkZrUpILWC+VDGJRf7kTWgmPMyViTKmbK7iRNapZT/rkt95QrLdyjC/bA7ndoRLy9wxVd+gQQckV9/0VRSRWf36Q11xHME0b5/ERAYqhE/mzDAPUFSDMxyw4T88tq7BbeH/FVtBEKYdiqngToXCW9BUb8P7ZQUQ5CueIdOGS3gZOcqDXki3IUAvjxFQYL59cR6yMGX2wJNIwbU9naxaWYX7XfT3gERInmgcE303RPu98bGi6BBT53PjkfsY1Pyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jypF+XZ5TJO5MGBaBe6ueZYxzrFZm4hXeUTs3lXLfRs=; b=ocGBuH+NpPc3is7hiDtEF4bDlhCK8WhQuOWbBEfXfbi39BBdFMQi3bVzVrfRkKYZ3qdY5X9FKwRLue0RGcW+B4bSuF7+BcZsMyxu8rnSdvc20BwPs98nBfNZAufikzE3/54CtPhPLUH+bAliQATcnAp0ecI30hngQNFKFwasfPMjAap5OUE+s/TmHTWfZzkVDR5ODZu6YGlmfrVuAXvL2lTkgAait3rEOj8i9tR5Tasi6jvyvOpU5VfowsrTJOP3vxNo597DwxPmV+K3xGsYOWbSbU0vCt8N2fGhFj8J7k7d4DVM4bkRyh1Ry7rgiOOEwNN5FS1fxCQu2Lt8Sfi1Iw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jypF+XZ5TJO5MGBaBe6ueZYxzrFZm4hXeUTs3lXLfRs=; b=pqd8qbAwOfEoKsNUW54KrAfJ8N7v3vZHwwT4M1J7wQLF5r1XfozNWGZ9CIHY1nIF9Xal/zefjwyhrfagb6Y69E8s/CWg8I9cbsNwlNASVDNm65UaAn4BFWEvMmCi4WX2Pq41APxilfkNi2gICtJZMtTD4xPwRhpykslOdEwiyvemTxKWtzbvGfXhIr3c6jKoXR8fYSEJqN7Vusc3FJ+WGQQETMyIsfd35Ji5oPaeZxwIHpLyzCSdmkqSVLP3DhU2QY3mk4jwrqWtdC/ZQ1Ye/ZFmAzHwJyPOAJVHvv4TVZOKXf9c27Qjk7jdnrqu6LQ7KnxGwzBIfeh8oWbdBFciUA== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 04/11] vfio/common: Change vfio_devices_all_running_and_saving() logic to equivalent one Date: Mon, 30 May 2022 20:07:32 +0300 Message-ID: <20220530170739.19072-5-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fcdc6f24-c8d3-4398-af4d-08da425ef252 X-MS-TrafficTypeDiagnostic: BN8PR12MB2994:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5pURG42hkpy7fL+ILNVx76YfBKVArihXaK/UYbyUiNLhw6K3ORuiGir1ECymPTH7G/RcH4uNPWFzLT42JvK8duGy3LLVrR6bk1+zNF3nhhm6ve0nV1cE44pvABShF7k+5DCmXRbZkhvRewJ52eKH7oey8e6VQug4vjnF9GWhK0EH7J7phf4NcGjY5NT/5qMKuV2JnpvyaHKTEV4esDpeh+W9c04VIFFFkNIplPEfnAeU1gsrmSkERaONB0b9z4qTxMsDQlF8Q5JYmC1m4CPYKYduFVhIh4rI4Y2UASZfYKcGTC4FQ3Khjfm60bclwLBvUFgMkrea0AeX/QgxesolZ0A5odC8FB+upLRtSWb3uCL0+dpvXmO9SUjJPcHCtSTZ3clLiCzH3Yu8j5kAU05HnyJ2/dX8gZiYGTplgI24cDUgAgQuoDM6bYbxCipWycYHn76GWuDpEgxT/0ASlyBjwcdYxbWd2WMxZ+Tx3hhPd9sL9ZP8dnOcgpom/QOKSujN5bkIVx20xpRc1EVhSvXJwFQ2pJUcWXgh3hTeLuoKa+BSik7y3PqbtnQj9tqNNEg+z3ABmj+8IbzwSGAbQW3P4+Ts9bqZLGeDAsmlVIiUEpanpfzCWy7je87wF1uSnuK3jDvZikUsivthkpAUzBqMnGKJwy96jtpaV5Hg8D9wTmwpkGV3pmC78g4mRmJJQ649E7/ZKGABu79fEUx9VAhW/w== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(316002)(4326008)(7696005)(2616005)(8676002)(70206006)(70586007)(26005)(82310400005)(2906002)(86362001)(5660300002)(508600001)(8936002)(40460700003)(426003)(356005)(336012)(47076005)(36860700001)(36756003)(6666004)(186003)(1076003)(107886003)(54906003)(81166007)(110136005)(83380400001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:07:58.5116 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fcdc6f24-c8d3-4398-af4d-08da425ef252 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT030.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB2994 Received-SPF: softfail client-ip=2a01:111:f400:7e88::630; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653933087726100001 Content-Type: text/plain; charset="utf-8" vfio_devices_all_running_and_saving() is used to check if migration is in pre-copy phase. This is done by checking if migration is in setup or active states and if all VFIO devices are in pre-copy state, i.e. _SAVING | _RUNNING. VFIO migration protocol v2 currently doesn't support pre-copy phase, so it doesn't have an equivalent VFIO pre-copy state like v1 has. As preparation for adding the v2 protocol and to make things easier, change vfio_devices_all_running_and_saving() logic to check if migration is active and if all VFIO devices are in running state. The new logic avoids using the VFIO pre-copy state and is equivalent to the previous one, indicating that migration is in pre-copy phase. No functional changes intended. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 29982c7af8..bbc6d375de 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -40,6 +40,7 @@ #include "trace.h" #include "qapi/error.h" #include "migration/migration.h" +#include "migration/misc.h" #include "sysemu/tpm.h" =20 VFIOGroupList vfio_group_list =3D @@ -363,13 +364,16 @@ static bool vfio_devices_all_dirty_tracking(VFIOConta= iner *container) return true; } =20 -static bool vfio_devices_all_running_and_saving(VFIOContainer *container) +/* + * Check if all VFIO devices are running and migration is active, which is + * essentially equivalent to the migration being in pre-copy phase. + */ +static bool vfio_devices_all_running_and_mig_active(VFIOContainer *contain= er) { VFIOGroup *group; VFIODevice *vbasedev; - MigrationState *ms =3D migrate_get_current(); =20 - if (!migration_is_setup_or_active(ms->state)) { + if (!migration_is_active(migrate_get_current())) { return false; } =20 @@ -381,8 +385,7 @@ static bool vfio_devices_all_running_and_saving(VFIOCon= tainer *container) return false; } =20 - if ((migration->device_state & VFIO_DEVICE_STATE_V1_SAVING) && - (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) { + if (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING) { continue; } else { return false; @@ -461,7 +464,7 @@ static int vfio_dma_unmap(VFIOContainer *container, }; =20 if (iotlb && container->dirty_pages_supported && - vfio_devices_all_running_and_saving(container)) { + vfio_devices_all_running_and_mig_active(container)) { return vfio_dma_unmap_bitmap(container, iova, size, iotlb); } =20 --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653934621; cv=pass; d=zohomail.com; s=zohoarc; b=lBmrY5199Jiej0QD1HwK38CQZX4ZIqAAwn6rN/uQaUzc+ruzJNHXAa3DGZJu5Q7bhdbALXkbbhvgnXSH5O5wR8zG3kVVYlWq2KA6j0mP6lsjMsVjt7xmLR5BaKLMuFYQXk5X4QAZR1IdKxBYwh4m6AXWF2uVlrPbsHD0rL05G64= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653934621; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DjlscHecdcMpLoybC6kEB/IIni+Nsq0JItrk38p+xG8=; b=JTwt7BRIPMlUokq0EswFFYF1BzfsWTmCSL+O/CdK16MnWFacg+y4cRzzStcVr8TWAzf6DinotyScIpn83hiFFrFb5dKjI7c/S2X4aLT+O/cZvRwzQLs0l/afiYzHV+oSiZ0Y9cTJ8P7uqJkLYQKvZuctCQ/xJnx0ekSlFv3OILU= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653934621842588.6832118366459; Mon, 30 May 2022 11:17:01 -0700 (PDT) Received: from localhost ([::1]:38440 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjwq-0005k0-4f for importer@patchew.org; Mon, 30 May 2022 14:17:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54278) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisB-0003tI-V7 for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:07 -0400 Received: from mail-co1nam11on2060e.outbound.protection.outlook.com ([2a01:111:f400:7eab::60e]:5216 helo=NAM11-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisA-0008MJ-9u for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:07 -0400 Received: from MW4PR04CA0314.namprd04.prod.outlook.com (2603:10b6:303:82::19) by BYAPR12MB4775.namprd12.prod.outlook.com (2603:10b6:a03:107::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:08:02 +0000 Received: from CO1NAM11FT059.eop-nam11.prod.protection.outlook.com (2603:10b6:303:82:cafe::d6) by MW4PR04CA0314.outlook.office365.com (2603:10b6:303:82::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:02 +0000 Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT059.mail.protection.outlook.com (10.13.174.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:01 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:08:01 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:08:01 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:07:58 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M5C3ICtLYHIUykVC7v40z6avDB5IaThnwxJrIczOcmqOA3a/Xau3XBit+zXreoW92iJH1MktQeM7kZTs6NU7zHLi1b82gERzaIl6F2Y1j9+Fa+DRHUKHUkz3EX+en1V4LfvtSvbTgK51wktNVrQBOFGMG6jpo+t72AzfzWuaVGYZ0nbPuNRty6KFagt2s4KDIBLTgaJ6WRjADv5DCYlgH3YR9MdcQ9kXmUSxr4A4DkSrfU2Z+a+y3FXkioV2jYIOvIMvAnQMv1wAiLfiDLetVwXkpdlCaI3iqdi6LzCJS427wY4ko0sPRDsHNUPZjN0gFZPh7e9qU/Ssj8UD/jymig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DjlscHecdcMpLoybC6kEB/IIni+Nsq0JItrk38p+xG8=; b=YV9kUbdr1QQ/R0y+wJIeo9qD+IeAgzhIkJdOjAhYk0oe4kAwwUISExvQu11Oor5wNP0J2gOTytmJFhUelGKhyhy9dU+Dtp+TULw+Xk6PegTOuGeUCfc1gP8yXOKvP+nz9a+jz0hvzKuY+cbOAVPhMUNVw+U80SyD27o+Xp2M8rWyI42F1C9m4BMsdZsyD4mMhGrymvS7/52Vx4Bq2vPDCbVu+Ur1M/5UE1pWOg9rCXLoq7+xowW1yzJF0eLqWTwYZLrhyBwDARwGVJWsW0QGOvYcnVIbzr8eHqMYBvAJvBENOevbtaXA2s660eA1LjztAPOW4FbgZIfABa/VtIKiSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DjlscHecdcMpLoybC6kEB/IIni+Nsq0JItrk38p+xG8=; b=p1KA81HzPakd1eUDtv9MYv3Hj0l964Xb2DX3SpQG7GP1akssFn0GA8Yc5STF31sFz7OMhJhPBlOcTQyP5IKjFxG/6M1MQ9XOh5zmiRpCpqDOfRyL1vJxixhQrSqzv9YmMO5nWdXuzhu/yDPjRKOC5JduELptamIgWS77av+yNIUMCjKe4ojbWqU2xGQlCB6aWYk88UWkvIpBMwqKElGlRhKa3g/YBcEldf1xScgYJ/oCh//qZkey+ZcN2rAwFj/uMZ3P4aFCCYpbuftrEFYtoFVW/HgBhtDEGx6u4MFgHsamTeW24ooUzPkrHnn/1MHkSG1Nzv1AP26FuKPh425CRQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 05/11] vfio/migration: Move migration v1 logic to vfio_migration_init() Date: Mon, 30 May 2022 20:07:33 +0300 Message-ID: <20220530170739.19072-6-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bfb53061-0f4a-4b08-67c8-08da425ef462 X-MS-TrafficTypeDiagnostic: BYAPR12MB4775:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FskSo0/GV3UtBtFwRCcnBZLb7VsUrtSFfSjUy0TwzS9kARXRqAtWnsu+AAx52pOt/2KSXgqweJkGKp9iRCrgMKm4tN3xzfS2fLRpv+WpvfIEM/IQYeajNuOszHqYZAC3sQmkyX5cKQjLTe7YfxyTEqxosnHVANFB2lRXcWiJrKzBpzW3Ud5fvse20rTYQt7S239vEPSGGeyo+AqT3eiHRxw7XJokOSkns9WXp4Xi/laDYdNzgc8qlj2G7DFCT3BebbPWQLY1EIyDDep/nTONavIDLOpRIN53qM01zm+ct1yEAfRPzi6he0GVH+WWtxF/iMfbTc2md9xZBWPTP+BONClS4zJLt59IggPC+xy2HwAdpV9HorXkCdoLESxF64Yi9+di3n/xoHPqwUqI2BLgwZde85SK93eAUhk6JlbdU/uwoPVJidWtEoz8dzq2GCupiDrvtCAtWX44H+oYkPpXB9SJp7E/7C8FbI9zO+p0VMBNinA4/L0myyQPacdhbF0LHrS/taWPVf+MWtvau02X7qh8fUufOvl2D9OUXbnMkPcRpEOuAJ038oE8aiB+Tv3qIXfRR9BVzrPrCY5AOsVIoiHBW8jXanQDS01B5nVWI772k8mEJNXFQURw5Wmg05eHbynX8dSwvtfRbjNtknpdpKlToBFjfc7WIpDMeaBc86fdTzlrYdmwnW3q0eX7YOu9HGryxr+9eFfdZcvV2fOH0A== X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(2616005)(26005)(2906002)(110136005)(54906003)(356005)(8676002)(316002)(36860700001)(7696005)(508600001)(47076005)(86362001)(6666004)(81166007)(336012)(186003)(40460700003)(82310400005)(8936002)(36756003)(426003)(83380400001)(107886003)(70206006)(70586007)(4326008)(1076003)(5660300002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:08:01.9545 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bfb53061-0f4a-4b08-67c8-08da425ef462 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT059.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB4775 Received-SPF: softfail client-ip=2a01:111:f400:7eab::60e; envelope-from=avihaih@nvidia.com; helo=NAM11-CO1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653934623082100001 Content-Type: text/plain; charset="utf-8" Move vfio_dev_get_region_info() logic from vfio_migration_probe() to vfio_migration_init(). This logic is specific to v1 protocol and moving it will make it easier to add the v2 protocol implementation later. No functional changes intended. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 30 +++++++++++++++--------------- hw/vfio/trace-events | 2 +- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index d8f9b086c2..8a0deed0e4 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -789,14 +789,14 @@ static void vfio_migration_exit(VFIODevice *vbasedev) vbasedev->migration =3D NULL; } =20 -static int vfio_migration_init(VFIODevice *vbasedev, - struct vfio_region_info *info) +static int vfio_migration_init(VFIODevice *vbasedev) { int ret; Object *obj; VFIOMigration *migration; char id[256] =3D ""; g_autofree char *path =3D NULL, *oid =3D NULL; + struct vfio_region_info *info =3D NULL; =20 if (!vbasedev->ops->vfio_get_object) { return -EINVAL; @@ -807,6 +807,14 @@ static int vfio_migration_init(VFIODevice *vbasedev, return -EINVAL; } =20 + ret =3D vfio_get_dev_region_info(vbasedev, + VFIO_REGION_TYPE_MIGRATION_DEPRECATED, + VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATE= D, + &info); + if (ret) { + return ret; + } + vbasedev->migration =3D g_new0(VFIOMigration, 1); =20 ret =3D vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, @@ -824,6 +832,8 @@ static int vfio_migration_init(VFIODevice *vbasedev, goto err; } =20 + g_free(info); + migration =3D vbasedev->migration; migration->vbasedev =3D vbasedev; =20 @@ -846,6 +856,7 @@ static int vfio_migration_init(VFIODevice *vbasedev, return 0; =20 err: + g_free(info); vfio_migration_exit(vbasedev); return ret; } @@ -860,7 +871,6 @@ int64_t vfio_mig_bytes_transferred(void) int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) { VFIOContainer *container =3D vbasedev->group->container; - struct vfio_region_info *info =3D NULL; int ret =3D -ENOTSUP; =20 if (!vbasedev->enable_migration) { @@ -874,27 +884,17 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error = **errp) migrate_get_current()->skip_precopy =3D true; } =20 - ret =3D vfio_get_dev_region_info(vbasedev, - VFIO_REGION_TYPE_MIGRATION_DEPRECATED, - VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATE= D, - &info); + ret =3D vfio_migration_init(vbasedev); if (ret) { goto add_blocker; } =20 - ret =3D vfio_migration_init(vbasedev, info); - if (ret) { - goto add_blocker; - } - - trace_vfio_migration_probe(vbasedev->name, info->index); - g_free(info); + trace_vfio_migration_probe(vbasedev->name); return 0; =20 add_blocker: error_setg(&vbasedev->migration_blocker, "VFIO device doesn't support migration"); - g_free(info); =20 ret =3D migrate_add_blocker(vbasedev->migration_blocker, errp); if (ret < 0) { diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 582882db91..438402b619 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -148,7 +148,7 @@ vfio_display_edid_update(uint32_t prefx, uint32_t prefy= ) "%ux%u" vfio_display_edid_write_error(void) "" =20 # migration.c -vfio_migration_probe(const char *name, uint32_t index) " (%s) Region %d" +vfio_migration_probe(const char *name) " (%s)" vfio_migration_set_state(const char *name, uint32_t state) " (%s) state %d" vfio_vmstate_change(const char *name, int running, const char *reason, uin= t32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) = state %s" --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653933532; cv=pass; d=zohomail.com; s=zohoarc; b=KKM6PCw/vKHDHvu+JIQ6NFm0steXEk93VOtcqL0yzRUwTi0EMKUCARI2FVK3VAnExEDSmYINbg3Qd6RQafIN9f4t3h9U8gRLRy/PGyzdyiawyW7kiUZw5yWS3tzT60PRHJ/lNitp8xMYR33LGX+25v1WGmZF4Ql5y4290WeOTLs= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653933532; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TAcKiWMX79rHiJ0N+9JONfz6MYJ8RIJrv7lY5JEVzKA=; b=PIui8LtCR5B6JnPixlqS40HH5z3/m8IAepOgJmoqGmonx4sRaIGTm/izqsuFiELypZOBIgo8qxLbG3Ilzn2mCMTpOagZJn8pO3baQXVXzsS5G18Mcun2sMc4IyNk2a3V2iu4q1KaXwHpLahJJdfdEmlu5NjyizYJwebdgjvmXI0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653933532480650.8493020171793; Mon, 30 May 2022 10:58:52 -0700 (PDT) Received: from localhost ([::1]:56680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjfH-0003Yr-79 for importer@patchew.org; Mon, 30 May 2022 13:58:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54300) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisF-00049y-K8 for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:11 -0400 Received: from mail-dm6nam10on20611.outbound.protection.outlook.com ([2a01:111:f400:7e88::611]:46689 helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisD-0008MT-5P for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:11 -0400 Received: from MWHPR10CA0069.namprd10.prod.outlook.com (2603:10b6:300:2c::31) by MWHPR12MB1360.namprd12.prod.outlook.com (2603:10b6:300:12::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:08:05 +0000 Received: from CO1NAM11FT024.eop-nam11.prod.protection.outlook.com (2603:10b6:300:2c:cafe::e6) by MWHPR10CA0069.outlook.office365.com (2603:10b6:300:2c::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:05 +0000 Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT024.mail.protection.outlook.com (10.13.174.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:05 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:08:05 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:08:04 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:08:01 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FSW6oANBiCDkEAh7KVmNV34FupFOe+8Kgu916NIGnMjeR1mZJvjdKPptL84uQC+5fWif83UyHAycAY9lJp1VkPkdPb9NO9hCUO3Xfs77c0gS/QNU3Ddel9IxC+B5M6MLCD5gEPugD60SOT9ObwHHTUWU145mZXNsYUw1Aa0d/s3BZ1Ul2MVmfqq1cAMifDj/8Uxks83H3xJCPyNj5frYt6T0ToCvSsc9K2YFJiVum7SdLUFo38YRkPc8SGpu/z1wXP/csSh6IIKTtHmmJri0q8NJH2T5G+knBX6DYnqqVUqvrFdWQGAopVe+ZzTnJKrtCoWmXeg5muwuXVrAUf5TZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TAcKiWMX79rHiJ0N+9JONfz6MYJ8RIJrv7lY5JEVzKA=; b=fH4R7SgwWx5ddFzh4GFmON/dZE1hBCpuoD7YS+7eNcrWfsPe5cHUPFKIk9hPQnACIoWzF/RDSyzj4nG6ufRt/ols5CINjoYOZGNtej2SQ+vwiUn8n6D0E/ay1DHD4HkJOuZDw+pCMRGXJADG9rhniSf4k5ch2GdqFuZ5s6OOJAyepYfF7sMe/S/Qit8sLIJucJNoPuazfqfFceG4iAwjr8qSLYRErU4DwAu72U661dDT8bH8B0w/ZwNvOgXl9+gPqDjwZiocPUeHfEcCDY1gt+dDvk6mLdGNH+Cm8xdnOuoUoXL+ytg+hgwFpaSdlAiY6vU/Yhk7Um+7wGko73P2DA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TAcKiWMX79rHiJ0N+9JONfz6MYJ8RIJrv7lY5JEVzKA=; b=cnS5mJF9GAPK59C32rAPRSIN0HHn6KlfL9vxg8VfTsztdajetkwi/LoZb8Jv7DrbD8Kfmzkmpk55eQaM2sF8snK2PO1jp04M9aSJtJlnCydhUA9denRS30m95d8jN0U668fFnL6SpLy1nxLJUsU4gYMopuQDRrVr4Z1RGw4JEPJYuxlwAXXJn8w+WiVWlHU4D6y0CQzDX+Z/zZ00iebAFOi6adiRIbhqhiqHjbcnfQrYcMBVkiai0kuYajMaDULBf1jkYPyB6MWOzhlkGoEKTtyOPuPevoXglrs5LpgQXIFQL85Rjmo7PBsgvbLzSG73CLBCGBGC38I8t77Fsrn7TQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 06/11] vfio/migration: Rename functions/structs related to v1 protocol Date: Mon, 30 May 2022 20:07:34 +0300 Message-ID: <20220530170739.19072-7-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 50a572a4-143a-4b83-e4aa-08da425ef69a X-MS-TrafficTypeDiagnostic: MWHPR12MB1360:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 14GiJsw5JtDs5YFpNx65qBCNPXJuOkUUYSV4RSTtgiukMS5KqxKI0LSlnY+ZaFwuY+t4H70Wr5Rfegs1lF5KZktJPu/p2NsWkIjeUDtSHTwSmEWiWnuuGaxW1Lc3nojqUME09ZfqYuYMO71ousFTbMdRjLN91c4kV26bT9wYkF3Q4pqo55S2jyEelJXYgyw46/kR7AwenYamhgOkUEsCWFEkEm6L1FyjmsdDBBSTnVx08VrHYNibnZyzQJk5vzDouqo6lUDuNZrQ8dvvyQJ6GnC1aLLdMAXQ5AA5AXZ/tLluibrpHoyVLIN9BYKzi4OdH8OXkDGmTVipH9HitXXNwKzlSaSFSffZ2MhVNjcfhqvbG3DWFhrcS2237A17Bm81E2P1ddbwvUAhPdDataCOOyCiuO5hbCCk1NnVObLe751/uwXjIMGTGZ2ZI0HsSa6vhJO/YDVA3omKMXSIqOGR1zkPbUNG2zYV8LO0FdwYz6opZsKOcgXgghlcsryRcxZ0oFfY1hBdGsp7cEGceaBON5bz/H0VbADUUdQZws50tPiYnQJur9EQFtxkOUn3x1gjGe3Ars/HeeLAuSF4yDIh4xHVrkkqdZzA1ymZzN7ftXe3aJDfJCuC0nqTYzCFUzx9V/zIxJBeeAxSAJ8AOWwP47BT1WJNW5LUH1SHaTGbMHMl+cMoP4FBN6vghlwXcRJz0Ohta3foe3nYBUQ46S83yA== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(40460700003)(186003)(86362001)(82310400005)(6666004)(81166007)(336012)(70206006)(107886003)(5660300002)(4326008)(70586007)(1076003)(426003)(36756003)(83380400001)(8936002)(30864003)(356005)(8676002)(2906002)(110136005)(54906003)(316002)(26005)(2616005)(508600001)(47076005)(36860700001)(7696005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:08:05.6801 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 50a572a4-143a-4b83-e4aa-08da425ef69a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT024.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1360 Received-SPF: softfail client-ip=2a01:111:f400:7e88::611; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653933533925100001 Content-Type: text/plain; charset="utf-8" To avoid name collisions, rename functions and structs related to VFIO migration protocol v1. This will allow the two protocols to co-exist when v2 protocol is added, until v1 is removed. No functional changes intended. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 6 +-- hw/vfio/migration.c | 82 +++++++++++++++++------------------ hw/vfio/trace-events | 2 +- include/hw/vfio/vfio-common.h | 2 +- 4 files changed, 46 insertions(+), 46 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index bbc6d375de..a3dd8221ed 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -355,8 +355,8 @@ static bool vfio_devices_all_dirty_tracking(VFIOContain= er *container) return false; } =20 - if ((vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) - && (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING= )) { + if ((vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) && + (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING= )) { return false; } } @@ -385,7 +385,7 @@ static bool vfio_devices_all_running_and_mig_active(VFI= OContainer *container) return false; } =20 - if (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING) { + if (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING)= { continue; } else { return false; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 8a0deed0e4..e40aa0ad80 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -107,8 +107,8 @@ static int vfio_mig_rw(VFIODevice *vbasedev, __u8 *buf,= size_t count, * an error is returned. */ =20 -static int vfio_migration_set_state(VFIODevice *vbasedev, uint32_t mask, - uint32_t value) +static int vfio_migration_v1_set_state(VFIODevice *vbasedev, uint32_t mask, + uint32_t value) { VFIOMigration *migration =3D vbasedev->migration; VFIORegion *region =3D &migration->region; @@ -145,7 +145,7 @@ static int vfio_migration_set_state(VFIODevice *vbasede= v, uint32_t mask, return ret; } =20 - migration->device_state =3D device_state; + migration->device_state_v1 =3D device_state; trace_vfio_migration_set_state(vbasedev->name, device_state); return 0; } @@ -260,8 +260,8 @@ static int vfio_save_buffer(QEMUFile *f, VFIODevice *vb= asedev, uint64_t *size) return ret; } =20 -static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, - uint64_t data_size) +static int vfio_v1_load_buffer(QEMUFile *f, VFIODevice *vbasedev, + uint64_t data_size) { VFIORegion *region =3D &vbasedev->migration->region; uint64_t data_offset =3D 0, size, report_size; @@ -288,7 +288,7 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *vb= asedev, data_size =3D 0; } =20 - trace_vfio_load_state_device_data(vbasedev->name, data_offset, siz= e); + trace_vfio_v1_load_state_device_data(vbasedev->name, data_offset, = size); =20 while (size) { void *buf; @@ -394,7 +394,7 @@ static int vfio_load_device_config_state(QEMUFile *f, v= oid *opaque) return qemu_file_get_error(f); } =20 -static void vfio_migration_cleanup(VFIODevice *vbasedev) +static void vfio_migration_v1_cleanup(VFIODevice *vbasedev) { VFIOMigration *migration =3D vbasedev->migration; =20 @@ -405,7 +405,7 @@ static void vfio_migration_cleanup(VFIODevice *vbasedev) =20 /* ---------------------------------------------------------------------- = */ =20 -static int vfio_save_setup(QEMUFile *f, void *opaque) +static int vfio_v1_save_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; @@ -431,8 +431,8 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) } } =20 - ret =3D vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_SAVING); + ret =3D vfio_migration_v1_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, + VFIO_DEVICE_STATE_V1_SAVING); if (ret) { error_report("%s: Failed to set state SAVING", vbasedev->name); return ret; @@ -448,11 +448,11 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) return 0; } =20 -static void vfio_save_cleanup(void *opaque) +static void vfio_v1_save_cleanup(void *opaque) { VFIODevice *vbasedev =3D opaque; =20 - vfio_migration_cleanup(vbasedev); + vfio_migration_v1_cleanup(vbasedev); trace_vfio_save_cleanup(vbasedev->name); } =20 @@ -524,15 +524,15 @@ static int vfio_save_iterate(QEMUFile *f, void *opaqu= e) return 0; } =20 -static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) +static int vfio_v1_save_complete_precopy(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; uint64_t data_size; int ret; =20 - ret =3D vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RUNNI= NG, - VFIO_DEVICE_STATE_V1_SAVING); + ret =3D vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RU= NNING, + VFIO_DEVICE_STATE_V1_SAVING); if (ret) { error_report("%s: Failed to set state STOP and SAVING", vbasedev->name); @@ -569,7 +569,8 @@ static int vfio_save_complete_precopy(QEMUFile *f, void= *opaque) return ret; } =20 - ret =3D vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SAVIN= G, 0); + ret =3D vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SA= VING, + 0); if (ret) { error_report("%s: Failed to set state STOPPED", vbasedev->name); return ret; @@ -592,7 +593,7 @@ static void vfio_save_state(QEMUFile *f, void *opaque) } } =20 -static int vfio_load_setup(QEMUFile *f, void *opaque) +static int vfio_v1_load_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; @@ -608,8 +609,8 @@ static int vfio_load_setup(QEMUFile *f, void *opaque) } } =20 - ret =3D vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_RESUMING); + ret =3D vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, + VFIO_DEVICE_STATE_V1_RESUMING); if (ret) { error_report("%s: Failed to set state RESUMING", vbasedev->name); if (migration->region.mmaps) { @@ -619,11 +620,11 @@ static int vfio_load_setup(QEMUFile *f, void *opaque) return ret; } =20 -static int vfio_load_cleanup(void *opaque) +static int vfio_v1_load_cleanup(void *opaque) { VFIODevice *vbasedev =3D opaque; =20 - vfio_migration_cleanup(vbasedev); + vfio_migration_v1_cleanup(vbasedev); trace_vfio_load_cleanup(vbasedev->name); return 0; } @@ -661,7 +662,7 @@ static int vfio_load_state(QEMUFile *f, void *opaque, i= nt version_id) uint64_t data_size =3D qemu_get_be64(f); =20 if (data_size) { - ret =3D vfio_load_buffer(f, vbasedev, data_size); + ret =3D vfio_v1_load_buffer(f, vbasedev, data_size); if (ret < 0) { return ret; } @@ -682,21 +683,21 @@ static int vfio_load_state(QEMUFile *f, void *opaque,= int version_id) return ret; } =20 -static SaveVMHandlers savevm_vfio_handlers =3D { - .save_setup =3D vfio_save_setup, - .save_cleanup =3D vfio_save_cleanup, +static SaveVMHandlers savevm_vfio_v1_handlers =3D { + .save_setup =3D vfio_v1_save_setup, + .save_cleanup =3D vfio_v1_save_cleanup, .save_live_pending =3D vfio_save_pending, .save_live_iterate =3D vfio_save_iterate, - .save_live_complete_precopy =3D vfio_save_complete_precopy, + .save_live_complete_precopy =3D vfio_v1_save_complete_precopy, .save_state =3D vfio_save_state, - .load_setup =3D vfio_load_setup, - .load_cleanup =3D vfio_load_cleanup, + .load_setup =3D vfio_v1_load_setup, + .load_cleanup =3D vfio_v1_load_cleanup, .load_state =3D vfio_load_state, }; =20 /* ---------------------------------------------------------------------- = */ =20 -static void vfio_vmstate_change(void *opaque, bool running, RunState state) +static void vfio_v1_vmstate_change(void *opaque, bool running, RunState st= ate) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; @@ -736,21 +737,21 @@ static void vfio_vmstate_change(void *opaque, bool ru= nning, RunState state) } } =20 - ret =3D vfio_migration_set_state(vbasedev, mask, value); + ret =3D vfio_migration_v1_set_state(vbasedev, mask, value); if (ret) { /* * Migration should be aborted in this case, but vm_state_notify() * currently does not support reporting failures. */ error_report("%s: Failed to set device state 0x%x", vbasedev->name, - (migration->device_state & mask) | value); + (migration->device_state_v1 & mask) | value); if (migrate_get_current()->to_dst_file) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); } } vbasedev->migration->vm_running =3D running; trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), - (migration->device_state & mask) | value); + (migration->device_state_v1 & mask) | value); } =20 static void vfio_migration_state_notifier(Notifier *notifier, void *data) @@ -769,10 +770,10 @@ static void vfio_migration_state_notifier(Notifier *n= otifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred =3D 0; - ret =3D vfio_migration_set_state(vbasedev, - ~(VFIO_DEVICE_STATE_V1_SAVING | - VFIO_DEVICE_STATE_V1_RESUMING), - VFIO_DEVICE_STATE_V1_RUNNING); + ret =3D vfio_migration_v1_set_state(vbasedev, + ~(VFIO_DEVICE_STATE_V1_SAVING | + VFIO_DEVICE_STATE_V1_RESUMING), + VFIO_DEVICE_STATE_V1_RUNNING); if (ret) { error_report("%s: Failed to set state RUNNING", vbasedev->name= ); } @@ -845,12 +846,11 @@ static int vfio_migration_init(VFIODevice *vbasedev) } strpadcpy(id, sizeof(id), path, '\0'); =20 - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_hand= lers, - vbasedev); + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, + &savevm_vfio_v1_handlers, vbasedev); =20 - migration->vm_state =3D qdev_add_vm_change_state_handler(vbasedev->dev, - vfio_vmstate_ch= ange, - vbasedev); + migration->vm_state =3D qdev_add_vm_change_state_handler( + vbasedev->dev, vfio_v1_vmstate_change, vbasedev); migration->migration_state.notify =3D vfio_migration_state_notifier; add_migration_state_change_notifier(&migration->migration_state); return 0; diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 438402b619..ac8b04f52a 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -162,7 +162,7 @@ vfio_save_iterate(const char *name, int data_size) " (%= s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_load_state_device_data(const char *name, uint64_t data_offset, uint64= _t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 +vfio_v1_load_state_device_data(const char *name, uint64_t data_offset, uin= t64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitma= p_size, uint64_t start) "container fd=3D%d, iova=3D0x%"PRIx64" size=3D 0x%"= PRIx64" bitmap_size=3D0x%"PRIx64" start=3D0x%"PRIx64 vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu= dirty @ 0x%"PRIx64" - 0x%"PRIx64 diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index e573f5a9f1..bbaf72ba00 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -62,7 +62,7 @@ typedef struct VFIOMigration { struct VFIODevice *vbasedev; VMChangeStateEntry *vm_state; VFIORegion region; - uint32_t device_state; + uint32_t device_state_v1; int vm_running; Notifier migration_state; uint64_t pending_bytes; --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653934888; cv=pass; d=zohomail.com; s=zohoarc; b=RO+iv9LHtD0whEvuoPU+V9tUYjlIhkUPwEN3kuKbsNTZmijUwsoRZY4b9v9m2sW3Gzb+i3QQ07qmMlevaiQFpn4BMCZGocqInLHFWG35TQT4Hl0nBBC2HbfKoJ8SUygOzY1Nk++yhcE1BAW3Lr6uJeWQo3KOcHfaBNYnxJViR+Y= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653934888; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9/zOSCKw7zFSSB+1LnE/2WjnkVIfVsyyLiFlMm4RF1o=; b=ieZCZrcd8XtylmwyoWggfcmdKc88LRwC/wu0/t6piptUwJX7ZY301tK44hQ4GpOih32tXWhVDXSjGGzt8i3fbiAEO0aGxh3zW/jl4f5edFAOLEPkYVrwpE2OUzAl3nMpUh2OcITI/vqPwZCg0uqDa0Lpinp0aqSAEOm4AQlJW0U= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653934888282105.08917228513292; Mon, 30 May 2022 11:21:28 -0700 (PDT) Received: from localhost ([::1]:47618 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvk19-0003u7-6c for importer@patchew.org; Mon, 30 May 2022 14:21:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54316) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisK-0004Nu-Bg for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:16 -0400 Received: from mail-sn1anam02on20614.outbound.protection.outlook.com ([2a01:111:f400:7ea9::614]:51678 helo=NAM02-SN1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisG-0008Ms-8X for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:16 -0400 Received: from CO1PR15CA0099.namprd15.prod.outlook.com (2603:10b6:101:21::19) by DM6PR12MB4235.namprd12.prod.outlook.com (2603:10b6:5:220::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:08:09 +0000 Received: from CO1NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:101:21:cafe::92) by CO1PR15CA0099.outlook.office365.com (2603:10b6:101:21::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.19 via Frontend Transport; Mon, 30 May 2022 17:08:09 +0000 Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT035.mail.protection.outlook.com (10.13.175.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:08 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:08:08 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:08:08 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:08:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h496DP5xAtWyVhUpAxmVSHIqNFqvEe3ocHJ5gZ8yucw7I5Bqj6Ie2jdvwrvvOsOazPDXcSqb3lAid+MKuabarYc0GjJv9Yy+ySsrgzUiHV+Dbfzl+nNmzT2MCZ0r1j2sl9NI9c526Nkn2RY2W7ImVw+N/xvtjA+5UC0LRcVNOfK+/UHdgQanMavKjQeJXRLfoRA/QhDPtVBjUAX+Dy6xyryhgc+qQeB4GXEv8eS/PbeRsrpEz7yBhjUzJGocuMy1eqjzPcOsgxjCJzYiQzxQh8JJm6zhr+NfYXOfC3Y59HGZrZ2m0G0YUStBCb5nHL5ICa5BKi/AT2XFk+dSEDxF3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9/zOSCKw7zFSSB+1LnE/2WjnkVIfVsyyLiFlMm4RF1o=; b=CuvRtPVztVq/7l6MCzPTmCZiV5jsHV7KsWHw26YM7v8orwL9klTySwGNKKEcms8qZfFs0jMvyo1P9WtZ+FPRYAF/oqXby5kgzeN4XjJX0J7jekoz50/OWwjDp8K5m3pCho1lB2QKw6l9X83AmtXuo34My6dRK1zXdhmx8Hd6LsLFzDN3vx09YqigZ1fx7/Nsl6+XLmtpDs3bU1dh+f5oakOLBODKQkzBpBlDTjMcnc5uDC9hQTsEafx2ZoaH14GNUm1UedDf/mHIw62MrlJ9B+Wa1/SQv/qmk/9wUr2HmyXsSrKp6Jg13s/L7SOww7+GJVT9qdpExPhutXKXY/jaCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9/zOSCKw7zFSSB+1LnE/2WjnkVIfVsyyLiFlMm4RF1o=; b=PGsteNYxN8zz4uu6ewkPkDq0kJjv4wDlZslRgmrH59r0IxF4nEXou8r1QMurIYJtqvLetpOzU/vfW2gISRjGCguUINMpvIcfLYtkJBtidZ4SaSs5joiZuPUh7YitWDEFtpA5h2dEbqdEjF8nLzzE/+aRfrn76109aeBKrrQaHIzZ8xqjw4AkumUiuvHt0ZJslqNJoywW4DoAd/+YkBvrutgS6oLJliZQ1aw9ryuWtGjWFICH9CatoRlvejLaY6MQMOKKIrq92LulJwytcv43CX2ETzRqOssVfwhEeFO/Ukvs1LyHDCMwObvRCJJQWFRtPLkRFo4KtfAdCViwgGNAEw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 07/11] vfio/migration: Implement VFIO migration protocol v2 Date: Mon, 30 May 2022 20:07:35 +0300 Message-ID: <20220530170739.19072-8-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7df47cd1-9995-4f66-4802-08da425ef884 X-MS-TrafficTypeDiagnostic: DM6PR12MB4235:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Yfp/bYY8hCIDni58U9PRYsQWhzQtMU0CZP6ThAYb6+x6uOjQ+cXZcl8uzFfTUaEpUSuDpbSA1v3VYAiZy9fJxkibYPCBELo7XaLB+RAcmRd6YUhbKHmgbL3+vtQBMj+LtLmAohQ2ttgQU5YC+ekyZsqCZ3ueky+L8JHbpX3Q9+0h+rI4X6wIMgPoPZT2orozACB0iUYPM49je9tCgsoMizl13yA8Z5asC/xiFUY3Z4M/gORnpzig0o37HLB01JC5Y/qzf8Q6vCmX9OzmmalxuG7JMIUsxerB1N+1LNk7bGmZQV67xHPt0wXlGfynqhEr46/WSaF35fEmbWK0rxQo8oQRXLjiF8UsVo2F6jw0FEDlzkEt7yRyJx5Z77SE0i7ajBpdW3eAVZ0CQymTB6OJ8zvMkq3FgIQfS5XuYH+kxhUfbw5MR+6qBGDxp675DoraAB8JpTZYf6ZAORORl0NAMFWd7xx/2jWiUZWTNVLiCphznpJ2RAriA0lXe2yCgrsYsAUvM8B2+dt4GbIuKQL6jHQjZaTOOck44YjWUl3nC0P8c07XxZRkzC7AewqJ6LyDMF8CMJhlsP2PDDBru4SPv/+vHCT67KYcNfbPLsvfMqxlDP9G9aLB8E5DPyKgq2k31EWGiWw3S2tLO+vQw8KYiSB7PNH+YixgbUho7h4th9D/t5nCnXuQNGnvX6DIpwI9uX0Q0VkXwUa9uV7h6sltyfmI0DkikpWt6JTl/WPI8frPg7vvQarYby3UztWQWZfn3KdCiJqpLGAiLJxS9IS6s0p25BflKWsPj6aywU3lt2U= X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(70206006)(82310400005)(4326008)(8676002)(70586007)(86362001)(5660300002)(2616005)(1076003)(107886003)(30864003)(36860700001)(186003)(6666004)(40460700003)(508600001)(966005)(26005)(54906003)(8936002)(7696005)(110136005)(2906002)(316002)(81166007)(47076005)(83380400001)(426003)(336012)(36756003)(356005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:08:08.8718 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7df47cd1-9995-4f66-4802-08da425ef884 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4235 Received-SPF: softfail client-ip=2a01:111:f400:7ea9::614; envelope-from=avihaih@nvidia.com; helo=NAM02-SN1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653934888945100003 Content-Type: text/plain; charset="utf-8" Add implementation of VFIO migration protocol v2. The two protocols, v1 and v2, will co-exist and in next patch v1 protocol will be removed. There are several main differences between v1 and v2 protocols: - VFIO device state is now represented as a finite state machine instead of a bitmap. - Migration interface with kernel is now done using VFIO_DEVICE_FEATURE ioctl and normal read() and write() instead of the migration region. - VFIO migration protocol v2 currently doesn't support the pre-copy phase of migration. Detailed information about VFIO migration protocol v2 and difference compared to v1 can be found here [1]. [1] https://lore.kernel.org/all/20220224142024.147653-10-yishaih@nvidia.com/ Signed-off-by: Avihai Horon --- hw/vfio/common.c | 19 +- hw/vfio/migration.c | 365 ++++++++++++++++++++++++++++++---- hw/vfio/trace-events | 2 + include/hw/vfio/vfio-common.h | 5 + 4 files changed, 354 insertions(+), 37 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index a3dd8221ed..5541133ec9 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -355,10 +355,18 @@ static bool vfio_devices_all_dirty_tracking(VFIOConta= iner *container) return false; } =20 - if ((vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) && + if (!migration->v2 && + (vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) && (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING= )) { return false; } + + if (migration->v2 && + (vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) && + (migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING = || + migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING_= P2P)) { + return false; + } } } return true; @@ -385,7 +393,14 @@ static bool vfio_devices_all_running_and_mig_active(VF= IOContainer *container) return false; } =20 - if (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING)= { + if (!migration->v2 && + migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING)= { + continue; + } + + if (migration->v2 && + (migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING = || + migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING_= P2P)) { continue; } else { return false; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index e40aa0ad80..de68eadb09 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -44,8 +44,83 @@ #define VFIO_MIG_FLAG_DEV_SETUP_STATE (0xffffffffef100003ULL) #define VFIO_MIG_FLAG_DEV_DATA_STATE (0xffffffffef100004ULL) =20 +#define VFIO_MIG_DATA_BUFFER_SIZE (1024 * 1024) + static int64_t bytes_transferred; =20 +static const char *mig_state_to_str(enum vfio_device_mig_state state) +{ + switch (state) { + case VFIO_DEVICE_STATE_ERROR: + return "ERROR"; + case VFIO_DEVICE_STATE_STOP: + return "STOP"; + case VFIO_DEVICE_STATE_RUNNING: + return "RUNNING"; + case VFIO_DEVICE_STATE_STOP_COPY: + return "STOP_COPY"; + case VFIO_DEVICE_STATE_RESUMING: + return "RESUMING"; + case VFIO_DEVICE_STATE_RUNNING_P2P: + return "RUNNING_P2P"; + default: + return "UNKNOWN STATE"; + } +} + +static int vfio_migration_set_state(VFIODevice *vbasedev, + enum vfio_device_mig_state new_state, + enum vfio_device_mig_state recover_sta= te) +{ + VFIOMigration *migration =3D vbasedev->migration; + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_mig_state), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (void *)buf; + struct vfio_device_feature_mig_state *mig_state =3D (void *)feature->d= ata; + int ret; + + feature->argsz =3D sizeof(buf); + feature->flags =3D + VFIO_DEVICE_FEATURE_SET | VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE; + mig_state->device_state =3D new_state; + ret =3D ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); + if (ret) { + /* Try to put the device in some good state */ + mig_state->device_state =3D recover_state; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + hw_error("%s: Device in error state, can't recover", + vbasedev->name); + } + + error_report("%s: Failed changing device state to %s", vbasedev->n= ame, + mig_state_to_str(new_state)); + migration->device_state =3D recover_state; + + return -1; + } + + if (mig_state->data_fd !=3D -1) { + if (migration->data_fd !=3D -1) { + /* + * This can happen if the device is asynchronously reset and + * terminates a data transfer. + */ + error_report("%s: data_fd out of sync", vbasedev->name); + close(mig_state->data_fd); + + return -1; + } + + migration->data_fd =3D mig_state->data_fd; + } + migration->device_state =3D new_state; + + trace_vfio_migration_set_state(vbasedev->name, new_state); + + return 0; +} + static inline int vfio_mig_access(VFIODevice *vbasedev, void *val, int cou= nt, off_t off, bool iswrite) { @@ -260,6 +335,22 @@ static int vfio_save_buffer(QEMUFile *f, VFIODevice *v= basedev, uint64_t *size) return ret; } =20 +static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, + uint64_t data_size) +{ + VFIOMigration *migration =3D vbasedev->migration; + int ret; + + ret =3D qemu_file_get_to_fd(f, migration->data_fd, data_size); + if (ret) { + return ret; + } + + trace_vfio_load_state_device_data(vbasedev->name, data_size); + + return 0; +} + static int vfio_v1_load_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t data_size) { @@ -394,6 +485,14 @@ static int vfio_load_device_config_state(QEMUFile *f, = void *opaque) return qemu_file_get_error(f); } =20 +static void vfio_migration_cleanup(VFIODevice *vbasedev) +{ + VFIOMigration *migration =3D vbasedev->migration; + + close(migration->data_fd); + migration->data_fd =3D -1; +} + static void vfio_migration_v1_cleanup(VFIODevice *vbasedev) { VFIOMigration *migration =3D vbasedev->migration; @@ -405,6 +504,18 @@ static void vfio_migration_v1_cleanup(VFIODevice *vbas= edev) =20 /* ---------------------------------------------------------------------- = */ =20 +static int vfio_save_setup(QEMUFile *f, void *opaque) +{ + VFIODevice *vbasedev =3D opaque; + + trace_vfio_save_setup(vbasedev->name); + + qemu_put_be64(f, VFIO_MIG_FLAG_DEV_SETUP_STATE); + qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); + + return qemu_file_get_error(f); +} + static int vfio_v1_save_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -448,6 +559,14 @@ static int vfio_v1_save_setup(QEMUFile *f, void *opaqu= e) return 0; } =20 +static void vfio_save_cleanup(void *opaque) +{ + VFIODevice *vbasedev =3D opaque; + + vfio_migration_cleanup(vbasedev); + trace_vfio_save_cleanup(vbasedev->name); +} + static void vfio_v1_save_cleanup(void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -524,6 +643,69 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) return 0; } =20 +/* Returns 1 if end-of-stream is reached, 0 if more data and -1 if error */ +static int vfio_save_block(QEMUFile *f, VFIOMigration *migration) +{ + ssize_t data_size; + + data_size =3D read(migration->data_fd, migration->data_buffer, + migration->data_buffer_size); + if (data_size < 0) { + return -1; + } + if (data_size =3D=3D 0) { + return 1; + } + + qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); + qemu_put_be64(f, data_size); + qemu_put_buffer_async(f, migration->data_buffer, data_size, false); + qemu_fflush(f); + bytes_transferred +=3D data_size; + + trace_vfio_save_block(migration->vbasedev->name, data_size); + + return qemu_file_get_error(f); +} + +static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) +{ + VFIODevice *vbasedev =3D opaque; + enum vfio_device_mig_state recover_state; + int ret; + + /* We reach here with device state STOP or STOP_COPY only */ + recover_state =3D VFIO_DEVICE_STATE_STOP; + ret =3D vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP_COPY, + recover_state); + if (ret) { + return ret; + } + + do { + ret =3D vfio_save_block(f, vbasedev->migration); + if (ret < 0) { + return ret; + } + } while (!ret); + + qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); + ret =3D qemu_file_get_error(f); + if (ret) { + return ret; + } + + ret =3D vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP, + recover_state); + if (ret) { + return ret; + } + + trace_vfio_save_complete_precopy(vbasedev->name); + + return 0; +} + static int vfio_v1_save_complete_precopy(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -593,6 +775,14 @@ static void vfio_save_state(QEMUFile *f, void *opaque) } } =20 +static int vfio_load_setup(QEMUFile *f, void *opaque) +{ + VFIODevice *vbasedev =3D opaque; + + return vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RESUMING, + vbasedev->migration->device_state); +} + static int vfio_v1_load_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -620,6 +810,15 @@ static int vfio_v1_load_setup(QEMUFile *f, void *opaqu= e) return ret; } =20 +static int vfio_load_cleanup(void *opaque) +{ + VFIODevice *vbasedev =3D opaque; + + vfio_migration_cleanup(vbasedev); + trace_vfio_load_cleanup(vbasedev->name); + return 0; +} + static int vfio_v1_load_cleanup(void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -662,7 +861,11 @@ static int vfio_load_state(QEMUFile *f, void *opaque, = int version_id) uint64_t data_size =3D qemu_get_be64(f); =20 if (data_size) { - ret =3D vfio_v1_load_buffer(f, vbasedev, data_size); + if (vbasedev->migration->v2) { + ret =3D vfio_load_buffer(f, vbasedev, data_size); + } else { + ret =3D vfio_v1_load_buffer(f, vbasedev, data_size); + } if (ret < 0) { return ret; } @@ -683,6 +886,16 @@ static int vfio_load_state(QEMUFile *f, void *opaque, = int version_id) return ret; } =20 +static SaveVMHandlers savevm_vfio_handlers =3D { + .save_setup =3D vfio_save_setup, + .save_cleanup =3D vfio_save_cleanup, + .save_live_complete_precopy =3D vfio_save_complete_precopy, + .save_state =3D vfio_save_state, + .load_setup =3D vfio_load_setup, + .load_cleanup =3D vfio_load_cleanup, + .load_state =3D vfio_load_state, +}; + static SaveVMHandlers savevm_vfio_v1_handlers =3D { .save_setup =3D vfio_v1_save_setup, .save_cleanup =3D vfio_v1_save_cleanup, @@ -697,6 +910,34 @@ static SaveVMHandlers savevm_vfio_v1_handlers =3D { =20 /* ---------------------------------------------------------------------- = */ =20 +static void vfio_vmstate_change(void *opaque, bool running, RunState state) +{ + VFIODevice *vbasedev =3D opaque; + enum vfio_device_mig_state new_state; + int ret; + + if (running) { + new_state =3D VFIO_DEVICE_STATE_RUNNING; + } else { + new_state =3D VFIO_DEVICE_STATE_STOP; + } + + ret =3D vfio_migration_set_state(vbasedev, new_state, + VFIO_DEVICE_STATE_ERROR); + if (ret) { + /* + * Migration should be aborted in this case, but vm_state_notify() + * currently does not support reporting failures. + */ + if (migrate_get_current()->to_dst_file) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + } + } + + trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), + new_state); +} + static void vfio_v1_vmstate_change(void *opaque, bool running, RunState st= ate) { VFIODevice *vbasedev =3D opaque; @@ -770,12 +1011,17 @@ static void vfio_migration_state_notifier(Notifier *= notifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred =3D 0; - ret =3D vfio_migration_v1_set_state(vbasedev, - ~(VFIO_DEVICE_STATE_V1_SAVING | - VFIO_DEVICE_STATE_V1_RESUMING), - VFIO_DEVICE_STATE_V1_RUNNING); - if (ret) { - error_report("%s: Failed to set state RUNNING", vbasedev->name= ); + if (migration->v2) { + vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RUNNING, + VFIO_DEVICE_STATE_ERROR); + } else { + ret =3D vfio_migration_v1_set_state(vbasedev, + ~(VFIO_DEVICE_STATE_V1_SAVIN= G | + VFIO_DEVICE_STATE_V1_RESUM= ING), + VFIO_DEVICE_STATE_V1_RUNNING= ); + if (ret) { + error_report("%s: Failed to set state RUNNING", vbasedev->= name); + } } } } @@ -784,12 +1030,35 @@ static void vfio_migration_exit(VFIODevice *vbasedev) { VFIOMigration *migration =3D vbasedev->migration; =20 - vfio_region_exit(&migration->region); - vfio_region_finalize(&migration->region); + if (migration->v2) { + g_free(migration->data_buffer); + } else { + vfio_region_exit(&migration->region); + vfio_region_finalize(&migration->region); + } g_free(vbasedev->migration); vbasedev->migration =3D NULL; } =20 +static int vfio_migration_query_flags(VFIODevice *vbasedev, uint64_t *mig_= flags) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_migrat= ion), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (void *)buf; + struct vfio_device_feature_migration *mig =3D (void *)feature->data; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_MIGRA= TION; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + return -EOPNOTSUPP; + } + + *mig_flags =3D mig->flags; + + return 0; +} + static int vfio_migration_init(VFIODevice *vbasedev) { int ret; @@ -798,6 +1067,7 @@ static int vfio_migration_init(VFIODevice *vbasedev) char id[256] =3D ""; g_autofree char *path =3D NULL, *oid =3D NULL; struct vfio_region_info *info =3D NULL; + uint64_t mig_flags; =20 if (!vbasedev->ops->vfio_get_object) { return -EINVAL; @@ -808,32 +1078,48 @@ static int vfio_migration_init(VFIODevice *vbasedev) return -EINVAL; } =20 - ret =3D vfio_get_dev_region_info(vbasedev, - VFIO_REGION_TYPE_MIGRATION_DEPRECATED, - VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATE= D, - &info); - if (ret) { - return ret; - } + ret =3D vfio_migration_query_flags(vbasedev, &mig_flags); + if (!ret) { + /* Migration v2 */ + /* Basic migration functionality must be supported */ + if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { + return -EOPNOTSUPP; + } + vbasedev->migration =3D g_new0(VFIOMigration, 1); + vbasedev->migration->data_buffer_size =3D VFIO_MIG_DATA_BUFFER_SIZ= E; + vbasedev->migration->data_buffer =3D + g_malloc0(vbasedev->migration->data_buffer_size); + vbasedev->migration->data_fd =3D -1; + vbasedev->migration->v2 =3D true; + } else { + /* Migration v1 */ + ret =3D vfio_get_dev_region_info(vbasedev, + VFIO_REGION_TYPE_MIGRATION_DEPRECAT= ED, + VFIO_REGION_SUBTYPE_MIGRATION_DEPRE= CATED, + &info); + if (ret) { + return ret; + } =20 - vbasedev->migration =3D g_new0(VFIOMigration, 1); + vbasedev->migration =3D g_new0(VFIOMigration, 1); =20 - ret =3D vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, - info->index, "migration"); - if (ret) { - error_report("%s: Failed to setup VFIO migration region %d: %s", - vbasedev->name, info->index, strerror(-ret)); - goto err; - } + ret =3D vfio_region_setup(obj, vbasedev, &vbasedev->migration->reg= ion, + info->index, "migration"); + if (ret) { + error_report("%s: Failed to setup VFIO migration region %d: %s= ", + vbasedev->name, info->index, strerror(-ret)); + goto err; + } =20 - if (!vbasedev->migration->region.size) { - error_report("%s: Invalid zero-sized VFIO migration region %d", - vbasedev->name, info->index); - ret =3D -EINVAL; - goto err; - } + if (!vbasedev->migration->region.size) { + error_report("%s: Invalid zero-sized VFIO migration region %d", + vbasedev->name, info->index); + ret =3D -EINVAL; + goto err; + } =20 - g_free(info); + g_free(info); + } =20 migration =3D vbasedev->migration; migration->vbasedev =3D vbasedev; @@ -846,11 +1132,20 @@ static int vfio_migration_init(VFIODevice *vbasedev) } strpadcpy(id, sizeof(id), path, '\0'); =20 - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, - &savevm_vfio_v1_handlers, vbasedev); + if (migration->v2) { + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, + &savevm_vfio_handlers, vbasedev); + + migration->vm_state =3D qdev_add_vm_change_state_handler( + vbasedev->dev, vfio_vmstate_change, vbasedev); + } else { + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, + &savevm_vfio_v1_handlers, vbasedev); + + migration->vm_state =3D qdev_add_vm_change_state_handler( + vbasedev->dev, vfio_v1_vmstate_change, vbasedev); + } =20 - migration->vm_state =3D qdev_add_vm_change_state_handler( - vbasedev->dev, vfio_v1_vmstate_change, vbasedev); migration->migration_state.notify =3D vfio_migration_state_notifier; add_migration_state_change_notifier(&migration->migration_state); return 0; diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index ac8b04f52a..6e8c5958b9 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -163,6 +163,8 @@ vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 vfio_v1_load_state_device_data(const char *name, uint64_t data_offset, uin= t64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 +vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) s= ize 0x%"PRIx64 vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitma= p_size, uint64_t start) "container fd=3D%d, iova=3D0x%"PRIx64" size=3D 0x%"= PRIx64" bitmap_size=3D0x%"PRIx64" start=3D0x%"PRIx64 vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu= dirty @ 0x%"PRIx64" - 0x%"PRIx64 +vfio_save_block(const char *name, int data_size) " (%s) data_size %d" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index bbaf72ba00..2ec3346fea 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -66,6 +66,11 @@ typedef struct VFIOMigration { int vm_running; Notifier migration_state; uint64_t pending_bytes; + enum vfio_device_mig_state device_state; + int data_fd; + void *data_buffer; + size_t data_buffer_size; + bool v2; } VFIOMigration; =20 typedef struct VFIOAddressSpace { --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653933370; cv=pass; d=zohomail.com; s=zohoarc; b=YBM7vGTGIo4DwILCncrLB1HXe8+paumjEwKyWtzKzfCOQSiMbzqGfff89S21SC5e11r3+eAQsc7XIUaSj0a/myFrlG2/zZQbZlWN4Vu7C6eNs+YY0GXXORb5XOUC8TEgytkjBuA8yCmLolfN7QZeHgsyy85EPpKD3UG1FJsYRPI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653933370; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZJuF1217+IQ4i7VgbZUxRxtNbyHUTT+Gl6kqze0jgY8=; b=MP0o3XCQxTU/3TPJ8ZexgkkXRYS/zUXg5HLpHP36oTJv/6T1dFRg9UQGXTquuwXIZy8DUhwDGi0lPtAaZ5NZ0D5TC9780R5NBWGcYC8vlrRYyJHC0qDwrR1ik3n9N4oxnT1RauytSKaRlmes0jj7Qd6lz5OboBTr7k26J4EGfzc= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653933370096386.2937120732131; Mon, 30 May 2022 10:56:10 -0700 (PDT) Received: from localhost ([::1]:49404 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjcd-00070r-V3 for importer@patchew.org; Mon, 30 May 2022 13:56:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54334) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisN-0004b3-JG for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:19 -0400 Received: from mail-dm6nam10on20604.outbound.protection.outlook.com ([2a01:111:f400:7e88::604]:32352 helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisK-0008N6-6o for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:19 -0400 Received: from CO1PR15CA0101.namprd15.prod.outlook.com (2603:10b6:101:21::21) by DM6PR12MB4337.namprd12.prod.outlook.com (2603:10b6:5:2a9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:08:12 +0000 Received: from CO1NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:101:21:cafe::bc) by CO1PR15CA0101.outlook.office365.com (2603:10b6:101:21::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.19 via Frontend Transport; Mon, 30 May 2022 17:08:12 +0000 Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT035.mail.protection.outlook.com (10.13.175.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:12 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:08:11 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:08:11 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:08:08 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T6jR/dtyQEL2NHiIdyMgk9IFWtEbAh/YsYngZuS8epNKYbYkfUuxElt7nhciRabcJCTOknLsae8RHidQhhfetntEpQmI1Yy/hnel+PMtLLLrqk0to+5SI8KmRn266pAJI3QUvQU/qpajVmZj0CgW/J3yB/mfXBEJYZbWIJAUyT27n4BnsXX2owPA/ZXAGwFZvrnS1ux6yUTPPXLIGUXX/XkkThi/obiYNguumIA2AccHLI6kyP1BUKcZaZoZ+0WPkdocO94hBhwrp70y/ky7VpD0nU12lCSEoeYOWh3qppK5VsOssAtZ7/k+0DtsH1VAwjcqp73p1d4GOrlaAqrHOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZJuF1217+IQ4i7VgbZUxRxtNbyHUTT+Gl6kqze0jgY8=; b=UYy2z+mLZRADTk/yQh7H2yNOQNRAR9LUbTGvx0jvo0JrOVLkJ3ubAXmo72v5+0CrzF3SEqGPsPYfl8UBIaeV0fHyUmK0HDQTu8LyCN3oDf8fbmokbCG7zYLUolgWYW/ipvgYsLV5Fekr82AM7ftu4E2Dxgv7LI0TRCvOjOjEZ7XZ8HJXV8u0+JWWwkifnwICdLkQl5+iBNHH8MzKjw++FX8LbHm9NaTUBsuWrghN8GPw0rGpS7/CsiRnuiJiTz0HVYmCeyFiZXz988LhGr41WN5a590vWbRc3yVxjFXIbQqVLf8YYiZSf1OKsBUBoTVmykZeQA7PCCDlN6gnMezKFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZJuF1217+IQ4i7VgbZUxRxtNbyHUTT+Gl6kqze0jgY8=; b=FpS1cZcHpYGkyRRJlpG9ww3yU9whsW73IJtyvZrErmkxVn9TWxX9kMGaQ48rCU7X3VU4sSyQQhdFrZ3O6fEWRdISMpe6aCwXdAADKBjemhrsM/gbt6SBf7yaUqsz9yY71cow3KCxXKTBRZAp6IopaFoDI2CFFAAmBt7v+FAIibHBf+MqODl6/lXBS0rLwRzS1kmDSZOeT9Ho7GdKAC1RSVdLwQI2eg1h35OZU4Li30i/qCmqOqLJZ+P3WW69M0eJXWbdLY32EcU6Cp9qIwRqSao77kflgBC4WqNnlPNoAkvsdEQZLaIOoVwF6bOmX1pIn610JTjrCHYIzgZb/Cy7zw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 08/11] vfio/migration: Remove VFIO migration protocol v1 Date: Mon, 30 May 2022 20:07:36 +0300 Message-ID: <20220530170739.19072-9-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ae243338-c926-4820-ebd3-08da425efa97 X-MS-TrafficTypeDiagnostic: DM6PR12MB4337:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EOB423/lKCJqqNH6HeB+NBrJZAraqJ86FbpSxbERmPOh8JXFgDwhaxw4qRp0PiFCJlubHZlA+0iOq8zIIt8nNC6atVFEf4yscgxMTig6DSamYjGuk0CGjfqjhr0dNgrqjEvG0Sz78MTis3jCSEEvu+Mj4j4bvSCguPKvqWXHIKYvRivwxT0ZxM0im1KyeJSW3BEi5AZUHXlrhG9GzalaruujzFyp1FXegB6xf4K/LJXrZjWQZD0xWSgJMlOdVK+9iqazq6q0vFhj/l0AdYC+Co2p3OQj5B5JWfgHztd6fAC7yE7ICSFqnpAsfZjNa40I/fEp1xJbzrWwMqq2YfWGnNh/x0JOaKUrdbB/NP5lZsmsP8fI5feKPgDSeOT2KtfJZY02uGiiQmq9ubj41s/9hTC4xAETfgDgeWfskrq5d995y4JUek99xMkmzF1OaRvBHaq5bPGk290QLpptpnS5+1dPra1JUeX4DE1SyLI7nGsmBrgHJKLEL5+A79Y/fx89x65YeMvPi4xTph5rRGIZWu+NG6yr4yEJgpHZllsKIhDmFr24I+woxRkUB3qG/mwiB1MQqhwep37r5DydOC5bkK5zvhuX09ECjW1WQ3yAkJHF2KCk/nGuClcMNFmuK+ypXvaLVoCkipu4YUgTV+uT3glqf6DKsAVgiuNBL02hrlP7aKgNcWZ7yrj2UfEyLQhWLzBgSiz1XyIvUcYBEfl5ww== X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(1076003)(356005)(47076005)(81166007)(7696005)(26005)(316002)(6666004)(70206006)(70586007)(54906003)(8676002)(30864003)(5660300002)(8936002)(40460700003)(82310400005)(336012)(186003)(426003)(36756003)(110136005)(508600001)(36860700001)(86362001)(107886003)(2616005)(2906002)(83380400001)(4326008)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:08:12.3559 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ae243338-c926-4820-ebd3-08da425efa97 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4337 Received-SPF: softfail client-ip=2a01:111:f400:7e88::604; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653933371174100001 Content-Type: text/plain; charset="utf-8" Now that v2 protocol implementation has been added, remove the deprecated v1 implementation. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 19 +- hw/vfio/migration.c | 698 +--------------------------------- hw/vfio/trace-events | 5 - include/hw/vfio/vfio-common.h | 5 - 4 files changed, 24 insertions(+), 703 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 5541133ec9..00c6cb0ffe 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -355,14 +355,7 @@ static bool vfio_devices_all_dirty_tracking(VFIOContai= ner *container) return false; } =20 - if (!migration->v2 && - (vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) && - (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING= )) { - return false; - } - - if (migration->v2 && - (vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) && + if ((vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) && (migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING = || migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING_= P2P)) { return false; @@ -393,14 +386,8 @@ static bool vfio_devices_all_running_and_mig_active(VF= IOContainer *container) return false; } =20 - if (!migration->v2 && - migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING)= { - continue; - } - - if (migration->v2 && - (migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING = || - migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING_= P2P)) { + if (migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING || + migration->device_state =3D=3D VFIO_DEVICE_STATE_RUNNING_P= 2P) { continue; } else { return false; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index de68eadb09..852759e6ca 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -121,220 +121,6 @@ static int vfio_migration_set_state(VFIODevice *vbase= dev, return 0; } =20 -static inline int vfio_mig_access(VFIODevice *vbasedev, void *val, int cou= nt, - off_t off, bool iswrite) -{ - int ret; - - ret =3D iswrite ? pwrite(vbasedev->fd, val, count, off) : - pread(vbasedev->fd, val, count, off); - if (ret < count) { - error_report("vfio_mig_%s %d byte %s: failed at offset 0x%" - HWADDR_PRIx", err: %s", iswrite ? "write" : "read", c= ount, - vbasedev->name, off, strerror(errno)); - return (ret < 0) ? ret : -EINVAL; - } - return 0; -} - -static int vfio_mig_rw(VFIODevice *vbasedev, __u8 *buf, size_t count, - off_t off, bool iswrite) -{ - int ret, done =3D 0; - __u8 *tbuf =3D buf; - - while (count) { - int bytes =3D 0; - - if (count >=3D 8 && !(off % 8)) { - bytes =3D 8; - } else if (count >=3D 4 && !(off % 4)) { - bytes =3D 4; - } else if (count >=3D 2 && !(off % 2)) { - bytes =3D 2; - } else { - bytes =3D 1; - } - - ret =3D vfio_mig_access(vbasedev, tbuf, bytes, off, iswrite); - if (ret) { - return ret; - } - - count -=3D bytes; - done +=3D bytes; - off +=3D bytes; - tbuf +=3D bytes; - } - return done; -} - -#define vfio_mig_read(f, v, c, o) vfio_mig_rw(f, (__u8 *)v, c, o, fa= lse) -#define vfio_mig_write(f, v, c, o) vfio_mig_rw(f, (__u8 *)v, c, o, tr= ue) - -#define VFIO_MIG_STRUCT_OFFSET(f) \ - offsetof(struct vfio_device_migration_inf= o, f) -/* - * Change the device_state register for device @vbasedev. Bits set in @mask - * are preserved, bits set in @value are set, and bits not set in either @= mask - * or @value are cleared in device_state. If the register cannot be access= ed, - * the resulting state would be invalid, or the device enters an error sta= te, - * an error is returned. - */ - -static int vfio_migration_v1_set_state(VFIODevice *vbasedev, uint32_t mask, - uint32_t value) -{ - VFIOMigration *migration =3D vbasedev->migration; - VFIORegion *region =3D &migration->region; - off_t dev_state_off =3D region->fd_offset + - VFIO_MIG_STRUCT_OFFSET(device_state); - uint32_t device_state; - int ret; - - ret =3D vfio_mig_read(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - if (ret < 0) { - return ret; - } - - device_state =3D (device_state & mask) | value; - - if (!VFIO_DEVICE_STATE_VALID(device_state)) { - return -EINVAL; - } - - ret =3D vfio_mig_write(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - if (ret < 0) { - int rret; - - rret =3D vfio_mig_read(vbasedev, &device_state, sizeof(device_stat= e), - dev_state_off); - - if ((rret < 0) || (VFIO_DEVICE_STATE_IS_ERROR(device_state))) { - hw_error("%s: Device in error state 0x%x", vbasedev->name, - device_state); - return rret ? rret : -EIO; - } - return ret; - } - - migration->device_state_v1 =3D device_state; - trace_vfio_migration_set_state(vbasedev->name, device_state); - return 0; -} - -static void *get_data_section_size(VFIORegion *region, uint64_t data_offse= t, - uint64_t data_size, uint64_t *size) -{ - void *ptr =3D NULL; - uint64_t limit =3D 0; - int i; - - if (!region->mmaps) { - if (size) { - *size =3D MIN(data_size, region->size - data_offset); - } - return ptr; - } - - for (i =3D 0; i < region->nr_mmaps; i++) { - VFIOMmap *map =3D region->mmaps + i; - - if ((data_offset >=3D map->offset) && - (data_offset < map->offset + map->size)) { - - /* check if data_offset is within sparse mmap areas */ - ptr =3D map->mmap + data_offset - map->offset; - if (size) { - *size =3D MIN(data_size, map->offset + map->size - data_of= fset); - } - break; - } else if ((data_offset < map->offset) && - (!limit || limit > map->offset)) { - /* - * data_offset is not within sparse mmap areas, find size of - * non-mapped area. Check through all list since region->mmaps= list - * is not sorted. - */ - limit =3D map->offset; - } - } - - if (!ptr && size) { - *size =3D limit ? MIN(data_size, limit - data_offset) : data_size; - } - return ptr; -} - -static int vfio_save_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t *s= ize) -{ - VFIOMigration *migration =3D vbasedev->migration; - VFIORegion *region =3D &migration->region; - uint64_t data_offset =3D 0, data_size =3D 0, sz; - int ret; - - ret =3D vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offs= et)); - if (ret < 0) { - return ret; - } - - ret =3D vfio_mig_read(vbasedev, &data_size, sizeof(data_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_si= ze)); - if (ret < 0) { - return ret; - } - - trace_vfio_save_buffer(vbasedev->name, data_offset, data_size, - migration->pending_bytes); - - qemu_put_be64(f, data_size); - sz =3D data_size; - - while (sz) { - void *buf; - uint64_t sec_size; - bool buf_allocated =3D false; - - buf =3D get_data_section_size(region, data_offset, sz, &sec_size); - - if (!buf) { - buf =3D g_try_malloc(sec_size); - if (!buf) { - error_report("%s: Error allocating buffer ", __func__); - return -ENOMEM; - } - buf_allocated =3D true; - - ret =3D vfio_mig_read(vbasedev, buf, sec_size, - region->fd_offset + data_offset); - if (ret < 0) { - g_free(buf); - return ret; - } - } - - qemu_put_buffer(f, buf, sec_size); - - if (buf_allocated) { - g_free(buf); - } - sz -=3D sec_size; - data_offset +=3D sec_size; - } - - ret =3D qemu_file_get_error(f); - - if (!ret && size) { - *size =3D data_size; - } - - bytes_transferred +=3D data_size; - return ret; -} - static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t data_size) { @@ -351,96 +137,6 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *v= basedev, return 0; } =20 -static int vfio_v1_load_buffer(QEMUFile *f, VFIODevice *vbasedev, - uint64_t data_size) -{ - VFIORegion *region =3D &vbasedev->migration->region; - uint64_t data_offset =3D 0, size, report_size; - int ret; - - do { - ret =3D vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offs= et)); - if (ret < 0) { - return ret; - } - - if (data_offset + data_size > region->size) { - /* - * If data_size is greater than the data section of migration = region - * then iterate the write buffer operation. This case can occu= r if - * size of migration region at destination is smaller than siz= e of - * migration region at source. - */ - report_size =3D size =3D region->size - data_offset; - data_size -=3D size; - } else { - report_size =3D size =3D data_size; - data_size =3D 0; - } - - trace_vfio_v1_load_state_device_data(vbasedev->name, data_offset, = size); - - while (size) { - void *buf; - uint64_t sec_size; - bool buf_alloc =3D false; - - buf =3D get_data_section_size(region, data_offset, size, &sec_= size); - - if (!buf) { - buf =3D g_try_malloc(sec_size); - if (!buf) { - error_report("%s: Error allocating buffer ", __func__); - return -ENOMEM; - } - buf_alloc =3D true; - } - - qemu_get_buffer(f, buf, sec_size); - - if (buf_alloc) { - ret =3D vfio_mig_write(vbasedev, buf, sec_size, - region->fd_offset + data_offset); - g_free(buf); - - if (ret < 0) { - return ret; - } - } - size -=3D sec_size; - data_offset +=3D sec_size; - } - - ret =3D vfio_mig_write(vbasedev, &report_size, sizeof(report_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_si= ze)); - if (ret < 0) { - return ret; - } - } while (data_size); - - return 0; -} - -static int vfio_update_pending(VFIODevice *vbasedev) -{ - VFIOMigration *migration =3D vbasedev->migration; - VFIORegion *region =3D &migration->region; - uint64_t pending_bytes =3D 0; - int ret; - - ret =3D vfio_mig_read(vbasedev, &pending_bytes, sizeof(pending_bytes), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(pending_byt= es)); - if (ret < 0) { - migration->pending_bytes =3D 0; - return ret; - } - - migration->pending_bytes =3D pending_bytes; - trace_vfio_update_pending(vbasedev->name, pending_bytes); - return 0; -} - static int vfio_save_device_config_state(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -493,15 +189,6 @@ static void vfio_migration_cleanup(VFIODevice *vbasede= v) migration->data_fd =3D -1; } =20 -static void vfio_migration_v1_cleanup(VFIODevice *vbasedev) -{ - VFIOMigration *migration =3D vbasedev->migration; - - if (migration->region.mmaps) { - vfio_region_unmap(&migration->region); - } -} - /* ---------------------------------------------------------------------- = */ =20 static int vfio_save_setup(QEMUFile *f, void *opaque) @@ -516,49 +203,6 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) return qemu_file_get_error(f); } =20 -static int vfio_v1_save_setup(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - VFIOMigration *migration =3D vbasedev->migration; - int ret; - - trace_vfio_save_setup(vbasedev->name); - - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_SETUP_STATE); - - if (migration->region.mmaps) { - /* - * Calling vfio_region_mmap() from migration thread. Memory API ca= lled - * from this function require locking the iothread when called from - * outside the main loop thread. - */ - qemu_mutex_lock_iothread(); - ret =3D vfio_region_mmap(&migration->region); - qemu_mutex_unlock_iothread(); - if (ret) { - error_report("%s: Failed to mmap VFIO migration region: %s", - vbasedev->name, strerror(-ret)); - error_report("%s: Falling back to slow path", vbasedev->name); - } - } - - ret =3D vfio_migration_v1_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_SAVING); - if (ret) { - error_report("%s: Failed to set state SAVING", vbasedev->name); - return ret; - } - - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - - ret =3D qemu_file_get_error(f); - if (ret) { - return ret; - } - - return 0; -} - static void vfio_save_cleanup(void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -567,82 +211,6 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } =20 -static void vfio_v1_save_cleanup(void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - - vfio_migration_v1_cleanup(vbasedev); - trace_vfio_save_cleanup(vbasedev->name); -} - -static void vfio_save_pending(QEMUFile *f, void *opaque, - uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) -{ - VFIODevice *vbasedev =3D opaque; - VFIOMigration *migration =3D vbasedev->migration; - int ret; - - ret =3D vfio_update_pending(vbasedev); - if (ret) { - return; - } - - *res_precopy_only +=3D migration->pending_bytes; - - trace_vfio_save_pending(vbasedev->name, *res_precopy_only, - *res_postcopy_only, *res_compatible); -} - -static int vfio_save_iterate(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - VFIOMigration *migration =3D vbasedev->migration; - uint64_t data_size; - int ret; - - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); - - if (migration->pending_bytes =3D=3D 0) { - ret =3D vfio_update_pending(vbasedev); - if (ret) { - return ret; - } - - if (migration->pending_bytes =3D=3D 0) { - qemu_put_be64(f, 0); - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - /* indicates data finished, goto complete phase */ - return 1; - } - } - - ret =3D vfio_save_buffer(f, vbasedev, &data_size); - if (ret) { - error_report("%s: vfio_save_buffer failed %s", vbasedev->name, - strerror(errno)); - return ret; - } - - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - - ret =3D qemu_file_get_error(f); - if (ret) { - return ret; - } - - /* - * Reset pending_bytes as .save_live_pending is not called during save= vm or - * snapshot case, in such case vfio_update_pending() at the start of t= his - * function updates pending_bytes. - */ - migration->pending_bytes =3D 0; - trace_vfio_save_iterate(vbasedev->name, data_size); - return 0; -} - /* Returns 1 if end-of-stream is reached, 0 if more data and -1 if error */ static int vfio_save_block(QEMUFile *f, VFIOMigration *migration) { @@ -706,62 +274,6 @@ static int vfio_save_complete_precopy(QEMUFile *f, voi= d *opaque) return 0; } =20 -static int vfio_v1_save_complete_precopy(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - VFIOMigration *migration =3D vbasedev->migration; - uint64_t data_size; - int ret; - - ret =3D vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RU= NNING, - VFIO_DEVICE_STATE_V1_SAVING); - if (ret) { - error_report("%s: Failed to set state STOP and SAVING", - vbasedev->name); - return ret; - } - - ret =3D vfio_update_pending(vbasedev); - if (ret) { - return ret; - } - - while (migration->pending_bytes > 0) { - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); - ret =3D vfio_save_buffer(f, vbasedev, &data_size); - if (ret < 0) { - error_report("%s: Failed to save buffer", vbasedev->name); - return ret; - } - - if (data_size =3D=3D 0) { - break; - } - - ret =3D vfio_update_pending(vbasedev); - if (ret) { - return ret; - } - } - - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - - ret =3D qemu_file_get_error(f); - if (ret) { - return ret; - } - - ret =3D vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SA= VING, - 0); - if (ret) { - error_report("%s: Failed to set state STOPPED", vbasedev->name); - return ret; - } - - trace_vfio_save_complete_precopy(vbasedev->name); - return ret; -} - static void vfio_save_state(QEMUFile *f, void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -783,33 +295,6 @@ static int vfio_load_setup(QEMUFile *f, void *opaque) vbasedev->migration->device_state); } =20 -static int vfio_v1_load_setup(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - VFIOMigration *migration =3D vbasedev->migration; - int ret =3D 0; - - if (migration->region.mmaps) { - ret =3D vfio_region_mmap(&migration->region); - if (ret) { - error_report("%s: Failed to mmap VFIO migration region %d: %s", - vbasedev->name, migration->region.nr, - strerror(-ret)); - error_report("%s: Falling back to slow path", vbasedev->name); - } - } - - ret =3D vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_RESUMING); - if (ret) { - error_report("%s: Failed to set state RESUMING", vbasedev->name); - if (migration->region.mmaps) { - vfio_region_unmap(&migration->region); - } - } - return ret; -} - static int vfio_load_cleanup(void *opaque) { VFIODevice *vbasedev =3D opaque; @@ -819,15 +304,6 @@ static int vfio_load_cleanup(void *opaque) return 0; } =20 -static int vfio_v1_load_cleanup(void *opaque) -{ - VFIODevice *vbasedev =3D opaque; - - vfio_migration_v1_cleanup(vbasedev); - trace_vfio_load_cleanup(vbasedev->name); - return 0; -} - static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) { VFIODevice *vbasedev =3D opaque; @@ -861,11 +337,7 @@ static int vfio_load_state(QEMUFile *f, void *opaque, = int version_id) uint64_t data_size =3D qemu_get_be64(f); =20 if (data_size) { - if (vbasedev->migration->v2) { - ret =3D vfio_load_buffer(f, vbasedev, data_size); - } else { - ret =3D vfio_v1_load_buffer(f, vbasedev, data_size); - } + ret =3D vfio_load_buffer(f, vbasedev, data_size); if (ret < 0) { return ret; } @@ -896,18 +368,6 @@ static SaveVMHandlers savevm_vfio_handlers =3D { .load_state =3D vfio_load_state, }; =20 -static SaveVMHandlers savevm_vfio_v1_handlers =3D { - .save_setup =3D vfio_v1_save_setup, - .save_cleanup =3D vfio_v1_save_cleanup, - .save_live_pending =3D vfio_save_pending, - .save_live_iterate =3D vfio_save_iterate, - .save_live_complete_precopy =3D vfio_v1_save_complete_precopy, - .save_state =3D vfio_save_state, - .load_setup =3D vfio_v1_load_setup, - .load_cleanup =3D vfio_v1_load_cleanup, - .load_state =3D vfio_load_state, -}; - /* ---------------------------------------------------------------------- = */ =20 static void vfio_vmstate_change(void *opaque, bool running, RunState state) @@ -938,70 +398,12 @@ static void vfio_vmstate_change(void *opaque, bool ru= nning, RunState state) new_state); } =20 -static void vfio_v1_vmstate_change(void *opaque, bool running, RunState st= ate) -{ - VFIODevice *vbasedev =3D opaque; - VFIOMigration *migration =3D vbasedev->migration; - uint32_t value, mask; - int ret; - - if (vbasedev->migration->vm_running =3D=3D running) { - return; - } - - if (running) { - /* - * Here device state can have one of _SAVING, _RESUMING or _STOP b= it. - * Transition from _SAVING to _RUNNING can happen if there is migr= ation - * failure, in that case clear _SAVING bit. - * Transition from _RESUMING to _RUNNING occurs during resuming - * phase, in that case clear _RESUMING bit. - * In both the above cases, set _RUNNING bit. - */ - mask =3D ~VFIO_DEVICE_STATE_MASK; - value =3D VFIO_DEVICE_STATE_V1_RUNNING; - } else { - /* - * Here device state could be either _RUNNING or _SAVING|_RUNNING.= Reset - * _RUNNING bit - */ - mask =3D ~VFIO_DEVICE_STATE_V1_RUNNING; - - /* - * When VM state transition to stop for savevm command, device sho= uld - * start saving data. - */ - if (state =3D=3D RUN_STATE_SAVE_VM) { - value =3D VFIO_DEVICE_STATE_V1_SAVING; - } else { - value =3D 0; - } - } - - ret =3D vfio_migration_v1_set_state(vbasedev, mask, value); - if (ret) { - /* - * Migration should be aborted in this case, but vm_state_notify() - * currently does not support reporting failures. - */ - error_report("%s: Failed to set device state 0x%x", vbasedev->name, - (migration->device_state_v1 & mask) | value); - if (migrate_get_current()->to_dst_file) { - qemu_file_set_error(migrate_get_current()->to_dst_file, ret); - } - } - vbasedev->migration->vm_running =3D running; - trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), - (migration->device_state_v1 & mask) | value); -} - static void vfio_migration_state_notifier(Notifier *notifier, void *data) { MigrationState *s =3D data; VFIOMigration *migration =3D container_of(notifier, VFIOMigration, migration_state); VFIODevice *vbasedev =3D migration->vbasedev; - int ret; =20 trace_vfio_migration_state_notifier(vbasedev->name, MigrationStatus_str(s->state)); @@ -1011,31 +413,14 @@ static void vfio_migration_state_notifier(Notifier *= notifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred =3D 0; - if (migration->v2) { - vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RUNNING, - VFIO_DEVICE_STATE_ERROR); - } else { - ret =3D vfio_migration_v1_set_state(vbasedev, - ~(VFIO_DEVICE_STATE_V1_SAVIN= G | - VFIO_DEVICE_STATE_V1_RESUM= ING), - VFIO_DEVICE_STATE_V1_RUNNING= ); - if (ret) { - error_report("%s: Failed to set state RUNNING", vbasedev->= name); - } - } + vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RUNNING, + VFIO_DEVICE_STATE_ERROR); } } =20 static void vfio_migration_exit(VFIODevice *vbasedev) { - VFIOMigration *migration =3D vbasedev->migration; - - if (migration->v2) { - g_free(migration->data_buffer); - } else { - vfio_region_exit(&migration->region); - vfio_region_finalize(&migration->region); - } + g_free(vbasedev->migration->data_buffer); g_free(vbasedev->migration); vbasedev->migration =3D NULL; } @@ -1066,7 +451,6 @@ static int vfio_migration_init(VFIODevice *vbasedev) VFIOMigration *migration; char id[256] =3D ""; g_autofree char *path =3D NULL, *oid =3D NULL; - struct vfio_region_info *info =3D NULL; uint64_t mig_flags; =20 if (!vbasedev->ops->vfio_get_object) { @@ -1079,48 +463,20 @@ static int vfio_migration_init(VFIODevice *vbasedev) } =20 ret =3D vfio_migration_query_flags(vbasedev, &mig_flags); - if (!ret) { - /* Migration v2 */ - /* Basic migration functionality must be supported */ - if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { - return -EOPNOTSUPP; - } - vbasedev->migration =3D g_new0(VFIOMigration, 1); - vbasedev->migration->data_buffer_size =3D VFIO_MIG_DATA_BUFFER_SIZ= E; - vbasedev->migration->data_buffer =3D - g_malloc0(vbasedev->migration->data_buffer_size); - vbasedev->migration->data_fd =3D -1; - vbasedev->migration->v2 =3D true; - } else { - /* Migration v1 */ - ret =3D vfio_get_dev_region_info(vbasedev, - VFIO_REGION_TYPE_MIGRATION_DEPRECAT= ED, - VFIO_REGION_SUBTYPE_MIGRATION_DEPRE= CATED, - &info); - if (ret) { - return ret; - } - - vbasedev->migration =3D g_new0(VFIOMigration, 1); - - ret =3D vfio_region_setup(obj, vbasedev, &vbasedev->migration->reg= ion, - info->index, "migration"); - if (ret) { - error_report("%s: Failed to setup VFIO migration region %d: %s= ", - vbasedev->name, info->index, strerror(-ret)); - goto err; - } - - if (!vbasedev->migration->region.size) { - error_report("%s: Invalid zero-sized VFIO migration region %d", - vbasedev->name, info->index); - ret =3D -EINVAL; - goto err; - } + if (ret) { + return ret; + } =20 - g_free(info); + /* Basic migration functionality must be supported */ + if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { + return -EOPNOTSUPP; } =20 + vbasedev->migration =3D g_new0(VFIOMigration, 1); + vbasedev->migration->data_buffer_size =3D VFIO_MIG_DATA_BUFFER_SIZE; + vbasedev->migration->data_buffer =3D + g_malloc0(vbasedev->migration->data_buffer_size); + vbasedev->migration->data_fd =3D -1; migration =3D vbasedev->migration; migration->vbasedev =3D vbasedev; =20 @@ -1132,28 +488,16 @@ static int vfio_migration_init(VFIODevice *vbasedev) } strpadcpy(id, sizeof(id), path, '\0'); =20 - if (migration->v2) { - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, - &savevm_vfio_handlers, vbasedev); - - migration->vm_state =3D qdev_add_vm_change_state_handler( - vbasedev->dev, vfio_vmstate_change, vbasedev); - } else { - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, - &savevm_vfio_v1_handlers, vbasedev); - - migration->vm_state =3D qdev_add_vm_change_state_handler( - vbasedev->dev, vfio_v1_vmstate_change, vbasedev); - } + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_hand= lers, + vbasedev); =20 + migration->vm_state =3D qdev_add_vm_change_state_handler(vbasedev->dev, + vfio_vmstate_ch= ange, + vbasedev); migration->migration_state.notify =3D vfio_migration_state_notifier; add_migration_state_change_notifier(&migration->migration_state); - return 0; =20 -err: - g_free(info); - vfio_migration_exit(vbasedev); - return ret; + return 0; } =20 /* ---------------------------------------------------------------------- = */ diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 6e8c5958b9..a24ea7d8b0 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -154,15 +154,10 @@ vfio_vmstate_change(const char *name, int running, co= nst char *reason, uint32_t vfio_migration_state_notifier(const char *name, const char *state) " (%s) = state %s" vfio_save_setup(const char *name) " (%s)" vfio_save_cleanup(const char *name) " (%s)" -vfio_save_buffer(const char *name, uint64_t data_offset, uint64_t data_siz= e, uint64_t pending) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64" pending 0x%= "PRIx64 -vfio_update_pending(const char *name, uint64_t pending) " (%s) pending 0x%= "PRIx64 vfio_save_device_config_state(const char *name) " (%s)" -vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy, u= int64_t compatible) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64" compati= ble 0x%"PRIx64 -vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_v1_load_state_device_data(const char *name, uint64_t data_offset, uin= t64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) s= ize 0x%"PRIx64 vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitma= p_size, uint64_t start) "container fd=3D%d, iova=3D0x%"PRIx64" size=3D 0x%"= PRIx64" bitmap_size=3D0x%"PRIx64" start=3D0x%"PRIx64 diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 2ec3346fea..76d470178f 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -61,16 +61,11 @@ typedef struct VFIORegion { typedef struct VFIOMigration { struct VFIODevice *vbasedev; VMChangeStateEntry *vm_state; - VFIORegion region; - uint32_t device_state_v1; - int vm_running; Notifier migration_state; - uint64_t pending_bytes; enum vfio_device_mig_state device_state; int data_fd; void *data_buffer; size_t data_buffer_size; - bool v2; } VFIOMigration; =20 typedef struct VFIOAddressSpace { --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653933547; cv=pass; d=zohomail.com; s=zohoarc; b=GR48XTdsJS7O3Smua7fe2dE7WZGcZzCbH4c+dek94KmZTqWma/zRbAu3bQU+O3B54GOGFKnjDAjY0tfs2TMuQO5bClfArY9Oj7LA1tZ1eEl/Cr2ArL4RZxKH3MCX6aK+jfEf7bgFgI+O9lVPK98wBTjFrjnE8GHngzg+0kkY+4w= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653933547; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LnU8qOv6i2xNavvVVrbqS7u4puQ5WHvKn7v45SwSF4E=; b=AK1jPngcxQxZhqQZsFKOGIURKfpXxd8LKA6bTqQy9sHC8fv4QUI7+WQ0OpKaHpj0BL/EqJdn5pNQIxBWB0evfH9lCGRI9XJirGK3j+av3B2NxlCgJYu6eP4j9fQs45hNzaCX6JtvlzFk65MJiKZCf4sRCgsnTx2AmKGrFYl4A+I= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 165393354709253.97464003419918; Mon, 30 May 2022 10:59:07 -0700 (PDT) Received: from localhost ([::1]:58210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjfV-0004bT-Uh for importer@patchew.org; Mon, 30 May 2022 13:59:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisS-0004vN-Pk for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:24 -0400 Received: from mail-mw2nam10on20603.outbound.protection.outlook.com ([2a01:111:f400:7e89::603]:31680 helo=NAM10-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisQ-0008NQ-3z for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:24 -0400 Received: from BN9PR03CA0233.namprd03.prod.outlook.com (2603:10b6:408:f8::28) by MWHPR1201MB0239.namprd12.prod.outlook.com (2603:10b6:301:50::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:08:16 +0000 Received: from BN8NAM11FT022.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f8:cafe::3e) by BN9PR03CA0233.outlook.office365.com (2603:10b6:408:f8::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:16 +0000 Received: from mail.nvidia.com (12.22.5.234) by BN8NAM11FT022.mail.protection.outlook.com (10.13.176.112) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:16 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:08:15 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:08:14 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:08:11 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LU7zy5+yN3odRNhjGvGXXNCqZ7LPVwSSDG8c3K9uO+/w/q7FPaCbXBwnQk039he8I/Rga/FoXeT1OUefwcKqII7BJdtGaVn+2ptjMpkWw9ANGWw8amJVzBLi/lUUTpwvXhS04YzY3SBKrjXVxLdhRuFl5fDnlkgNG6t0tEjYrOLvNUbyvJfr3tK3LYXpyQwRpht1EKByOEOFnwxr607k84YMamQyJoQxIxzKZjVQqbgJ6ykEtNif7V577Vt+ykOdhMWT2c6vrv56fq83RWNgt8ZaRGQfH8fCxq2djZWx2H6J/IFHJbQYNGzxWFqA/6m5o4xTYXSXLmJOK+q2GKXu+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LnU8qOv6i2xNavvVVrbqS7u4puQ5WHvKn7v45SwSF4E=; b=UvvzX9f1IvE5cXLO3A40kXlDTDaxW8u3Rhoqof63RDL5fDEKwU3xuZfLbyFVQt2nncZAt8iMSHO/G0w8BCOm+3hUTPuPA17RkUhrwejYzlU+rqurtJJZWkfau+WFKHDuAJ7/u+xqR/FQspZuAnqJy0P4wtoAmH6HbxZKa1m//41Kwrq9ct+hs4cNwVV4Y6ElL+Y64C6OptVpubY3eWnEdOzNxCkzlJ9VcKaNkYOUQI1iVwpuhEdwi+z5Prv/bjLkNr70A61ut/c/Wb8FmdReV8QfHIpm4krMsq4LVRkCGfqeCGjb4scLBlC7+6zEQEWJbV0O8J3UJTgSiGpe3kCWew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LnU8qOv6i2xNavvVVrbqS7u4puQ5WHvKn7v45SwSF4E=; b=ORMG2osqX5n3hbvYLGmycn0FijNS02WIEGCxbpPF86EO5erj6vgnJETrQxhfrhVellaS85VHULAnUvUNL+XSNBcN7xeQrDPj8F1JQJHB9KGvQEz2kOb5YuNSzjGr7iW7dEML5RJFW4HGck81hujhKpH9hGquDi6Kg7O9AzWJaae+q5uOlEvOdTdGge52Xu58XpINBLzdSoB4bMJI+suHXoPvaaoTJr1cwtg+4fXUbc+qWzYZV38+ofIxfYk5RsiwPUNAmTDE+DE0RYJZ1P2ben+i1QnKEdOOr+cM8BYeLO4osdNmyYi5RftyQLe9S6mSSvyFHrO5fhs/mdfDbGS/YQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 09/11] vfio/migration: Reset device if setting recover state fails Date: Mon, 30 May 2022 20:07:37 +0300 Message-ID: <20220530170739.19072-10-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b7ca6b28-4a36-4305-ffb0-08da425efcca X-MS-TrafficTypeDiagnostic: MWHPR1201MB0239:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vbremTIWLXZyELrVxs+ewf/bJ3Er3TeBaoBxXGgEGR9TbyJyEJ3+Ej4M8ASmXDaLo21kpdfEfdJiA38PSgEbbAq68U7g+j0rUGPmF/zqX0qWM0RSknPRSZyUnHgxLjEWzzeTuOg0GNcFGOD3oWbI1aS9pqR7IwRnHPqF7oOk1mHCGHzqVlwqMX19MSHtkS/kAl0Xlj4X+SZAVJmkxgMkEQq6shwPexevv17FcZixDNniFJZVqa+OxqmcpVhDWfH+1+WSbsIhgi1Sb5J7jGh6c7+NqSaXuG2LDDiLxQ6bAEVsGPJIIrxijyauxa8t5KLPotJoNgt7iZRyRBQwJ7b/FfwV19Bd5TtIFpXM7X4jRLROqkFmtBvoUun1XEaGrhIQMGbYdGsxbtymYRWcBDvKFBoTZalRaLyvlHhnCRojnnXRHMPiqwwAZuCZeRfiLzoyua2Mn39M8o5lhelBshmoSCCXX0XQsA4kDYHO/ZXIPB4isRI2mMEovrHVg0S8m/v3Fol/T0izVIlS3Xcjz2EjDClRLZuBqnW8l4QtChY2Mj3TDuLIY1iBl8PWnBE0CjYyDnLHDudIe6HKJSX715aN+bsmyHPCYe474BpsNO91lUgK8PPKA/tu9EwfOUrLT/HgxSVmVRGVOnEWwFKTI7NorP9qKOo8D9NgYq02AaGkisG8MwExdQhK8dAlfJvM0SnPjvoRCDHbsU/jvAyMa6NNjw== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(316002)(4326008)(54906003)(110136005)(8936002)(2906002)(8676002)(5660300002)(426003)(336012)(47076005)(7696005)(26005)(83380400001)(1076003)(6666004)(186003)(2616005)(107886003)(86362001)(36756003)(40460700003)(36860700001)(356005)(82310400005)(81166007)(70206006)(70586007)(508600001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:08:16.0149 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b7ca6b28-4a36-4305-ffb0-08da425efcca X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT022.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0239 Received-SPF: softfail client-ip=2a01:111:f400:7e89::603; envelope-from=avihaih@nvidia.com; helo=NAM10-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653933547918100001 Content-Type: text/plain; charset="utf-8" If vfio_migration_set_state() fails to set the device in the requested state it tries to put it in a recover state. If setting the device in the recover state fails as well, hw_error is triggered and the VM is aborted. To improve user experience and avoid VM data loss, reset the device with VFIO_RESET_DEVICE instead of aborting the VM. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 852759e6ca..6c34502611 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -89,8 +89,16 @@ static int vfio_migration_set_state(VFIODevice *vbasedev, /* Try to put the device in some good state */ mig_state->device_state =3D recover_state; if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { - hw_error("%s: Device in error state, can't recover", - vbasedev->name); + if (ioctl(vbasedev->fd, VFIO_DEVICE_RESET)) { + hw_error("%s: Device in error state, can't recover", + vbasedev->name); + } + + error_report( + "%s: Device was reset due to failure in changing device st= ate to recover state %s", + vbasedev->name, mig_state_to_str(recover_state)); + + return -1; } =20 error_report("%s: Failed changing device state to %s", vbasedev->n= ame, --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653933851; cv=pass; d=zohomail.com; s=zohoarc; b=YYTnG2BXQ6iJy8sw6Bdfupco8U80nyhKdrOJZ2NhVjmOaQhAcj9SzU0q/rmAIycCwnVM2OIeYb83voInYRmlcZI1YOkBAhFunp9228aJL9ZgSQ0ha2M/qkUe8c5CrNW9eHbvKWdny9jQSwuP5tdwgClDsHAh/J7QEUzCxyb07ec= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653933851; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=liYUGVmuX0kZmO+aQDDBrQ7rfJPkkF+ldMbOwennimQ=; b=ie2eYRboqRNc/iepZbuP0NY9j0S9vmvsK3YCL5ZyOeij4chuOmnksQou3tqkSOCt0z25nQUQ1Lm9fo0jnhwU8LxUQLLhxb4wq4PzSIvR8s+1o5hpPfkHM+Ux+Uh3JLCdS7x3088JnqmCpKc5jmpwcL1HYl8yEaosGpvI8FvL+T0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653933851860787.9021551505442; Mon, 30 May 2022 11:04:11 -0700 (PDT) Received: from localhost ([::1]:38546 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjkQ-0002EH-H5 for importer@patchew.org; Mon, 30 May 2022 14:04:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54402) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisS-0004wI-Vl for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:25 -0400 Received: from mail-mw2nam12on20625.outbound.protection.outlook.com ([2a01:111:f400:fe5a::625]:48705 helo=NAM12-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisQ-0008NW-QN for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:24 -0400 Received: from CO2PR04CA0178.namprd04.prod.outlook.com (2603:10b6:104:4::32) by IA1PR12MB6307.namprd12.prod.outlook.com (2603:10b6:208:3e5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:08:19 +0000 Received: from CO1NAM11FT048.eop-nam11.prod.protection.outlook.com (2603:10b6:104:4:cafe::b6) by CO2PR04CA0178.outlook.office365.com (2603:10b6:104:4::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:19 +0000 Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT048.mail.protection.outlook.com (10.13.175.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:19 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:08:18 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:08:18 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:08:15 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nZsd2uUZIckUGe1kXxzsIR38hntlhHto05qtRPAp6duYpwk+J+n2GniC1uifXpQmArhkY4WmibVV4XhEY0/+SASgCk82FPcCgYKQNCq5gH7dlmli3a8tw3OC5JfQtpOJKbno8enW1KSyDXLVudV9b8mS7THtOkSmwpF+2ykZQwMG+uNfiK5IFNb+Ag0ln7Jksrw9STeM/s8lskRsvAfvUhN0GBdnk0nCVZJsRFFHjeEFK3GL4YxS2Ttlka9a+HeRTaFBxHO3vKy5p3zXhhNFxgU0uZXotTiFPbMxL1tx0+QPy9BNvC5DArGUrFGd15bjGo8G4VrHYgswZeNOnlXKxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=liYUGVmuX0kZmO+aQDDBrQ7rfJPkkF+ldMbOwennimQ=; b=CfrBcTi2qdBQgKU85kN1dNiEk/DPgzy24gvV/4TGyvOJnRB584dEqLGphL/CjyGB+mFibELDbm3O1keAlj1EpV7Uj9Nf2tNs3E05theLXJgVvdhV2fUEcRpDS70EE1IZEzLYfpi0rgsL3JgSIXeAJbBP2b/CJ1t1i2O7kY+/v9NO87EDQRKL7WwQPgvB194hrlMHvn+LCiPH9WWQ374zc7yQMr95fXrQZRMyT5dKvaGIHKyVwuEiDZrbeTcZivJ4pdxNng4PNhjOIxvdEVw4pp+3FzvbtUBuD33krKnuPMLSwrCGyO9bW1aGnuVzIUvMvMRn/ACOm5AY3atWNnfWTg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=liYUGVmuX0kZmO+aQDDBrQ7rfJPkkF+ldMbOwennimQ=; b=DRmACLktMqyd6b58g9loM1Zh04F/lWdPv65BvJHvY/PvCrCyi+RQwSP3L9Zcmx/Aywmw0bT+afPd5AI+7yz+Ile8LZaND/lhPzsVkUh60jfxA3ltUmqthBN0A6XmQTa7VY4cteocL2JMa3XEIyyECGBRQIvqeVIX/UVjhF9J5Y513msYdbkWJqTyU5oCy9Df73tB+Uh2e+t+CYR/qWy6SVIXbXxZ3yPsF38KdOcjqZlhB9o88KXMvsLdjucQmVh+v2PZvpyxpawqPs6KYwKbSujLphg4yzwekx1dK9RSJyhcifFa3/WSm2noR2lwAzKXSX9be+7AcjNYUmwg7CS1iQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 10/11] vfio: Alphabetize migration section of VFIO trace-events file Date: Mon, 30 May 2022 20:07:38 +0300 Message-ID: <20220530170739.19072-11-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 82af9299-0d8b-438a-017a-08da425efeae X-MS-TrafficTypeDiagnostic: IA1PR12MB6307:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uVYozSL64rFJXRj4TB81gQhLBNsuoLgUXUuVxRXF+MAtnMkOn+kxICHsRditEjuEQkAW0v9T0E7Ar9enlBM91MTSvKwcgdPyrzCEzckEB1qE3sMZ5qpvsB+2WO2p0G3WSosbk5Bq7cNdV/G2998kuH7LRS95bp+pmdB+ZUUl7Zd5myhqQIi0qE8SBBUKewqTtRaipDPNX/3wbT+Jsteo2Fj/rD3Vu+XV26bJIpFSiqFocpPqMehToN8nGPBuGIGx8DzMQJ2MuApjoVEN9jplcADmZE2nmnu1PEDAFZd4YAaJFGQ7TiR3sKyVtP1G36+Omlsyqh+H0si+Cha1t63Fz95oj32FSHlhUpGz3FCmfWEbCGK9g6my9ZVGrPgwAV9XtYNpwxx65wCMFeyM6hgopFb638nHzDvG3TP+lykzE8G7xqWNwwO06ql1VIVpmKJQBWciwJ4HhqHD3GzIBZiAQy03e8ReAB7eqAoRxdRgcScFM1pcHCPknXLfvU/ndMiSDfNVwfFBqWyhW4luCjNEgLrSNti3sKYl57HeIv28sZV/DXsWNNOthdakRzrhRzyEKOPAKfC9BE9YxBA8ZP8e0sQ4aOpXjF77bl4tA1d+RYmpZSXlf5FLTYMwpywpwpNMsagb8WIEWW3CV51tOImUb9TrRwBDMFUerveHuVWaMlh5x1tBzW34MDyJoTqwDdZhgZuNy6u1KCEAnKv040/ZNw== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(54906003)(70206006)(8936002)(110136005)(70586007)(1076003)(8676002)(2616005)(316002)(4326008)(86362001)(107886003)(356005)(7696005)(6666004)(26005)(81166007)(36756003)(426003)(336012)(83380400001)(82310400005)(5660300002)(36860700001)(40460700003)(2906002)(186003)(47076005)(508600001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:08:19.2272 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 82af9299-0d8b-438a-017a-08da425efeae X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT048.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6307 Received-SPF: softfail client-ip=2a01:111:f400:fe5a::625; envelope-from=avihaih@nvidia.com; helo=NAM12-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653933853715100001 Content-Type: text/plain; charset="utf-8" Sort the migration section of VFIO trace events file alphabetically and move two misplaced traces to common.c section. Signed-off-by: Avihai Horon --- hw/vfio/trace-events | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index a24ea7d8b0..d3cba59bfd 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -119,6 +119,8 @@ vfio_region_sparse_mmap_header(const char *name, int in= dex, int nr_areas) "Devic vfio_region_sparse_mmap_entry(int i, unsigned long start, unsigned long en= d) "sparse entry %d [0x%lx - 0x%lx]" vfio_get_dev_region(const char *name, int index, uint32_t type, uint32_t s= ubtype) "%s index %d, %08x/%0x8" vfio_dma_unmap_overflow_workaround(void) "" +vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitma= p_size, uint64_t start) "container fd=3D%d, iova=3D0x%"PRIx64" size=3D 0x%"= PRIx64" bitmap_size=3D0x%"PRIx64" start=3D0x%"PRIx64 +vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu= dirty @ 0x%"PRIx64" - 0x%"PRIx64 =20 # platform.c vfio_platform_base_device_init(char *name, int groupid) "%s belongs to gro= up #%d" @@ -148,18 +150,16 @@ vfio_display_edid_update(uint32_t prefx, uint32_t pre= fy) "%ux%u" vfio_display_edid_write_error(void) "" =20 # migration.c +vfio_load_cleanup(const char *name) " (%s)" +vfio_load_device_config_state(const char *name) " (%s)" +vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 +vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) s= ize 0x%"PRIx64 vfio_migration_probe(const char *name) " (%s)" vfio_migration_set_state(const char *name, uint32_t state) " (%s) state %d" -vfio_vmstate_change(const char *name, int running, const char *reason, uin= t32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) = state %s" -vfio_save_setup(const char *name) " (%s)" +vfio_save_block(const char *name, int data_size) " (%s) data_size %d" vfio_save_cleanup(const char *name) " (%s)" -vfio_save_device_config_state(const char *name) " (%s)" vfio_save_complete_precopy(const char *name) " (%s)" -vfio_load_device_config_state(const char *name) " (%s)" -vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) s= ize 0x%"PRIx64 -vfio_load_cleanup(const char *name) " (%s)" -vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitma= p_size, uint64_t start) "container fd=3D%d, iova=3D0x%"PRIx64" size=3D 0x%"= PRIx64" bitmap_size=3D0x%"PRIx64" start=3D0x%"PRIx64 -vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu= dirty @ 0x%"PRIx64" - 0x%"PRIx64 -vfio_save_block(const char *name, int data_size) " (%s) data_size %d" +vfio_save_device_config_state(const char *name) " (%s)" +vfio_save_setup(const char *name) " (%s)" +vfio_vmstate_change(const char *name, int running, const char *reason, uin= t32_t dev_state) " (%s) running %d reason %s device state %d" --=20 2.21.3 From nobody Sun May 19 02:06:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1653934056; cv=pass; d=zohomail.com; s=zohoarc; b=H+TivWQW2nfBFxUzW6P+Uf50vQs0RTVkV0TVobrOotBoagf4y7WPAmdtXtivXgz2phjwJdY+fOYtAe9W1sMTEnxPnt4e0fp5JMQI6vrHcFxc8uMd2oyOF1xJpxAf6qNcH2IF9ShUU6qJaTiWyTTeHBrx7Fs2NF9ZFVFnqKKmWLM= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653934056; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PSPaPjjDN9b8FTrPUpEHjY11iJitw0hUj9yIciMkLkE=; b=LysnounfSGMMVF+Zo1H4YjPOYjoNlvN+j9MirU5U6GSB7xrdFYUIRIJwAkKAQam/Zyg+r/Yw0tWfipDcY5fQjvW/kw3+MPs6lOV1Ry1VBqUCnPdQRgJktb/1N/leRn0QqN6HxeLCPwsoMCY07MQSwxQwIs60pHRMx54H8frFlJA= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1653934056462429.7088279921228; Mon, 30 May 2022 11:07:36 -0700 (PDT) Received: from localhost ([::1]:47082 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvjnj-0000TZ-AG for importer@patchew.org; Mon, 30 May 2022 14:07:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54430) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisX-0005BA-4j for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:29 -0400 Received: from mail-co1nam11on20630.outbound.protection.outlook.com ([2a01:111:f400:7eab::630]:31904 helo=NAM11-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvisU-0008PY-UA for qemu-devel@nongnu.org; Mon, 30 May 2022 13:08:28 -0400 Received: from CO1PR15CA0084.namprd15.prod.outlook.com (2603:10b6:101:20::28) by BN6PR1201MB0004.namprd12.prod.outlook.com (2603:10b6:404:ac::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Mon, 30 May 2022 17:08:23 +0000 Received: from CO1NAM11FT046.eop-nam11.prod.protection.outlook.com (2603:10b6:101:20:cafe::67) by CO1PR15CA0084.outlook.office365.com (2603:10b6:101:20::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.19 via Frontend Transport; Mon, 30 May 2022 17:08:23 +0000 Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT046.mail.protection.outlook.com (10.13.174.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5293.13 via Frontend Transport; Mon, 30 May 2022 17:08:22 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 30 May 2022 17:08:22 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 30 May 2022 10:08:21 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Mon, 30 May 2022 10:08:18 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dZYnjr76hgpYD6IFlWEtsdYjv2uDA25QwAxZ7U/fM0iixgAa2CykHFJVMX8n6/EFCEUK/oWUqzAuuLlGGIIIZzxVaw2sr65EAXj5qslpTxp4HPg+LhfQABNpcion9szslSVo5Bs3Lap44H/PCmz4nkyLqpzcCbYeOjI7R/IAnrNFQ8i0YOQTAfH4GaeiFUdjsyB5TnSQINtnkg8ZUKWU2rwUV6SnxBuuJozK4b33nF+o5fhAyqJyGa1y6RNDRLlRfMrz97PtnKTthMtqwAvaM3By12Y8W0yUVa7BV9M5SnYBhVxPnhfzxmlTCixTDu/A+FvbK4Smxn9aHd5oJ4VGBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PSPaPjjDN9b8FTrPUpEHjY11iJitw0hUj9yIciMkLkE=; b=Uh73WDhN6QdIsjAL3kuyDcxiB4YKD/jyRFcyigsMcdTlO7RTbBKGnFV4bqpRKiGaox6JLTuJ6ci7RQJzUhzoiNOd/cV8kt+/f8mU3SU+a+7mFQDy44T6yd6BtOFs7QGDGw9oGIs+yCzI1kqtYDbU2SB3fHT01uK26ksS330sgU4XOxnJ427PnmAgufHV39JY0b0KeV6Y7R9HxRJzBPimjZcdkj2LHTq7xwhKQuaZ6TE6SrD/frSmFP+bLJSOXSdDGkHXV46jDoNbaoO9JUQGTCzIQfmR9dYp+o95PzGnp3JpfksAWXEUUUbX7K+FpS3r2AdxP70wbJH1teLwvxhFoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PSPaPjjDN9b8FTrPUpEHjY11iJitw0hUj9yIciMkLkE=; b=OncJ0ubDiPz1OwAQZ3YsQgk6xWIM+gPZDcnAV7FODnCmhALKdN1kc2ylb8v2rYMGt65Nlc4iCzDY4WfDHWJtaL4gCY9A+O3Dczt8hImAJJxcbs62zugDT8oEm0SqTCqxVTyUiHh3Vq5uAlMsoNexu5VkKtz7aEfDYLbe4oAemkCjS35n7aRFKkR422RRz78ao9KX+ObWdsWKUfPq9HaxSYjJmmHnrQGontYBE6Sr5cqrPCImnf+9yKHBmpppQ7oRhYvOK6gFrSqJ21OZUk8QdVt7oBJHAG82QbNhHURBluSG28MXVXn5e3QDefylWRQmrSbSy1s5EttnOklr/NTFPA== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; pr=C From: Avihai Horon To: , Cornelia Huck , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Joao Martins , Yishai Hadas , Jason Gunthorpe , Mark Bloch , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v2 11/11] docs/devel: Align vfio-migration docs to VFIO migration v2 Date: Mon, 30 May 2022 20:07:39 +0300 Message-ID: <20220530170739.19072-12-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220530170739.19072-1-avihaih@nvidia.com> References: <20220530170739.19072-1-avihaih@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 430a0034-c8de-4553-6081-08da425f00e5 X-MS-TrafficTypeDiagnostic: BN6PR1201MB0004:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eiNSNyNT7JTsj5T07pRLZSk1W5QZXPBMKb0D9MbZkWiI22cV/dU1NeC8i003lROnF+5PGmhunOSmmfAHjE+N9gERj2AlqEdnQJugp14OogYTZUUwakEeQIOa0Q0Vsd6Z3uMnY2SFSKf/7JDKjmPSFiKNhxABKhzXoPLdDSMrOQOYIViJdgeHh8mSDxlyd9zuqkhWSt4IZgNTZL6OC71s50+gISAgg9ywt21aUxlMFuxPnsBjgfahvJtYR8YekNLmV2NNzCpEBpeqD231rmH6rKk+Kgji0wqLePjeFfeOkpwQPbMewTGfSG7ODAjkIuRfHb2/34FI8m9bFMz688Bg2xqbiTJPxFTh/0yl3YMqoLmN9Jd6FT5A55L4qTlNzlAvN7rxx0xX3b9EHJ2wYFPS44KCSY/NN1nDIXCM/p/U6aND1ilRwChtt3uc+PoGfofQdu3uy3yMyNw2kfAohnNSJ0EOPdeK37IvFF9iEjSlyBngrBUku7ZrPkk81aY4Q2ErXVfhv/LEnlzlGYLJ/bkrl+LfnA8akZMlr9CYDYNWG7VaFd4wireZ2dc72D1aHq7LHC7kbNk1k2lyJ6qfZR9pCq1j+xQmfkLAkxKPhNcbo0AyUhDhLuabN3aqxHmaZqGux9Ul3nU1HFZu64ALf/6nRc8eebtvuJHJdJyEX/GTpzE0Qd8ZLD5lCw3fdF6KmMTm2Cw02XjJt8SPFDGIttMhkg== X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(26005)(86362001)(40460700003)(6666004)(508600001)(8676002)(356005)(7696005)(316002)(5660300002)(4326008)(110136005)(54906003)(36756003)(83380400001)(36860700001)(336012)(426003)(186003)(8936002)(2906002)(1076003)(2616005)(82310400005)(70586007)(70206006)(107886003)(81166007)(47076005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2022 17:08:22.9455 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 430a0034-c8de-4553-6081-08da425f00e5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT046.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB0004 Received-SPF: softfail client-ip=2a01:111:f400:7eab::630; envelope-from=avihaih@nvidia.com; helo=NAM11-CO1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1653934058569100001 Content-Type: text/plain; charset="utf-8" Align the vfio-migration documentation to VFIO migration protocol v2. Signed-off-by: Avihai Horon --- docs/devel/vfio-migration.rst | 77 +++++++++++++++-------------------- 1 file changed, 33 insertions(+), 44 deletions(-) diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index 9ff6163c88..09744af5a6 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -7,46 +7,35 @@ the guest is running on source host and restoring this sa= ved state on the destination host. This document details how saving and restoring of VFIO devices is done in QEMU. =20 -Migration of VFIO devices consists of two phases: the optional pre-copy ph= ase, -and the stop-and-copy phase. The pre-copy phase is iterative and allows to -accommodate VFIO devices that have a large amount of data that needs to be -transferred. The iterative pre-copy phase of migration allows for the gues= t to -continue whilst the VFIO device state is transferred to the destination, t= his -helps to reduce the total downtime of the VM. VFIO devices can choose to s= kip -the pre-copy phase of migration by returning pending_bytes as zero during = the -pre-copy phase. +Migration of VFIO devices currently consists of a single stop-and-copy pha= se. +During the stop-and-copy phase the guest is stopped and the entire VFIO de= vice +data is transferred to the destination. + +The pre-copy phase of migration is currently not supported for VFIO device= s, +so VFIO device data is not transferred during pre-copy phase. =20 A detailed description of the UAPI for VFIO device migration can be found = in -the comment for the ``vfio_device_migration_info`` structure in the header -file linux-headers/linux/vfio.h. +the comment for the ``vfio_device_mig_state`` structure in the header file +linux-headers/linux/vfio.h. =20 VFIO implements the device hooks for the iterative approach as follows: =20 -* A ``save_setup`` function that sets up the migration region and sets _SA= VING - flag in the VFIO device state. - -* A ``load_setup`` function that sets up the migration region on the - destination and sets _RESUMING flag in the VFIO device state. - -* A ``save_live_pending`` function that reads pending_bytes from the vendor - driver, which indicates the amount of data that the vendor driver has ye= t to - save for the VFIO device. +* A ``save_setup`` function that sets up migration on the source. =20 -* A ``save_live_iterate`` function that reads the VFIO device's data from = the - vendor driver through the migration region during iterative phase. +* A ``load_setup`` function that sets the VFIO device on the destination in + _RESUMING state. =20 * A ``save_state`` function to save the device config space if it is prese= nt. =20 -* A ``save_live_complete_precopy`` function that resets _RUNNING flag from= the - VFIO device state and iteratively copies the remaining data for the VFIO - device until the vendor driver indicates that no data remains (pending b= ytes - is zero). +* A ``save_live_complete_precopy`` function that sets the VFIO device in + _STOP_COPY state and iteratively copies the data for the VFIO device unt= il + the vendor driver indicates that no data remains. =20 * A ``load_state`` function that loads the config section and the data - sections that are generated by the save functions above + sections that are generated by the save functions above. =20 * ``cleanup`` functions for both save and load that perform any migration - related cleanup, including unmapping the migration region + related cleanup. =20 =20 The VFIO migration code uses a VM state change handler to change the VFIO @@ -71,13 +60,13 @@ tracking can identify dirtied pages, but any page pinne= d by the vendor driver can also be written by the device. There is currently no device or IOMMU support for dirty page tracking in hardware. =20 -By default, dirty pages are tracked when the device is in pre-copy as well= as -stop-and-copy phase. So, a page pinned by the vendor driver will be copied= to -the destination in both phases. Copying dirty pages in pre-copy phase helps -QEMU to predict if it can achieve its downtime tolerances. If QEMU during -pre-copy phase keeps finding dirty pages continuously, then it understands -that even in stop-and-copy phase, it is likely to find dirty pages and can -predict the downtime accordingly. +By default, dirty pages are tracked during pre-copy as well as stop-and-co= py +phase. So, a page pinned by the vendor driver will be copied to the destin= ation +in both phases. Copying dirty pages in pre-copy phase helps QEMU to predic= t if +it can achieve its downtime tolerances. If QEMU during pre-copy phase keeps +finding dirty pages continuously, then it understands that even in stop-an= d-copy +phase, it is likely to find dirty pages and can predict the downtime +accordingly. =20 QEMU also provides a per device opt-out option ``pre-copy-dirty-page-track= ing`` which disables querying the dirty bitmap during pre-copy phase. If it is s= et to @@ -111,23 +100,23 @@ Live migration save path | migrate_init spawns migration_thread Migration thread then calls each device's .save_setup() - (RUNNING, _SETUP, _RUNNING|_SAVING) + (RUNNING, _SETUP, _RUNNING) | - (RUNNING, _ACTIVE, _RUNNING|_SAVING) - If device is active, get pending_bytes by .save_live_pending() - If total pending_bytes >=3D threshold_size, call .save_live_iter= ate() - Data of VFIO device for pre-copy phase is copied + (RUNNING, _ACTIVE, _RUNNING) + Migration thread calls each .save_live_pending() handler + If total pending_bytes >=3D threshold_size, call each .save_live_iterate= () handler + Data of this iteration for pre-copy phase is copied Iterate till total pending bytes converge and are less than thresh= old | On migration completion, vCPU stops and calls .save_live_complete_precop= y for - each active device. The VFIO device is then transitioned into _SAVING s= tate - (FINISH_MIGRATE, _DEVICE, _SAVING) + each active device. The VFIO device is then transitioned into _STOP_COPY= state + (FINISH_MIGRATE, _DEVICE, _STOP_COPY) | For the VFIO device, iterate in .save_live_complete_precopy until pending data is 0 - (FINISH_MIGRATE, _DEVICE, _STOPPED) + (FINISH_MIGRATE, _DEVICE, _STOP) | - (FINISH_MIGRATE, _COMPLETED, _STOPPED) + (FINISH_MIGRATE, _COMPLETED, _STOP) Migraton thread schedules cleanup bottom half and exits =20 Live migration resume path @@ -136,7 +125,7 @@ Live migration resume path :: =20 Incoming migration calls .load_setup for each device - (RESTORE_VM, _ACTIVE, _STOPPED) + (RESTORE_VM, _ACTIVE, _STOP) | For each device, .load_state is called for that device section data (RESTORE_VM, _ACTIVE, _RESUMING) --=20 2.21.3