From nobody Sat Apr 11 23:08:08 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1772615609; cv=none; d=zohomail.com; s=zohoarc; b=NsPkaFGuykRG5ypvM8ayWIFwyEBKs8d70+8oFLzWPQ1AVeRmefjKa/uW8K5vM/ByonA88Q+F7IUtbdV4xPUD5Yk4x+Z52x/2b0c7Tlj5UkJ7PixdO0LWRiLtopC7CaRGc28smBHiT7c+bGctpXzZifkaK1SnX4+MVgCUdiSQnE4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772615609; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZTBRdCe4EKhZ7QFgzj0snqvNaQKXZdaqV+19ND1Hpu0=; b=Hzyc3kaYJohaez/J/h7y6aBX/rEE0NnY8M0+IFpFOSqykyk6M+hat+y7U3UHWuGJ95+S+DsBdQwMIsR7aiN91XMudSVMHw+sejTfF02k/vjFN/ymw1F6sdee2G8EBC1e+w4xZCcLmk/RxvzCtrrcWDVww5/jjbpWbvLnYPCvmCc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772615609274106.12787968334203; Wed, 4 Mar 2026 01:13:29 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vxiHk-00014G-Ic; Wed, 04 Mar 2026 04:12:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vxiHa-00010l-1j for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:43 -0500 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vxiHX-0000IJ-B3 for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:41 -0500 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-483bd7354efso88409655e9.2 for ; Wed, 04 Mar 2026 01:12:35 -0800 (PST) Received: from alex-laptop.lan (p200300cf57228c00bac66d1b6d99e025.dip0.t-ipconnect.de. [2003:cf:5722:8c00:bac6:6d1b:6d99:e025]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-439c4489279sm9674234f8f.20.2026.03.04.01.12.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Mar 2026 01:12:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1772615554; x=1773220354; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZTBRdCe4EKhZ7QFgzj0snqvNaQKXZdaqV+19ND1Hpu0=; b=JEMIQTkxcNFbViMZZuM6etBjv2XFFbd2KKEQqsy3PM81A7ikHo2d86EIsi9bL3qgP6 SQzCaik6czem7wG+qgubtpBqh/w5DIbhyb2fw+iBvcfe8WyYV4/31dqWJmKhFPHfUb42 Ybijd6jWICI4ee1Q5ulNq/s26Kv2Ua63q/dT0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772615554; x=1773220354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ZTBRdCe4EKhZ7QFgzj0snqvNaQKXZdaqV+19ND1Hpu0=; b=jgpUFWGlW6EUHF0jb71fw/G80bkBCrn8VVCfALsjmV6My0Tfxel/Ts9wLKlYRr6w0N WDgyFEciRkxwNxeERF97n+stcCvIFMn9PJhqNnnbFsD0DvfI8tPAeMlp7iDC6I2hWYnJ rX/GuVZ19c2I4ErzfB3841czWjvH79ZZbObz8QGKFT0le7LBrbtZkjxC5RM2EbNSC0Oe bmIBs1oD9azyKUXpiqDkmE88c7e+u1sI09tHloSMulxEAw+e4pwtZ0mtJiiPtq7ccSva I1Uizw9XvzblAJhyxHHdMYRGm7m0fC8LD9mf36s+3UKQr1aIicbLovtyctyxIYFE/Tcn uchw== X-Gm-Message-State: AOJu0YwARjQTnqOLSTNaiVJlAVE8qNK9pwNTK9jGBwVzbvfewBfI5KOK Ao9pB4cxDdZTqkBP+GNhAfRFVY2FShLIRI8d6UeF7uFMGJ4ZTMH9GXEPGVUYMWf7STZvam8+n3H HWO59xfU= X-Gm-Gg: ATEYQzwZd0wbTpUYHzrEmZQoN/xPn5JOa+ZrQjK5BuzRNAnflS9SlxtiPLy6Y85Ppgf lFLs9i3xY6/QCWlm+ARe3zku8JQ5xgRJEwxxpY9AgAM/E1QVwYAaWs583MSPqH2Ce0w5Cxybov6 EDf1UtK3vnnQbTcXmzHd+/5TISncgV5Xa7iZw6P2/kitaCAwQl0O5edivX1/CFkv/yjpd60dUYs o0bh9IpTDS+EbLif/Bj42wEHnGKJw64ZhfJ0hi924QMMuPKScLQKG5/uDPV+fryHFXzkoI2oaEa dgYQKE6aWENusEpKcmv/zDlOY0JBRz+Qsc9Tpr9bxE8jFRioeB7bpL5F12sqonlv5X/UlEkP/YT biGRfdsBj+CJLfW8B7eR/d9Xpbtp/6DfWgaAOmN8cOiYtSVFbBX7PWUYBQ7NXWh9JH6hoQQn3vN oniP98gUtkkOEizz4Ai1qIwko+47Ntv5/nDAikqD/2bKSiR80eXAxwvRKZLk696vHgVXMCiXfqA hqGhDW5ELHLiysuuZp+n4k= X-Received: by 2002:a05:600c:a03:b0:47d:264e:b35a with SMTP id 5b1f17b1804b1-48519854e97mr23177625e9.13.1772615553822; Wed, 04 Mar 2026 01:12:33 -0800 (PST) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Klaus Jensen , Jesper Devantier , Peter Xu , Alexander Mikhalitsyn , Keith Busch , Fabiano Rosas , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Alexander Mikhalitsyn Subject: [PATCH v2 1/4] hw/nvme: add migration blockers for non-supported cases Date: Wed, 4 Mar 2026 10:12:26 +0100 Message-ID: <20260304091229.80725-2-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260304091229.80725-1-alexander@mihalicyn.com> References: <20260304091229.80725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::330; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x330.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1772615612000158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Let's block migration for cases we don't support: - SR-IOV - CMB - PMR - SPDM No functional changes here, because NVMe migration is not supported at all as of this commit. Signed-off-by: Alexander Mikhalitsyn --- hw/nvme/ctrl.c | 35 +++++++++++++++++++++++++++++++++++ hw/nvme/nvme.h | 3 +++ 2 files changed, 38 insertions(+) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index cc4593cd427..4694bdb4d02 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -207,6 +207,7 @@ #include "hw/pci/msix.h" #include "hw/pci/pcie_sriov.h" #include "system/spdm-socket.h" +#include "migration/blocker.h" #include "migration/vmstate.h" =20 #include "nvme.h" @@ -8962,6 +8963,14 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pc= i_dev, Error **errp) pcie_endpoint_cap_init(pci_dev, 0x80); pcie_cap_flr_init(pci_dev); if (n->params.sriov_max_vfs) { + if (n->migration_blocker =3D=3D NULL) { + error_setg(&n->migration_blocker, + "Migration is disabled when SR-IOV capability is se= t"); + if (migrate_add_blocker(&n->migration_blocker, errp) < 0) { + return false; + } + } + pcie_ari_init(pci_dev, 0x100); } =20 @@ -9025,6 +9034,14 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pc= i_dev, Error **errp) if (pci_dev->spdm_port) { uint16_t doe_offset =3D PCI_CONFIG_SPACE_SIZE; =20 + if (n->migration_blocker =3D=3D NULL) { + error_setg(&n->migration_blocker, + "Migration is disabled when SPDM responder is used"= ); + if (migrate_add_blocker(&n->migration_blocker, errp) < 0) { + return false; + } + } + switch (pci_dev->spdm_trans) { case SPDM_SOCKET_TRANSPORT_TYPE_PCI_DOE: if (n->params.sriov_max_vfs) { @@ -9053,10 +9070,26 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *p= ci_dev, Error **errp) } =20 if (n->params.cmb_size_mb) { + if (n->migration_blocker =3D=3D NULL) { + error_setg(&n->migration_blocker, + "Migration is disabled when CMB feature is used"); + if (migrate_add_blocker(&n->migration_blocker, errp) < 0) { + return false; + } + } + nvme_init_cmb(n, pci_dev); } =20 if (n->pmr.dev) { + if (n->migration_blocker =3D=3D NULL) { + error_setg(&n->migration_blocker, + "Migration is disabled when PMR feature is used"); + if (migrate_add_blocker(&n->migration_blocker, errp) < 0) { + return false; + } + } + if (!nvme_init_pmr(n, pci_dev, errp)) { return false; } @@ -9365,6 +9398,8 @@ static void nvme_exit(PCIDevice *pci_dev) } =20 memory_region_del_subregion(&n->bar0, &n->iomem); + + migrate_del_blocker(&n->migration_blocker); } =20 static const Property nvme_props[] =3D { diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index d66f7dc82d5..457b6637249 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -666,6 +666,9 @@ typedef struct NvmeCtrl { =20 /* Socket mapping to SPDM over NVMe Security In/Out commands */ int spdm_socket; + + /* Migration-related stuff */ + Error *migration_blocker; } NvmeCtrl; =20 typedef enum NvmeResetType { --=20 2.47.3 From nobody Sat Apr 11 23:08:08 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1772615583; cv=none; d=zohomail.com; s=zohoarc; b=ZvcfAybnvVRtFGdAjT8z7QhvEeKnSMKb5EsOs1hvEX5j03M6zBaP4PVnM+pjDikVa2K9vDfev6sk8xGL2Mdb+4F+nkg6oq1JNZRJVZoP1Ipe3gAxE3sD8+Sm45ueTS1ygpreXIaWvBrEjkYmcEF1B0chOPdG7Hz3D4n3jOvUYY4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772615583; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=gpP959L10hBsg4INOewg8w8g5XRXZUc7QBJng71AuUU=; b=aTIry/7UWG0XxXRYYHeTmYG6wZyGJ619EfFS03RwYB58EMq4+COhRkC3Q2xEJP8fkpw9XHphlyMoQkM7nZreboprPuLTT5vm7wfdZtEuKrkWRqwpl2qJhGAWZKheG19j+TM3rdRVG+vzoVKEDKyizmuko3UQZviTu0vPkKRdVJk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772615583326811.670494435332; Wed, 4 Mar 2026 01:13:03 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vxiHi-00013q-1U; Wed, 04 Mar 2026 04:12:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vxiHa-00010n-3W for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:43 -0500 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vxiHX-0000IP-BH for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:41 -0500 Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-439b78b638eso3177689f8f.2 for ; Wed, 04 Mar 2026 01:12:36 -0800 (PST) Received: from alex-laptop.lan (p200300cf57228c00bac66d1b6d99e025.dip0.t-ipconnect.de. [2003:cf:5722:8c00:bac6:6d1b:6d99:e025]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-439c4489279sm9674234f8f.20.2026.03.04.01.12.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Mar 2026 01:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1772615555; x=1773220355; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gpP959L10hBsg4INOewg8w8g5XRXZUc7QBJng71AuUU=; b=TiTEL0nGnMVnEbYs6tYd/Qk97RpZBWQEmikzZLKWM4gY9ogL3MwgMfHoUlvIr/OtND Hz+u5bEccJpUrOxmPrpAg5bPPwC5/mLGsiYH+gsWi7/m6y6f2W+gX5vOQm4/l/gOiBx1 Es9aNbRUn+e+vjSTZq0niYHt4DGotPNcj2jFs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772615555; x=1773220355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=gpP959L10hBsg4INOewg8w8g5XRXZUc7QBJng71AuUU=; b=hoDN05hzKVcA7AcvgICntN58XM/3Yges8FZ2mp8v0XWbXQYNH17rBaIl+lVaZZ5fPL tLMvYsw4d1qh1yYftr1r57txVhGVAl4RaH12DXf41T7NHvEANEAuuXM9Gz03IfAk3Wf1 ZsviBhCIpgoA4fCPW1cIBzqjQuPV5S5Ztg0MZkp8ofq/l7CSbYHV7W9YZKmffp8cce93 zGxM9yVhl65d94wgsQ+VVxQHLQ/I3H33cDsnzt2xgP9kTjVQHO/4nN8hEzpI0dcLH/T2 u/Dno5bAs8T+hpqLrXVr9P2purOhMDiri3xwdpDrLWaJl2cZnbr/QucO7fHtesxtyDqM ekSA== X-Gm-Message-State: AOJu0Yx+4WmC/cqM9A5043W+lq7F3Zp3B0V35nJhKUvaWVjD2lfBxSa3 zrpmNH8t4dmMVjsQ/cma+a0vKD/uWq803fftzN8GGKSWEU8CbzClezRhZ0WB2YiUboPPuOdGdgA 5fmoS8QU= X-Gm-Gg: ATEYQzx0kMv0rnEKRBUdxJxEmBcHxnme2v9BLDjUYheUh6lp4xBEc2DfuHFMeY+nUYd wb/fNWoiM0bCsLOlQGJiOnbNVxQeH3FB1s5s4/Nk/lAnZKpWFdcF38BN/h1ttPwFQlIK6GmmsJ/ lLZ0ihhgP+0jJPzTR6LetRVZ8vr77s57GKvaN86JKzQp44k2ds/Ioz6pSMXZuN7FwNnDWBWxf/i LzDDM6kwb0SbzRXKzNrOp6aWPVZFl+yY2+YCRhO5KxsBzZr8BXbP0mvLFKjvGK5pXN+25EdbObr ssD8tSShI1QGnYv65zI0bfax6taCiJDz5HZDZXvirtrMjnNL5MAWm3qwK/u2WL2buAMPvaYx1qK kn/bSeRgUPciTjN6n2/gWMk8XZioDBBxML8sEqUPNajSGFYdZ+RNkrnm+fPOSubF7ERxjVYz1xx wFO/rEdRW9O3aUHzC7+QEo8vBPaWyGcoWdnNuyo5J6TOmWPkXfUhFO0s+e/snj/psgsPDoVDXzE opTpmEMMGrwUHdQRuRKBGs= X-Received: by 2002:a05:6000:200c:b0:439:9282:e728 with SMTP id ffacd0b85a97d-439c7f64ff6mr2512939f8f.2.1772615554982; Wed, 04 Mar 2026 01:12:34 -0800 (PST) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Klaus Jensen , Jesper Devantier , Peter Xu , Alexander Mikhalitsyn , Keith Busch , Fabiano Rosas , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Alexander Mikhalitsyn Subject: [PATCH v2 2/4] hw/nvme: split nvme_init_sq/nvme_init_cq into helpers Date: Wed, 4 Mar 2026 10:12:27 +0100 Message-ID: <20260304091229.80725-3-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260304091229.80725-1-alexander@mihalicyn.com> References: <20260304091229.80725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::430; envelope-from=alexander@mihalicyn.com; helo=mail-wr1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1772615586020158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Signed-off-by: Alexander Mikhalitsyn --- hw/nvme/ctrl.c | 57 +++++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 21 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 4694bdb4d02..89cc26d745b 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -4852,18 +4852,14 @@ static uint16_t nvme_del_sq(NvmeCtrl *n, NvmeReques= t *req) return NVME_SUCCESS; } =20 -static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, uint64_t dma_addr, - uint16_t sqid, uint16_t cqid, uint16_t size) +static void __nvme_init_sq(NvmeSQueue *sq) { + NvmeCtrl *n =3D sq->ctrl; + uint16_t sqid =3D sq->sqid; + uint16_t cqid =3D sq->cqid; int i; NvmeCQueue *cq; =20 - sq->ctrl =3D n; - sq->dma_addr =3D dma_addr; - sq->sqid =3D sqid; - sq->size =3D size; - sq->cqid =3D cqid; - sq->head =3D sq->tail =3D 0; sq->io_req =3D g_new0(NvmeRequest, sq->size); =20 QTAILQ_INIT(&sq->req_list); @@ -4893,6 +4889,18 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n= , uint64_t dma_addr, n->sq[sqid] =3D sq; } =20 +static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, uint64_t dma_addr, + uint16_t sqid, uint16_t cqid, uint16_t size) +{ + sq->ctrl =3D n; + sq->dma_addr =3D dma_addr; + sq->sqid =3D sqid; + sq->size =3D size; + sq->cqid =3D cqid; + sq->head =3D sq->tail =3D 0; + __nvme_init_sq(sq); +} + static uint16_t nvme_create_sq(NvmeCtrl *n, NvmeRequest *req) { NvmeSQueue *sq; @@ -5553,24 +5561,16 @@ static uint16_t nvme_del_cq(NvmeCtrl *n, NvmeReques= t *req) return NVME_SUCCESS; } =20 -static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr, - uint16_t cqid, uint16_t vector, uint16_t size, - uint16_t irq_enabled) +static void __nvme_init_cq(NvmeCQueue *cq) { + NvmeCtrl *n =3D cq->ctrl; PCIDevice *pci =3D PCI_DEVICE(n); + uint16_t cqid =3D cq->cqid; =20 - if (msix_enabled(pci) && irq_enabled) { - msix_vector_use(pci, vector); + if (msix_enabled(pci) && cq->irq_enabled) { + msix_vector_use(pci, cq->vector); } =20 - cq->ctrl =3D n; - cq->cqid =3D cqid; - cq->size =3D size; - cq->dma_addr =3D dma_addr; - cq->phase =3D 1; - cq->irq_enabled =3D irq_enabled; - cq->vector =3D vector; - cq->head =3D cq->tail =3D 0; QTAILQ_INIT(&cq->req_list); QTAILQ_INIT(&cq->sq_list); if (n->dbbuf_enabled) { @@ -5588,6 +5588,21 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n= , uint64_t dma_addr, &DEVICE(cq->ctrl)->mem_reentrancy_guard); } =20 +static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr, + uint16_t cqid, uint16_t vector, uint16_t size, + uint16_t irq_enabled) +{ + cq->ctrl =3D n; + cq->cqid =3D cqid; + cq->size =3D size; + cq->dma_addr =3D dma_addr; + cq->phase =3D 1; + cq->irq_enabled =3D irq_enabled; + cq->vector =3D vector; + cq->head =3D cq->tail =3D 0; + __nvme_init_cq(cq); +} + static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeRequest *req) { NvmeCQueue *cq; --=20 2.47.3 From nobody Sat Apr 11 23:08:08 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1772615580; cv=none; d=zohomail.com; s=zohoarc; b=YLnCk/YSXGOizbGClsXM4rbBVaVCxLY5SvmVnsfo1BYN13Z7AQ3LbYD7oglK2f9Fd2cv81v5eDt7dDG2OAzY0B4+FzaDikcVd9SYjXgM4/7c3B4If1JceTY57josJTco0Gy6btaSg8j54KDSdJRDaH+mDemgKsssDgRBcv0kxPo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772615580; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Aa7YS8YRAQ8yRhQF9Whg1Qcoxr3p3yRf12L+u0hXtWc=; b=OfCe/o4JHBBG1tECzkKF/UIKD9SlYkHwYE7GMIa4EJWjYFUfCbOqzMrdIE3n9ENAzeedP4jg3/qhaXVcD/vJ0do174dZDhDgvbvMQVADvj3rw0or4OSPwuurfBM/J4VGgXWAd7MaB/lRF3c+2efhTuzVDE8R/FXZhlxv5sbNhDs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772615580241766.5910994707074; Wed, 4 Mar 2026 01:13:00 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vxiHg-00012u-Sk; Wed, 04 Mar 2026 04:12:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vxiHa-00010o-EH for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:43 -0500 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vxiHX-0000Ie-Ap for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:42 -0500 Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-4836f363d0dso58338165e9.3 for ; Wed, 04 Mar 2026 01:12:37 -0800 (PST) Received: from alex-laptop.lan (p200300cf57228c00bac66d1b6d99e025.dip0.t-ipconnect.de. [2003:cf:5722:8c00:bac6:6d1b:6d99:e025]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-439c4489279sm9674234f8f.20.2026.03.04.01.12.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Mar 2026 01:12:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1772615556; x=1773220356; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Aa7YS8YRAQ8yRhQF9Whg1Qcoxr3p3yRf12L+u0hXtWc=; b=V8sOSHHoGkICQW31sdmwjyIlSkdMZpg6GaA/89QAEwu7jmvG70eO9o39U7tlMQIBws 3ki00JeRkVo/IF1e1RfOs3bLHUn/5u4i0CfNewbA9vxRiA5EqOzDyezdSM2aBLv3FOsm 4B5LyD+PNJlmVH47s0sl5wND8g/yxIRY2GiWc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772615556; x=1773220356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Aa7YS8YRAQ8yRhQF9Whg1Qcoxr3p3yRf12L+u0hXtWc=; b=XJAIHtGDzDcYymvNcDvor/RwL5nbwFKZ9QLxEv1DWsADxWRebUxOqDylWegTqC0bqY 0rVCKPfKUsYm8IqPxZv3MpznNldijEcxfpNUvqumfJiBx4QGR7vGog2XzU9w/mPnJIvL S4AbAnW2kfIouyrWQYrqpmhJg9/dnXDB5Nvmiy1oRl1q8Qy13T4B/oIutkC8xSpsVMCU UbahtcXzVGBwN7SJ3y5bLnoWi8XO+pIyu3UgNig7S4kkPFd2i3o00X+11JBuquL7Q+NZ XjqQZ+6DNOUmdqmzfbt5CDMka78CfTJRSXOm7rRnR5vFgpIV3sFsGw3yRVyeZMtFE8XO W8kg== X-Gm-Message-State: AOJu0Yxjkqc/KYOv3dwMRFwBH9qX9RTRpXwD9SEiqbMQhaK50IK/GwoS APdZzod2f8FTXB51EM6nPsoFCJnfEroKJhjXceAXlfN0YNvtirWtHbIhs1Z5sizWgYjCYjh8kKi Xw1gYJ1k= X-Gm-Gg: ATEYQzwL/rqvD2LsPTdUhPc6N29iSzuFIIssV6pwLJ9a/U1zQCrAkcB+yimhW6Px0Qy 855flS2D/EnzXncud49CUiG5hv9spjcN3rIgL8PBmUtPmXYeTnM98tXj2OLZjEivo62y+2jRFLR QFo2ITcSTSi8FfyZ3CQM7jWZIn22IJtyjIGYnWNa8xumAsIztj3HzfG+74fXhPtiWF91py0j9Mi 4sXne+EAgL58wz5PF8q305TyTU8ww1wlitimeGHz7f4zG3sZFH84qAH7CiQUugjEg5VPKO4Jeqz ulZvTm/bqoHlS+6JBzuWvkJ3+exFMGvA3PqHYZwUZsAG0ZxzTemvW0dEAx8kVDq+3BARIQubpON D858j17OPdRZDPJmflgSenb8s8mH0wyTLbjgLLUeykZGkFzSi+r69t/RIm1JaTloHebePgqXyqB CyRYZ+xNk/MHCMsNQESaZK1R4HHMQg095wDyvR5urHN0NbY/hTPgRBP6/TlbzstxD019BLJ/tg7 feVhGf3oxq0ibpEnKfQS2I= X-Received: by 2002:a05:600c:4e0f:b0:483:8f0f:36fe with SMTP id 5b1f17b1804b1-48519837c8cmr20689175e9.1.1772615555873; Wed, 04 Mar 2026 01:12:35 -0800 (PST) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Klaus Jensen , Jesper Devantier , Peter Xu , Alexander Mikhalitsyn , Keith Busch , Fabiano Rosas , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Alexander Mikhalitsyn Subject: [PATCH v2 3/4] migration: add VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT{8, 32}_ALLOC Date: Wed, 4 Mar 2026 10:12:28 +0100 Message-ID: <20260304091229.80725-4-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260304091229.80725-1-alexander@mihalicyn.com> References: <20260304091229.80725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::334; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x334.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1772615584506139100 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Add VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT{8, 32}_ALLOC, which helps to save/restore a dynamic array of pointers to structures. Signed-off-by: Alexander Mikhalitsyn v2: - added VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT8_ALLOC --- include/migration/vmstate.h | 33 ++++++++++++++ migration/vmstate-types.c | 88 +++++++++++++++++++++++++++++++++++++ 2 files changed, 121 insertions(+) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 89f9f49d20a..0bbe3317740 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -265,6 +265,7 @@ extern const VMStateInfo vmstate_info_bitmap; extern const VMStateInfo vmstate_info_qtailq; extern const VMStateInfo vmstate_info_gtree; extern const VMStateInfo vmstate_info_qlist; +extern const VMStateInfo vmstate_info_ptrs_array_entry; =20 #define type_check_2darray(t1,t2,n,m) ((t1(*)[n][m])0 - (t2*)0) /* @@ -537,6 +538,38 @@ extern const VMStateInfo vmstate_info_qlist; .offset =3D vmstate_offset_array(_s, _f, _type*, _n), \ } =20 +/* + * For migrating a dynamically allocated uint32-indexed array + * of pointers to structures (with NULL entries and with auto memory alloc= ation). + * + * _type: type of structure pointed to + * _vmsd: VMSD for structure + * start: size of structure pointed to (for auto memory allocation) + */ +#define VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT8_ALLOC(_field, _state, _f= ield_num, _version, _vmsd, _type) { \ + .name =3D (stringify(_field)), \ + .version_id =3D (_version), \ + .num_offset =3D vmstate_offset_value(_state, _field_num, uint8_t), \ + .info =3D &vmstate_info_ptrs_array_entry, \ + .vmsd =3D &(_vmsd), \ + .start =3D sizeof(_type), \ + .size =3D sizeof(_type *), \ + .flags =3D VMS_VARRAY_UINT8|VMS_POINTER, \ + .offset =3D vmstate_offset_pointer(_state, _field, _type *), \ +} + +#define VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC(_field, _state, _= field_num, _version, _vmsd, _type) { \ + .name =3D (stringify(_field)), \ + .version_id =3D (_version), \ + .num_offset =3D vmstate_offset_value(_state, _field_num, uint32_t), \ + .info =3D &vmstate_info_ptrs_array_entry, \ + .vmsd =3D &(_vmsd), \ + .start =3D sizeof(_type), \ + .size =3D sizeof(_type *), \ + .flags =3D VMS_VARRAY_UINT32|VMS_POINTER, \ + .offset =3D vmstate_offset_pointer(_state, _field, _type *), \ +} + #define VMSTATE_VARRAY_OF_POINTER_UINT32(_field, _state, _field_num, _vers= ion, _info, _type) { \ .name =3D (stringify(_field)), = \ .version_id =3D (_version), = \ diff --git a/migration/vmstate-types.c b/migration/vmstate-types.c index 89cb2114721..3335377cd07 100644 --- a/migration/vmstate-types.c +++ b/migration/vmstate-types.c @@ -942,3 +942,91 @@ const VMStateInfo vmstate_info_qlist =3D { .get =3D get_qlist, .put =3D put_qlist, }; + +static int put_ptrs_array_entry(QEMUFile *f, void *ppv, size_t unused_size, + const VMStateField *field, JSONWriter *vmd= esc) +{ + const VMStateDescription *vmsd =3D field->vmsd; + int ret; + Error *local_err =3D NULL; + void *pv; + + /* + * (ppv) is an address of an i-th element of a dynamic array. + * + * (ppv) can not be NULL unless we have some regression/bug in + * vmstate_save_state_v(), because it is result of pointer arithemic l= ike: + * first_elem + size * i. + */ + if (ppv =3D=3D NULL) { + error_report("vmstate: put_ptrs_array_entry must be called with pp= v !=3D NULL"); + return -EINVAL; + } + + /* get a pointer to a structure */ + pv =3D *(void **)ppv; + + if (pv =3D=3D NULL) { + /* write a mark telling that there was a NULL pointer */ + qemu_put_byte(f, false); + return 0; + } + + /* if pointer is not NULL, dump the structure contents with help of vm= sd */ + qemu_put_byte(f, true); + ret =3D vmstate_save_state(f, vmsd, pv, vmdesc, &local_err); + if (ret) { + error_report_err(local_err); + return ret; + } + + return 0; +} + +static int get_ptrs_array_entry(QEMUFile *f, void *ppv, size_t unused_size, + const VMStateField *field) +{ + int ret =3D 0; + Error *local_err =3D NULL; + const VMStateDescription *vmsd =3D field->vmsd; + /* size of structure pointed to by elements of array */ + size_t size =3D field->start; + + if (ppv =3D=3D NULL) { + error_report("vmstate: get_ptrs_array_entry must be called with pp= v !=3D NULL"); + return -EINVAL; + } + + /* + * We start from a clean array, all elements must be NULL, unless + * something we haven't prepared for has changed in vmstate_save_state= _v(). + * Let's check for this just in case. + */ + if (*(void **)ppv !=3D NULL) { + error_report("vmstate: get_ptrs_array_entry must be called with *p= pv =3D=3D NULL"); + return -EINVAL; + } + + if (qemu_get_byte(f)) { + void *pv; + + /* allocate memory for structure */ + pv =3D g_malloc0(size); + ret =3D vmstate_load_state(f, vmsd, pv, vmsd->version_id, &local_e= rr); + if (ret) { + error_report_err(local_err); + g_free(pv); + return ret; + } + + *(void **)ppv =3D pv; + } + + return ret; +} + +const VMStateInfo vmstate_info_ptrs_array_entry =3D { + .name =3D "ptrs_array_entry", + .get =3D get_ptrs_array_entry, + .put =3D put_ptrs_array_entry, +}; --=20 2.47.3 From nobody Sat Apr 11 23:08:09 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1772615585; cv=none; d=zohomail.com; s=zohoarc; b=W6fthY6q3FbajOVGyp1tzUGHGN3Igl518Eloe3S5M19PE13ykZ59le1nFHjTQhHTCYMESrWdFlurEqYfubwtOWKz8QO3qOXkfA3tpdlhKHYkijmbu/oMMvTRfTOkaX9o4oDiUFOuGOlvJwkISk/zh7OJAwTpJ+sPPW9RWBfxRqw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772615585; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xj9/f+SaUMvAcrT6CuXG1fVnLMkvBtAqefrCAJGy/GQ=; b=XNrlOHw2CGsFm64068nYsQrBz+t/XssHkKZA8xIRS5quRrFr7TR6kPHFr2b5eqWbZ7zloOrSgowXjQ4bs0HoXymCy1iVjF+C7gVv0Gf1tB5icdj4LxsJKWZS8glCNLy9Mbeq9N0A8Yz55eGDOSWyXwarrnx8uEmMA83jRJxY2yA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772615585538833.3614722287341; Wed, 4 Mar 2026 01:13:05 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vxiHk-00014L-IP; Wed, 04 Mar 2026 04:12:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vxiHc-00011q-Tp for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:46 -0500 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vxiHX-0000Iq-Hi for qemu-devel@nongnu.org; Wed, 04 Mar 2026 04:12:44 -0500 Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-439c5cce2c6so648084f8f.3 for ; Wed, 04 Mar 2026 01:12:39 -0800 (PST) Received: from alex-laptop.lan (p200300cf57228c00bac66d1b6d99e025.dip0.t-ipconnect.de. [2003:cf:5722:8c00:bac6:6d1b:6d99:e025]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-439c4489279sm9674234f8f.20.2026.03.04.01.12.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Mar 2026 01:12:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1772615558; x=1773220358; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xj9/f+SaUMvAcrT6CuXG1fVnLMkvBtAqefrCAJGy/GQ=; b=LVghETrMkMgLeI6lz2Vf73jpzAQqqwrSb/aLCn/pfjCUps8PjmzePHZmY7ZWN8vA07 WTheiykIewuB1MGd/EoSkmiXTq/OzS0jA52MnfD9wGAESTQhse1ZLEEu5+WZ8ESg9C8e xI/aXs1prNUWEcCi/Uxf9hvkuLkhjV+3VB+84= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772615558; x=1773220358; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=xj9/f+SaUMvAcrT6CuXG1fVnLMkvBtAqefrCAJGy/GQ=; b=YYYXAX/+yF2IU8nezybw5o2fzz61dms6TiW9xfc1PP/CWgqFnz6y+uXeJLaAPXkHo7 icgp9E89MdRtxSjN173r+TmDtFh5+2Mb8BSqMD94jSFC51xhC8shmRK4dC7dIqfdoE+R pjgYbIljHs5lbxgJcb/VuD05wcyHj6aGfGwlrmNVIdOvJAWDeDU7sbKrgPdvw5jHx0wQ XknRvk7ivMW0sFDJBCJLUbt6BnSlO8svcr3SC/G7O46QCpFWsnm68oUEIVHRRi6YwxLi NOXEOM6t1v5xW2jJgEHR4rXbvK4ELkwOBtERzKJxbJXgoG9JRQPKmbUoSWtTPCe5yCjX LI3Q== X-Gm-Message-State: AOJu0YzWjkF2KsMwoYmgntL8WmxJJZ+aGITZRFheYrBgm4VtM196GWBH lbflAna3z3YLxIO1RuN2b46yD3aP6jjT9T9Rq5rAKsSMaXKCBRQA2CJ/mMypR1CcKMPcnfJOnyK W+bt7w8A= X-Gm-Gg: ATEYQzwquNs+qi4+1U0/XSFsxokMAC4EqOVVAo41RfsvHpyFlIRIh4rV44fvY7CMPxF kOS6aoBnkllrRNxI6BYChPk0rtu8kFsdoZdv1UCT6SXNccIrk5dbBkXtmalyh/iavEuyTUqMvMH dfJIYnfdkV8iNXB9mqZNGpa9yphWnMuw2zBXLiUyCpIOscDovl1ywW8uANU35+OgwrYP9vAAAJ2 i5XILaSpU9zzRz4mhB9ZTc9N9iLfB4iKWR86R3tCLZFxn0F6T4+L0eB0UazdrD+uaT6jY874rJB rCxS1Kxufi6wihInA1HUCTXBxvHOcb4MohsYr06YS4lStV8yf7uwc9d15rAWdf6qO7kdCxkLmsV iJL2JjglCpNyaceyd08R8kZwvGEhiIPFjkczL+w50LD4wdSyc/yg2RQH++0e7Ga3QOJ0KOxOFir m3ruYdsSnKVZtrszejPbYHWEDp4Ohet+lBJPO4epkzu0Jw8iwvJ0ZtGvCI/SqULBmw8rbBJuLIb KDuYUlVSNiMFtQXuxDrgtM= X-Received: by 2002:a05:6000:2313:b0:439:ba50:f0b3 with SMTP id ffacd0b85a97d-439c7f90245mr2501065f8f.26.1772615557749; Wed, 04 Mar 2026 01:12:37 -0800 (PST) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Klaus Jensen , Jesper Devantier , Peter Xu , Alexander Mikhalitsyn , Keith Busch , Fabiano Rosas , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Alexander Mikhalitsyn Subject: [PATCH v2 4/4] hw/nvme: add basic live migration support Date: Wed, 4 Mar 2026 10:12:29 +0100 Message-ID: <20260304091229.80725-5-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260304091229.80725-1-alexander@mihalicyn.com> References: <20260304091229.80725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alexander@mihalicyn.com; helo=mail-wr1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1772615587529139100 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn It has some limitations: - only one NVMe namespace is supported - SMART counters are not preserved - CMB is not supported - PMR is not supported - SPDM is not supported - SR-IOV is not supported Signed-off-by: Alexander Mikhalitsyn v2: - AERs are now fully supported --- hw/nvme/ctrl.c | 573 ++++++++++++++++++++++++++++++++++++++++++- hw/nvme/nvme.h | 2 + hw/nvme/trace-events | 11 + 3 files changed, 577 insertions(+), 9 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 89cc26d745b..cdfda1d3be8 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -208,6 +208,7 @@ #include "hw/pci/pcie_sriov.h" #include "system/spdm-socket.h" #include "migration/blocker.h" +#include "migration/qemu-file-types.h" #include "migration/vmstate.h" =20 #include "nvme.h" @@ -4901,6 +4902,25 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n= , uint64_t dma_addr, __nvme_init_sq(sq); } =20 +static void nvme_restore_sq(NvmeSQueue *sq_from) +{ + NvmeCtrl *n =3D sq_from->ctrl; + NvmeSQueue *sq =3D sq_from; + + if (sq_from->sqid =3D=3D 0) { + sq =3D &n->admin_sq; + sq->ctrl =3D n; + sq->dma_addr =3D sq_from->dma_addr; + sq->sqid =3D sq_from->sqid; + sq->size =3D sq_from->size; + sq->cqid =3D sq_from->cqid; + sq->head =3D sq_from->head; + sq->tail =3D sq_from->tail; + } + + __nvme_init_sq(sq); +} + static uint16_t nvme_create_sq(NvmeCtrl *n, NvmeRequest *req) { NvmeSQueue *sq; @@ -5603,6 +5623,27 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n= , uint64_t dma_addr, __nvme_init_cq(cq); } =20 +static void nvme_restore_cq(NvmeCQueue *cq_from) +{ + NvmeCtrl *n =3D cq_from->ctrl; + NvmeCQueue *cq =3D cq_from; + + if (cq_from->cqid =3D=3D 0) { + cq =3D &n->admin_cq; + cq->ctrl =3D n; + cq->cqid =3D cq_from->cqid; + cq->size =3D cq_from->size; + cq->dma_addr =3D cq_from->dma_addr; + cq->phase =3D cq_from->phase; + cq->irq_enabled =3D cq_from->irq_enabled; + cq->vector =3D cq_from->vector; + cq->head =3D cq_from->head; + cq->tail =3D cq_from->tail; + } + + __nvme_init_cq(cq); +} + static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeRequest *req) { NvmeCQueue *cq; @@ -7291,7 +7332,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const = NvmeRequest *req) n->dbbuf_eis =3D eis_addr; n->dbbuf_enabled =3D true; =20 - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { NvmeSQueue *sq =3D n->sq[i]; NvmeCQueue *cq =3D n->cq[i]; =20 @@ -7731,7 +7772,7 @@ static int nvme_atomic_write_check(NvmeCtrl *n, NvmeC= md *cmd, /* * Walk the queues to see if there are any atomic conflicts. */ - for (i =3D 1; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 1; i < n->num_queues; i++) { NvmeSQueue *sq; NvmeRequest *req; NvmeRwCmd *req_rw; @@ -7801,6 +7842,10 @@ static void nvme_process_sq(void *opaque) NvmeCmd cmd; NvmeRequest *req; =20 + if (qatomic_read(&n->stop_processing_sq)) { + return; + } + if (n->dbbuf_enabled) { nvme_update_sq_tail(sq); } @@ -7809,6 +7854,10 @@ static void nvme_process_sq(void *opaque) NvmeAtomic *atomic; bool cmd_is_atomic; =20 + if (qatomic_read(&n->stop_processing_sq)) { + return; + } + addr =3D sq->dma_addr + (sq->head << NVME_SQES); if (nvme_addr_read(n, addr, (void *)&cmd, sizeof(cmd))) { trace_pci_nvme_err_addr_read(addr); @@ -7917,12 +7966,12 @@ static void nvme_ctrl_reset(NvmeCtrl *n, NvmeResetT= ype rst) nvme_ns_drain(ns); } =20 - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { if (n->sq[i] !=3D NULL) { nvme_free_sq(n->sq[i], n); } } - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { if (n->cq[i] !=3D NULL) { nvme_free_cq(n->cq[i], n); } @@ -8592,6 +8641,8 @@ static bool nvme_check_params(NvmeCtrl *n, Error **er= rp) params->max_ioqpairs =3D params->num_queues - 1; } =20 + n->num_queues =3D params->max_ioqpairs + 1; + if (n->namespace.blkconf.blk && n->subsys) { error_setg(errp, "subsystem support is unavailable with legacy " "namespace ('drive' property)"); @@ -8746,8 +8797,8 @@ static void nvme_init_state(NvmeCtrl *n) n->conf_msix_qsize =3D n->params.msix_qsize; } =20 - n->sq =3D g_new0(NvmeSQueue *, n->params.max_ioqpairs + 1); - n->cq =3D g_new0(NvmeCQueue *, n->params.max_ioqpairs + 1); + n->sq =3D g_new0(NvmeSQueue *, n->num_queues); + n->cq =3D g_new0(NvmeCQueue *, n->num_queues); n->temperature =3D NVME_TEMPERATURE; n->features.temp_thresh_hi =3D NVME_TEMPERATURE_WARNING; n->starttime_ms =3D qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); @@ -8990,7 +9041,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci= _dev, Error **errp) } =20 if (n->params.msix_exclusive_bar && !pci_is_vf(pci_dev)) { - bar_size =3D nvme_mbar_size(n->params.max_ioqpairs + 1, 0, NULL, N= ULL); + bar_size =3D nvme_mbar_size(n->num_queues, 0, NULL, NULL); memory_region_init_io(&n->iomem, OBJECT(n), &nvme_mmio_ops, n, "nv= me", bar_size); pci_register_bar(pci_dev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY | @@ -9002,7 +9053,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci= _dev, Error **errp) /* add one to max_ioqpairs to account for the admin queue pair */ if (!pci_is_vf(pci_dev)) { nr_vectors =3D n->params.msix_qsize; - bar_size =3D nvme_mbar_size(n->params.max_ioqpairs + 1, + bar_size =3D nvme_mbar_size(n->num_queues, nr_vectors, &msix_table_offset, &msix_pba_offset); } else { @@ -9552,9 +9603,513 @@ static uint32_t nvme_pci_read_config(PCIDevice *dev= , uint32_t address, int len) return pci_default_read_config(dev, address, len); } =20 +static bool nvme_ctrl_pre_save(void *opaque, Error **errp) +{ + NvmeCtrl *n =3D opaque; + int i; + + trace_pci_nvme_pre_save_enter(n); + + /* ask SQ processing code not to take new requests */ + qatomic_set(&n->stop_processing_sq, true); + + /* prevent new in-flight IO from appearing */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + qemu_bh_cancel(sq->bh); + } + + /* drain all IO */ + for (i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns; + + ns =3D nvme_ns(n, i); + if (!ns) { + continue; + } + + trace_pci_nvme_pre_save_ns_drain(n, i); + nvme_ns_drain(ns); + } + + /* + * Now, we should take care of AERs. + * + * 1. Save all queued events (n->aer_queue). + * This is done automatically, see nvme_vmstate VMStateDescription. + * Here we only need to print them for debugging purpose. + * 2. Go over outstanding AER requests (n->aer_reqs) and check they are + * all have expected opcode (NVME_ADM_CMD_ASYNC_EV_REQ) and other f= ields. + * + * We must be really careful here, because in case of further QEMU NVM= e changes, + * we may break migration without noticing it, or worse, introduce sil= ent + * data corruption during migration. + */ + if (n->aer_queued) { + NvmeAsyncEvent *event; + + QTAILQ_FOREACH(event, &n->aer_queue, entry) { + trace_pci_nvme_pre_save_aer(event->result.event_type, event->r= esult.event_info, + event->result.log_page); + } + } + + for (i =3D 0; i < n->outstanding_aers; i++) { + NvmeRequest *re =3D n->aer_reqs[i]; + + /* + * Can't use assert() here, because we don't want + * to just crash QEMU when user requests a migration. + */ + if (!(re->cmd.opcode =3D=3D NVME_ADM_CMD_ASYNC_EV_REQ)) { + error_setg(errp, "re->cmd.opcode (%u) !=3D NVME_ADM_CMD_ASYNC_= EV_REQ", re->cmd.opcode); + goto err; + } + + if (!(re->ns =3D=3D NULL)) { + error_setg(errp, "re->ns !=3D NULL"); + goto err; + } + + if (!(re->sq =3D=3D &n->admin_sq)) { + error_setg(errp, "re->sq !=3D &n->admin_sq"); + goto err; + } + + if (!(re->aiocb =3D=3D NULL)) { + error_setg(errp, "re->aiocb !=3D NULL"); + goto err; + } + + if (!(re->opaque =3D=3D NULL)) { + error_setg(errp, "re->opaque !=3D NULL"); + goto err; + } + + if (!(re->atomic_write =3D=3D false)) { + error_setg(errp, "re->atomic_write !=3D false"); + goto err; + } + + if (re->sg.flags & NVME_SG_ALLOC) { + error_setg(errp, "unexpected NVME_SG_ALLOC flag in re->sg.flag= s"); + goto err; + } + } + + /* wait when all in-flight IO requests (except NVME_ADM_CMD_ASYNC_EV_R= EQ) are processed */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeRequest *req; + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + trace_pci_nvme_pre_save_sq_out_req_drain_wait(n, i, sq->head, sq->= tail, sq->size); + +wait_out_reqs: + QTAILQ_FOREACH(req, &sq->out_req_list, entry) { + if (req->cmd.opcode !=3D NVME_ADM_CMD_ASYNC_EV_REQ) { + cpu_relax(); + goto wait_out_reqs; + } + } + + trace_pci_nvme_pre_save_sq_out_req_drain_wait_end(n, i, sq->head, = sq->tail); + } + + /* wait when all IO requests completions are written to guest memory */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeCQueue *cq =3D n->cq[i]; + + if (!cq) + continue; + + trace_pci_nvme_pre_save_cq_req_drain_wait(n, i, cq->head, cq->tail= , cq->size); + + while (!QTAILQ_EMPTY(&cq->req_list)) { + /* + * nvme_post_cqes() can't do its job of cleaning cq->req_list + * when CQ is full, it means that we need to save what we have= in + * cq->req_list and restore it back on VM resume. + * + * Good thing is that this can only happen when guest hasn't + * processed CQ for a long time and at the same time, many SQEs + * are in flight. + * + * For now, let's just block migration in this rare case. + */ + if (nvme_cq_full(cq)) { + error_setg(errp, "no free space in CQ (not supported)"); + goto err; + } + + cpu_relax(); + } + + trace_pci_nvme_pre_save_cq_req_drain_wait_end(n, i, cq->head, cq->= tail); + } + + for (uint32_t nsid =3D 0; nsid <=3D NVME_MAX_NAMESPACES; nsid++) { + NvmeNamespace *ns =3D n->namespaces[nsid]; + + if (!ns) + continue; + + if (ns !=3D &n->namespace) { + error_setg(errp, "only one NVMe namespace is supported for mig= ration"); + goto err; + } + } + + return true; + +err: + /* restore sq processing back to normal */ + qatomic_set(&n->stop_processing_sq, false); + return false; +} + +static bool nvme_ctrl_post_load(void *opaque, int version_id, Error **errp) +{ + NvmeCtrl *n =3D opaque; + int i; + + trace_pci_nvme_post_load_enter(n); + + /* restore CQs first */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeCQueue *cq =3D n->cq[i]; + + if (!cq) + continue; + + cq->ctrl =3D n; + nvme_restore_cq(cq); + trace_pci_nvme_post_load_restore_cq(n, i, cq->head, cq->tail, cq->= size); + + if (i =3D=3D 0) { + /* + * Admin CQ lives in n->admin_cq, we don't need + * memory allocated for it in get_ptrs_array_entry() anymore. + * + * nvme_restore_cq() also takes care of: + * n->cq[0] =3D &n->admin_cq; + * so n->cq[0] remains valid. + */ + g_free(cq); + } + } + + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + sq->ctrl =3D n; + nvme_restore_sq(sq); + trace_pci_nvme_post_load_restore_sq(n, i, sq->head, sq->tail, sq->= size); + + if (i =3D=3D 0) { + /* same as for CQ */ + g_free(sq); + } + } + + if (n->aer_queued) { + NvmeAsyncEvent *event; + + QTAILQ_FOREACH(event, &n->aer_queue, entry) { + trace_pci_nvme_post_load_aer(event->result.event_type, event->= result.event_info, + event->result.log_page); + } + } + + for (i =3D 0; i < n->outstanding_aers; i++) { + NvmeSQueue *sq =3D &n->admin_sq; + NvmeRequest *req_from =3D n->aer_reqs[i]; + NvmeRequest *req; + + /* + * We use nvme_vmstate VMStateDescription to save/restore + * NvmeRequest structures, but tricky thing here is that + * memory for each n->aer_reqs[i] will be allocated separately + * during restore. It doesn't work for us. We need to take + * an existing NvmeRequest structure from SQ's req_list + * and fill it with data from the newly allocated one (req_from). + * Then, we can safely release allocated memory for it. + */ + + /* take an NvmeRequest struct from SQ */ + req =3D QTAILQ_FIRST(&sq->req_list); + QTAILQ_REMOVE(&sq->req_list, req, entry); + QTAILQ_INSERT_TAIL(&sq->out_req_list, req, entry); + nvme_req_clear(req); + + /* copy data from the source NvmeRequest */ + req->status =3D req_from->status; + memcpy(&req->cqe, &req_from->cqe, sizeof(NvmeCqe)); + memcpy(&req->cmd, &req_from->cmd, sizeof(NvmeCmd)); + + n->aer_reqs[i] =3D req; + g_free(req_from); + } + + /* + * We need to attach namespaces (currently, only one namespace is + * supported for migration). + * This logic comes from nvme_start_ctrl(). + */ + for (i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + + if (!ns || (!ns->params.shared && ns->ctrl !=3D n)) { + continue; + } + + if (nvme_csi_supported(n, ns->csi) && !ns->params.detached) { + if (!ns->attached || ns->params.shared) { + nvme_attach_ns(n, ns); + } + } + } + + /* schedule SQ processing */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + qemu_bh_schedule(sq->bh); + } + + /* + * We ensured in pre_save() that cq->req_list was empty, + * so we don't need to schedule BH for CQ processing. + */ + + return true; +} + +static const VMStateDescription nvme_vmstate_bar =3D { + .name =3D "nvme-bar", + .minimum_version_id =3D 1, + .version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(cap, NvmeBar), + VMSTATE_UINT32(vs, NvmeBar), + VMSTATE_UINT32(intms, NvmeBar), + VMSTATE_UINT32(intmc, NvmeBar), + VMSTATE_UINT32(cc, NvmeBar), + VMSTATE_UINT8_ARRAY(rsvd24, NvmeBar, 4), + VMSTATE_UINT32(csts, NvmeBar), + VMSTATE_UINT32(nssr, NvmeBar), + VMSTATE_UINT32(aqa, NvmeBar), + VMSTATE_UINT64(asq, NvmeBar), + VMSTATE_UINT64(acq, NvmeBar), + VMSTATE_UINT32(cmbloc, NvmeBar), + VMSTATE_UINT32(cmbsz, NvmeBar), + VMSTATE_UINT32(bpinfo, NvmeBar), + VMSTATE_UINT32(bprsel, NvmeBar), + VMSTATE_UINT64(bpmbl, NvmeBar), + VMSTATE_UINT64(cmbmsc, NvmeBar), + VMSTATE_UINT32(cmbsts, NvmeBar), + VMSTATE_UINT8_ARRAY(rsvd92, NvmeBar, 3492), + VMSTATE_UINT32(pmrcap, NvmeBar), + VMSTATE_UINT32(pmrctl, NvmeBar), + VMSTATE_UINT32(pmrsts, NvmeBar), + VMSTATE_UINT32(pmrebs, NvmeBar), + VMSTATE_UINT32(pmrswtp, NvmeBar), + VMSTATE_UINT32(pmrmscl, NvmeBar), + VMSTATE_UINT32(pmrmscu, NvmeBar), + VMSTATE_UINT8_ARRAY(css, NvmeBar, 484), + VMSTATE_END_OF_LIST() + }, +}; + +static const VMStateDescription nvme_vmstate_cqueue =3D { + .name =3D "nvme-cq", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(phase, NvmeCQueue), + VMSTATE_UINT16(cqid, NvmeCQueue), + VMSTATE_UINT16(irq_enabled, NvmeCQueue), + VMSTATE_UINT32(head, NvmeCQueue), + VMSTATE_UINT32(tail, NvmeCQueue), + VMSTATE_UINT32(vector, NvmeCQueue), + VMSTATE_UINT32(size, NvmeCQueue), + VMSTATE_UINT64(dma_addr, NvmeCQueue), + /* db_addr, ei_addr, etc will be recalculated */ + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_squeue =3D { + .name =3D "nvme-sq", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(sqid, NvmeSQueue), + VMSTATE_UINT16(cqid, NvmeSQueue), + VMSTATE_UINT32(head, NvmeSQueue), + VMSTATE_UINT32(tail, NvmeSQueue), + VMSTATE_UINT32(size, NvmeSQueue), + VMSTATE_UINT64(dma_addr, NvmeSQueue), + /* db_addr, ei_addr, etc will be recalculated */ + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_async_event_result =3D { + .name =3D "nvme-async-event-result", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(event_type, NvmeAerResult), + VMSTATE_UINT8(event_info, NvmeAerResult), + VMSTATE_UINT8(log_page, NvmeAerResult), + VMSTATE_UINT8(resv, NvmeAerResult), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_async_event =3D { + .name =3D "nvme-async-event", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_STRUCT(result, NvmeAsyncEvent, 0, nvme_vmstate_async_event= _result, NvmeAerResult), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cqe =3D { + .name =3D "nvme-cqe", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT32(result, NvmeCqe), + VMSTATE_UINT32(dw1, NvmeCqe), + VMSTATE_UINT16(sq_head, NvmeCqe), + VMSTATE_UINT16(sq_id, NvmeCqe), + VMSTATE_UINT16(cid, NvmeCqe), + VMSTATE_UINT16(status, NvmeCqe), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd_dptr_sgl =3D { + .name =3D "nvme-request-cmd-dptr-sgl", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(addr, NvmeSglDescriptor), + VMSTATE_UINT32(len, NvmeSglDescriptor), + VMSTATE_UINT8_ARRAY(rsvd, NvmeSglDescriptor, 3), + VMSTATE_UINT8(type, NvmeSglDescriptor), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd_dptr =3D { + .name =3D "nvme-request-cmd-dptr", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(prp1, NvmeCmdDptr), + VMSTATE_UINT64(prp2, NvmeCmdDptr), + VMSTATE_STRUCT(sgl, NvmeCmdDptr, 0, nvme_vmstate_cmd_dptr_sgl, Nvm= eSglDescriptor), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd =3D { + .name =3D "nvme-request-cmd", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(opcode, NvmeCmd), + VMSTATE_UINT8(flags, NvmeCmd), + VMSTATE_UINT16(cid, NvmeCmd), + VMSTATE_UINT32(nsid, NvmeCmd), + VMSTATE_UINT64(res1, NvmeCmd), + VMSTATE_UINT64(mptr, NvmeCmd), + VMSTATE_STRUCT(dptr, NvmeCmd, 0, nvme_vmstate_cmd_dptr, NvmeCmdDpt= r), + VMSTATE_UINT32(cdw10, NvmeCmd), + VMSTATE_UINT32(cdw11, NvmeCmd), + VMSTATE_UINT32(cdw12, NvmeCmd), + VMSTATE_UINT32(cdw13, NvmeCmd), + VMSTATE_UINT32(cdw14, NvmeCmd), + VMSTATE_UINT32(cdw15, NvmeCmd), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_request =3D { + .name =3D "nvme-request", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(status, NvmeRequest), + VMSTATE_STRUCT(cqe, NvmeRequest, 0, nvme_vmstate_cqe, NvmeCqe), + VMSTATE_STRUCT(cmd, NvmeRequest, 0, nvme_vmstate_cmd, NvmeCmd), + VMSTATE_END_OF_LIST() + } +}; + static const VMStateDescription nvme_vmstate =3D { .name =3D "nvme", - .unmigratable =3D 1, + .minimum_version_id =3D 1, + .version_id =3D 1, + .pre_save_errp =3D nvme_ctrl_pre_save, + .post_load_errp =3D nvme_ctrl_post_load, + .fields =3D (const VMStateField[]) { + VMSTATE_PCI_DEVICE(parent_obj, NvmeCtrl), + VMSTATE_MSIX(parent_obj, NvmeCtrl), + VMSTATE_STRUCT(bar, NvmeCtrl, 0, nvme_vmstate_bar, NvmeBar), + + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC( + sq, NvmeCtrl, num_queues, 0, nvme_vmstate_squeue, NvmeSQueue), + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC( + cq, NvmeCtrl, num_queues, 0, nvme_vmstate_cqueue, NvmeCQueue), + + VMSTATE_BOOL(qs_created, NvmeCtrl), + VMSTATE_UINT32(page_size, NvmeCtrl), + VMSTATE_UINT16(page_bits, NvmeCtrl), + VMSTATE_UINT16(max_prp_ents, NvmeCtrl), + VMSTATE_UINT32(max_q_ents, NvmeCtrl), + VMSTATE_UINT8(outstanding_aers, NvmeCtrl), + VMSTATE_UINT32(irq_status, NvmeCtrl), + VMSTATE_INT32(cq_pending, NvmeCtrl), + + VMSTATE_UINT64(host_timestamp, NvmeCtrl), + VMSTATE_UINT64(timestamp_set_qemu_clock_ms, NvmeCtrl), + VMSTATE_UINT64(starttime_ms, NvmeCtrl), + VMSTATE_UINT16(temperature, NvmeCtrl), + VMSTATE_UINT8(smart_critical_warning, NvmeCtrl), + + VMSTATE_UINT32(conf_msix_qsize, NvmeCtrl), + VMSTATE_UINT32(conf_ioqpairs, NvmeCtrl), + VMSTATE_UINT64(dbbuf_dbs, NvmeCtrl), + VMSTATE_UINT64(dbbuf_eis, NvmeCtrl), + VMSTATE_BOOL(dbbuf_enabled, NvmeCtrl), + + VMSTATE_UINT8(aer_mask, NvmeCtrl), + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT8_ALLOC( + aer_reqs, NvmeCtrl, outstanding_aers, 0, nvme_vmstate_request,= NvmeRequest), + VMSTATE_QTAILQ_V(aer_queue, NvmeCtrl, 1, nvme_vmstate_async_event, + NvmeAsyncEvent, entry), + VMSTATE_INT32(aer_queued, NvmeCtrl), + + VMSTATE_END_OF_LIST() + }, }; =20 static void nvme_class_init(ObjectClass *oc, const void *data) diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 457b6637249..9c5f53c688c 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -638,6 +638,7 @@ typedef struct NvmeCtrl { =20 NvmeNamespace namespace; NvmeNamespace *namespaces[NVME_MAX_NAMESPACES + 1]; + uint32_t num_queues; NvmeSQueue **sq; NvmeCQueue **cq; NvmeSQueue admin_sq; @@ -669,6 +670,7 @@ typedef struct NvmeCtrl { =20 /* Migration-related stuff */ Error *migration_blocker; + bool stop_processing_sq; } NvmeCtrl; =20 typedef enum NvmeResetType { diff --git a/hw/nvme/trace-events b/hw/nvme/trace-events index 6be0bfa1c1f..8e5544e0008 100644 --- a/hw/nvme/trace-events +++ b/hw/nvme/trace-events @@ -7,6 +7,17 @@ pci_nvme_dbbuf_config(uint64_t dbs_addr, uint64_t eis_addr= ) "dbs_addr=3D0x%"PRIx64 pci_nvme_map_addr(uint64_t addr, uint64_t len) "addr 0x%"PRIx64" len %"PRI= u64"" pci_nvme_map_addr_cmb(uint64_t addr, uint64_t len) "addr 0x%"PRIx64" len %= "PRIu64"" pci_nvme_map_prp(uint64_t trans_len, uint32_t len, uint64_t prp1, uint64_t= prp2, int num_prps) "trans_len %"PRIu64" len %"PRIu32" prp1 0x%"PRIx64" pr= p2 0x%"PRIx64" num_prps %d" +pci_nvme_pre_save_enter(void *n) "n=3D%p" +pci_nvme_pre_save_ns_drain(void *n, int i) "n=3D%p i=3D%d" +pci_nvme_pre_save_sq_out_req_drain_wait(void *n, int i, uint32_t head, uin= t32_t tail, uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRI= x32" size=3D0x%"PRIx32"" +pci_nvme_pre_save_sq_out_req_drain_wait_end(void *n, int i, uint32_t head,= uint32_t tail) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32"" +pci_nvme_pre_save_cq_req_drain_wait(void *n, int i, uint32_t head, uint32_= t tail, uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32"= size=3D0x%"PRIx32"" +pci_nvme_pre_save_cq_req_drain_wait_end(void *n, int i, uint32_t head, uin= t32_t tail) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32"" +pci_nvme_pre_save_aer(uint8_t typ, uint8_t info, uint8_t log_page) "type 0= x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" +pci_nvme_post_load_enter(void *n) "n=3D%p" +pci_nvme_post_load_restore_cq(void *n, int i, uint32_t head, uint32_t tail= , uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" size= =3D0x%"PRIx32"" +pci_nvme_post_load_restore_sq(void *n, int i, uint32_t head, uint32_t tail= , uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" size= =3D0x%"PRIx32"" +pci_nvme_post_load_aer(uint8_t typ, uint8_t info, uint8_t log_page) "type = 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" pci_nvme_map_sgl(uint8_t typ, uint64_t len) "type 0x%"PRIx8" len %"PRIu64"" pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode= , const char *opname) "cid %"PRIu16" nsid 0x%"PRIx32" sqid %"PRIu16" opc 0x= %"PRIx8" opname '%s'" pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char= *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" --=20 2.47.3