From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743370; cv=none; d=zohomail.com; s=zohoarc; b=m9RyN5b2FxQWAudhbpQSyOgticULH0WthkJb+8UOcaZZMFhPewWbk8h5P3OqzdWT4caDSYQtt5Qv/bGeO+AJrHW8QJ+5gzmE4mMwNyM3qXroC7DJ63H1FeGIEqQHPOdnOvYK/Xc/1WHD8UPG9Zlpu9FvH/jePFWsfqs+Ou9iqwI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743370; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=wa1h6U50bGiB8meyYuSt02vtuuLdduSF5Ef9ithEKUc=; b=XW3QdjpXpwEna848wKx2L/ftzazLC7Iqz0GZDvlmVrWRcRp2i868iVaAqFtlPTjldIJ9is9KkEFfojMZ02YqVWuZSS/FBjTUDs0i7N7/uUNZx1MOU63g9Kx92/693dA89rmsvIpN2Ery/DMdzgih+aVBPCr9EU9txebOCq1e36o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 177374337038872.22780395210009; Tue, 17 Mar 2026 03:29:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2ReR-00022j-Lp; Tue, 17 Mar 2026 06:27:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Rdx-0001vh-Cd for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:25 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rdr-0006VQ-Jw for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:19 -0400 Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-485392de558so33159435e9.1 for ; Tue, 17 Mar 2026 03:27:12 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743232; x=1774348032; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wa1h6U50bGiB8meyYuSt02vtuuLdduSF5Ef9ithEKUc=; b=S6YoQYoHs99WZnpbUgmzayF8tEd3ntAKBAo+1WqcnlSMk3zePp9gDGTTkQY8rul3J1 eeUtGnCUGTztY01FGgmPXxg/5QPKdnJi7hz5lU+wL0lI1M5+ZmNPwwpvfyYZxvHu9daL wKUn8YsQ2bTJnuBE4pHmHdXQ5rFra+/uvEBI4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743232; x=1774348032; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=wa1h6U50bGiB8meyYuSt02vtuuLdduSF5Ef9ithEKUc=; b=OmWEeAJyXN56s6NNiIcqq/X5au7I30k1zvqqwDDIjm0ILJmSUTnGtU4ZZzIaCPL+as C4R4auoIbxczwgljMjqWrQQrLW1RlfWPIOt2l4tfaHIplaaCPO3GtvCS/YskQYSUxmHO 00/9sqbAFaPNTapRK9j7otSjXK+vC+mlfg+sOV6OiunMkTLCkf7bTOXQqSK7aSLxoucb SqBF7P2HjYgqPMQolaGZ+BB/JC5GoKkF/PM+ktkKRVphTlp0RgUjO5YDKgOJrA4FTSEU meU+jCx0R3XjTVJ1As2A7ymRtbQF9ng/yTEk6edYUdgajTQfPqSUgT7hFSSrEmncO5AG 8C+A== X-Gm-Message-State: AOJu0YxSleTZ5Blkq2eTxfrR4PUeyqXTOzaD1hDNS72S784CUgiUQ12I lHVdOiU6nQ1eeeMMQEmwqJ1K+g4VnylxlKuV5zTp/Rzpapl8t8a85vBZJBJ9lfFNxIkasYn5DhP JnUre X-Gm-Gg: ATEYQzz00QYO9uJu378sLzsUz4cSTQMDwKd4kuA6aYndAAa94oM7pN3HFOtj/Taeu+f rDUgeAyFM1eBCeVFQ7ff3TWs9xpqivzumak5O+UwVOf1kmgtGiZgrYPz0xrDT+Uzdqg1kxnhmmJ LNdMIUe0fl68FnTPw2s45wlI3yrA8itjX9PSXrsRaRIctCfwifIM7fqT+chBx5wz9SIcJXgJUyr BGyHDXQIbtydPMwuX8GQZrOLP33uJmPZf5G4Ex8+dLng3lSeIe5e4vYkNxd869+JJYEioeG0FM9 fZLcsmeyhpLuj6gAaBB0K8kDsodcTDCYTZcTQmryPkmehjkglV2tZ/yWQy6ylnExcgQcZnXQY4Y H83mYUlE45lAVgzE0RtXCKbjryJueZ7/Zjs3sVuccR2CaWva3ehNvy3PfXdaAB/SpePzozxCqAF FuN/8raH6VHg9czNHfDiF/1KEMtzVbVf12BXoe6lrgffCL/Xjm8iMjtUx7uZ95djlxA+3BN4HS/ JL+Be9t8lnkjCLwK4vYYpA= X-Received: by 2002:a05:600c:4e08:b0:483:8062:b2f with SMTP id 5b1f17b1804b1-485566ca9b4mr264800365e9.6.1773743230899; Tue, 17 Mar 2026 03:27:10 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 1/8] migration/vmstate: export vmstate_{load, save}_field helpers Date: Tue, 17 Mar 2026 11:27:01 +0100 Message-ID: <20260317102708.126725-2-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743371790158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Let's export vmstate_{load,save}_field() helpers, they will be used in next patches to support fully-dynamic arrays with NULLs. Signed-off-by: Alexander Mikhalitsyn --- include/migration/vmstate.h | 6 ++++++ migration/vmstate.c | 10 +++++----- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index d4a39aa7944..7ed4a0742b2 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -1255,6 +1255,12 @@ extern const VMStateInfo vmstate_info_qlist; .flags =3D VMS_END, \ } =20 +bool vmstate_load_field(QEMUFile *f, void *pv, size_t size, + const VMStateField *field, Error **errp); +bool vmstate_save_field(QEMUFile *f, void *pv, size_t size, + const VMStateField *field, + JSONWriter *vmdesc, Error **errp); + /* * vmstate_load_state() and vmstate_save_state() are * depreacated, use vmstate_load_vmsd() and vmstate_save_vmsd() diff --git a/migration/vmstate.c b/migration/vmstate.c index e98b5f5346c..616eb310e61 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -161,8 +161,8 @@ static bool vmstate_pre_load(const VMStateDescription *= vmsd, void *opaque, return true; } =20 -static bool vmstate_load_field(QEMUFile *f, void *pv, size_t size, - const VMStateField *field, Error **errp) +bool vmstate_load_field(QEMUFile *f, void *pv, size_t size, + const VMStateField *field, Error **errp) { if (field->flags & VMS_STRUCT) { return vmstate_load_vmsd(f, field->vmsd, pv, field->vmsd->version_= id, @@ -485,9 +485,9 @@ static bool vmstate_pre_save(const VMStateDescription *= vmsd, void *opaque, return true; } =20 -static bool vmstate_save_field(QEMUFile *f, void *pv, size_t size, - const VMStateField *field, - JSONWriter *vmdesc, Error **errp) +bool vmstate_save_field(QEMUFile *f, void *pv, size_t size, + const VMStateField *field, + JSONWriter *vmdesc, Error **errp) { if (field->flags & VMS_STRUCT) { return vmstate_save_vmsd(f, field->vmsd, pv, vmdesc, errp); --=20 2.47.3 From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743274; cv=none; d=zohomail.com; s=zohoarc; b=lI4nqFxYSVlKJHP4kYcUjO9fZ9Y7f8fdxKamRzopnRTaDz4wGqv2kzRntD1tx2ZqhPOK4ctnPm4arHRziHZjH0HFTLtF1zOcbMJGbJGYwQfI+DpOL41TrZlm6jFqsMduFqNU6sF/kFeR5SddwnuXfPTFxst+jdh6tf0KZHHoOa4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743274; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=KZugiD/Qj6JgHPutbm5MIWrUpWCKjPH2a1AseJPIji4=; b=fGtMHp1q4zGwRS0bKT76J3AUiCnXvzKmhBu2x/hO7oefE2VRdGPVsDe8FV+AdL80aCR6pbBYPakK3TA4tqCfMm7/+9IMnqYpZKpYlAdcq0ovJNR8fFMVdglMabWymVHTF9wJVt+PXZCf1yr3QJnG3LUuWR/y/+/2cQfL80BSXMI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773743274111470.23440573862956; Tue, 17 Mar 2026 03:27:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2Re9-0001x7-Cn; Tue, 17 Mar 2026 06:27:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Rdu-0001vJ-Bp for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:18 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rdr-0006VZ-K0 for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:17 -0400 Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-4852f8ac7e9so66479655e9.1 for ; Tue, 17 Mar 2026 03:27:13 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743232; x=1774348032; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KZugiD/Qj6JgHPutbm5MIWrUpWCKjPH2a1AseJPIji4=; b=W8/bp033B8Zu8FjZrfy31phkagPxwRZzt36MErwqwJe85031jJHexLnKxWwC3FZpjs TByr01E+QDQgW2DBiwsUrEWsXUTxUKKGeKJVbL74F45lTZkUBkp+iloU1GpTCXuoJoDP /RjzQaZkbAo7TjJ0/QQJwomc8/ZeOmph70G04= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743232; x=1774348032; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=KZugiD/Qj6JgHPutbm5MIWrUpWCKjPH2a1AseJPIji4=; b=Z2A/SDN+za8DCeIUvEDD8KB2nAy4qYB5ByHfkns2KC3KzqjY3Y4ZpsDNkec5AzdyLN JB+FZyilNeQ/gX84czNe7OufAV77KlDFSJO2VucbYBK+t2bXZLVccGxgN/Bo/xGtzAi3 fk/YDInjls+DwU8SfgA+vCMW58FsXn+nJIb8IctB++miGhwjYgLZEObsh1c63BhuNRHx F6wmdMPZ+LSPwrUOeV/wHiAAa9rxoXNBch75MZaQEQKWi92Lwv5ViVYtRK+UXH0zW2FO jJUfbykpD+0QcKl/uDQfUCGbiRgPf0k1UZl5C6lFStuluctF1mvDuH3jvP/OIuQu7jW2 WjBw== X-Gm-Message-State: AOJu0Ywf+3evSFWAD79zQ+m8OHR3yydx0zkYLVVJlVzoKDxYhwQs0Zx+ /8ZRveaBlZiSXiYFViVP39CtDDkhuNrV/+oLhLSHK/jl9TobRQSF2UCDTa4d5UqjUjLFjfPzkZm nKEmQSqo= X-Gm-Gg: ATEYQzyizf2RnfkcRGsGyNEqLnXa+2u41CHXR7bwgfaq1ng5GLVXNOyQaLbwg0l0SPK 2jVPrIFKVV1+he0L0wwOn6Y5dV2/JbowSRJw3ypgg4AwFPY7V9lizOVnazXW5wVM4hGTiHGJO71 pz9LJjzzlaHoNJIMgg2be4jE00k9BYDTXX+rz2AksSWBcifwQVXf11q7HLQUhnVG34YQknzqaOR r+veLFACBLgvVgO/9aAAzeCb0wntP6eCQECRXs5GpmafnqUkwqbljbts/Wi6vF25xXV1+fxwo8A 44rRSB8tt8JGlMFhbTTmS2L64nbPKcJM/IoR5n7wusJ7m/AbCmMzOL9eJnsUwU0vbACvBOTyD8b gRMy+3J4wXgXg7JGxgpHjyaQnMJDlfedWRpRezSzm4i4smmdPxkTA66GF2qwIfZxBUBa6JoJjzk zj99K6e5pouexj17ZOa8yXJxx3G59X1c97TSoADsNaNuisSS7iHMdjxM3A8mIh2vMcLgAqkXKJ5 qd7kF8vd0mmRtmtPD5ripQ= X-Received: by 2002:a05:600c:45d5:b0:485:3ff1:d5c3 with SMTP id 5b1f17b1804b1-485566cfa3cmr259038945e9.5.1773743231990; Tue, 17 Mar 2026 03:27:11 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 2/8] migration: add VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT{8, 32}_ALLOC Date: Tue, 17 Mar 2026 11:27:02 +0100 Message-ID: <20260317102708.126725-3-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743276781154100 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Add VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT{8, 32}_ALLOC, which helps to save/restore a dynamic array of pointers to structures. Suggested-by: Peter Xu Signed-off-by: Alexander Mikhalitsyn v2: - added VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT8_ALLOC v4: - almost completely reworked, new flag VMS_ARRAY_OF_POINTER_ALLOW_NULL was introduced as suggested by Peter v5: - rebased on top of https://lore.kernel.org/all/20260304212303.667141-1-vse= mentsov@yandex-team.ru/ --- include/migration/vmstate.h | 77 ++++++++++++++++++++++++++++++- migration/savevm.c | 26 +++++++++++ migration/vmstate-types.c | 91 +++++++++++++++++++++++++++++++++++++ migration/vmstate.c | 54 ++++++++++++++++++---- 4 files changed, 236 insertions(+), 12 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 7ed4a0742b2..0a409700598 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -162,7 +162,19 @@ enum VMStateFlags { VMS_VSTRUCT =3D 0x8000, =20 /* Marker for end of list */ - VMS_END =3D 0x10000 + VMS_END =3D 0x10000, + + /* The field is a (fixed-size or variable-size) array of pointers + * (e.g. struct a { uint8_t **b; }) that can contain NULL values. + * This instructs vmstate engine to: + * - Dereference each array entry before using it. + * - Assume that array is initialized with NULLs on load phase + * - Automatically allocate memory for array entries (with size + * specified in (VMStateField).start) on load phase + * - Produce NULL/not-NULL markers in migration stream + * + * Note: Does not imply VMS_ARRAY_OF_POINTER; it needs to be set expli= citly. */ + VMS_ARRAY_OF_POINTER_ALLOW_NULL =3D 0x20000, }; =20 typedef enum { @@ -194,6 +206,7 @@ struct VMStateField { int version_id; int struct_version_id; bool (*field_exists)(void *opaque, int version_id); + const struct VMStateField *real_field; }; =20 struct VMStateDescription { @@ -268,8 +281,10 @@ extern const VMStateInfo vmstate_info_uint64; extern const VMStateInfo vmstate_info_fd; =20 /** Put this in the stream when migrating a null pointer.*/ -#define VMS_NULLPTR_MARKER (0x30U) /* '0' */ +#define VMS_NULLPTR_MARKER (0x30U) /* '0' */ +#define VMS_NOTNULLPTR_MARKER (0x31U) /* '1' */ extern const VMStateInfo vmstate_info_nullptr; +extern const VMStateInfo vmstate_info_maybeptr; =20 extern const VMStateInfo vmstate_info_cpudouble; =20 @@ -281,6 +296,7 @@ extern const VMStateInfo vmstate_info_bitmap; extern const VMStateInfo vmstate_info_qtailq; extern const VMStateInfo vmstate_info_gtree; extern const VMStateInfo vmstate_info_qlist; +extern const VMStateInfo vmstate_info_ptrs_array_entry; =20 #define type_check_2darray(t1,t2,n,m) ((t1(*)[n][m])0 - (t2*)0) /* @@ -562,6 +578,63 @@ extern const VMStateInfo vmstate_info_qlist; .offset =3D vmstate_offset_array(_s, _f, _type*, _n), \ } =20 +/* + * For migrating a dynamically allocated uint{8,32}-indexed array + * of pointers to structures (with NULL entries and with auto memory alloc= ation). + * + * _type: type of structure pointed to + * _vmsd: VMSD for structure _type (when VMS_STRUCT is set) + * _info: VMStateInfo for _type (when VMS_STRUCT is not set) + * start: size of (_type) pointed to (for auto memory allocation) + */ +#define VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT8_ALLOC(_field, _state, _f= ield_num, _version, _vmsd, _type) { \ + .name =3D (stringify(_field)), \ + .version_id =3D (_version), \ + .num_offset =3D vmstate_offset_value(_state, _field_num, uint8_t), \ + .vmsd =3D &(_vmsd), \ + .start =3D sizeof(_type), \ + .size =3D sizeof(_type *), \ + .flags =3D VMS_POINTER|VMS_VARRAY_UINT8|VMS_ARRAY_OF_POINTER| \ + VMS_ARRAY_OF_POINTER_ALLOW_NULL|VMS_STRUCT, \ + .offset =3D vmstate_offset_pointer(_state, _field, _type *), \ +} + +#define VMSTATE_VARRAY_OF_POINTER_UINT8_ALLOC(_field, _state, _field_num, = _version, _info, _type) { \ + .name =3D (stringify(_field)), \ + .version_id =3D (_version), \ + .num_offset =3D vmstate_offset_value(_state, _field_num, uint8_t), \ + .info =3D &(_info), \ + .start =3D sizeof(_type), \ + .size =3D sizeof(_type *), \ + .flags =3D VMS_POINTER|VMS_VARRAY_UINT8|VMS_ARRAY_OF_POINTER| \ + VMS_ARRAY_OF_POINTER_ALLOW_NULL, \ + .offset =3D vmstate_offset_pointer(_state, _field, _type *), \ +} + +#define VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC(_field, _state, _= field_num, _version, _vmsd, _type) { \ + .name =3D (stringify(_field)), \ + .version_id =3D (_version), \ + .num_offset =3D vmstate_offset_value(_state, _field_num, uint32_t), \ + .vmsd =3D &(_vmsd), \ + .start =3D sizeof(_type), \ + .size =3D sizeof(_type *), \ + .flags =3D VMS_POINTER|VMS_VARRAY_UINT32|VMS_ARRAY_OF_POINTER| \ + VMS_ARRAY_OF_POINTER_ALLOW_NULL|VMS_STRUCT, \ + .offset =3D vmstate_offset_pointer(_state, _field, _type *), \ +} + +#define VMSTATE_VARRAY_OF_POINTER_UINT32_ALLOC(_field, _state, _field_num,= _version, _info, _type) { \ + .name =3D (stringify(_field)), \ + .version_id =3D (_version), \ + .num_offset =3D vmstate_offset_value(_state, _field_num, uint32_t), \ + .info =3D &(_info), \ + .start =3D sizeof(_type), \ + .size =3D sizeof(_type *), \ + .flags =3D VMS_POINTER|VMS_VARRAY_UINT32|VMS_ARRAY_OF_POINTER| \ + VMS_ARRAY_OF_POINTER_ALLOW_NULL, \ + .offset =3D vmstate_offset_pointer(_state, _field, _type *), \ +} + #define VMSTATE_VARRAY_OF_POINTER_UINT32(_field, _state, _field_num, _vers= ion, _info, _type) { \ .name =3D (stringify(_field)), = \ .version_id =3D (_version), = \ diff --git a/migration/savevm.c b/migration/savevm.c index 8115203b518..882c882f684 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -868,6 +868,32 @@ static void vmstate_check(const VMStateDescription *vm= sd) =20 if (field) { while (field->name) { + /* + * VMS_ARRAY_OF_POINTER must be used only together + * with one of VMS_(V)ARRAY* flags. + */ + assert(!(field->flags & VMS_ARRAY_OF_POINTER) || + ((field->flags & (VMS_ARRAY | VMS_VARRAY_INT32 | + VMS_VARRAY_UINT16 | VMS_VARRAY_UINT8 | VMS_VARRAY_UIN= T32)))); + + /* + * When VMS_ARRAY_OF_POINTER_ALLOW_NULL is used, we must: + * 1. have VMS_ARRAY_OF_POINTER set too; + * 2. have ->start field set and it should tell us a size + * of memory chunk we should allocate for every array membe= r. + */ + assert(!(field->flags & VMS_ARRAY_OF_POINTER_ALLOW_NULL) || + (field->flags & VMS_ARRAY_OF_POINTER)); + assert(!(field->flags & VMS_ARRAY_OF_POINTER_ALLOW_NULL) || + field->start); + + /* + * (VMStateField).real_field is only for internal purposes + * and should never be used by any user-defined VMStateField. + * Currently, it is only used by vmsd_create_fake_nullptr_fiel= d(). + */ + assert(!field->real_field); + if (field->flags & (VMS_STRUCT | VMS_VSTRUCT)) { /* Recurse to sub structures */ vmstate_check(field->vmsd); diff --git a/migration/vmstate-types.c b/migration/vmstate-types.c index 23f34336964..a55f4d51f4b 100644 --- a/migration/vmstate-types.c +++ b/migration/vmstate-types.c @@ -391,6 +391,97 @@ const VMStateInfo vmstate_info_nullptr =3D { .save =3D save_nullptr, }; =20 +static bool load_maybeptr(QEMUFile *f, void *ppv, size_t unused_size, + const VMStateField *field, Error **errp) +{ + bool ok =3D false; + const VMStateField *real_field =3D field->real_field; + /* size of structure pointed to by elements of array */ + size_t size =3D real_field->start; + int marker; + + assert(size); + + if (ppv =3D=3D NULL) { + error_setg(errp, "vmstate: get_maybeptr must be called with ppv != =3D NULL"); + return false; + } + + /* + * We start from a clean array, all elements must be NULL, unless + * something we haven't prepared for has changed in vmstate_save_state= _v(). + * Let's check for this just in case. + */ + if (*(void **)ppv !=3D NULL) { + error_setg(errp, "vmstate: get_maybeptr must be called with *ppv = =3D=3D NULL"); + return false; + } + + marker =3D qemu_get_byte(f); + assert(marker =3D=3D VMS_NULLPTR_MARKER || marker =3D=3D VMS_NOTNULLPT= R_MARKER); + + if (marker =3D=3D VMS_NOTNULLPTR_MARKER) { + void *pv; + + /* allocate memory for structure */ + pv =3D g_malloc0(size); + + ok =3D vmstate_load_field(f, pv, size, real_field, errp); + if (!ok) { + g_free(pv); + return false; + } + + *(void **)ppv =3D pv; + } + + return true; +} + +static bool save_maybeptr(QEMUFile *f, void *ppv, size_t unused_size, + const VMStateField *field, JSONWriter *vmdesc, + Error **errp) +{ + const VMStateField *real_field =3D field->real_field; + /* size of structure pointed to by elements of array */ + size_t size =3D real_field->start; + void *pv; + + assert(size); + + /* + * (ppv) is an address of an i-th element of a dynamic array. + * + * (ppv) can not be NULL unless we have some regression/bug in + * vmstate_save_state_v(), because it is result of pointer arithemic l= ike: + * first_elem + size * i. + */ + if (ppv =3D=3D NULL) { + error_setg(errp, "vmstate: put_maybeptr must be called with ppv != =3D NULL"); + return false; + } + + /* get a pointer to a structure */ + pv =3D *(void **)ppv; + + if (pv =3D=3D NULL) { + /* write a mark telling that there was a NULL pointer */ + qemu_put_byte(f, VMS_NULLPTR_MARKER); + return true; + } + + /* if pv is not NULL, write a marker and save field using vmstate_save= _field() */ + qemu_put_byte(f, VMS_NOTNULLPTR_MARKER); + + return vmstate_save_field(f, pv, size, real_field, vmdesc, errp); +} + +const VMStateInfo vmstate_info_maybeptr =3D { + .name =3D "maybeptr", + .load =3D load_maybeptr, + .save =3D save_maybeptr, +}; + /* 64 bit unsigned int. See that the received value is the same than the o= ne in the field */ =20 diff --git a/migration/vmstate.c b/migration/vmstate.c index 616eb310e61..29e63751105 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -74,10 +74,15 @@ vmsd_create_fake_nullptr_field(const VMStateField *fiel= d) /* Do not need "field_exists" check as it always exists (which is null= ) */ fake->field_exists =3D NULL; =20 - /* See vmstate_info_nullptr - use 1 byte to represent nullptr */ - fake->size =3D 1; - fake->info =3D &vmstate_info_nullptr; - fake->flags =3D VMS_SINGLE; + if (!(field->flags & VMS_ARRAY_OF_POINTER_ALLOW_NULL)) { + /* See vmstate_info_nullptr - use 1 byte to represent nullptr */ + fake->size =3D 1; + fake->info =3D &vmstate_info_nullptr; + fake->flags =3D VMS_SINGLE; + } else { + fake->real_field =3D field; + fake->info =3D &vmstate_info_maybeptr; + } =20 /* All the rest fields shouldn't matter.. */ =20 @@ -258,13 +263,28 @@ bool vmstate_load_vmsd(QEMUFile *f, const VMStateDesc= ription *vmsd, for (i =3D 0; i < n_elems; i++) { bool ok; void *curr_elem =3D first_elem + size * i; + bool need_fake_field =3D false; const VMStateField *inner_field; =20 if (field->flags & VMS_ARRAY_OF_POINTER) { - curr_elem =3D *(void **)curr_elem; + if (!(field->flags & VMS_ARRAY_OF_POINTER_ALLOW_NULL))= { + assert(curr_elem); + curr_elem =3D *(void **)curr_elem; + need_fake_field =3D !curr_elem; + } else { + /* + * We expect array of pointers to be initialized. + * We don't want to overwrite curr_elem with it's + * dereferenced value, because we may need to + * allocate memory (depending on what is in the mi= gration + * stream) and write to it later. + */ + assert(!*(void **)curr_elem); + need_fake_field =3D true; + } } =20 - if (!curr_elem && size) { + if (need_fake_field && size) { /* * If null pointer found (which should only happen in * an array of pointers), use null placeholder and do @@ -272,6 +292,7 @@ bool vmstate_load_vmsd(QEMUFile *f, const VMStateDescri= ption *vmsd, */ inner_field =3D vmsd_create_fake_nullptr_field(field); } else { + assert(curr_elem || !size); inner_field =3D field; } =20 @@ -546,25 +567,38 @@ static bool vmstate_save_vmsd_v(QEMUFile *f, const VM= StateDescription *vmsd, =20 for (i =3D 0; i < n_elems; i++) { void *curr_elem =3D first_elem + size * i; + bool need_fake_field =3D false; const VMStateField *inner_field; bool is_null; int max_elems =3D n_elems - i; =20 old_offset =3D qemu_file_transferred(f); if (field->flags & VMS_ARRAY_OF_POINTER) { - assert(curr_elem); - curr_elem =3D *(void **)curr_elem; + if (!(field->flags & VMS_ARRAY_OF_POINTER_ALLOW_NULL))= { + assert(curr_elem); + curr_elem =3D *(void **)curr_elem; + need_fake_field =3D !curr_elem; + } else { + /* + * We always need a fake field to properly handle + * VMS_ARRAY_OF_POINTER_ALLOW_NULL case, because + * even if pointer is not NULL, we still want to + * write a marker in the migration stream. + */ + need_fake_field =3D true; + } } =20 - if (!curr_elem && size) { + if (need_fake_field && size) { /* * If null pointer found (which should only happen in * an array of pointers), use null placeholder and do * not follow. */ inner_field =3D vmsd_create_fake_nullptr_field(field); - is_null =3D true; + is_null =3D !curr_elem; } else { + assert(curr_elem || !size); inner_field =3D field; is_null =3D false; } --=20 2.47.3 From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743383; cv=none; d=zohomail.com; s=zohoarc; b=DkyAlNYgw6o4qmLbxbxqZdPjRgHx8cJ+fkb6sAVYtjdOKsBfrlEHpuaNTbOelP3c94C/Kw2n4PgTV6zisvRcMd4LtioHPUfO34aHf5M5i+edUHOy6RodpJkieey2bhK/dKMJVofZh1gS7e0GT4of2vSMZrjtHYaaG+5h3EAi21w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743383; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tVGscBXdfGh8fDEwqpswNHuWUNoSPZVqRG0/M7ov0/8=; b=KGguo4W2sIPxfA9+rHVawuE8RixHdt24Cm0SR8/dPq3lI8WgzRi4henNcTvWZAtBn2qsumApxuwH+EAhWZkRM5LMxkyoJOktANxB/l9l3WSyF+KQc65EGeLeTXaiV3V3WyR7GHtBvmRB23F5c1J8KtQHFUW7EuGbprX8cmdB67c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773743383160920.6146388156253; Tue, 17 Mar 2026 03:29:43 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2ReR-00021I-CM; Tue, 17 Mar 2026 06:27:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Rdx-0001vg-CD for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:22 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rdr-0006Vs-Jk for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:18 -0400 Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-48334ee0aeaso48942005e9.1 for ; Tue, 17 Mar 2026 03:27:14 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743233; x=1774348033; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tVGscBXdfGh8fDEwqpswNHuWUNoSPZVqRG0/M7ov0/8=; b=X48yjoPjfBdx4KRlgS0Zn4GjSEvTMREBi0L5KN2CIIxfZY8c2mZjqvfOyC5nfgcrUL IKwP1sG3bR00VYAine5Ft2Bc0vsqefqQ0o3vHWw/rud+zF+DQxFKmNm0GZFqOZAoZmMA NIlLMwGBXrt0HdDidcKRnVwQQhfe4MdjyfyEQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743233; x=1774348033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=tVGscBXdfGh8fDEwqpswNHuWUNoSPZVqRG0/M7ov0/8=; b=IUAUElQG8eQ0d0cPJLjodhG+cWpC38Oi2/jwmq9KjQ/HkNGyPAeYBxr3AbX0e2/86g TmGPRKYD777RKhVwnNHOuyFn2gpt+MnvPrVbp3Ba4ZgFYdpWahlsDYwlcHriWZi5bkdN s8oNIjzoiSFcogxpyi0dZtbQsixibf6bFDj8azSDb5zHahJfIyUkoxECfIgkfvJvDH+O +tdjDfUU6RMppoZ7uFlKypqh5RLcc8o+lDkK7bKXvIhN47xhkZDRx5SsH9UYNdaB94N7 AJjjOths3NBNMVRvDA7Qz+6j+y9sD/vxWHw402+RaUb6OkOJ/v5iCGjnxkFx2hKax255 eLxw== X-Gm-Message-State: AOJu0YyChrWLs66LvX0UaFBiLsd0yu1iLeY1s6PGGTUvdYR9+RqAL7ck WaQz4qyl/bOFKTUgqoU73yPwNinSj5czvz1dmT8FfN92eWCVic5BeNlWeJ9FDxzbcH3a4wNnNhe Sb6+00Ls= X-Gm-Gg: ATEYQzz2uFD5Cc3PmwYz5jEb1CzGiUjSlkXIcFHQpeZMf0nK2PVGpqTKlQVyv/5LRkj jOA4EDQ0ymXFeO94YzZ5WdeHsssEsxJcDXhf6RgA40Pc3cGSJSSoreIvZtk5lo65fEkh9njLOXN gkI62fDsxZNT0Xa3NMThMPZ8ZLkMWRVh8fhkC8570vAVeH1ObhKAHGrtp2NXXebHJJc+Vf98FHm qBQVg3rjj1mPayvXn6BD7D/CoPDOi8fp50j74L3i0FzI0CS4YLRMtaMGpRx4VRB6JIaxrFlpmeB acsJ7oGNKFulNrdQYPEHJgpMS9k/O7ymdb3RMizCFDqPacRf0ZMGjgj2mscfz6MxNFT3V+PK7N+ RXPzZwwdTt+TjlXFDZvFeCh6s09jd33V78oO515Eb36rVO0EFfQLZFwclCHdyGtbYFr78VBlmY2 47JUpYf0UPU3oh4yGT7jwzhGeYkdAzWrvoj0WFKA17ALDEVgZHqswmXpwLpVrsFlmIPeUCq0Zio b8KDoFXU249BKNgojpKEQo= X-Received: by 2002:a05:600c:1913:b0:483:6a8d:b2f9 with SMTP id 5b1f17b1804b1-485566e2d50mr277093135e9.5.1773743233329; Tue, 17 Mar 2026 03:27:13 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 3/8] tests/unit/test-vmstate: add tests for VMS_ARRAY_OF_POINTER_ALLOW_NULL Date: Tue, 17 Mar 2026 11:27:03 +0100 Message-ID: <20260317102708.126725-4-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743385969154100 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Add tests for VMSTATE_VARRAY_OF_POINTER{,_TO_STRUCT}_UINT32_ALLOC. Signed-off-by: Alexander Mikhalitsyn --- tests/unit/test-vmstate.c | 157 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 157 insertions(+) diff --git a/tests/unit/test-vmstate.c b/tests/unit/test-vmstate.c index cadbab3c5e2..d10cf34fc75 100644 --- a/tests/unit/test-vmstate.c +++ b/tests/unit/test-vmstate.c @@ -702,6 +702,155 @@ static void test_arr_ptr_prim_0_load(void) } } =20 +static uint8_t wire_arr_ptr_with_nulls[] =3D { + VMS_NOTNULLPTR_MARKER, + 0x00, 0x00, 0x00, 0x00, + VMS_NULLPTR_MARKER, + VMS_NOTNULLPTR_MARKER, + 0x00, 0x00, 0x00, 0x02, + VMS_NOTNULLPTR_MARKER, + 0x00, 0x00, 0x00, 0x03, + QEMU_VM_EOF +}; + +typedef struct { + uint32_t ar_items_num; + TestStructTriv **ar; +} TestVArrayOfPtrToStuctWithNULLs; + +const VMStateDescription vmsd_arps_with_nulls =3D { + .name =3D "test/arps_with_nulls", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC( + ar, TestVArrayOfPtrToStuctWithNULLs, ar_items_num, 0, vmsd_tst= , TestStructTriv), + VMSTATE_END_OF_LIST() + } +}; + +static void test_arr_ptr_nulls_str_save(void) +{ + TestStructTriv ar[AR_SIZE] =3D { {.i =3D 0}, {.i =3D 1}, {.i =3D 2}, {= .i =3D 3} }; + TestVArrayOfPtrToStuctWithNULLs sample =3D {}; + int idx; + + sample.ar_items_num =3D AR_SIZE; + sample.ar =3D g_new0(TestStructTriv*, sample.ar_items_num); + sample.ar[0] =3D g_new0(TestStructTriv, 1); + *sample.ar[0] =3D ar[0]; + /* note, sample.ar[1] remains NULL */ + sample.ar[2] =3D g_new0(TestStructTriv, 1); + *sample.ar[2] =3D ar[2]; + sample.ar[3] =3D g_new0(TestStructTriv, 1); + *sample.ar[3] =3D ar[3]; + + save_vmstate(&vmsd_arps_with_nulls, &sample); + compare_vmstate(wire_arr_ptr_with_nulls, sizeof(wire_arr_ptr_with_null= s)); + + for (idx =3D 0; idx < AR_SIZE; ++idx) { + g_free(sample.ar[idx]); + } + g_free(sample.ar); +} + +static void test_arr_ptr_nulls_str_load(void) +{ + TestStructTriv ar_gt[AR_SIZE] =3D {{.i =3D 0}, {.i =3D 0}, {.i =3D 2},= {.i =3D 3} }; + TestVArrayOfPtrToStuctWithNULLs obj =3D {}; + int idx; + + obj.ar_items_num =3D AR_SIZE; + obj.ar =3D g_new0(TestStructTriv*, obj.ar_items_num); + + save_buffer(wire_arr_ptr_with_nulls, sizeof(wire_arr_ptr_with_nulls)); + SUCCESS(load_vmstate_one(&vmsd_arps_with_nulls, &obj, 1, + wire_arr_ptr_with_nulls, sizeof(wire_arr_ptr_wit= h_nulls))); + + for (idx =3D 0; idx < AR_SIZE; ++idx) { + if (idx =3D=3D 1) { + g_assert_cmpint((uintptr_t)(obj.ar[idx]), =3D=3D, 0); + } else { + /* compare the target array ar with the ground truth array ar_= gt */ + g_assert_cmpint(ar_gt[idx].i, =3D=3D, obj.ar[idx]->i); + } + } + + for (idx =3D 0; idx < AR_SIZE; ++idx) { + g_free(obj.ar[idx]); + } + g_free(obj.ar); +} + +typedef struct { + uint32_t ar_items_num; + int32_t **ar; +} TestVArrayOfPtrToIntWithNULLs; + +const VMStateDescription vmsd_arpp_with_nulls =3D { + .name =3D "test/arpp_with_nulls", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_VARRAY_OF_POINTER_UINT32_ALLOC( + ar, TestVArrayOfPtrToIntWithNULLs, ar_items_num, 0, vmstate_in= fo_int32, int32_t), + VMSTATE_END_OF_LIST() + } +}; + +static void test_arr_ptr_nulls_prim_save(void) +{ + int32_t ar[AR_SIZE] =3D { 0, 1, 2, 3 }; + TestVArrayOfPtrToIntWithNULLs sample =3D {}; + int idx; + + sample.ar_items_num =3D AR_SIZE; + sample.ar =3D g_new0(int32_t*, sample.ar_items_num); + sample.ar[0] =3D g_new0(int32_t, 1); + *sample.ar[0] =3D ar[0]; + /* note, sample.ar[1] remains NULL */ + sample.ar[2] =3D g_new0(int32_t, 1); + *sample.ar[2] =3D ar[2]; + sample.ar[3] =3D g_new0(int32_t, 1); + *sample.ar[3] =3D ar[3]; + + save_vmstate(&vmsd_arpp_with_nulls, &sample); + compare_vmstate(wire_arr_ptr_with_nulls, sizeof(wire_arr_ptr_with_null= s)); + + for (idx =3D 0; idx < AR_SIZE; ++idx) { + g_free(sample.ar[idx]); + } + g_free(sample.ar); +} + +static void test_arr_ptr_nulls_prim_load(void) +{ + int32_t ar_gt[AR_SIZE] =3D { 0, 0, 2, 3 }; + TestVArrayOfPtrToIntWithNULLs obj =3D {}; + int idx; + + obj.ar_items_num =3D AR_SIZE; + obj.ar =3D g_new0(int32_t*, obj.ar_items_num); + + save_buffer(wire_arr_ptr_with_nulls, sizeof(wire_arr_ptr_with_nulls)); + SUCCESS(load_vmstate_one(&vmsd_arpp_with_nulls, &obj, 1, + wire_arr_ptr_with_nulls, sizeof(wire_arr_ptr_wit= h_nulls))); + + for (idx =3D 0; idx < AR_SIZE; ++idx) { + if (idx =3D=3D 1) { + g_assert_cmpint((uintptr_t)(obj.ar[idx]), =3D=3D, 0); + } else { + /* compare the target array ar with the ground truth array ar_= gt */ + g_assert_cmpint(ar_gt[idx], =3D=3D, *obj.ar[idx]); + } + } + + for (idx =3D 0; idx < AR_SIZE; ++idx) { + g_free(obj.ar[idx]); + } + g_free(obj.ar); +} + /* test QTAILQ migration */ typedef struct TestQtailqElement TestQtailqElement; =20 @@ -1568,6 +1717,14 @@ int main(int argc, char **argv) test_arr_ptr_prim_0_save); g_test_add_func("/vmstate/array/ptr/prim/0/load", test_arr_ptr_prim_0_load); + g_test_add_func("/vmstate/array/ptr-nulls/str/save", + test_arr_ptr_nulls_str_save); + g_test_add_func("/vmstate/array/ptr-nulls/str/load", + test_arr_ptr_nulls_str_load); + g_test_add_func("/vmstate/array/ptr-nulls/prim/save", + test_arr_ptr_nulls_prim_save); + g_test_add_func("/vmstate/array/ptr-nulls/prim/load", + test_arr_ptr_nulls_prim_load); g_test_add_func("/vmstate/qtailq/save/saveq", test_save_q); g_test_add_func("/vmstate/qtailq/load/loadq", test_load_q); g_test_add_func("/vmstate/gtree/save/savedomain", test_gtree_save_doma= in); --=20 2.47.3 From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743320; cv=none; d=zohomail.com; s=zohoarc; b=VSXUMZ4lvR9zMnmXHaKOTVePdArY+aVW91gmemYfDk8leK2NDzW/j2yGJV4C5JGEhvKV6Oq2FDbKPAsZP3byHANeY5RDNNIqJOFg4fjPzFDRysH1+Svy7XYgOcdiuXLFUyS+hatZAQak8PgZMbrrIWKNYLWNP9GH74YoyBl4cs0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743320; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2mdV95Xn1uRZNv+wMaqt5uMFd58yUD2RuxYvZm/pTuU=; b=GzmaayT2JFw9es4gjwcm/WwdbeGJPUcQK4onfdZooEcf6y/KfPtN0LddhzouCW1aRAoK0mnryht0l1c2+0LrecIAami/0/FqvnZ2XAcaGmCHglYK5HYQtfkjnPKVMrQSdwgyeJLH3byfoXhScojbVltqupzpmwx6O/qr4itPaXY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773743320446125.64411228640847; Tue, 17 Mar 2026 03:28:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2ReS-00023J-50; Tue, 17 Mar 2026 06:27:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Rdx-0001va-B8 for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:22 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rds-0006W2-2z for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:19 -0400 Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-4853c3c2fe7so32668925e9.0 for ; Tue, 17 Mar 2026 03:27:15 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743235; x=1774348035; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2mdV95Xn1uRZNv+wMaqt5uMFd58yUD2RuxYvZm/pTuU=; b=BT4WZMfRk2VhKDke9wpMnwkiEAhVQjrvX673zFan8dCTyGOic7FpJ+2lXVbEa+mxqL D1g7JRPQM9LP261LM0SaGmTp+pWjiqoEzOt4HPr6Tei8UjsTiW2b/6SA+gvFqvn1Fo6p olnxGzhofadI3j7E1KwfIcNU5PaKBJjNre2bw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743235; x=1774348035; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=2mdV95Xn1uRZNv+wMaqt5uMFd58yUD2RuxYvZm/pTuU=; b=FYBvKLTa1vMNv93MMiZ/lZt5iJWxpQTMnNq0OTcQqQkA8L8Z3GLBcDeS1sS2DEbWes YGxY7kl7sw08askjuMu6HH/gmLGNEiUEIcPVk2ERbrpDRzIFAZYcCrU/oMPNhQ1mkU9j x1DSvZxuk+6pvSP65x+UB77AF4lvxeVjjmj/fmiRwCqPL+mz+HKefOoSa1GkgNJ+aVx2 FjZYsAGrnakzF4vnQ1eACe+FiNLu5RWVpaSemys0ODV/2Y8WxFpVPdHDNIxFTH2vel+P jolKkiYtO1fBI3F6flrC03iJ9L77uuHiExELSeNjk3zz6ORzgohwHCzq2EDoh3QnDRqD aq6g== X-Gm-Message-State: AOJu0YyrZldsZ87jBaXlX3S3gSd5movGkQc9OYcZWhlL6WhZjRwOS2e5 e7uIBVBNrPf2zZaDGJ2aAZg0w4ihFhkQ2U2LHIPAOZas2J0HB/7jk959/njTDFwn+EYN0e5WueT c56Rxefk= X-Gm-Gg: ATEYQzyGeBjg+McFgFaRKt6q+O4InedLnoYZ2YeF3lZNxn9k9EmnwxEvpB7DLM0pQql 5ni7IW4qKkwUdlqMs3cn4s8qcaF9YqYmrxJO8Yot6M8oyGqfwIRcc1Mxm3a8DFqyi7Ahf0IkiSR Fd9fyRTFdBtg5M6ZUSRjh/+esjibCoafI+z8ErtJwhrSq71dfGhIxpwyaasBAaZbWjTje9gjR7c IihCZ/fttTwXG52eDNq9yZkp0Lt/d0juFyxXIH3X4Fmrb857a6QJ0WzI70jXGTh9gwj1aasW49d OsuOx7FJ8Pe/1rpHhfxMSfUV5IvlJSfP8wkyzG7Sf1dQPCw1TOMQHU2o+gGlYMWdSQyWvu/yv6n D5pKDSvdbxexX2Xf1YWH+EHtoxX871lfpBZwwrvk82rfykUtGHULC0GElN2YcAyJV6dHn6q/7fG h4JQ7IVdjqQCAgAKJpAFP5qK3TUBl1yuw/k+U3tTY4zUACjcGIYzLxzQKKA06G3cgPmzi/GDiq/ UBXj+VkNVvkTOYKxO8kXE0= X-Received: by 2002:a05:600d:8488:10b0:480:3ad0:93bf with SMTP id 5b1f17b1804b1-485567060a1mr198689335e9.24.1773743234402; Tue, 17 Mar 2026 03:27:14 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 4/8] tests/functional/migration: add VM launch/configure hooks Date: Tue, 17 Mar 2026 11:27:04 +0100 Message-ID: <20260317102708.126725-5-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743323647158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Introduce configure_machine, launch_source_vm and assert_dest_vm methods to allow child classes to override some pieces of source/dest VMs creation, start and check logic. Signed-off-by: Alexander Mikhalitsyn --- tests/functional/migration.py | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/tests/functional/migration.py b/tests/functional/migration.py index e995328e833..40e9c094c46 100644 --- a/tests/functional/migration.py +++ b/tests/functional/migration.py @@ -40,19 +40,35 @@ def assert_migration(self, src_vm, dst_vm): self.assertEqual(dst_vm.cmd('query-status')['status'], 'running') self.assertEqual(src_vm.cmd('query-status')['status'],'postmigrate= ') =20 + # Can be overridden by subclasses to configure both source/dest VMs. + def configure_machine(self, vm): + vm.add_args('-nodefaults') + + # Can be overridden by subclasses to prepare the source VM before migr= ation, + # e.g. by running some workload inside the source VM to see if it cont= inues + # to run properly after migration. + def launch_source_vm(self, vm): + vm.launch() + + # Can be overridden by subclasses to check the destination VM after mi= gration, + # e.g. by checking if the workload is still running after migration. + def assert_dest_vm(self, vm): + pass + def migrate_vms(self, dst_uri, src_uri, dst_vm, src_vm): dst_vm.qmp('migrate-incoming', uri=3Ddst_uri) src_vm.qmp('migrate', uri=3Dsrc_uri) self.assert_migration(src_vm, dst_vm) + self.assert_dest_vm(dst_vm) =20 def migrate(self, dst_uri, src_uri=3DNone): dst_vm =3D self.get_vm('-incoming', 'defer', name=3D"dst-qemu") - dst_vm.add_args('-nodefaults') + self.configure_machine(dst_vm) dst_vm.launch() =20 src_vm =3D self.get_vm(name=3D"src-qemu") - src_vm.add_args('-nodefaults') - src_vm.launch() + self.configure_machine(src_vm) + self.launch_source_vm(src_vm) =20 if src_uri is None: src_uri =3D dst_uri --=20 2.47.3 From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743400; cv=none; d=zohomail.com; s=zohoarc; b=SgOeJp0VtE2EcGxYb1AtWdiyRg6ubUWG90QZ+rGfox7xvZdhXXEq2wp+WGPllpxCydzTdBPu3dc7cJNuVyl5Avjpoaq4Ver1856crFLgVYQOcqisPKwgtDaDiWs5XXy3NgXohM9PAezdZlvPeNVmYP4mLxpm++TEDxGCUbDInxQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743400; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SEO6/lrYdm4dOs+1aF7CdwIzHMn2tXQZYpY2v/1+t2k=; b=HCdTgvz9vvqwUHurvgmVgeG/ZnrkUX/Lp5qp3z+aU9uEwJ+tHHkBjXRY/lBHxwROqT6LCbG/jDLWzSoqFCTbBokUlk0OucsdrqFBOWg0O3OthVyFKpGpc4EJAjf7bwChTEhz4YNda6xbjVjLKK+V0LNrqWQfEJcRQO2Af3acDZs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773743400714804.0565034131673; Tue, 17 Mar 2026 03:30:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2ReQ-000219-Il; Tue, 17 Mar 2026 06:27:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Rdx-0001vf-C8 for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:22 -0400 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rdt-0006WC-Jr for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:21 -0400 Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-48535a0ef86so46447755e9.1 for ; Tue, 17 Mar 2026 03:27:17 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743236; x=1774348036; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SEO6/lrYdm4dOs+1aF7CdwIzHMn2tXQZYpY2v/1+t2k=; b=TKfFQDo0Eg1KJ2wJSfghTebvstlxZP+w8F36V9LPfJECGaM4V1kwujvj3DhgKc+nGf r/bOnBvertx1J0LWlqZSZ0a5sBJHeNKWPsUEcAV2sFbcA40nKz6oHxG9Tbj8bAiHKZXx VRso9IoLeYGZWjhwUfkjH0l71Q5lgqRGF8xUY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743236; x=1774348036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=SEO6/lrYdm4dOs+1aF7CdwIzHMn2tXQZYpY2v/1+t2k=; b=lk6/T8bkA1v4h+n6E63gJBzz0Lt5p1DSpYXssTl0PJGpLFowquCmevQa37SlnDQcVS WHJoRYDhbeOybu3CMCuqVXUMGOJbsN1F3hNAq8WYENVjhjc4B3Af/U0Bhsemoiu3GpTS nuRaoEl1ZcRsk5g0Zl90bseXyOU0kDD/7ZoL9gP82iuZWmNtq2dUT7czhDDc6wpIqid4 VkXWvs0UHxAKAeoBSMoOnT466ImFT0qfB3oVBWtiQGar1QYP0vhPTkSZonOO4W6eUUfh 1p6IyBT0fiLxUBRl+G9QWkSEOPnJRMg/K9hV4OUv/zd8sFkudgLvZqQMad/uLDpiy+q0 iGJg== X-Gm-Message-State: AOJu0Yy34iTL66ChUSIvYzCcZgtLXcZF9/7oTWP/pYdIDhNmihB04XOz AxvBnPovt0tR8e7Wtcqi5UxClUzDZDyzB0nHDhbZHsesmTsQr2Ri6Nmgi7ndaaZiF3VOMgVgVhK JUFt1OEw= X-Gm-Gg: ATEYQzwzdGP/msmPSWhILKPryEdNAibT5lI+qlEnCdEiNUcZ0cg/lTg8tgdROOsGcOO mNCblH5XPdmE8JU18umaW7fWEyP0ZCMcp2H7WTZGjOBksBAqx7UGnZoQjriyLOpYW7BUnsjbdki 4U8vwDrZkXjaq00i2AURqXNamzCYV3iagFN2p+FD91KG/fLJYCVFSx1ykm2AqvqrxczXeeYQbRv JpXgIhTaXXDgp51rjNYsLny++xufh31P/O/lEG0kUWvKeJCg60+8bB/pq7e4Q+Ifl1PBLxk6W8A X/3fdupYxB0ZM5ILH8CbNUnovOwKM8D4G3TokwNiWrvWG1WJeetOARK4IvvfonNh4+1h41k5tQF XL+yOaHdeEFuwtMcx8xUX+22hS3j73+x6KHiBP5ForSGRvwsaya3kFBpXA23sJJxVvcHwEecQz/ QRvheJr4Wz04eOzcae2Wk3vB98zKnJ4B1JjrFwHpvRtD2aVuWAxS3ctWQSRAfw/Stx+CNsg+SVu IjsdRt2s2oxRVXIWkKyiG0= X-Received: by 2002:a05:600c:4ed1:b0:485:3d43:7c9a with SMTP id 5b1f17b1804b1-4855670e6dcmr285492505e9.25.1773743235525; Tue, 17 Mar 2026 03:27:15 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 5/8] hw/nvme: add migration blockers for non-supported cases Date: Tue, 17 Mar 2026 11:27:05 +0100 Message-ID: <20260317102708.126725-6-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::335; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743402187158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Let's block migration for cases we don't support: - SR-IOV - CMB - PMR - SPDM No functional changes here, because NVMe migration is not supported at all as of this commit. Signed-off-by: Alexander Mikhalitsyn --- hw/nvme/ctrl.c | 206 +++++++++++++++++++++++++++++++++++++++++++ hw/nvme/nvme.h | 3 + include/block/nvme.h | 12 +++ 3 files changed, 221 insertions(+) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index cc4593cd427..9f9c9bcbead 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -207,6 +207,7 @@ #include "hw/pci/msix.h" #include "hw/pci/pcie_sriov.h" #include "system/spdm-socket.h" +#include "migration/blocker.h" #include "migration/vmstate.h" =20 #include "nvme.h" @@ -250,6 +251,7 @@ static const bool nvme_feature_support[NVME_FID_MAX] = =3D { [NVME_COMMAND_SET_PROFILE] =3D true, [NVME_FDP_MODE] =3D true, [NVME_FDP_EVENTS] =3D true, + /* if you add something here, please update nvme_set_migration_blocker= s() */ }; =20 static const uint32_t nvme_feature_cap[NVME_FID_MAX] =3D { @@ -4601,6 +4603,7 @@ static uint16_t nvme_io_mgmt_send(NvmeCtrl *n, NvmeRe= quest *req) return 0; case NVME_IOMS_MO_RUH_UPDATE: return nvme_io_mgmt_send_ruh_update(n, req); + /* if you add something here, please update nvme_set_migration_blocker= s() */ default: return NVME_INVALID_FIELD | NVME_DNR; }; @@ -7518,6 +7521,10 @@ static uint16_t nvme_security_receive(NvmeCtrl *n, N= vmeRequest *req) =20 static uint16_t nvme_directive_send(NvmeCtrl *n, NvmeRequest *req) { + /* + * When adding a new dtype handling here, + * please also update nvme_set_migration_blockers(). + */ return NVME_INVALID_FIELD | NVME_DNR; } =20 @@ -9208,6 +9215,199 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *= pci_dev) } } =20 +#define BLOCKER_FEATURES_MAX_LEN 256 + +static inline void nvme_add_blocker_feature(char *blocker_features, const = char *feature) +{ + if (strlen(blocker_features) > 0) { + g_strlcat(blocker_features, ", ", BLOCKER_FEATURES_MAX_LEN); + } + g_strlcat(blocker_features, feature, BLOCKER_FEATURES_MAX_LEN); +} + +static bool nvme_set_migration_blockers(NvmeCtrl *n, PCIDevice *pci_dev, E= rror **errp) +{ + uint64_t unsupported_cap, cap =3D ldq_le_p(&n->bar.cap); + char blocker_features[BLOCKER_FEATURES_MAX_LEN] =3D ""; + bool adm_cmd_security_checked =3D false; + bool cmd_io_mgmt_checked =3D false; + bool cmd_zone_checked =3D false; + + /* + * Idea of this function is simple, we iterate over all Command Sets a= nd + * for each supported command we provide a special handling logic to + * determine if we should block migration or not. + * + * For instance, we have NVME_ADM_CMD_NS_ATTACHMENT and it is always + * available to the guest, but if there is only 1 namespace, then it is + * safe to allow migration, but if there are more, then we need to blo= ck + * migration because we don't handle this in migration code yet. + */ + for (int opcode =3D 0; opcode < sizeof(n->cse.acs) / sizeof(n->cse.acs= [0]); opcode++) { + /* Is command supported? */ + if (!n->cse.acs[opcode]) { + continue; + } + + switch (opcode) { + case NVME_ADM_CMD_DELETE_SQ: + case NVME_ADM_CMD_CREATE_SQ: + case NVME_ADM_CMD_GET_LOG_PAGE: + case NVME_ADM_CMD_DELETE_CQ: + case NVME_ADM_CMD_CREATE_CQ: + case NVME_ADM_CMD_IDENTIFY: + case NVME_ADM_CMD_ABORT: + case NVME_ADM_CMD_SET_FEATURES: + case NVME_ADM_CMD_GET_FEATURES: + case NVME_ADM_CMD_ASYNC_EV_REQ: + case NVME_ADM_CMD_DBBUF_CONFIG: + case NVME_ADM_CMD_FORMAT_NVM: + case NVME_ADM_CMD_DIRECTIVE_SEND: + case NVME_ADM_CMD_DIRECTIVE_RECV: + break; + case NVME_ADM_CMD_NS_ATTACHMENT: + int namespaces_num =3D 0; + for (int i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + if (!ns) { + continue; + } + + namespaces_num++; + } + + if (namespaces_num > 1) { + nvme_add_blocker_feature(blocker_features, "Namespace Atta= chment"); + } + + break; + case NVME_ADM_CMD_VIRT_MNGMT: + if (n->params.sriov_max_vfs) { + nvme_add_blocker_feature(blocker_features, "SR-IOV"); + } + + break; + case NVME_ADM_CMD_SECURITY_SEND: + case NVME_ADM_CMD_SECURITY_RECV: + if (adm_cmd_security_checked) { + break; + } + + if (pci_dev->spdm_port) { + nvme_add_blocker_feature(blocker_features, "SPDM"); + } + + adm_cmd_security_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + for (int opcode =3D 0; opcode < sizeof(n->cse.iocs.nvm) / sizeof(n->cs= e.iocs.nvm[0]); opcode++) { + if (!n->cse.iocs.nvm[opcode]) { + continue; + } + + switch (opcode) { + case NVME_CMD_FLUSH: + case NVME_CMD_WRITE: + case NVME_CMD_READ: + case NVME_CMD_COMPARE: + case NVME_CMD_WRITE_ZEROES: + case NVME_CMD_DSM: + case NVME_CMD_VERIFY: + case NVME_CMD_COPY: + break; + case NVME_CMD_IO_MGMT_RECV: + case NVME_CMD_IO_MGMT_SEND: + if (cmd_io_mgmt_checked) { + break; + } + + /* check for NVME_IOMS_MO_RUH_UPDATE */ + if (n->subsys->params.fdp.enabled) { + nvme_add_blocker_feature(blocker_features, "FDP"); + } + + cmd_io_mgmt_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + for (int opcode =3D 0; opcode < sizeof(n->cse.iocs.zoned) / sizeof(n->= cse.iocs.zoned[0]); opcode++) { + /* + * If command isn't supported or we have the same command + * in n->cse.iocs.nvm, then we can skip it here. + */ + if (!n->cse.iocs.zoned[opcode] || n->cse.iocs.nvm[opcode]) { + continue; + } + + switch (opcode) { + case NVME_CMD_ZONE_APPEND: + case NVME_CMD_ZONE_MGMT_SEND: + case NVME_CMD_ZONE_MGMT_RECV: + if (cmd_zone_checked) { + break; + } + + for (int i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + if (!ns) { + continue; + } + + if (ns->params.zoned) { + nvme_add_blocker_feature(blocker_features, "Zoned Name= space"); + break; + } + } + + cmd_zone_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + /* + * Try our best to explicitly detect all not supported caps, + * to let users know what features cause migration to be blocked, + * but in case we miss handling here, everything else will be + * covered by unsupported_cap check. + */ + if (NVME_CAP_CMBS(cap)) { + nvme_add_blocker_feature(blocker_features, "CMB"); + cap &=3D ~((uint64_t)CAP_CMBS_MASK << CAP_CMBS_SHIFT); + } + + if (NVME_CAP_PMRS(cap)) { + nvme_add_blocker_feature(blocker_features, "PMR"); + cap &=3D ~((uint64_t)CAP_PMRS_MASK << CAP_PMRS_SHIFT); + } + + unsupported_cap =3D cap & ~NVME_MIGRATION_SUPPORTED_CAP_BITS; + if (unsupported_cap) { + nvme_add_blocker_feature(blocker_features, "unknown capability"); + } + + assert(n->migration_blocker =3D=3D NULL); + if (strlen(blocker_features) > 0) { + error_setg(&n->migration_blocker, "Migration is not supported for = %s", blocker_features); + if (migrate_add_blocker(&n->migration_blocker, errp) < 0) { + return false; + } + } + + return true; +} + static int nvme_init_subsys(NvmeCtrl *n, Error **errp) { int cntlid; @@ -9313,6 +9513,10 @@ static void nvme_realize(PCIDevice *pci_dev, Error *= *errp) =20 n->subsys->namespaces[ns->params.nsid] =3D ns; } + + if (!nvme_set_migration_blockers(n, pci_dev, errp)) { + return; + } } =20 static void nvme_exit(PCIDevice *pci_dev) @@ -9365,6 +9569,8 @@ static void nvme_exit(PCIDevice *pci_dev) } =20 memory_region_del_subregion(&n->bar0, &n->iomem); + + migrate_del_blocker(&n->migration_blocker); } =20 static const Property nvme_props[] =3D { diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index d66f7dc82d5..457b6637249 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -666,6 +666,9 @@ typedef struct NvmeCtrl { =20 /* Socket mapping to SPDM over NVMe Security In/Out commands */ int spdm_socket; + + /* Migration-related stuff */ + Error *migration_blocker; } NvmeCtrl; =20 typedef enum NvmeResetType { diff --git a/include/block/nvme.h b/include/block/nvme.h index 9d7159ed7a7..a7f586fc801 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -141,6 +141,18 @@ enum NvmeCapMask { #define NVME_CAP_SET_CMBS(cap, val) \ ((cap) |=3D (uint64_t)((val) & CAP_CMBS_MASK) << CAP_CMBS_SHIFT) =20 +#define NVME_MIGRATION_SUPPORTED_CAP_BITS ( \ + ((uint64_t)CAP_MQES_MASK << CAP_MQES_SHIFT) \ + | ((uint64_t)CAP_CQR_MASK << CAP_CQR_SHIFT) \ + | ((uint64_t)CAP_AMS_MASK << CAP_AMS_SHIFT) \ + | ((uint64_t)CAP_TO_MASK << CAP_TO_SHIFT) \ + | ((uint64_t)CAP_DSTRD_MASK << CAP_DSTRD_SHIFT) \ + | ((uint64_t)CAP_NSSRS_MASK << CAP_NSSRS_SHIFT) \ + | ((uint64_t)CAP_CSS_MASK << CAP_CSS_SHIFT) \ + | ((uint64_t)CAP_MPSMIN_MASK << CAP_MPSMIN_SHIFT) \ + | ((uint64_t)CAP_MPSMAX_MASK << CAP_MPSMAX_SHIFT) \ +) + enum NvmeCapCss { NVME_CAP_CSS_NCSS =3D 1 << 0, NVME_CAP_CSS_IOCSS =3D 1 << 6, --=20 2.47.3 From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743345; cv=none; d=zohomail.com; s=zohoarc; b=beFiubZm6grkzQb/im3CZ6DOicorv9CkAEv/MeZPOfPePUcin8R651geveX4ulMGE5Utue+1fS2RTlHhFvy+2zZEgA5HP9lMkqYE/u60Mrd4R2+LrxuBWTJca1s/Q+g6Xyyb1vapqCHe+U0wNHeH01cBndTbICV+wz+PB/RUSvM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743345; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lv8cYOpNkSeeqxeKbJ1G68OYSr/qDeNt7sRa838SjpY=; b=RhurDRGP2WWsTxtJz/ROnH4tkWCIMAcccyrn47IG0KMYIhNyqOmz7hDLnAJ4Z4ZJBxYoTKAUDK824S2SVnffEVR4eO07eYh/S0/zzUF3g9vluUYLINHFohHEXPz1ilxWWOUJcmt0FtcmFAmpL6XSmWbme25s0tb0a/tTkijZQ2I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773743345604189.1799127498266; Tue, 17 Mar 2026 03:29:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2ReQ-00020Z-1I; Tue, 17 Mar 2026 06:27:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Rdy-0001vr-Cm for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:27 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rdu-0006WP-4d for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:21 -0400 Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-4853aec185aso45702115e9.1 for ; Tue, 17 Mar 2026 03:27:17 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743237; x=1774348037; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lv8cYOpNkSeeqxeKbJ1G68OYSr/qDeNt7sRa838SjpY=; b=cFDRZc/lq4f6DtLkQi84W9LCVfaym3NbV8dycj06eL0A6vUjO7lbWAnW2NMwoUxJxL CZYr8u2YR7Iy5zCUyseEe8/QJKVNsIV1EtpRjwvoxms5/O/jSr4Iq2ZY/PZsafGRT1SZ bEjnfjdgFPoSz4X54E9V03BnQk+fNxhg/IqC0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743237; x=1774348037; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=lv8cYOpNkSeeqxeKbJ1G68OYSr/qDeNt7sRa838SjpY=; b=S5xziEzGZUh24IeAjBotD0pY1vC+ic9MxXzlAX4tz0seBSgG4ZRn6CqzsydR3tfznr 2rvLsxrXAs4wC+23i0QD5q+/3ZxOxS3qm7wQqQFOx3oVFJ3im4Y0Ig4eFqCFO0hxyBCt BpZ129NZT1Xgm/7WBSjn60BRpr6qIHVsX23Rwmn700Z5EArj9ounDuR7sbos4NrvEhVE oG17c1/p34AMXOs/JI5iY7OXloxBfJurvi3dgjvdKrLfoifhh6tB2xvrzfrrZbqXzO0G AmmdSMzFkfdP8H2ZvE9Y0zECu9Y7+TOilvl9EWei7aywoKMSXiZFpS3ABANYe4kszUVh z9HQ== X-Gm-Message-State: AOJu0YyToVxlLJ7yOxpwtyFypjmVkQwoStV5ZMuH9+fd8JdLPUWH7gx9 ORs0B0xuVQm8ptBaQA5hMG6RdeKGRogNURiwElFmNKLtjeq0n6vZLK3iTjbKDALoGl93gGbgY90 CVRitNCk= X-Gm-Gg: ATEYQzwOUd2ZZF0BC6QrlTKNALu5coSXpa2wuWipNwoZbqtmC4m/CFh9rnAU7iBrBHd m5ZWwmNQIdguVkweDM8lQousBFfAMWoue6QjUpO//JOPDvyqCtVv+i02TBzdp6aQJV9et0YG9M0 21ZgSBLtQwZJT28uJkVDaGY91GPhWWzBnoMToxdNzKGqmbINC70uaTX+OzW02o/pMHpiQPPNdJ7 knIy4IS27IiNmKzwrJzsL506B/kBg42SU84PqalJSMSefzZIJJPz5KQm9E/qEuvfR7y1/F2LNcR wm1XptrH53MedFxDnXpXUDowZqGwYGfZLRVPxDx2uKxh3Zs7yQON121/yQzyQT+OoxXvKpphFXb pqUXBBCpdHjRk/Ib84pXq12MjeFuyQrROzqvPmnjt/BT0QpMVt6bB4fJFFyRRAkiNaYX1mJNz3K Rv29h3MrC0mgy08HOI44FODbGCw9MaOLjhneYyZqU7BKKYCbGmUgSvdvT/0fa1QitdcwAIzVpUY KOL4HgAS+Fs7rGEp3/IEDU= X-Received: by 2002:a05:600c:5247:b0:485:3c8f:e4c5 with SMTP id 5b1f17b1804b1-485566fcaaemr284271005e9.17.1773743236605; Tue, 17 Mar 2026 03:27:16 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 6/8] hw/nvme: split nvme_init_sq/nvme_init_cq into helpers Date: Tue, 17 Mar 2026 11:27:06 +0100 Message-ID: <20260317102708.126725-7-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743347533154100 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Signed-off-by: Alexander Mikhalitsyn --- hw/nvme/ctrl.c | 57 +++++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 21 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 9f9c9bcbead..68ce9802505 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -4854,18 +4854,14 @@ static uint16_t nvme_del_sq(NvmeCtrl *n, NvmeReques= t *req) return NVME_SUCCESS; } =20 -static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, uint64_t dma_addr, - uint16_t sqid, uint16_t cqid, uint16_t size) +static void __nvme_init_sq(NvmeSQueue *sq) { + NvmeCtrl *n =3D sq->ctrl; + uint16_t sqid =3D sq->sqid; + uint16_t cqid =3D sq->cqid; int i; NvmeCQueue *cq; =20 - sq->ctrl =3D n; - sq->dma_addr =3D dma_addr; - sq->sqid =3D sqid; - sq->size =3D size; - sq->cqid =3D cqid; - sq->head =3D sq->tail =3D 0; sq->io_req =3D g_new0(NvmeRequest, sq->size); =20 QTAILQ_INIT(&sq->req_list); @@ -4895,6 +4891,18 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n= , uint64_t dma_addr, n->sq[sqid] =3D sq; } =20 +static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, uint64_t dma_addr, + uint16_t sqid, uint16_t cqid, uint16_t size) +{ + sq->ctrl =3D n; + sq->dma_addr =3D dma_addr; + sq->sqid =3D sqid; + sq->size =3D size; + sq->cqid =3D cqid; + sq->head =3D sq->tail =3D 0; + __nvme_init_sq(sq); +} + static uint16_t nvme_create_sq(NvmeCtrl *n, NvmeRequest *req) { NvmeSQueue *sq; @@ -5555,24 +5563,16 @@ static uint16_t nvme_del_cq(NvmeCtrl *n, NvmeReques= t *req) return NVME_SUCCESS; } =20 -static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr, - uint16_t cqid, uint16_t vector, uint16_t size, - uint16_t irq_enabled) +static void __nvme_init_cq(NvmeCQueue *cq) { + NvmeCtrl *n =3D cq->ctrl; PCIDevice *pci =3D PCI_DEVICE(n); + uint16_t cqid =3D cq->cqid; =20 - if (msix_enabled(pci) && irq_enabled) { - msix_vector_use(pci, vector); + if (msix_enabled(pci) && cq->irq_enabled) { + msix_vector_use(pci, cq->vector); } =20 - cq->ctrl =3D n; - cq->cqid =3D cqid; - cq->size =3D size; - cq->dma_addr =3D dma_addr; - cq->phase =3D 1; - cq->irq_enabled =3D irq_enabled; - cq->vector =3D vector; - cq->head =3D cq->tail =3D 0; QTAILQ_INIT(&cq->req_list); QTAILQ_INIT(&cq->sq_list); if (n->dbbuf_enabled) { @@ -5590,6 +5590,21 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n= , uint64_t dma_addr, &DEVICE(cq->ctrl)->mem_reentrancy_guard); } =20 +static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr, + uint16_t cqid, uint16_t vector, uint16_t size, + uint16_t irq_enabled) +{ + cq->ctrl =3D n; + cq->cqid =3D cqid; + cq->size =3D size; + cq->dma_addr =3D dma_addr; + cq->phase =3D 1; + cq->irq_enabled =3D irq_enabled; + cq->vector =3D vector; + cq->head =3D cq->tail =3D 0; + __nvme_init_cq(cq); +} + static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeRequest *req) { NvmeCQueue *cq; --=20 2.47.3 From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743326; cv=none; d=zohomail.com; s=zohoarc; b=VAcQNDGQT2DptugNWVBKWNvOg3hiHltie9B9/+yw0nD0bx8qwzDICj3brsW2hSR3lZnn32mhttdva026XpeBIrMl8D1s8dsxsRGlsnZNaMDD7ZEq0fF7cWYVfB2t/0EJ0LkntWx2W7jY4MKCqp0VOYmsi3G13Q3XmJ+GN9z0Xpo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743326; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5oDn1DyyxJp+kkjF+9H5BBpPByqnknuxzyFDSX/Voqs=; b=UMcBvHwrMvr6FZTaOV9C5WrBa8FgXEJ7jUhJNKNebxsxKMDOihqEpL0FKkpUPQWJbqXUkQsPEc3WnYEJFZZFzR3qGRhJWKZKjcH/BxDELCB9bCbZSrETkrgT+I8BHFfID4uT/aKtYJ3jZ0dA/wBRDoPP1AC5yey0aRbz+bil3aA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773743326658542.1724733883785; Tue, 17 Mar 2026 03:28:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2ReP-00020L-T8; Tue, 17 Mar 2026 06:27:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Re2-0001vz-9J for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:27 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rdx-0006Wj-05 for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:25 -0400 Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-48539d21b76so39600835e9.1 for ; Tue, 17 Mar 2026 03:27:19 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743238; x=1774348038; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5oDn1DyyxJp+kkjF+9H5BBpPByqnknuxzyFDSX/Voqs=; b=W+hDNJ6kLCzMvItAr2+HuFyZgFrTwCuFIwEG7cbZAmCHnYaYjHIaM8CRdpip4LZBxQ Tv2nWqDLHYv6whLLGRm2CVpXUpTyYE4YPzkWforoD0YgQNhEZVJ26LeaNv6YoMH53Aax XRxhTFyzPhKM238AkoOhlpNudj9LAtTHSqkLs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743238; x=1774348038; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=5oDn1DyyxJp+kkjF+9H5BBpPByqnknuxzyFDSX/Voqs=; b=BI4JjX6mzyBxQ492/FRk5xxFLVjANtoIiMIl1GY7m6edxn4BILmGeVRFCe8ENsJtuP 46R5hwaORhLpkj9pCoBF0QhM7G3Py1JN5DOxPYkPIgTa9av5gzNGAgfWQQ6HWqP4l1BC UH4Im/ZglFn1o8YT5Cjh1V/u1M6JImc3RJmVU5HJhzQ10BJUK4zpW9OkATfMMO9PJocW 9EwZXHUdhPKP4Lj3vSWKsMJNjF+C1XB1VmrunpAet5cqV8mQ27z4CJJAW2L4hXuTSAwB ZTPbbojw850S5kE1MnIV1ZocJA2xueJYJvB2zpm65nyaRd6ES7NPKw67pyH1x/mN3FNV MaCA== X-Gm-Message-State: AOJu0YzLArlpg/1yvgzuJOzxvMAFe+GaAE6d3W1KYODSoFpRnTS364OY vlBX0gq/d85XYzL+BghuZdFsjgLmTFv3c7ChACRXJqwMrcRLfcAOq5yNgAVdQI6aUR6uqqcN0Fo cqYyFGVc= X-Gm-Gg: ATEYQzxH+SzoRG7dvMibnQ+Ur6mOzeBhBQDmhdlVTmPv6tkrkMfKB0IljMKoi8Qr6bD FPe09ujEOROuy+rtZOeu1qwJCKkBAKRhFVMSHTjT0uc4ZB3pREWm3oFCmQHyLqngZ7tHj6ntTEp 31rzLGjFRM9/g4q87AB3+trKV/50C9yNLMcHIqT0tbPnfJDbsrmbX3tCc2sqK6JNqea9IU7Dl6/ XT9MGZP0r/6XgUahXhzqksHchvdFwWv4+lDUIO4OBoMrmefILFjkiWUUnGNJnHbE5lPESKAV2FQ BmMsD3twCVDLgWLh403lcm/w/WMTXEGfKqtO32KFfyWbxC5O5U0bXNthqg8JKzHMxISHKNlayHF byqs+yKzG//d7g49MwzOY5aZFp31irxZO8wr2VegpqQ0cyA9FI8IZc9zalKZX5rum/I0xoEy8Ui nhDvzJ8KggnavjP8GtrrubTVtdX2yb6qjuSHodT3AVrlNFH9ZUbDqzs1KTxQoKN0jiS9BagDgxl m+zXy7hwN9KAaNFeqmSSH/iUyAGjh/1Vw== X-Received: by 2002:a05:600c:4583:b0:485:439b:683f with SMTP id 5b1f17b1804b1-48556700c23mr251512865e9.20.1773743237765; Tue, 17 Mar 2026 03:27:17 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 7/8] hw/nvme: add basic live migration support Date: Tue, 17 Mar 2026 11:27:07 +0100 Message-ID: <20260317102708.126725-8-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743329186154100 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn It has some limitations: - only one NVMe namespace is supported - SMART counters are not preserved - CMB is not supported - PMR is not supported - SPDM is not supported - SR-IOV is not supported Signed-off-by: Alexander Mikhalitsyn v2: - AERs are now fully supported --- hw/nvme/ctrl.c | 611 ++++++++++++++++++++++++++++++++++++++++++- hw/nvme/ns.c | 160 +++++++++++ hw/nvme/nvme.h | 5 + hw/nvme/trace-events | 11 + 4 files changed, 778 insertions(+), 9 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 68ce9802505..e3030bfadcb 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -208,6 +208,7 @@ #include "hw/pci/pcie_sriov.h" #include "system/spdm-socket.h" #include "migration/blocker.h" +#include "migration/qemu-file-types.h" #include "migration/vmstate.h" =20 #include "nvme.h" @@ -4903,6 +4904,25 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n= , uint64_t dma_addr, __nvme_init_sq(sq); } =20 +static void nvme_restore_sq(NvmeSQueue *sq_from) +{ + NvmeCtrl *n =3D sq_from->ctrl; + NvmeSQueue *sq =3D sq_from; + + if (sq_from->sqid =3D=3D 0) { + sq =3D &n->admin_sq; + sq->ctrl =3D n; + sq->dma_addr =3D sq_from->dma_addr; + sq->sqid =3D sq_from->sqid; + sq->size =3D sq_from->size; + sq->cqid =3D sq_from->cqid; + sq->head =3D sq_from->head; + sq->tail =3D sq_from->tail; + } + + __nvme_init_sq(sq); +} + static uint16_t nvme_create_sq(NvmeCtrl *n, NvmeRequest *req) { NvmeSQueue *sq; @@ -5605,6 +5625,27 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n= , uint64_t dma_addr, __nvme_init_cq(cq); } =20 +static void nvme_restore_cq(NvmeCQueue *cq_from) +{ + NvmeCtrl *n =3D cq_from->ctrl; + NvmeCQueue *cq =3D cq_from; + + if (cq_from->cqid =3D=3D 0) { + cq =3D &n->admin_cq; + cq->ctrl =3D n; + cq->cqid =3D cq_from->cqid; + cq->size =3D cq_from->size; + cq->dma_addr =3D cq_from->dma_addr; + cq->phase =3D cq_from->phase; + cq->irq_enabled =3D cq_from->irq_enabled; + cq->vector =3D cq_from->vector; + cq->head =3D cq_from->head; + cq->tail =3D cq_from->tail; + } + + __nvme_init_cq(cq); +} + static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeRequest *req) { NvmeCQueue *cq; @@ -7293,7 +7334,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const = NvmeRequest *req) n->dbbuf_eis =3D eis_addr; n->dbbuf_enabled =3D true; =20 - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { NvmeSQueue *sq =3D n->sq[i]; NvmeCQueue *cq =3D n->cq[i]; =20 @@ -7737,7 +7778,7 @@ static int nvme_atomic_write_check(NvmeCtrl *n, NvmeC= md *cmd, /* * Walk the queues to see if there are any atomic conflicts. */ - for (i =3D 1; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 1; i < n->num_queues; i++) { NvmeSQueue *sq; NvmeRequest *req; NvmeRwCmd *req_rw; @@ -7807,6 +7848,10 @@ static void nvme_process_sq(void *opaque) NvmeCmd cmd; NvmeRequest *req; =20 + if (qatomic_read(&n->stop_processing_sq)) { + return; + } + if (n->dbbuf_enabled) { nvme_update_sq_tail(sq); } @@ -7815,6 +7860,10 @@ static void nvme_process_sq(void *opaque) NvmeAtomic *atomic; bool cmd_is_atomic; =20 + if (qatomic_read(&n->stop_processing_sq)) { + return; + } + addr =3D sq->dma_addr + (sq->head << NVME_SQES); if (nvme_addr_read(n, addr, (void *)&cmd, sizeof(cmd))) { trace_pci_nvme_err_addr_read(addr); @@ -7923,12 +7972,12 @@ static void nvme_ctrl_reset(NvmeCtrl *n, NvmeResetT= ype rst) nvme_ns_drain(ns); } =20 - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { if (n->sq[i] !=3D NULL) { nvme_free_sq(n->sq[i], n); } } - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { if (n->cq[i] !=3D NULL) { nvme_free_cq(n->cq[i], n); } @@ -8598,6 +8647,8 @@ static bool nvme_check_params(NvmeCtrl *n, Error **er= rp) params->max_ioqpairs =3D params->num_queues - 1; } =20 + n->num_queues =3D params->max_ioqpairs + 1; + if (n->namespace.blkconf.blk && n->subsys) { error_setg(errp, "subsystem support is unavailable with legacy " "namespace ('drive' property)"); @@ -8752,8 +8803,8 @@ static void nvme_init_state(NvmeCtrl *n) n->conf_msix_qsize =3D n->params.msix_qsize; } =20 - n->sq =3D g_new0(NvmeSQueue *, n->params.max_ioqpairs + 1); - n->cq =3D g_new0(NvmeCQueue *, n->params.max_ioqpairs + 1); + n->sq =3D g_new0(NvmeSQueue *, n->num_queues); + n->cq =3D g_new0(NvmeCQueue *, n->num_queues); n->temperature =3D NVME_TEMPERATURE; n->features.temp_thresh_hi =3D NVME_TEMPERATURE_WARNING; n->starttime_ms =3D qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); @@ -8988,7 +9039,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci= _dev, Error **errp) } =20 if (n->params.msix_exclusive_bar && !pci_is_vf(pci_dev)) { - bar_size =3D nvme_mbar_size(n->params.max_ioqpairs + 1, 0, NULL, N= ULL); + bar_size =3D nvme_mbar_size(n->num_queues, 0, NULL, NULL); memory_region_init_io(&n->iomem, OBJECT(n), &nvme_mmio_ops, n, "nv= me", bar_size); pci_register_bar(pci_dev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY | @@ -9000,7 +9051,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci= _dev, Error **errp) /* add one to max_ioqpairs to account for the admin queue pair */ if (!pci_is_vf(pci_dev)) { nr_vectors =3D n->params.msix_qsize; - bar_size =3D nvme_mbar_size(n->params.max_ioqpairs + 1, + bar_size =3D nvme_mbar_size(n->num_queues, nr_vectors, &msix_table_offset, &msix_pba_offset); } else { @@ -9723,9 +9774,551 @@ static uint32_t nvme_pci_read_config(PCIDevice *dev= , uint32_t address, int len) return pci_default_read_config(dev, address, len); } =20 +static bool nvme_ctrl_pre_save(void *opaque, Error **errp) +{ + NvmeCtrl *n =3D opaque; + int i; + + trace_pci_nvme_pre_save_enter(n); + + /* ask SQ processing code not to take new requests */ + qatomic_set(&n->stop_processing_sq, true); + + /* prevent new in-flight IO from appearing */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + qemu_bh_cancel(sq->bh); + } + + /* drain all IO */ + for (i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns; + + ns =3D nvme_ns(n, i); + if (!ns) { + continue; + } + + trace_pci_nvme_pre_save_ns_drain(n, i); + nvme_ns_drain(ns); + } + + /* + * Now, we should take care of AERs. + * + * 1. Save all queued events (n->aer_queue). + * This is done automatically, see nvme_vmstate VMStateDescription. + * Here we only need to print them for debugging purpose. + * 2. Go over outstanding AER requests (n->aer_reqs) and check they are + * all have expected opcode (NVME_ADM_CMD_ASYNC_EV_REQ) and other f= ields. + * + * We must be really careful here, because in case of further QEMU NVM= e changes, + * we may break migration without noticing it, or worse, introduce sil= ent + * data corruption during migration. + */ + if (n->aer_queued) { + NvmeAsyncEvent *event; + + QTAILQ_FOREACH(event, &n->aer_queue, entry) { + trace_pci_nvme_pre_save_aer(event->result.event_type, event->r= esult.event_info, + event->result.log_page); + } + } + + for (i =3D 0; i < n->outstanding_aers; i++) { + NvmeRequest *re =3D n->aer_reqs[i]; + + /* + * Can't use assert() here, because we don't want + * to just crash QEMU when user requests a migration. + */ + if (!(re->cmd.opcode =3D=3D NVME_ADM_CMD_ASYNC_EV_REQ)) { + error_setg(errp, "re->cmd.opcode (%u) !=3D NVME_ADM_CMD_ASYNC_= EV_REQ", re->cmd.opcode); + goto err; + } + + if (!(re->ns =3D=3D NULL)) { + error_setg(errp, "re->ns !=3D NULL"); + goto err; + } + + if (!(re->sq =3D=3D &n->admin_sq)) { + error_setg(errp, "re->sq !=3D &n->admin_sq"); + goto err; + } + + if (!(re->aiocb =3D=3D NULL)) { + error_setg(errp, "re->aiocb !=3D NULL"); + goto err; + } + + if (!(re->opaque =3D=3D NULL)) { + error_setg(errp, "re->opaque !=3D NULL"); + goto err; + } + + if (!(re->atomic_write =3D=3D false)) { + error_setg(errp, "re->atomic_write !=3D false"); + goto err; + } + + if (re->sg.flags & NVME_SG_ALLOC) { + error_setg(errp, "unexpected NVME_SG_ALLOC flag in re->sg.flag= s"); + goto err; + } + } + + /* wait when all in-flight IO requests (except NVME_ADM_CMD_ASYNC_EV_R= EQ) are processed */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeRequest *req; + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + trace_pci_nvme_pre_save_sq_out_req_drain_wait(n, i, sq->head, sq->= tail, sq->size); + +wait_out_reqs: + QTAILQ_FOREACH(req, &sq->out_req_list, entry) { + if (req->cmd.opcode !=3D NVME_ADM_CMD_ASYNC_EV_REQ) { + cpu_relax(); + goto wait_out_reqs; + } + } + + trace_pci_nvme_pre_save_sq_out_req_drain_wait_end(n, i, sq->head, = sq->tail); + } + + /* wait when all IO requests completions are written to guest memory */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeCQueue *cq =3D n->cq[i]; + + if (!cq) + continue; + + trace_pci_nvme_pre_save_cq_req_drain_wait(n, i, cq->head, cq->tail= , cq->size); + + while (!QTAILQ_EMPTY(&cq->req_list)) { + /* + * nvme_post_cqes() can't do its job of cleaning cq->req_list + * when CQ is full, it means that we need to save what we have= in + * cq->req_list and restore it back on VM resume. + * + * Good thing is that this can only happen when guest hasn't + * processed CQ for a long time and at the same time, many SQEs + * are in flight. + * + * For now, let's just block migration in this rare case. + */ + if (nvme_cq_full(cq)) { + error_setg(errp, "no free space in CQ (not supported)"); + goto err; + } + + cpu_relax(); + } + + trace_pci_nvme_pre_save_cq_req_drain_wait_end(n, i, cq->head, cq->= tail); + } + + for (uint32_t nsid =3D 0; nsid <=3D NVME_MAX_NAMESPACES; nsid++) { + NvmeNamespace *ns =3D n->namespaces[nsid]; + + if (!ns) + continue; + + if (ns !=3D &n->namespace) { + error_setg(errp, "only one NVMe namespace is supported for mig= ration"); + goto err; + } + } + + return true; + +err: + /* restore sq processing back to normal */ + qatomic_set(&n->stop_processing_sq, false); + return false; +} + +static bool nvme_ctrl_post_load(void *opaque, int version_id, Error **errp) +{ + NvmeCtrl *n =3D opaque; + int i; + + trace_pci_nvme_post_load_enter(n); + + /* restore CQs first */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeCQueue *cq =3D n->cq[i]; + + if (!cq) + continue; + + cq->ctrl =3D n; + nvme_restore_cq(cq); + trace_pci_nvme_post_load_restore_cq(n, i, cq->head, cq->tail, cq->= size); + + if (i =3D=3D 0) { + /* + * Admin CQ lives in n->admin_cq, we don't need + * memory allocated for it in get_ptrs_array_entry() anymore. + * + * nvme_restore_cq() also takes care of: + * n->cq[0] =3D &n->admin_cq; + * so n->cq[0] remains valid. + */ + g_free(cq); + } + } + + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + sq->ctrl =3D n; + nvme_restore_sq(sq); + trace_pci_nvme_post_load_restore_sq(n, i, sq->head, sq->tail, sq->= size); + + if (i =3D=3D 0) { + /* same as for CQ */ + g_free(sq); + } + } + + if (n->aer_queued) { + NvmeAsyncEvent *event; + + QTAILQ_FOREACH(event, &n->aer_queue, entry) { + trace_pci_nvme_post_load_aer(event->result.event_type, event->= result.event_info, + event->result.log_page); + } + } + + for (i =3D 0; i < n->outstanding_aers; i++) { + NvmeSQueue *sq =3D &n->admin_sq; + NvmeRequest *req_from =3D n->aer_reqs[i]; + NvmeRequest *req; + + /* + * We use nvme_vmstate VMStateDescription to save/restore + * NvmeRequest structures, but tricky thing here is that + * memory for each n->aer_reqs[i] will be allocated separately + * during restore. It doesn't work for us. We need to take + * an existing NvmeRequest structure from SQ's req_list + * and fill it with data from the newly allocated one (req_from). + * Then, we can safely release allocated memory for it. + */ + + /* take an NvmeRequest struct from SQ */ + req =3D QTAILQ_FIRST(&sq->req_list); + QTAILQ_REMOVE(&sq->req_list, req, entry); + QTAILQ_INSERT_TAIL(&sq->out_req_list, req, entry); + nvme_req_clear(req); + + /* copy data from the source NvmeRequest */ + req->status =3D req_from->status; + memcpy(&req->cqe, &req_from->cqe, sizeof(NvmeCqe)); + memcpy(&req->cmd, &req_from->cmd, sizeof(NvmeCmd)); + + n->aer_reqs[i] =3D req; + g_free(req_from); + } + + /* + * We need to attach namespaces (currently, only one namespace is + * supported for migration). + * This logic comes from nvme_start_ctrl(). + */ + for (i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + + if (!ns || (!ns->params.shared && ns->ctrl !=3D n)) { + continue; + } + + if (nvme_csi_supported(n, ns->csi) && !ns->params.detached) { + if (!ns->attached || ns->params.shared) { + nvme_attach_ns(n, ns); + } + } + } + + /* schedule SQ processing */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) + continue; + + qemu_bh_schedule(sq->bh); + } + + /* + * We ensured in pre_save() that cq->req_list was empty, + * so we don't need to schedule BH for CQ processing. + */ + + return true; +} + +static const VMStateDescription nvme_vmstate_bar =3D { + .name =3D "nvme-bar", + .minimum_version_id =3D 1, + .version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(cap, NvmeBar), + VMSTATE_UINT32(vs, NvmeBar), + VMSTATE_UINT32(intms, NvmeBar), + VMSTATE_UINT32(intmc, NvmeBar), + VMSTATE_UINT32(cc, NvmeBar), + VMSTATE_UINT8_ARRAY(rsvd24, NvmeBar, 4), + VMSTATE_UINT32(csts, NvmeBar), + VMSTATE_UINT32(nssr, NvmeBar), + VMSTATE_UINT32(aqa, NvmeBar), + VMSTATE_UINT64(asq, NvmeBar), + VMSTATE_UINT64(acq, NvmeBar), + VMSTATE_UINT32(cmbloc, NvmeBar), + VMSTATE_UINT32(cmbsz, NvmeBar), + VMSTATE_UINT32(bpinfo, NvmeBar), + VMSTATE_UINT32(bprsel, NvmeBar), + VMSTATE_UINT64(bpmbl, NvmeBar), + VMSTATE_UINT64(cmbmsc, NvmeBar), + VMSTATE_UINT32(cmbsts, NvmeBar), + VMSTATE_UINT8_ARRAY(rsvd92, NvmeBar, 3492), + VMSTATE_UINT32(pmrcap, NvmeBar), + VMSTATE_UINT32(pmrctl, NvmeBar), + VMSTATE_UINT32(pmrsts, NvmeBar), + VMSTATE_UINT32(pmrebs, NvmeBar), + VMSTATE_UINT32(pmrswtp, NvmeBar), + VMSTATE_UINT32(pmrmscl, NvmeBar), + VMSTATE_UINT32(pmrmscu, NvmeBar), + VMSTATE_UINT8_ARRAY(css, NvmeBar, 484), + VMSTATE_END_OF_LIST() + }, +}; + +static const VMStateDescription nvme_vmstate_cqueue =3D { + .name =3D "nvme-cq", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(phase, NvmeCQueue), + VMSTATE_UINT16(cqid, NvmeCQueue), + VMSTATE_UINT16(irq_enabled, NvmeCQueue), + VMSTATE_UINT32(head, NvmeCQueue), + VMSTATE_UINT32(tail, NvmeCQueue), + VMSTATE_UINT32(vector, NvmeCQueue), + VMSTATE_UINT32(size, NvmeCQueue), + VMSTATE_UINT64(dma_addr, NvmeCQueue), + /* db_addr, ei_addr, etc will be recalculated */ + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_squeue =3D { + .name =3D "nvme-sq", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(sqid, NvmeSQueue), + VMSTATE_UINT16(cqid, NvmeSQueue), + VMSTATE_UINT32(head, NvmeSQueue), + VMSTATE_UINT32(tail, NvmeSQueue), + VMSTATE_UINT32(size, NvmeSQueue), + VMSTATE_UINT64(dma_addr, NvmeSQueue), + /* db_addr, ei_addr, etc will be recalculated */ + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_async_event_result =3D { + .name =3D "nvme-async-event-result", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(event_type, NvmeAerResult), + VMSTATE_UINT8(event_info, NvmeAerResult), + VMSTATE_UINT8(log_page, NvmeAerResult), + VMSTATE_UINT8(resv, NvmeAerResult), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_async_event =3D { + .name =3D "nvme-async-event", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_STRUCT(result, NvmeAsyncEvent, 0, nvme_vmstate_async_event= _result, NvmeAerResult), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cqe =3D { + .name =3D "nvme-cqe", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT32(result, NvmeCqe), + VMSTATE_UINT32(dw1, NvmeCqe), + VMSTATE_UINT16(sq_head, NvmeCqe), + VMSTATE_UINT16(sq_id, NvmeCqe), + VMSTATE_UINT16(cid, NvmeCqe), + VMSTATE_UINT16(status, NvmeCqe), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd_dptr_sgl =3D { + .name =3D "nvme-request-cmd-dptr-sgl", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(addr, NvmeSglDescriptor), + VMSTATE_UINT32(len, NvmeSglDescriptor), + VMSTATE_UINT8_ARRAY(rsvd, NvmeSglDescriptor, 3), + VMSTATE_UINT8(type, NvmeSglDescriptor), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd_dptr =3D { + .name =3D "nvme-request-cmd-dptr", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(prp1, NvmeCmdDptr), + VMSTATE_UINT64(prp2, NvmeCmdDptr), + VMSTATE_STRUCT(sgl, NvmeCmdDptr, 0, nvme_vmstate_cmd_dptr_sgl, Nvm= eSglDescriptor), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd =3D { + .name =3D "nvme-request-cmd", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(opcode, NvmeCmd), + VMSTATE_UINT8(flags, NvmeCmd), + VMSTATE_UINT16(cid, NvmeCmd), + VMSTATE_UINT32(nsid, NvmeCmd), + VMSTATE_UINT64(res1, NvmeCmd), + VMSTATE_UINT64(mptr, NvmeCmd), + VMSTATE_STRUCT(dptr, NvmeCmd, 0, nvme_vmstate_cmd_dptr, NvmeCmdDpt= r), + VMSTATE_UINT32(cdw10, NvmeCmd), + VMSTATE_UINT32(cdw11, NvmeCmd), + VMSTATE_UINT32(cdw12, NvmeCmd), + VMSTATE_UINT32(cdw13, NvmeCmd), + VMSTATE_UINT32(cdw14, NvmeCmd), + VMSTATE_UINT32(cdw15, NvmeCmd), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_request =3D { + .name =3D "nvme-request", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(status, NvmeRequest), + VMSTATE_STRUCT(cqe, NvmeRequest, 0, nvme_vmstate_cqe, NvmeCqe), + VMSTATE_STRUCT(cmd, NvmeRequest, 0, nvme_vmstate_cmd, NvmeCmd), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_hbs =3D { + .name =3D "nvme-hbs", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(acre, NvmeHostBehaviorSupport), + VMSTATE_UINT8(etdas, NvmeHostBehaviorSupport), + VMSTATE_UINT8(lbafee, NvmeHostBehaviorSupport), + VMSTATE_UINT8(rsvd3, NvmeHostBehaviorSupport), + VMSTATE_UINT16(cdfe, NvmeHostBehaviorSupport), + VMSTATE_UINT8_ARRAY(rsvd6, NvmeHostBehaviorSupport, 506), + VMSTATE_END_OF_LIST() + } +}; + +const VMStateDescription nvme_vmstate_atomic =3D { + .name =3D "nvme-atomic", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT32(atomic_max_write_size, NvmeAtomic), + VMSTATE_UINT64(atomic_boundary, NvmeAtomic), + VMSTATE_UINT64(atomic_nabo, NvmeAtomic), + VMSTATE_BOOL(atomic_writes, NvmeAtomic), + VMSTATE_END_OF_LIST() + } +}; + static const VMStateDescription nvme_vmstate =3D { .name =3D "nvme", - .unmigratable =3D 1, + .minimum_version_id =3D 1, + .version_id =3D 1, + .pre_save_errp =3D nvme_ctrl_pre_save, + .post_load_errp =3D nvme_ctrl_post_load, + .fields =3D (const VMStateField[]) { + VMSTATE_PCI_DEVICE(parent_obj, NvmeCtrl), + VMSTATE_MSIX(parent_obj, NvmeCtrl), + VMSTATE_STRUCT(bar, NvmeCtrl, 0, nvme_vmstate_bar, NvmeBar), + + VMSTATE_BOOL(qs_created, NvmeCtrl), + VMSTATE_UINT32(page_size, NvmeCtrl), + VMSTATE_UINT16(page_bits, NvmeCtrl), + VMSTATE_UINT16(max_prp_ents, NvmeCtrl), + VMSTATE_UINT32(max_q_ents, NvmeCtrl), + VMSTATE_UINT8(outstanding_aers, NvmeCtrl), + VMSTATE_UINT32(irq_status, NvmeCtrl), + VMSTATE_INT32(cq_pending, NvmeCtrl), + + VMSTATE_UINT64(host_timestamp, NvmeCtrl), + VMSTATE_UINT64(timestamp_set_qemu_clock_ms, NvmeCtrl), + VMSTATE_UINT64(starttime_ms, NvmeCtrl), + VMSTATE_UINT16(temperature, NvmeCtrl), + VMSTATE_UINT8(smart_critical_warning, NvmeCtrl), + + VMSTATE_UINT32(conf_msix_qsize, NvmeCtrl), + VMSTATE_UINT32(conf_ioqpairs, NvmeCtrl), + VMSTATE_UINT64(dbbuf_dbs, NvmeCtrl), + VMSTATE_UINT64(dbbuf_eis, NvmeCtrl), + VMSTATE_BOOL(dbbuf_enabled, NvmeCtrl), + + VMSTATE_UINT8(aer_mask, NvmeCtrl), + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT8_ALLOC( + aer_reqs, NvmeCtrl, outstanding_aers, 0, nvme_vmstate_request,= NvmeRequest), + VMSTATE_QTAILQ_V(aer_queue, NvmeCtrl, 1, nvme_vmstate_async_event, + NvmeAsyncEvent, entry), + VMSTATE_INT32(aer_queued, NvmeCtrl), + + VMSTATE_STRUCT(namespace, NvmeCtrl, 0, nvme_vmstate_ns, NvmeNamesp= ace), + + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC( + sq, NvmeCtrl, num_queues, 0, nvme_vmstate_squeue, NvmeSQueue), + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC( + cq, NvmeCtrl, num_queues, 0, nvme_vmstate_cqueue, NvmeCQueue), + + VMSTATE_UINT16(features.temp_thresh_hi, NvmeCtrl), + VMSTATE_UINT16(features.temp_thresh_low, NvmeCtrl), + VMSTATE_UINT32(features.async_config, NvmeCtrl), + VMSTATE_STRUCT(features.hbs, NvmeCtrl, 0, nvme_vmstate_hbs, NvmeHo= stBehaviorSupport), + + VMSTATE_UINT32(dn, NvmeCtrl), + VMSTATE_STRUCT(atomic, NvmeCtrl, 0, nvme_vmstate_atomic, NvmeAtomi= c), + + VMSTATE_END_OF_LIST() + }, }; =20 static void nvme_class_init(ObjectClass *oc, const void *data) diff --git a/hw/nvme/ns.c b/hw/nvme/ns.c index 38f86a17268..dd374677078 100644 --- a/hw/nvme/ns.c +++ b/hw/nvme/ns.c @@ -20,6 +20,7 @@ #include "qemu/bitops.h" #include "system/system.h" #include "system/block-backend.h" +#include "migration/vmstate.h" =20 #include "nvme.h" #include "trace.h" @@ -886,6 +887,164 @@ static void nvme_ns_realize(DeviceState *dev, Error *= *errp) } } =20 +static const VMStateDescription nvme_vmstate_lbaf =3D { + .name =3D "nvme_lbaf", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(ms, NvmeLBAF), + VMSTATE_UINT8(ds, NvmeLBAF), + VMSTATE_UINT8(rp, NvmeLBAF), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_id_ns =3D { + .name =3D "nvme_id_ns", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(nsze, NvmeIdNs), + VMSTATE_UINT64(ncap, NvmeIdNs), + VMSTATE_UINT64(nuse, NvmeIdNs), + VMSTATE_UINT8(nsfeat, NvmeIdNs), + VMSTATE_UINT8(nlbaf, NvmeIdNs), + VMSTATE_UINT8(flbas, NvmeIdNs), + VMSTATE_UINT8(mc, NvmeIdNs), + VMSTATE_UINT8(dpc, NvmeIdNs), + VMSTATE_UINT8(dps, NvmeIdNs), + VMSTATE_UINT8(nmic, NvmeIdNs), + VMSTATE_UINT8(rescap, NvmeIdNs), + VMSTATE_UINT8(fpi, NvmeIdNs), + VMSTATE_UINT8(dlfeat, NvmeIdNs), + VMSTATE_UINT16(nawun, NvmeIdNs), + VMSTATE_UINT16(nawupf, NvmeIdNs), + VMSTATE_UINT16(nacwu, NvmeIdNs), + VMSTATE_UINT16(nabsn, NvmeIdNs), + VMSTATE_UINT16(nabo, NvmeIdNs), + VMSTATE_UINT16(nabspf, NvmeIdNs), + VMSTATE_UINT16(noiob, NvmeIdNs), + VMSTATE_UINT8_ARRAY(nvmcap, NvmeIdNs, 16), + VMSTATE_UINT16(npwg, NvmeIdNs), + VMSTATE_UINT16(npwa, NvmeIdNs), + VMSTATE_UINT16(npdg, NvmeIdNs), + VMSTATE_UINT16(npda, NvmeIdNs), + VMSTATE_UINT16(nows, NvmeIdNs), + VMSTATE_UINT16(mssrl, NvmeIdNs), + VMSTATE_UINT32(mcl, NvmeIdNs), + VMSTATE_UINT8(msrc, NvmeIdNs), + VMSTATE_UINT8_ARRAY(rsvd81, NvmeIdNs, 18), + VMSTATE_UINT8(nsattr, NvmeIdNs), + VMSTATE_UINT16(nvmsetid, NvmeIdNs), + VMSTATE_UINT16(endgid, NvmeIdNs), + VMSTATE_UINT8_ARRAY(nguid, NvmeIdNs, 16), + VMSTATE_UINT64(eui64, NvmeIdNs), + VMSTATE_STRUCT_ARRAY(lbaf, NvmeIdNs, NVME_MAX_NLBAF, 1, + nvme_vmstate_lbaf, NvmeLBAF), + VMSTATE_UINT8_ARRAY(vs, NvmeIdNs, 3712), + + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_id_ns_nvm =3D { + .name =3D "nvme_id_ns_nvm", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(lbstm, NvmeIdNsNvm), + VMSTATE_UINT8(pic, NvmeIdNsNvm), + VMSTATE_UINT8_ARRAY(rsvd9, NvmeIdNsNvm, 3), + VMSTATE_UINT32_ARRAY(elbaf, NvmeIdNsNvm, NVME_MAX_NLBAF), + VMSTATE_UINT32(npdgl, NvmeIdNsNvm), + VMSTATE_UINT32(nprg, NvmeIdNsNvm), + VMSTATE_UINT32(npra, NvmeIdNsNvm), + VMSTATE_UINT32(nors, NvmeIdNsNvm), + VMSTATE_UINT32(npdal, NvmeIdNsNvm), + VMSTATE_UINT8_ARRAY(rsvd288, NvmeIdNsNvm, 3808), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_id_ns_ind =3D { + .name =3D "nvme_id_ns_ind", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(nsfeat, NvmeIdNsInd), + VMSTATE_UINT8(nmic, NvmeIdNsInd), + VMSTATE_UINT8(rescap, NvmeIdNsInd), + VMSTATE_UINT8(fpi, NvmeIdNsInd), + VMSTATE_UINT32(anagrpid, NvmeIdNsInd), + VMSTATE_UINT8(nsattr, NvmeIdNsInd), + VMSTATE_UINT8(rsvd9, NvmeIdNsInd), + VMSTATE_UINT16(nvmsetid, NvmeIdNsInd), + VMSTATE_UINT16(endgrpid, NvmeIdNsInd), + VMSTATE_UINT8(nstat, NvmeIdNsInd), + VMSTATE_UINT8_ARRAY(rsvd15, NvmeIdNsInd, 4081), + VMSTATE_END_OF_LIST() + } +}; + +typedef struct TmpNvmeNamespace { + NvmeNamespace *parent; + bool enable_write_cache; +} TmpNvmeNamespace; + +static bool nvme_ns_tmp_pre_save(void *opaque, Error **errp) +{ + struct TmpNvmeNamespace *tns =3D opaque; + + tns->enable_write_cache =3D blk_enable_write_cache(tns->parent->blkcon= f.blk); + + return true; +} + +static bool nvme_ns_tmp_post_load(void *opaque, int version_id, Error **er= rp) +{ + struct TmpNvmeNamespace *tns =3D opaque; + + blk_set_enable_write_cache(tns->parent->blkconf.blk, tns->enable_write= _cache); + + return true; +} + +static const VMStateDescription nvme_vmstate_ns_tmp =3D { + .name =3D "nvme_ns_tmp", + .pre_save_errp =3D nvme_ns_tmp_pre_save, + .post_load_errp =3D nvme_ns_tmp_post_load, + .fields =3D (const VMStateField[]) { + VMSTATE_BOOL(enable_write_cache, TmpNvmeNamespace), + VMSTATE_END_OF_LIST() + } +}; + +const VMStateDescription nvme_vmstate_ns =3D { + .name =3D "nvme_ns", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_WITH_TMP(NvmeNamespace, TmpNvmeNamespace, nvme_vmstate_ns_= tmp), + + VMSTATE_STRUCT(id_ns, NvmeNamespace, 0, nvme_vmstate_id_ns, NvmeId= Ns), + VMSTATE_STRUCT(id_ns_nvm, NvmeNamespace, 0, nvme_vmstate_id_ns_nvm= , NvmeIdNsNvm), + VMSTATE_STRUCT(id_ns_ind, NvmeNamespace, 0, nvme_vmstate_id_ns_ind= , NvmeIdNsInd), + VMSTATE_STRUCT(lbaf, NvmeNamespace, 0, nvme_vmstate_lbaf, NvmeLBAF= ), + VMSTATE_UINT32(nlbaf, NvmeNamespace), + VMSTATE_UINT8(csi, NvmeNamespace), + VMSTATE_UINT16(status, NvmeNamespace), + VMSTATE_UINT8(pif, NvmeNamespace), + + VMSTATE_UINT16(zns.zrwas, NvmeNamespace), + VMSTATE_UINT16(zns.zrwafg, NvmeNamespace), + VMSTATE_UINT32(zns.numzrwa, NvmeNamespace), + + VMSTATE_UINT32(features.err_rec, NvmeNamespace), + VMSTATE_STRUCT(atomic, NvmeNamespace, 0, nvme_vmstate_atomic, Nvme= Atomic), + VMSTATE_END_OF_LIST() + } +}; + static const Property nvme_ns_props[] =3D { DEFINE_BLOCK_PROPERTIES(NvmeNamespace, blkconf), DEFINE_PROP_BOOL("detached", NvmeNamespace, params.detached, false), @@ -937,6 +1096,7 @@ static void nvme_ns_class_init(ObjectClass *oc, const = void *data) dc->bus_type =3D TYPE_NVME_BUS; dc->realize =3D nvme_ns_realize; dc->unrealize =3D nvme_ns_unrealize; + dc->vmsd =3D &nvme_vmstate_ns; device_class_set_props(dc, nvme_ns_props); dc->desc =3D "Virtual NVMe namespace"; } diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 457b6637249..03d6aecb7d7 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -638,6 +638,7 @@ typedef struct NvmeCtrl { =20 NvmeNamespace namespace; NvmeNamespace *namespaces[NVME_MAX_NAMESPACES + 1]; + uint32_t num_queues; NvmeSQueue **sq; NvmeCQueue **cq; NvmeSQueue admin_sq; @@ -669,6 +670,7 @@ typedef struct NvmeCtrl { =20 /* Migration-related stuff */ Error *migration_blocker; + bool stop_processing_sq; } NvmeCtrl; =20 typedef enum NvmeResetType { @@ -749,4 +751,7 @@ void nvme_atomic_configure_max_write_size(bool dn, uint= 16_t awun, void nvme_ns_atomic_configure_boundary(bool dn, uint16_t nabsn, uint16_t nabspf, NvmeAtomic *atomic= ); =20 +extern const VMStateDescription nvme_vmstate_atomic; +extern const VMStateDescription nvme_vmstate_ns; + #endif /* HW_NVME_NVME_H */ diff --git a/hw/nvme/trace-events b/hw/nvme/trace-events index 6be0bfa1c1f..8e5544e0008 100644 --- a/hw/nvme/trace-events +++ b/hw/nvme/trace-events @@ -7,6 +7,17 @@ pci_nvme_dbbuf_config(uint64_t dbs_addr, uint64_t eis_addr= ) "dbs_addr=3D0x%"PRIx64 pci_nvme_map_addr(uint64_t addr, uint64_t len) "addr 0x%"PRIx64" len %"PRI= u64"" pci_nvme_map_addr_cmb(uint64_t addr, uint64_t len) "addr 0x%"PRIx64" len %= "PRIu64"" pci_nvme_map_prp(uint64_t trans_len, uint32_t len, uint64_t prp1, uint64_t= prp2, int num_prps) "trans_len %"PRIu64" len %"PRIu32" prp1 0x%"PRIx64" pr= p2 0x%"PRIx64" num_prps %d" +pci_nvme_pre_save_enter(void *n) "n=3D%p" +pci_nvme_pre_save_ns_drain(void *n, int i) "n=3D%p i=3D%d" +pci_nvme_pre_save_sq_out_req_drain_wait(void *n, int i, uint32_t head, uin= t32_t tail, uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRI= x32" size=3D0x%"PRIx32"" +pci_nvme_pre_save_sq_out_req_drain_wait_end(void *n, int i, uint32_t head,= uint32_t tail) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32"" +pci_nvme_pre_save_cq_req_drain_wait(void *n, int i, uint32_t head, uint32_= t tail, uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32"= size=3D0x%"PRIx32"" +pci_nvme_pre_save_cq_req_drain_wait_end(void *n, int i, uint32_t head, uin= t32_t tail) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32"" +pci_nvme_pre_save_aer(uint8_t typ, uint8_t info, uint8_t log_page) "type 0= x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" +pci_nvme_post_load_enter(void *n) "n=3D%p" +pci_nvme_post_load_restore_cq(void *n, int i, uint32_t head, uint32_t tail= , uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" size= =3D0x%"PRIx32"" +pci_nvme_post_load_restore_sq(void *n, int i, uint32_t head, uint32_t tail= , uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" size= =3D0x%"PRIx32"" +pci_nvme_post_load_aer(uint8_t typ, uint8_t info, uint8_t log_page) "type = 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" pci_nvme_map_sgl(uint8_t typ, uint64_t len) "type 0x%"PRIx8" len %"PRIu64"" pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode= , const char *opname) "cid %"PRIu16" nsid 0x%"PRIx32" sqid %"PRIu16" opc 0x= %"PRIx8" opname '%s'" pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char= *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" --=20 2.47.3 From nobody Mon Apr 6 23:46:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773743352; cv=none; d=zohomail.com; s=zohoarc; b=IbB8bf950cl0MX4ZuTtZOWC1dtnfai3Wd67pXhIk6P/7kO50rmi89dc1jzGyrRGmEB+ioDA+VBFEILcAZhwPmH2n848QkCXyasaKo6j/nYzhC6lJFR1EXAo7XfUCVqj1Q40/bV5tRkmMC3vpvbwxJ5GfDm89N4I3ZebQwem6AOM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773743352; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=sBpcKOoEk+DQ0qOwh5vSLzEuS5T0qQxiJFTEUttDnck=; b=OdYukwq5AHmS/Rw5chGfFYL+2BC1zjrMZ7BXJmgWy4JvHHPtZI/zQlNhzQVQrH8vBmGPwMCQRSIJF1VEoIFRndFUFWSkHyLWH2W1qlpSLhGDIVzspHrPqVPgNzAAkDsCcaPem7npjbuUAcCBL1DC9DF/cCEPB6Qoks2FMinWjJM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 177374335268356.877922014094906; Tue, 17 Mar 2026 03:29:12 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w2ReT-00025q-CB; Tue, 17 Mar 2026 06:27:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w2Re2-0001vy-9F for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:27 -0400 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w2Rdx-0006Wv-0I for qemu-devel@nongnu.org; Tue, 17 Mar 2026 06:27:23 -0400 Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-48539d21b76so39600905e9.1 for ; Tue, 17 Mar 2026 03:27:19 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c0051af80c54da1a9bc.dip0.t-ipconnect.de. [2003:cf:5722:8c00:51af:80c5:4da1:a9bc]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4856ea9c36bsm61977665e9.9.2026.03.17.03.27.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2026 03:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773743239; x=1774348039; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sBpcKOoEk+DQ0qOwh5vSLzEuS5T0qQxiJFTEUttDnck=; b=PNYRmLzne/u/cxds08bAWoh6ANAXvWUkBh8H9ZL/PvI/zTJfT78v5JBjrW0eduWrJe rFGpc5WNpQIZNu0aGIdAundLG4RWXh+7Mef+4CnqLMIp8NzTK8TgMFQ/SaPc7AMSVAb5 gJS//rAb3SR6pgPOpxAelJRHPG8FgTR8uY8oc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773743239; x=1774348039; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sBpcKOoEk+DQ0qOwh5vSLzEuS5T0qQxiJFTEUttDnck=; b=smbPT7en8KIVCyMSTh5dODmYtoYzLoiF5WBEmCCvVfHKm/4pNjfkXVU/ilOjbYdKCr eAWYj6wAi7N++NlHFqMkJ5jHgSZ/sxSjTuHFZICE8v4EdIFs8EMsap0E2SPkrOfnE15c SqkAHyN3cCI7ENYP3QAdrcHYQBuA+P8Ddv4bLedYTN+c7WHaJp7M4HSQY1xx6vS0Z6sP gQSGHKRqhRDmrcmiWPToxylqUMo9FGnNIqWlFxdyhhCxNdb0PK7yL75UTBL6MfED/Vyd QxjXrioRLQbihP9uYW6p/8yFDLNmKOIgH8btjIpnk01793U1OZ9WJpDRgoqqjOK8ALYn 4vvQ== X-Gm-Message-State: AOJu0YwWkpO/hajDq+sFs+sIQkJfWvOMBzl1fy7JlNIlr7nf54iHnPIf LZx0hEcMAhtLTRSZtKPUl0exbTN3pgPsPyr4Oafu4l0c0U14Q5e/jPKIDBzwr4fefLqRa2kR1mK o8NgCpwQ= X-Gm-Gg: ATEYQzwpcsXxFBVrJ7Pu2zbyoagwIzR6/W3TVKbeR+9QpAQfJOpxQFCZd4RlJmk5nXi FiKvPjNfBXPhysmChF7iRurc/xzRqc7aunB4JDqqZcbBxw7JcYA3YKOr5mbK3u6Kqtxc2ZeuK1O 9+S0viP85PU34qWmUQ08hlCsxwtYShsWhLdJlK1fB8AdLriNHic0YJ0IYoToXBJ0w1i4lz8p9Mg sHBH+LcujBZ47Y3nmx7bAqBT1WeWkXyUaH4kph1DfwjKDIRshNeiZubeNtxwVYaWxv8SChkm0Oc JRA69H9j5gnHghNjDLxNM44jbJPoMCY5NYnT8lHTjxJFoF+cIxS9WbCN7R++QYk5Dbu4QVxA7Fa 4D9+9uus0ZkQeZE/EagBzPS6g0ieEWqbSTmN71d6ycgO2mvqzeqQvwF9u/UW0r39qvFdRf3PhBt eNcufmwTDdUPkZ+vKsFvBvZTYVrrZy+QBkBOfq+u13ntrHPNr5pm5wky1apFLnyC3kGfQqPM/4g 1i2FjXmgMaual6W9WTQzv8= X-Received: by 2002:a05:600c:4f54:b0:485:3baa:af14 with SMTP id 5b1f17b1804b1-48556700c18mr277470205e9.18.1773743238801; Tue, 17 Mar 2026 03:27:18 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Alexander Mikhalitsyn , Peter Xu , Fabiano Rosas , Jesper Devantier , Klaus Jensen , =?UTF-8?q?St=C3=A9phane=20Graber?= , qemu-block@nongnu.org, Stefan Hajnoczi , Hanna Reitz , Paolo Bonzini , Keith Busch , Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Zhao Liu , Kevin Wolf , Alexander Mikhalitsyn Subject: [PATCH v5 8/8] tests/functional/x86_64: add migration test for NVMe device Date: Tue, 17 Mar 2026 11:27:08 +0100 Message-ID: <20260317102708.126725-9-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260317102708.126725-1-alexander@mihalicyn.com> References: <20260317102708.126725-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=alexander@mihalicyn.com; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773743355995154100 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Introduce a very simple test to ensure that NVMe device migration works fine. Test plan is simple: 1. prepare VM with NVMe device 2. run workload that produces relatively heavy IO on the device 3. migrate VM 4. ensure that workload is alive and finishes without errors Test can be run as simple as: $ meson test 'func-x86_64-nvme_migration' --setup thorough -C build In the future we can extend this approach, and introduce some fio-based tests. And probably, it makes sense to make this test to apply not only to NVMe device, but also virtio-{blk,scsi}, ide, sata and other migratable devices. Signed-off-by: Alexander Mikhalitsyn --- tests/functional/x86_64/meson.build | 1 + .../functional/x86_64/test_nvme_migration.py | 159 ++++++++++++++++++ 2 files changed, 160 insertions(+) create mode 100755 tests/functional/x86_64/test_nvme_migration.py diff --git a/tests/functional/x86_64/meson.build b/tests/functional/x86_64/= meson.build index 1ed10ad6c29..fd77f19d726 100644 --- a/tests/functional/x86_64/meson.build +++ b/tests/functional/x86_64/meson.build @@ -37,6 +37,7 @@ tests_x86_64_system_thorough =3D [ 'linux_initrd', 'multiprocess', 'netdev_ethtool', + 'nvme_migration', 'replay', 'reverse_debug', 'tuxrun', diff --git a/tests/functional/x86_64/test_nvme_migration.py b/tests/functio= nal/x86_64/test_nvme_migration.py new file mode 100755 index 00000000000..3788a8e3473 --- /dev/null +++ b/tests/functional/x86_64/test_nvme_migration.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# x86_64 NVMe migration test + +from migration import MigrationTest +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern +from qemu_test import exec_command, exec_command_and_wait_for_pattern + + +class X8664NVMeMigrationTest(MigrationTest): + ASSET_KERNEL =3D Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/rele= ases' + '/31/Server/x86_64/os/images/pxeboot/vmlinuz'), + 'd4738d03dbbe083ca610d0821d0a8f1488bebbdccef54ce33e3adb35fda00129') + + ASSET_INITRD =3D Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/rele= ases' + '/31/Server/x86_64/os/images/pxeboot/initrd.img'), + '277cd6c7adf77c7e63d73bbb2cded8ef9e2d3a2f100000e92ff1f8396513cd8b') + + ASSET_DISKIMAGE =3D Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/rele= ases' + '/31/Cloud/x86_64/images/Fedora-Cloud-Base-31-1.9.x86_64.qcow2'), + 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0') + + DEFAULT_KERNEL_PARAMS =3D ('root=3D/dev/nvme0n1p1 console=3DttyS0 net.= ifnames=3D0 ' + 'rd.rescue quiet') + + def wait_for_console_pattern(self, success_message, vm): + wait_for_console_pattern( + self, + success_message, + failure_message=3D"Kernel panic - not syncing", + vm=3Dvm, + ) + + def exec_command_and_check(self, command, vm): + prompt =3D '# ' + exec_command_and_wait_for_pattern(self, + f"{command} && echo OK || echo FAI= L", + 'FAIL', vm=3Dvm) + # Note, that commands we send to the console are echo-ed back, so = if we have a word "FAIL" + # in the command itself, we should expect to see it once. + wait_for_console_pattern(self, 'OK', failure_message=3D"FAIL", vm= =3Dvm) + self.wait_for_console_pattern(prompt, vm) + + def configure_machine(self, vm): + kernel_path =3D self.ASSET_KERNEL.fetch() + initrd_path =3D self.ASSET_INITRD.fetch() + diskimage_path =3D self.ASSET_DISKIMAGE.fetch() + + vm.set_console() + vm.add_args("-cpu", "max") + vm.add_args("-m", "2G") + vm.add_args("-accel", "kvm") + + vm.add_args('-drive', + f'file=3D{diskimage_path},if=3Dnone,id=3Ddrv0,sna= pshot=3Don') + vm.add_args('-device', 'nvme,bus=3Dpcie.0,' + + 'drive=3Ddrv0,id=3Dnvme-disk0,serial=3Dnvmemigrat= etest,bootindex=3D1') + + vm.add_args( + "-kernel", + kernel_path, + "-initrd", + initrd_path, + "-append", + self.DEFAULT_KERNEL_PARAMS + ) + + def launch_source_vm(self, vm): + vm.launch() + + self.wait_for_console_pattern('Entering emergency mode.', vm) + prompt =3D '# ' + self.wait_for_console_pattern(prompt, vm) + + # Synchronize on NVMe driver creating the root device + exec_command_and_wait_for_pattern(self, + "while ! (dmesg -c | grep nvme0n1:) ; do sleep 1 ;= done", + "nvme0n1", vm=3Dvm) + self.wait_for_console_pattern(prompt, vm) + + # prepare system + exec_command_and_wait_for_pattern(self, 'mount /dev/nvme0n1p1 /sys= root', + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, 'chroot /sysroot', + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, 'mount -t proc proc /proc', + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, 'mount -t sysfs sysfs /sys= ', + prompt, vm=3Dvm) + + # Run workload before migration to check if it continues to run pr= operly after migration + # + # Workload is simple: it continuously calculates checksums of all = files in /usr/bin + # to generate some I/O load on the NVMe disk and at the same time = it drops caches to + # make sure that we have some read I/O on the disk as well. + # If there are any issues with the migration of the NVMe device, w= e should see errors + # in dmesg and consequently in the workload log. + exec_command_and_wait_for_pattern(self, + "(while [ ! -f /tmp/test_nvme_migr= ation_workload.stop ]; do \ + rm -f /tmp/test_nvme_migration= _workload.iteration_finished; \ + echo 3 > /proc/sys/vm/drop_cac= hes; \ + find /usr/bin -type f -exec ck= sum {} \\;; \ + touch /tmp/test_nvme_migration= _workload.iteration_finished; \ + done) > /dev/null 2> /tmp/test_nvm= e_migration_workload.errors &", + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, 'echo $! > /tmp/test_nvme_= migration_workload.pid', + prompt, vm=3Dvm) + + # check if process is alive and running + self.exec_command_and_check("kill -0 $(cat /tmp/test_nvme_migratio= n_workload.pid)", vm) + + def assert_dest_vm(self, vm): + prompt =3D '# ' + + # check if process is alive and running after migration, if not - = fail the test + self.exec_command_and_check("kill -0 $(cat /tmp/test_nvme_migratio= n_workload.pid)", vm) + + # signal workload to stop + exec_command_and_wait_for_pattern(self, 'touch /tmp/test_nvme_migr= ation_workload.stop', + prompt, vm=3Dvm) + + # wait workload to finish, because we want to examine log to see i= f there are any errors + exec_command_and_wait_for_pattern(self, + "while [ ! -f /tmp/test_nvme_migra= tion_workload.iteration_finished ]; do sleep 1; done;", + prompt, vm=3Dvm) + + exec_command_and_wait_for_pattern(self, 'cat /tmp/test_nvme_migrat= ion_workload.errors', + prompt, vm=3Dvm) + + # fail the test if non-empty + self.exec_command_and_check("[ ! -s /tmp/test_nvme_migration_workl= oad.errors ]", vm) + + def test_migration_with_tcp_localhost(self): + self.set_machine('q35') + self.require_accelerator("kvm") + + self.migration_with_tcp_localhost() + + def test_migration_with_unix(self): + self.set_machine('q35') + self.require_accelerator("kvm") + + self.migration_with_unix() + + def test_migration_with_exec(self): + self.set_machine('q35') + self.require_accelerator("kvm") + + self.migration_with_exec() + + +if __name__ =3D=3D '__main__': + MigrationTest.main() --=20 2.47.3