From nobody Tue Apr 7 12:56:58 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=mihalicyn.com ARC-Seal: i=1; a=rsa-sha256; t=1773411249; cv=none; d=zohomail.com; s=zohoarc; b=ldDF3HxZXv5t08F258W/ih5n26OmepCIsgazWJUz0l/a+PnVGsjDFyVjGlt63kaZf8E4rlTrFsCQni9moq492RZNLtksmmM3vAhKSl8GAqZ4ki3Yjlp90lETV2Sfh0jCjgaXoFZmOZu1YVTpeScjq/eoAhJzeMS5qTflOp7j2Xs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773411249; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SEO6/lrYdm4dOs+1aF7CdwIzHMn2tXQZYpY2v/1+t2k=; b=n/BKXFUTfAf2hTamqQm0keBgWjZKXp8ltjW6MDV+S8uMRWTDjR4NQ//pv4xc5UjNStXBFFeS6fzOMnZEBlvOKxjLnAHi6rJtZREpBfQvQRNjFvTqmUUKHWS4XU+XmIkwym+DluVhCDtOm++60WFoqhZuDtVnVF6duw9UBvxyL0U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17734112491841002.9126863636183; Fri, 13 Mar 2026 07:14:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w13GA-0005TD-EF; Fri, 13 Mar 2026 10:13:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w13Fw-0005QX-C6 for qemu-devel@nongnu.org; Fri, 13 Mar 2026 10:12:51 -0400 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1w13Fl-0006Mx-Tj for qemu-devel@nongnu.org; Fri, 13 Mar 2026 10:12:45 -0400 Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-439bcec8613so1806668f8f.3 for ; Fri, 13 Mar 2026 07:12:31 -0700 (PDT) Received: from alex-laptop.lan (p200300cf57228c00321ddb23f48fbb73.dip0.t-ipconnect.de. [2003:cf:5722:8c00:321d:db23:f48f:bb73]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-439fe19ad9asm18580939f8f.7.2026.03.13.07.12.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Mar 2026 07:12:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mihalicyn.com; s=mihalicyn; t=1773411150; x=1774015950; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SEO6/lrYdm4dOs+1aF7CdwIzHMn2tXQZYpY2v/1+t2k=; b=IyrcrCu8qOoA/cORk1FlyMFdX6dOxxhtyA3cI/iIcPxnJYKmvqLqqWuzgYLjU4n1Uc VgDEYnOSd2cviuJgO2KSXtM08P1H3xNk1ajq4iiv9FX/Bcde/tEQVeh4KbpwvpFf8RtK Sj/Y8QxYFz7ZPaWLD1V2823PrDdtVgD0cbFy4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773411150; x=1774015950; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=SEO6/lrYdm4dOs+1aF7CdwIzHMn2tXQZYpY2v/1+t2k=; b=VlmCIh8VoQ0uwMRmTzfo37aC0QKdeAUjdo9DLZPsqvY3ntASpvmeA/bJCUrRA6h/34 b4sPOtN48oqhi2a914LWvxG1nBzFJ6cfoNHnJJ6LbwB/YSjLsz2FaqPZwu8i1jo8KO2+ u98NBlqoNK6sxsir9K/+Shx+f/VBfoy4YofKcjXIC3ZClthQe7bv4XBntfblosqL9gFO POvwq9ZIM1Ctn1vuG3eMAqSwuuMQZPKte7cHrXQCd0LiKdJiz/sOuFgLf7ozvEsdf1bK bKfMyie75Fg/RPoU79ogElJ6QUQsmYT4c4Nr/XsotcQ5+GgKbiXE8r5Ee30/LpEEisUM qoBg== X-Gm-Message-State: AOJu0YyJDQyBn0DvjRWpEcviNYykEgGIkCjmV0VluRCkXOdGZhcU0xLD bcycEmPL2l2qSrmWgLXM+XOolKZ/7AHpLTrp7fGk1GPfUgmkmT6nJk2n3LRfW9x2FUBXrE9wdx9 EHeWkUl0= X-Gm-Gg: ATEYQzxxAoXvzNCKxLUGXGN8ROUaBg8LI0LumZ+J4x5H80Xb6jwiPHwbRuZJJEyVfPs xM8SfyXuJVDQwf+Q6wmUN4zSP9YwEhhFRDtkkMyn57xHS1aAthobATSQwokoMwxOQ/AE9WPWmaZ AVAl3KeW+hYyWgMVe/iHhniS6ugQxUSpf9PAV96ePPCOLiaq8O5QJwyh+QFrAPnohU1V1qhryFd PZSxeWtwKJfByczwtco+KklQVCUAGOawNxyGQrs28SnuNzHELQxM1jbqMkG/OZfCt37K7plhCUj KAZmm2anEYl40H0wrvZCk+DQBGwRGeLO7rJJsTf+eeVtg4PhwBkoNVYTX1bpQiG9IgJnqP6NZJZ 7aMejlw7Po0uePumZNp6qEjM9EwExdGksoHJSsL5caqag7zKJuMYU1gshXiRA5Pgthc5B7U7VhB kG3d31iyyUkdyS5hBGgIbA4Z4MXWoLB6AGwBNbmCT2iLPlkBvc8ooTbip5yLgOJTCcOhgqnRpbt RGXC5IwSTs4/mfFwoIety/ZbE/jUGmQbw== X-Received: by 2002:adf:f20e:0:b0:439:fbbc:d593 with SMTP id ffacd0b85a97d-43a04db7d5amr4936542f8f.27.1773411150091; Fri, 13 Mar 2026 07:12:30 -0700 (PDT) From: Alexander Mikhalitsyn To: qemu-devel@nongnu.org Cc: Peter Xu , Jesper Devantier , Kevin Wolf , Fabiano Rosas , Paolo Bonzini , Hanna Reitz , qemu-block@nongnu.org, =?UTF-8?q?St=C3=A9phane=20Graber?= , Keith Busch , Stefan Hajnoczi , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Alexander Mikhalitsyn , Fam Zheng , Klaus Jensen , Zhao Liu , Alexander Mikhalitsyn Subject: [PATCH v4 5/8] hw/nvme: add migration blockers for non-supported cases Date: Fri, 13 Mar 2026 15:12:18 +0100 Message-ID: <20260313141221.359503-6-alexander@mihalicyn.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260313141221.359503-1-alexander@mihalicyn.com> References: <20260313141221.359503-1-alexander@mihalicyn.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42f; envelope-from=alexander@mihalicyn.com; helo=mail-wr1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @mihalicyn.com) X-ZM-MESSAGEID: 1773411249695158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Let's block migration for cases we don't support: - SR-IOV - CMB - PMR - SPDM No functional changes here, because NVMe migration is not supported at all as of this commit. Signed-off-by: Alexander Mikhalitsyn --- hw/nvme/ctrl.c | 206 +++++++++++++++++++++++++++++++++++++++++++ hw/nvme/nvme.h | 3 + include/block/nvme.h | 12 +++ 3 files changed, 221 insertions(+) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index cc4593cd427..9f9c9bcbead 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -207,6 +207,7 @@ #include "hw/pci/msix.h" #include "hw/pci/pcie_sriov.h" #include "system/spdm-socket.h" +#include "migration/blocker.h" #include "migration/vmstate.h" =20 #include "nvme.h" @@ -250,6 +251,7 @@ static const bool nvme_feature_support[NVME_FID_MAX] = =3D { [NVME_COMMAND_SET_PROFILE] =3D true, [NVME_FDP_MODE] =3D true, [NVME_FDP_EVENTS] =3D true, + /* if you add something here, please update nvme_set_migration_blocker= s() */ }; =20 static const uint32_t nvme_feature_cap[NVME_FID_MAX] =3D { @@ -4601,6 +4603,7 @@ static uint16_t nvme_io_mgmt_send(NvmeCtrl *n, NvmeRe= quest *req) return 0; case NVME_IOMS_MO_RUH_UPDATE: return nvme_io_mgmt_send_ruh_update(n, req); + /* if you add something here, please update nvme_set_migration_blocker= s() */ default: return NVME_INVALID_FIELD | NVME_DNR; }; @@ -7518,6 +7521,10 @@ static uint16_t nvme_security_receive(NvmeCtrl *n, N= vmeRequest *req) =20 static uint16_t nvme_directive_send(NvmeCtrl *n, NvmeRequest *req) { + /* + * When adding a new dtype handling here, + * please also update nvme_set_migration_blockers(). + */ return NVME_INVALID_FIELD | NVME_DNR; } =20 @@ -9208,6 +9215,199 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *= pci_dev) } } =20 +#define BLOCKER_FEATURES_MAX_LEN 256 + +static inline void nvme_add_blocker_feature(char *blocker_features, const = char *feature) +{ + if (strlen(blocker_features) > 0) { + g_strlcat(blocker_features, ", ", BLOCKER_FEATURES_MAX_LEN); + } + g_strlcat(blocker_features, feature, BLOCKER_FEATURES_MAX_LEN); +} + +static bool nvme_set_migration_blockers(NvmeCtrl *n, PCIDevice *pci_dev, E= rror **errp) +{ + uint64_t unsupported_cap, cap =3D ldq_le_p(&n->bar.cap); + char blocker_features[BLOCKER_FEATURES_MAX_LEN] =3D ""; + bool adm_cmd_security_checked =3D false; + bool cmd_io_mgmt_checked =3D false; + bool cmd_zone_checked =3D false; + + /* + * Idea of this function is simple, we iterate over all Command Sets a= nd + * for each supported command we provide a special handling logic to + * determine if we should block migration or not. + * + * For instance, we have NVME_ADM_CMD_NS_ATTACHMENT and it is always + * available to the guest, but if there is only 1 namespace, then it is + * safe to allow migration, but if there are more, then we need to blo= ck + * migration because we don't handle this in migration code yet. + */ + for (int opcode =3D 0; opcode < sizeof(n->cse.acs) / sizeof(n->cse.acs= [0]); opcode++) { + /* Is command supported? */ + if (!n->cse.acs[opcode]) { + continue; + } + + switch (opcode) { + case NVME_ADM_CMD_DELETE_SQ: + case NVME_ADM_CMD_CREATE_SQ: + case NVME_ADM_CMD_GET_LOG_PAGE: + case NVME_ADM_CMD_DELETE_CQ: + case NVME_ADM_CMD_CREATE_CQ: + case NVME_ADM_CMD_IDENTIFY: + case NVME_ADM_CMD_ABORT: + case NVME_ADM_CMD_SET_FEATURES: + case NVME_ADM_CMD_GET_FEATURES: + case NVME_ADM_CMD_ASYNC_EV_REQ: + case NVME_ADM_CMD_DBBUF_CONFIG: + case NVME_ADM_CMD_FORMAT_NVM: + case NVME_ADM_CMD_DIRECTIVE_SEND: + case NVME_ADM_CMD_DIRECTIVE_RECV: + break; + case NVME_ADM_CMD_NS_ATTACHMENT: + int namespaces_num =3D 0; + for (int i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + if (!ns) { + continue; + } + + namespaces_num++; + } + + if (namespaces_num > 1) { + nvme_add_blocker_feature(blocker_features, "Namespace Atta= chment"); + } + + break; + case NVME_ADM_CMD_VIRT_MNGMT: + if (n->params.sriov_max_vfs) { + nvme_add_blocker_feature(blocker_features, "SR-IOV"); + } + + break; + case NVME_ADM_CMD_SECURITY_SEND: + case NVME_ADM_CMD_SECURITY_RECV: + if (adm_cmd_security_checked) { + break; + } + + if (pci_dev->spdm_port) { + nvme_add_blocker_feature(blocker_features, "SPDM"); + } + + adm_cmd_security_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + for (int opcode =3D 0; opcode < sizeof(n->cse.iocs.nvm) / sizeof(n->cs= e.iocs.nvm[0]); opcode++) { + if (!n->cse.iocs.nvm[opcode]) { + continue; + } + + switch (opcode) { + case NVME_CMD_FLUSH: + case NVME_CMD_WRITE: + case NVME_CMD_READ: + case NVME_CMD_COMPARE: + case NVME_CMD_WRITE_ZEROES: + case NVME_CMD_DSM: + case NVME_CMD_VERIFY: + case NVME_CMD_COPY: + break; + case NVME_CMD_IO_MGMT_RECV: + case NVME_CMD_IO_MGMT_SEND: + if (cmd_io_mgmt_checked) { + break; + } + + /* check for NVME_IOMS_MO_RUH_UPDATE */ + if (n->subsys->params.fdp.enabled) { + nvme_add_blocker_feature(blocker_features, "FDP"); + } + + cmd_io_mgmt_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + for (int opcode =3D 0; opcode < sizeof(n->cse.iocs.zoned) / sizeof(n->= cse.iocs.zoned[0]); opcode++) { + /* + * If command isn't supported or we have the same command + * in n->cse.iocs.nvm, then we can skip it here. + */ + if (!n->cse.iocs.zoned[opcode] || n->cse.iocs.nvm[opcode]) { + continue; + } + + switch (opcode) { + case NVME_CMD_ZONE_APPEND: + case NVME_CMD_ZONE_MGMT_SEND: + case NVME_CMD_ZONE_MGMT_RECV: + if (cmd_zone_checked) { + break; + } + + for (int i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + if (!ns) { + continue; + } + + if (ns->params.zoned) { + nvme_add_blocker_feature(blocker_features, "Zoned Name= space"); + break; + } + } + + cmd_zone_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + /* + * Try our best to explicitly detect all not supported caps, + * to let users know what features cause migration to be blocked, + * but in case we miss handling here, everything else will be + * covered by unsupported_cap check. + */ + if (NVME_CAP_CMBS(cap)) { + nvme_add_blocker_feature(blocker_features, "CMB"); + cap &=3D ~((uint64_t)CAP_CMBS_MASK << CAP_CMBS_SHIFT); + } + + if (NVME_CAP_PMRS(cap)) { + nvme_add_blocker_feature(blocker_features, "PMR"); + cap &=3D ~((uint64_t)CAP_PMRS_MASK << CAP_PMRS_SHIFT); + } + + unsupported_cap =3D cap & ~NVME_MIGRATION_SUPPORTED_CAP_BITS; + if (unsupported_cap) { + nvme_add_blocker_feature(blocker_features, "unknown capability"); + } + + assert(n->migration_blocker =3D=3D NULL); + if (strlen(blocker_features) > 0) { + error_setg(&n->migration_blocker, "Migration is not supported for = %s", blocker_features); + if (migrate_add_blocker(&n->migration_blocker, errp) < 0) { + return false; + } + } + + return true; +} + static int nvme_init_subsys(NvmeCtrl *n, Error **errp) { int cntlid; @@ -9313,6 +9513,10 @@ static void nvme_realize(PCIDevice *pci_dev, Error *= *errp) =20 n->subsys->namespaces[ns->params.nsid] =3D ns; } + + if (!nvme_set_migration_blockers(n, pci_dev, errp)) { + return; + } } =20 static void nvme_exit(PCIDevice *pci_dev) @@ -9365,6 +9569,8 @@ static void nvme_exit(PCIDevice *pci_dev) } =20 memory_region_del_subregion(&n->bar0, &n->iomem); + + migrate_del_blocker(&n->migration_blocker); } =20 static const Property nvme_props[] =3D { diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index d66f7dc82d5..457b6637249 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -666,6 +666,9 @@ typedef struct NvmeCtrl { =20 /* Socket mapping to SPDM over NVMe Security In/Out commands */ int spdm_socket; + + /* Migration-related stuff */ + Error *migration_blocker; } NvmeCtrl; =20 typedef enum NvmeResetType { diff --git a/include/block/nvme.h b/include/block/nvme.h index 9d7159ed7a7..a7f586fc801 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -141,6 +141,18 @@ enum NvmeCapMask { #define NVME_CAP_SET_CMBS(cap, val) \ ((cap) |=3D (uint64_t)((val) & CAP_CMBS_MASK) << CAP_CMBS_SHIFT) =20 +#define NVME_MIGRATION_SUPPORTED_CAP_BITS ( \ + ((uint64_t)CAP_MQES_MASK << CAP_MQES_SHIFT) \ + | ((uint64_t)CAP_CQR_MASK << CAP_CQR_SHIFT) \ + | ((uint64_t)CAP_AMS_MASK << CAP_AMS_SHIFT) \ + | ((uint64_t)CAP_TO_MASK << CAP_TO_SHIFT) \ + | ((uint64_t)CAP_DSTRD_MASK << CAP_DSTRD_SHIFT) \ + | ((uint64_t)CAP_NSSRS_MASK << CAP_NSSRS_SHIFT) \ + | ((uint64_t)CAP_CSS_MASK << CAP_CSS_SHIFT) \ + | ((uint64_t)CAP_MPSMIN_MASK << CAP_MPSMIN_SHIFT) \ + | ((uint64_t)CAP_MPSMAX_MASK << CAP_MPSMAX_SHIFT) \ +) + enum NvmeCapCss { NVME_CAP_CSS_NCSS =3D 1 << 0, NVME_CAP_CSS_IOCSS =3D 1 << 6, --=20 2.47.3