From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783377901; cv=none; d=zohomail.com; s=zohoarc; b=SE1/LRjHT8A9RpR+NL6pRbHTX+Wj3X2OoWq2McG89khLKG8HzYExn5/c0Sx2WJD15QJcz2RLsZ9n/CvYvV9g6FIRLwTM3k8C5uNWnBzsnLBF8kaA9OLsrBtAVp3e4RXaxSZ6ntFGCudOEcgmt/wFt9s4QPgpb9gIdvfo7qZtrWA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783377901; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=01r+ZCPo4PvZmuDTdlJehJjsdM0SEGXP4aeZBzGR2Kg=; b=kCHPGx9gtF5kT0ystyw5K6odFKou9BgtPszG3kNQ6y7E77Ca8MX/dt9EYR3aeYJytyeHD9bLF5FRfgb+CKQudL9vu4drLZepoLtk8AACsbjAntsyV8wpbTYq+3O4a6K5w3v7mEZPkQ98UfGj0to1IM2YyrREy7VYzGxMthITQCo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783377901358340.7927176250172; Mon, 6 Jul 2026 15:45:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3Y-0001sG-Vj; Mon, 06 Jul 2026 18:44:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3W-0001j5-59; Mon, 06 Jul 2026 18:44:50 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3U-0000wF-6t; Mon, 06 Jul 2026 18:44:49 -0400 Received: from phl-compute-01.internal (phl-compute-01.internal [10.202.2.41]) by mailfhigh.stl.internal (Postfix) with ESMTP id B13677A01DD; Mon, 6 Jul 2026 18:44:46 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Mon, 06 Jul 2026 18:44:46 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:44:44 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377886; x= 1783464286; bh=01r+ZCPo4PvZmuDTdlJehJjsdM0SEGXP4aeZBzGR2Kg=; b=U 2D3KqnapzskqeZXUm4neikSg/UeWfF2fV8XSg2bMPGAMj7wM0Kv/cjuOhpKHLiKx UQpL1PNWKKb5BkIR+YH16guMcbia5MhAFjsRDbLVyi7qcra7qLhjaN+iXgNe1xXJ gHEb/NZj7kJkBrNPf5M3asGmwZmQfMrRHMd2EsE0Zib0qsNwPcwwWUvD6RzsmQlD BlFeP9aY4zfyBF5KoT+ykv2TyAXUh+/YOKXkuhNgQaoVnyyId+ialN/+x/WpbeTy Lgs9jJGRria1AquLOuWIbKyRrV5n7WKIsY7reZWqbPtkRQOg65ohWkUt41lb36ud g0lVzxeVvf0s+ZrDgveDA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377886; x=1783464286; bh=0 1r+ZCPo4PvZmuDTdlJehJjsdM0SEGXP4aeZBzGR2Kg=; b=aSCjByYr12qhp0zWB q+erJLd9SDJNwjRCFrl5KctFxdq6jx9uBKLad7AgX8uYUVMn1o4vnF1EZJlUAbww XTnYq7+milgLgtdYfYwzHmwoXJVbQeTq582ULQ9/T3yf79nZljtnbAeAV6ONJc7P o60gbUpnzKMnTdf4b/E+CT3prTJIgQvoV6jMbpbXv7Op6Sbwo9BTZ18oNsNO8Os6 yvgqne9G/A1iyRs3jloQ9z5YXz1mzlS1I4zReCtZrYYvn8qIzyqk0xBQqM0s6LqW ToewyP0eShW4K83+ghQAlH0kn4JhVbjdmK/mhfaGJz1tE0j3VTOUkiQdJbm6tn11 uVbxg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTE7v849ilEkFjUpDUSaiq1uMOu/R/xvF5klO3DvrAI67IkU8xQbZDLRCZPCZZpXDZ 8IVKKHf3bURWtK1H7DUwaXYv3vqf6fJmJiV7VYlGtvWMpLfpKXnnn/Wu6+a2NLJxfuIdFs s5jMRUhwn8Ox0g3k8pgQSZHfcNR/ft2gFinCk8awAzyiaa6sO81d1EQpyql7nyXV6WnUvO P/B/f2HNVaOTGbrzdG02lmCkzzrC6VhvavK+aYDnZ9M4MM2GWaUjxSC2Qy7esF9vrsEzeP OGYzuX8S63Q0tuMS0LImlTPNY6W7TfP5vYgULcx+rZyG1i5W71Tve2PJJ3/RGeJ9RMSzjy 4VIubdX05dsIqSc29Z72Q8fIqSr2F1LzY0xzPN5I4jbeB5K/sMnK70J9RwP91I1R21B//o nG3apUA723RhGHEpmSM+uUx8Pz9mS4laodNOUSFMCpt1PZtmCwShfk5VneZpPirIx+MifI CXWRipLom3xj53BNrMFWu1plg/t0K2gOOHDsyV1qp232/3GGMIH2Y5AJP5NrGt5v2gkae9 wtuhFcJxAaaitm/lXjefRpSgrIXHiItDC90DZEyhmi5y8sK5SrzmBle2ix9azde5UEmqBP 39ih2bq+6yeEL9wvi9mWm5oJpUgPZGqmYH41wnANTwdwOwhnbMxXAg6wVqnQ X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Jesper Wendel Devantier , qemu-stable@nongnu.org, jaeyeong , Jesper Wendel Devantier , Klaus Jensen , Keith Busch , Klaus Jensen , qemu-block@nongnu.org Subject: [PULL 01/11] hw/nvme: fix FDP set FDP events Date: Tue, 7 Jul 2026 00:44:13 +0200 Message-ID: <20260706224426.14156-2-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.159; envelope-from=its@irrelevant.dk; helo=fhigh-b8-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783377904065158500 Content-Type: text/plain; charset="utf-8" From: Jesper Wendel Devantier Addresses an issue reported whereby user-provided event type values could trigger two issues: 1. if provided event_type =3D=3D 0xff -> out-of-bounds access 2. if provided event_type > 7 -> generate a value too large for the u8 event mask. This patch fixes (1) by correctly adjusting the length of the look-up array to be 256 values. This patch fixes (2) by: a. changing the event_type mask to 64bit, matching NvmeRuHandle.event_filter b. Matching the behavior of Get Feature - FDP Events by skipping event type values which we do not support. 5.2.26.1.21 of the 2.3 Base specification does not explicitly tell us to reject unsupported event type values. c. Documenting in the event type lookup table, that supporting event types greater than 63 requires refactoring the masking code. Cc: qemu-stable@nongnu.org Reported-by: jaeyeong Resolves: https://gitlab.com/qemu-project/qemu/-/work_items/3631 Signed-off-by: Jesper Wendel Devantier Reviewed-by: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 22 ++++++++++++++++------ hw/nvme/nvme.h | 9 ++++++++- 2 files changed, 24 insertions(+), 7 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 815f39173c8a..d60a680dbc15 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -6244,10 +6244,6 @@ static uint16_t nvme_get_feature_fdp_events(NvmeCtrl= *n, NvmeNamespace *ns, for (uint8_t event_type =3D 0; event_type < FDP_EVT_MAX; event_type++)= { uint8_t shift =3D nvme_fdp_evf_shifts[event_type]; if (!shift && event_type) { - /* - * only first entry (event_type =3D=3D 0) has a shift value of= 0 - * other entries are simply unpopulated. - */ continue; } =20 @@ -6492,9 +6488,9 @@ static uint16_t nvme_set_feature_fdp_events(NvmeCtrl = *n, NvmeNamespace *ns, uint8_t noet =3D (cdw11 >> 16) & 0xff; uint16_t ret, ruhid; uint8_t enable =3D le32_to_cpu(cmd->cdw12) & 0x1; - uint8_t event_mask =3D 0; + uint64_t event_mask =3D 0; unsigned int i; - g_autofree uint8_t *events =3D g_malloc0(noet); + g_autofree uint8_t *events =3D NULL; NvmeRuHandle *ruh =3D NULL; =20 assert(ns); @@ -6507,15 +6503,29 @@ static uint16_t nvme_set_feature_fdp_events(NvmeCtr= l *n, NvmeNamespace *ns, return NVME_INVALID_FIELD | NVME_DNR; } =20 + if (unlikely(noet =3D=3D 0)) { + return NVME_SUCCESS; + } + ruhid =3D ns->fdp.phs[ph]; ruh =3D &n->subsys->endgrp.fdp.ruhs[ruhid]; =20 + events =3D g_malloc0(noet); + ret =3D nvme_h2c(n, events, noet, req); if (ret) { return ret; } =20 for (i =3D 0; i < noet; i++) { + /* + * We ignore requests to enable tracking of unsupported FDP event = types + */ + uint8_t event_type =3D events[i]; + uint8_t shift =3D nvme_fdp_evf_shifts[event_type]; + if (!shift && event_type) { + continue; + } event_mask |=3D (1 << nvme_fdp_evf_shifts[events[i]]); } =20 diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 5ef3ebee29e5..9de9f347c542 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -160,7 +160,14 @@ typedef struct NvmeZone { #define NVME_FDP_MAX_NS_RUHS 32u #define FDPVSS 0 =20 -static const uint8_t nvme_fdp_evf_shifts[FDP_EVT_MAX] =3D { +/* + * NOTE: Apart from event type 0, any event type with a shift value of 0 is + * considered unsupported and thus skipped in get/set features calls. + * + * NOTE: NvmeRuHandle uses a 64bit event mask - refactor to support event = types + * of 63 or greater. + */ +static const uint8_t nvme_fdp_evf_shifts[FDP_EVT_MAX + 1] =3D { /* Host events */ [FDP_EVT_RU_NOT_FULLY_WRITTEN] =3D 0, [FDP_EVT_RU_ATL_EXCEEDED] =3D 1, --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783378006; cv=none; d=zohomail.com; s=zohoarc; b=a87NkK7n4MDwdaw50HdEyoZDEc4OYbn1Ts7O5seBhfjrVS235Yqd65ZeeQi0vm9wbd9meyz7Sl6XWiHqZ1+flDtE33W5x0RDpM/2E7kxVtBveIC6wDbQYUcg3LvODFokdTsbyxBQd9HU1vnVRM1kEPHMtcEqu+pBAlNog59yHZM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783378006; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LWbOE8SiKBTE7BDcBZIXw60DedsTrZ4otCCNMbHXrKQ=; b=fcBljU92OTNJAOilorPyEnBtoP6wUMwYcEKVjbhZDtdL26dagCjr47tkBKmSYzqY6XBSSW5P5KDdk0MdgW81Eqyu0H1RmlE4XBgSK+mI8TUXCH1LBi2hqzRyqo67QALzD8A4eMWw/ygma7Vr1lB4zenlQ7175biedEsMK/2b+oA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783378006686168.25941883572875; Mon, 6 Jul 2026 15:46:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3a-0001tV-B3; Mon, 06 Jul 2026 18:44:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3X-0001pt-TI; Mon, 06 Jul 2026 18:44:51 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3W-0000wZ-A5; Mon, 06 Jul 2026 18:44:51 -0400 Received: from phl-compute-06.internal (phl-compute-06.internal [10.202.2.46]) by mailfhigh.stl.internal (Postfix) with ESMTP id CB55D7A01E4; Mon, 6 Jul 2026 18:44:48 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-06.internal (MEProxy); Mon, 06 Jul 2026 18:44:49 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:44:47 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377888; x= 1783464288; bh=LWbOE8SiKBTE7BDcBZIXw60DedsTrZ4otCCNMbHXrKQ=; b=l Tz7TNrnnKcyasUW9E1cITzj1RtSMx8kHL48Bk7HuJ0BZQtpMfcEiDUAQ8JzAyX/J DX7QKvy0T8ykFA/LSB5DsJVX2VnTGLKoirgDIkcoKfNv5rfP/HKB9u+m+/bPDSUp 2jtRwQ5F3hb6QxYIL08b3bRsm0tJtbo/XR+An9PDQO8SXMa7BWe4r1aBtOL/LGfu yQwqetj+ZQ+5nuTQ99kp35FT8j6vH/C1HP5DombFhd907Efw3hh4kTOz10tZ2cDZ 6MpAZHvkDRlE4ijRuPetzdSvz72kidj16TAHcySfDsBOjYR/9YBplwCuhzP7LvqW zmphcAVmE7vwPIeR4WteQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377888; x=1783464288; bh=L WbOE8SiKBTE7BDcBZIXw60DedsTrZ4otCCNMbHXrKQ=; b=aeSdOOEtdHhWdooMs TqTE4nv99540P/ImacjmZFf5GBRSLorZXUWogh5cCScDXayczO+2QHnKGodQVtZw 4WSSBsOEcSdCC5muP4lCSwCQcmBO1EqP17cVAYj92aVchh2ochfGKWfxppBUlEAa 3aXpjjJ5TYNkz2OWhHnHDIratPytPFby+4Oq0CsUX9xAQYFnmVHflEbfqd2WzVxi NtoC5F1CVw3X4TDIeCnXmyf5Iz8ecRNGWFaknMMfDmECxdD+R9WSQigajEFhXAEM gKs9azjZ/fbMgf/XhEAs9eZiLF07OqBgzmrTcNQGtdpJJTxLnrANop0eaJ6LH+EL VrHqg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTGFBpywmipUNxqz7T25qNH2oFLWY3g2g2IYbd+a9NZ5PpDTzOXx5JVDMxv2l0k/rP Op8DOnueC7GUN1f+KdSrev4NwwyMoYqNOwq40rgiBIuAngiPnVcWGmD4iyZmLQQexgV30/ AqGZVGLAhMshNTq3tK5SWUxKXqUvvJz6ST9lexsuPYigR1n/8VpuvdjBzoGXsJTB2VXzWk OMUszQP3oXhgh2d3zJFmqzGaKLeum1kObyrB8pjHEvd6sPFHvINGSxiUijAx/xjYnxdftn 6ngxU68dld0btl9LKTMjBW9EkFIflcc2TLWmzEn0lttUhZM8u5JLx6ngc+Q/iURs3LZ+nL SQRYNA8yYb2QNvhbZiF2LcFjRH74/UEmHoSf3j2xMaSN0IO7wiVj0OTdIhcId3xOwDSlfn iyUfMZ2qZxz8UX0eSbH82eMURTH8OhKo7aizWP7NY3u1OJWxepfKASy+VZVm9wtOzdZQmE DQEtx6KrlLW6dphF27aBDtJ8zrrAFhlhYYl+Xye3Y71YxlTaccSQMLR9NedEUdCo29qyPJ mTLHuUVGtL2YBbu5HmjcFp1qUsYsJ7dB6rcNUCoRQ1OwrLojL1RYuaf3JQX3SgB0JZSyzr UYGy+K+6TfVMgEipcgsXE78AU3UqBxDwOgfl/xLLS9vl4OethBKtJlSZh+jA X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Keith Busch , qemu-stable@nongnu.org, Feifan Qian , boy juju , Klaus Jensen , Klaus Jensen , Jesper Devantier , qemu-block@nongnu.org Subject: [PULL 02/11] hw/nvme: ensure sgl forward progress Date: Tue, 7 Jul 2026 00:44:14 +0200 Message-ID: <20260706224426.14156-3-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.159; envelope-from=its@irrelevant.dk; helo=fhigh-b8-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783378007875158500 Content-Type: text/plain; charset="utf-8" From: Keith Busch A degenerate host can create segment loops of zero-byte data descriptors that the controller never breaks out of. While the spec allows zero length segments, it provides no guidance on handling loops. It makes no sense for a host to submit such a descriptor anyway since it can and trivially should point to the next transfer segment, so don't even try to work with such behavior. Just reject the command, terminating the loop. Cc: qemu-stable@nongnu.org Reported-by: Feifan Qian Reported-by: boy juju Signed-off-by: Keith Busch Reviewed-by: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index d60a680dbc15..7845e59e4b08 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -1090,6 +1090,8 @@ static uint16_t nvme_map_sgl(NvmeCtrl *n, NvmeSg *sg,= NvmeSglDescriptor sgl, } =20 for (;;) { + size_t prev_len =3D len; + switch (NVME_SGL_TYPE(sgld->type)) { case NVME_SGL_DESCR_TYPE_SEGMENT: case NVME_SGL_DESCR_TYPE_LAST_SEGMENT: @@ -1170,6 +1172,17 @@ static uint16_t nvme_map_sgl(NvmeCtrl *n, NvmeSg *sg= , NvmeSglDescriptor sgl, if (status) { goto unmap; } + + /* + * Reject if this segment made no forward progress. The host should + * have skipped linking an empty segment. While not strictly spec + * compliant, allowing this makes it easy for a pathological host = to + * create an infinite loop. + */ + if (len =3D=3D prev_len) { + status =3D NVME_INVALID_SGL_SEG_DESCR | NVME_DNR; + goto unmap; + } } =20 out: --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783377999; cv=none; d=zohomail.com; s=zohoarc; b=bmzg3K15CiK4jw+3/bDi/ECBCmZXZ7bTBJecgwjKZMBt57Xs+/C3tth4CwPSpHaNCvJZcMIjbvutyvD0jtQSodGp/x5ZfFk2UrefvDWoJ8j1Vdbyu3DW2dylvsk+/OId2JfWhnmpBNSAQKDMxGOunEiSuWamiFPCduPm9xqZ+S4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783377999; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3APf4nwAux95GbU1jGRozkoBxabTHnbgGvUjZhM48/k=; b=VQoOTfKE7ITMJgE/mud9n1IijONJTEnNf7jxAsi8aZVd6VEey9atEfZnDwUh3t6MWLu/DTKIbc86SUV3nPY/wDbOzYczXFRN+o43z1K+wGdLdx5XqQ5bHyQYIAI8QkE3gkSjis9WQoUIUf0Lwof7/3+CMcp7MuqsGWDaCZYCLLY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783377999827651.2166186753793; Mon, 6 Jul 2026 15:46:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3a-0001tW-CA; Mon, 06 Jul 2026 18:44:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3Z-0001sp-KH for qemu-devel@nongnu.org; Mon, 06 Jul 2026 18:44:53 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3Y-0000x0-4B for qemu-devel@nongnu.org; Mon, 06 Jul 2026 18:44:53 -0400 Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.stl.internal (Postfix) with ESMTP id 508F47A01DD; Mon, 6 Jul 2026 18:44:51 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Mon, 06 Jul 2026 18:44:51 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:44:49 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377891; x= 1783464291; bh=3APf4nwAux95GbU1jGRozkoBxabTHnbgGvUjZhM48/k=; b=A U0JahiMDag9vowZr8hklBxoaCRirBuytn6AjN4c1bxQQ751vrUlbwTdcNhuL0pvm y5slbgknHUkLQkKaFUkNXEtI/Tg3sKMRlxKtosfwXuVMScqEMhlu4fU43SV9DuNu N4W4j99ape538KlkMbfiRRNhyI1QJUZzf24f7Velf611zQtTjJ5XsY8eFbNKaSGi pOEUC1hDfacDBHZ8rD3yjbm99CKGm1qk76L2yhHzss4T1hAsES/39iCmP3WxkChE DFJLH4pVDQxdVCJPa0yPM9xgbTi0CX7fcxAyFz+m/IPPZEOVmJMABsO4GElzq1NG as7skRID1U3zB8mbwLSAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377891; x=1783464291; bh=3 APf4nwAux95GbU1jGRozkoBxabTHnbgGvUjZhM48/k=; b=FlhRjctmtm9r0hDku eKMzYQGNy9UH5yReNZieW2DZtgloUiJdv34LeCEG+uAt8kAftn5F2oC1KFz3tpwl FYWMPkhwbJmqy2a0EBlu3r6zfa3/Y6qflDhdSQO8tJ+U/ITkCumqhjA7xFXcIlaj pn5WjkO9tZwCozVO8x8lt/uzEEKZ2p+cVg3a3Vc7o/kzihG6Mz5TZBkuTGFXiv25 IeiDQ1hTroRpDKaAXUy6HeC6RyRTD9qjWHwaMOqgioW6UUVc192E2M8+hphUC0nW sKVgzhzW+L6eU7sYBoBZ1Wz6SE4TMgNGNweNutBj+0POnUOKTENwpsdsMn8q4Yl2 fNJUA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTEBar+xcPJ6TGk2BRFD+/PQr7WZKLLnzl1lVSCdlSIyG0uCVLY82Gs7M5B5GndyFB O/P0+eD2zh/XrQgDHxqg00p15q5Qzw7PIbLGVkgQfI5ah8WSj2rr7xsbk02s3QNueEyg1P uHxCWWE7GNu317gk6JJEmbd3+/oen3y+m0Z15/F78rk451OEixAYafgzZVoxR9YsiHTnsi pkl5vDKm1IAyDDaEn/THuzeN7lX4xDLhr5j8Z6HscItNeJ4zDsnoQOgY57OGYvnzGv3L7c Ix4dWml9kPIpVOZvBZLF5TfBYx6q2yDYcVFI9eF2o7kl9f8NN4MyRbYzFs1rZrzVp7Rfps 0QSzfeKXNci3ClHwdXmBPdy1XLY+biLGq/yWixuUXbxv8iHBHOBjKQgBZ/E07SeIGvYrcV y5zDCRqFINEiRbN8hGLvpczjPy8W6C3P7GbjsRLxU9AdWG6xGQFxvu4gCJJqAEYsrkij6Q NXkJs8SV7Fg8YoHZgJx9XPZbQ1n2GpNGdTo27i1Jrc7sKnZ+Lr5hHxfOwz3JXZw+OeEutq vY3LmgNnRvVSGSrjyATn/9jB90ln/tlFRPUqd0DM9zhmaQ+GgfUAAKuXOmdvZDDRbo19NZ Y6bWusunLNE8at1bwcHgkeTyrnkS0WD2qXJVQNKei4QeuFsbxD0vSZGrPHrA X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Peter Xu , Stefan Hajnoczi , Klaus Jensen , Fabiano Rosas Subject: [PULL 03/11] tests/functional/migration: add VM launch/configure hooks Date: Tue, 7 Jul 2026 00:44:15 +0200 Message-ID: <20260706224426.14156-4-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.159; envelope-from=its@irrelevant.dk; helo=fhigh-b8-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783378001721158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Introduce configure_machine, launch_source_vm and assert_dest_vm methods to allow child classes to override some pieces of source/dest VMs creation, start and check logic. Reviewed-by: Peter Xu Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Signed-off-by: Klaus Jensen --- tests/functional/migration.py | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/tests/functional/migration.py b/tests/functional/migration.py index 3b7674af3b68..4344e03be417 100644 --- a/tests/functional/migration.py +++ b/tests/functional/migration.py @@ -40,19 +40,36 @@ def assert_migration(self, src_vm, dst_vm): self.assertEqual(dst_vm.cmd('query-status')['status'], 'running') self.assertEqual(src_vm.cmd('query-status')['status'],'postmigrate= ') =20 + # Can be overridden by subclasses to configure both source/dest VMs. + def configure_machine(self, vm): + vm.add_args('-nodefaults') + + # Can be overridden by subclasses to prepare the source VM before + # migration, e.g. by running some workload inside the source VM + # to see if it continues to run properly after migration. + def launch_source_vm(self, vm): + vm.launch() + + # Can be overridden by subclasses to check the destination VM after + # migration, e.g. by checking if the workload is still running after + # migration. + def assert_dest_vm(self, vm): + pass + def migrate_vms(self, dst_uri, src_uri, dst_vm, src_vm): dst_vm.qmp('migrate-incoming', uri=3Ddst_uri) src_vm.qmp('migrate', uri=3Dsrc_uri) self.assert_migration(src_vm, dst_vm) + self.assert_dest_vm(dst_vm) =20 def migrate(self, dst_uri, src_uri=3DNone): dst_vm =3D self.get_vm('-incoming', 'defer', name=3D"dst-qemu") - dst_vm.add_args('-nodefaults') + self.configure_machine(dst_vm) dst_vm.launch() =20 src_vm =3D self.get_vm(name=3D"src-qemu") - src_vm.add_args('-nodefaults') - src_vm.launch() + self.configure_machine(src_vm) + self.launch_source_vm(src_vm) =20 if src_uri is None: src_uri =3D dst_uri --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783377931; cv=none; d=zohomail.com; s=zohoarc; b=FIHfRXOsy685DlV2GGTNARba1kfS/ZInLX1u/c0yDwDuBnS7sRGB3oTNoQ9o5v++DhzJKVn+fuTwMpzpX4oJXJjI7GnMmTLySPw61ewRaSC4OIagoWd6EMUGMtjX07YjX4oum6OWQwEJ73hW2NN9MUbmUKwWfoz5OZ+QoCsTBpw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783377931; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5VWvCrV1Tc3tNKrgFFIAsHgM/VO34Wk/bRBMZdeiKMM=; b=YORHw5nbDOV/lqv/XHcS2I3DQlQHSLX7vf9FZ1S6XHHGJ/bLPsav1ezRsUQ/OHkIea3tI9dqJ3xd3DzKlEl8ykSUB/mawzniauPxARrm5Uo0bZnrMnjE8rbSzMDFeYXjWPHFaHN8pkQWB2XChEprthYigJVOUubWIH+QiIuWMeg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783377931081675.5584380011736; Mon, 6 Jul 2026 15:45:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3f-0001v7-7q; Mon, 06 Jul 2026 18:44:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3d-0001uB-Fq; Mon, 06 Jul 2026 18:44:57 -0400 Received: from fout-b2-smtp.messagingengine.com ([202.12.124.145]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3b-0000yW-A5; Mon, 06 Jul 2026 18:44:57 -0400 Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfout.stl.internal (Postfix) with ESMTP id 0BF061D0019A; Mon, 6 Jul 2026 18:44:54 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Mon, 06 Jul 2026 18:44:54 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:44:51 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377893; x= 1783464293; bh=5VWvCrV1Tc3tNKrgFFIAsHgM/VO34Wk/bRBMZdeiKMM=; b=B q+uwpTEN4nB1K7E5+kBLCNyxYQScxECqHdmo5LBGfmudcboNp2Myq5RTTepklpJg Fv4g3UnIjOTZrdZi58XY2Z6g2L9FgqSi0n1LcmnSDSWXwQnjrZDheuB9IjsI7Ywz 5kKc2gJp810TKlXqm5h4yuLciAMh91hyQN+0DsDvNE/KWb0vskb6Fj4g28I2tHQU SiVn9XzHccqITslSuUMb/kmevuGK9P8MAr1V2JH7MhqE9fLNPZnA5+LxAnb+Iq/T CGgaQEtkG/7649f2a6HbWSjPIlscdUgQCuhBwzDMrznIlme3en30lCF7hmkJXUI6 q0fx/5MdD3bJn6NxuFHAA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377893; x=1783464293; bh=5 VWvCrV1Tc3tNKrgFFIAsHgM/VO34Wk/bRBMZdeiKMM=; b=bgRc2dwTWoxK9B66J vanr2mKLPoo/zr0AUNtopMsIJcRLkRFSj3IgVXfLYYvU2twInghzLZDDLyiZPEGm WhVUp7wWe0NhlV0pL8vl7rncxZN+ZxMBBkPRFSWSAVkF4G7c/4lVTgMuTnFZc8Hf Gev2I2Y1256rD+M3daob0ijZJ8UyK1ok3Wzbw1QOGBcn5o7/p9Uh9SqmcrRE9uuR GSw2y4tN9eLL1uF8CrY19IO+R2anpT0psRu2qIf43+O5PDjHnqC1hF4I6QMLFsN4 wmabCJZxbWlOXIuWMFQplYeu0tyfH8wvAMBEP8FCdUWon07BX0MEx3ybC/YFyXU+ s+XuA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTECAh4m0OxgoGkgiRC+isJUP1GDqnIO5UvUOWif0bQdjPXB8qf1GZ0hHupTDw4R0Y g4mitjEJUEwEak/fEVpJ3XdKJwV0M29eYINll4VGi0GLuYpFdQvU7g3fLsndCqVAoXlSA2 fJ9DNBS8taF/f1VxYX64HMHD2uh/IXyGFce7JruI/qWIL7fAJ/T2UfbBi2iBDDO/ZrArI0 UH2dyvWfd6nJlU49f2ukXEjjFgACuhiWXgWCQrE4kDYzmMHRkIN+qEu9HbnfildN69oXiZ PxVlZ9B9zMrq0W1FRie4V10xhzbLCTAL6A1M8qp7WD5EzbKNNgklY03slvRtLYDyRub/On 6FsL4XPn/mp5oU0rqJnPsAOCCLfh7Ad1ldvIEC/54tMUOgqZ6O/cFr4ndNXWcTYPgsjH0B voSG5cVhUwwF1I5ivlKHctYa5BgIvsX/UWos2I+ErVyf/+JIsOkZvxxsp36v27ornmwJ3l PrOdoRmqghBonzFpuo/EK+bOc52Joao8700To8vpPOtfTdx0AbwfdH/VexqTFGzlTY+3Ca ZgYNlrY5LbnYczL/2ZgjlLFZR+YKsRQ/Pw+NWcO55RxbjSpjTxlBn6kjLrrGMPaad12+RX AoKg7Jk/cxop07Vqy9KKPAbbXVIyFD2xJ3jN0eA6V0SJpnEG7kVoKe+bwxeg X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Klaus Jensen , Stefan Hajnoczi , Keith Busch , Klaus Jensen , Jesper Devantier , Fam Zheng , Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org Subject: [PULL 04/11] hw/nvme: add migration blockers for non-supported cases Date: Tue, 7 Jul 2026 00:44:16 +0200 Message-ID: <20260706224426.14156-5-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.145; envelope-from=its@irrelevant.dk; helo=fout-b2-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783377933783158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Let's block migration for cases we don't support: - SR-IOV - CMB - PMR - SPDM No functional changes here, because NVMe migration is not supported at all as of this commit. Reviewed-by: Klaus Jensen Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 212 +++++++++++++++++++++++++++++++++++++++++++ hw/nvme/nvme.h | 3 + include/block/nvme.h | 12 +++ 3 files changed, 227 insertions(+) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 7845e59e4b08..d2096eb49d97 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -209,6 +209,7 @@ #include "hw/pci/msix.h" #include "hw/pci/pcie_sriov.h" #include "system/spdm-socket.h" +#include "migration/blocker.h" #include "migration/vmstate.h" =20 #include "nvme.h" @@ -252,6 +253,7 @@ static const bool nvme_feature_support[NVME_FID_MAX] = =3D { [NVME_COMMAND_SET_PROFILE] =3D true, [NVME_FDP_MODE] =3D true, [NVME_FDP_EVENTS] =3D true, + /* if you add something here, please update nvme_set_migration_blocker= s() */ }; =20 static const uint32_t nvme_feature_cap[NVME_FID_MAX] =3D { @@ -4616,6 +4618,7 @@ static uint16_t nvme_io_mgmt_send(NvmeCtrl *n, NvmeRe= quest *req) return 0; case NVME_IOMS_MO_RUH_UPDATE: return nvme_io_mgmt_send_ruh_update(n, req); + /* if you add something here, please update nvme_set_migration_blocker= s() */ default: return NVME_INVALID_FIELD | NVME_DNR; }; @@ -7545,6 +7548,10 @@ static uint16_t nvme_security_receive(NvmeCtrl *n, N= vmeRequest *req) =20 static uint16_t nvme_directive_send(NvmeCtrl *n, NvmeRequest *req) { + /* + * When adding a new dtype handling here, + * please also update nvme_set_migration_blockers(). + */ return NVME_INVALID_FIELD | NVME_DNR; } =20 @@ -9256,6 +9263,205 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *= pci_dev) } } =20 +#define BLOCKER_FEATURES_MAX_LEN 256 + +static inline void nvme_add_blocker_feature(char *blocker_features, + const char *feature) +{ + if (strlen(blocker_features) > 0) { + g_strlcat(blocker_features, ", ", BLOCKER_FEATURES_MAX_LEN); + } + g_strlcat(blocker_features, feature, BLOCKER_FEATURES_MAX_LEN); +} + +static bool nvme_set_migration_blockers(NvmeCtrl *n, PCIDevice *pci_dev, + Error **errp) +{ + uint64_t unsupported_cap, cap =3D ldq_le_p(&n->bar.cap); + char blocker_features[BLOCKER_FEATURES_MAX_LEN] =3D ""; + bool adm_cmd_security_checked =3D false; + bool cmd_io_mgmt_checked =3D false; + bool cmd_zone_checked =3D false; + + /* + * Idea of this function is simple, we iterate over all Command Sets a= nd + * for each supported command we provide a special handling logic to + * determine if we should block migration or not. + * + * For instance, we have NVME_ADM_CMD_NS_ATTACHMENT and it is always + * available to the guest, but if there is only 1 namespace, then it is + * safe to allow migration, but if there are more, then we need to blo= ck + * migration because we don't handle this in migration code yet. + */ + for (int opcode =3D 0; opcode < ARRAY_SIZE(n->cse.acs); opcode++) { + /* Is command supported? */ + if (!n->cse.acs[opcode]) { + continue; + } + + switch (opcode) { + case NVME_ADM_CMD_DELETE_SQ: + case NVME_ADM_CMD_CREATE_SQ: + case NVME_ADM_CMD_GET_LOG_PAGE: + case NVME_ADM_CMD_DELETE_CQ: + case NVME_ADM_CMD_CREATE_CQ: + case NVME_ADM_CMD_IDENTIFY: + case NVME_ADM_CMD_ABORT: + case NVME_ADM_CMD_SET_FEATURES: + case NVME_ADM_CMD_GET_FEATURES: + case NVME_ADM_CMD_ASYNC_EV_REQ: + case NVME_ADM_CMD_DBBUF_CONFIG: + case NVME_ADM_CMD_FORMAT_NVM: + case NVME_ADM_CMD_DIRECTIVE_SEND: + case NVME_ADM_CMD_DIRECTIVE_RECV: + break; + case NVME_ADM_CMD_NS_ATTACHMENT: { + int namespaces_num =3D 0; + for (int i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + if (!ns) { + continue; + } + + namespaces_num++; + } + + if (namespaces_num > 1) { + nvme_add_blocker_feature(blocker_features, + "Namespace Attachment"); + } + + break; + } + case NVME_ADM_CMD_VIRT_MNGMT: + if (n->params.sriov_max_vfs) { + nvme_add_blocker_feature(blocker_features, "SR-IOV"); + } + + break; + case NVME_ADM_CMD_SECURITY_SEND: + case NVME_ADM_CMD_SECURITY_RECV: + if (adm_cmd_security_checked) { + break; + } + + if (pci_dev->spdm_port) { + nvme_add_blocker_feature(blocker_features, "SPDM"); + } + + adm_cmd_security_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + for (int opcode =3D 0; opcode < ARRAY_SIZE(n->cse.iocs.nvm); opcode++)= { + if (!n->cse.iocs.nvm[opcode]) { + continue; + } + + switch (opcode) { + case NVME_CMD_FLUSH: + case NVME_CMD_WRITE: + case NVME_CMD_READ: + case NVME_CMD_COMPARE: + case NVME_CMD_WRITE_ZEROES: + case NVME_CMD_DSM: + case NVME_CMD_VERIFY: + case NVME_CMD_COPY: + break; + case NVME_CMD_IO_MGMT_RECV: + case NVME_CMD_IO_MGMT_SEND: + if (cmd_io_mgmt_checked) { + break; + } + + /* check for NVME_IOMS_MO_RUH_UPDATE */ + if (n->subsys->params.fdp.enabled) { + nvme_add_blocker_feature(blocker_features, "FDP"); + } + + cmd_io_mgmt_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + for (int opcode =3D 0; opcode < ARRAY_SIZE(n->cse.iocs.zoned); opcode+= +) { + /* + * If command isn't supported or we have the same command + * in n->cse.iocs.nvm, then we can skip it here. + */ + if (!n->cse.iocs.zoned[opcode] || n->cse.iocs.nvm[opcode]) { + continue; + } + + switch (opcode) { + case NVME_CMD_ZONE_APPEND: + case NVME_CMD_ZONE_MGMT_SEND: + case NVME_CMD_ZONE_MGMT_RECV: + if (cmd_zone_checked) { + break; + } + + for (int i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + if (!ns) { + continue; + } + + if (ns->params.zoned) { + nvme_add_blocker_feature(blocker_features, + "Zoned Namespace"); + break; + } + } + + cmd_zone_checked =3D true; + + break; + default: + g_assert_not_reached(); + } + } + + /* + * Try our best to explicitly detect all not supported caps, + * to let users know what features cause migration to be blocked, + * but in case we miss handling here, everything else will be + * covered by unsupported_cap check. + */ + if (NVME_CAP_CMBS(cap)) { + nvme_add_blocker_feature(blocker_features, "CMB"); + cap &=3D ~((uint64_t)CAP_CMBS_MASK << CAP_CMBS_SHIFT); + } + + if (NVME_CAP_PMRS(cap)) { + nvme_add_blocker_feature(blocker_features, "PMR"); + cap &=3D ~((uint64_t)CAP_PMRS_MASK << CAP_PMRS_SHIFT); + } + + unsupported_cap =3D cap & ~NVME_MIGRATION_SUPPORTED_CAP_BITS; + if (unsupported_cap) { + nvme_add_blocker_feature(blocker_features, "unknown capability"); + } + + assert(n->migration_blocker =3D=3D NULL); + if (strlen(blocker_features) > 0) { + error_setg(&n->migration_blocker, + "Migration is not supported for %s", blocker_features); + if (migrate_add_blocker(&n->migration_blocker, errp) < 0) { + return false; + } + } + + return true; +} + static int nvme_init_subsys(NvmeCtrl *n, Error **errp) { int cntlid; @@ -9361,6 +9567,10 @@ static void nvme_realize(PCIDevice *pci_dev, Error *= *errp) =20 n->subsys->namespaces[ns->params.nsid] =3D ns; } + + if (!nvme_set_migration_blockers(n, pci_dev, errp)) { + return; + } } =20 static void nvme_exit(PCIDevice *pci_dev) @@ -9413,6 +9623,8 @@ static void nvme_exit(PCIDevice *pci_dev) } =20 memory_region_del_subregion(&n->bar0, &n->iomem); + + migrate_del_blocker(&n->migration_blocker); } =20 static const Property nvme_props[] =3D { diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 9de9f347c542..40a9726546dc 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -675,6 +675,9 @@ typedef struct NvmeCtrl { =20 /* Socket mapping to SPDM over NVMe Security In/Out commands */ int spdm_socket; + + /* Migration-related stuff */ + Error *migration_blocker; } NvmeCtrl; =20 typedef enum NvmeResetType { diff --git a/include/block/nvme.h b/include/block/nvme.h index e4e7be51205e..17a7c7818d77 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -141,6 +141,18 @@ enum NvmeCapMask { #define NVME_CAP_SET_CMBS(cap, val) \ ((cap) |=3D (uint64_t)((val) & CAP_CMBS_MASK) << CAP_CMBS_SHIFT) =20 +#define NVME_MIGRATION_SUPPORTED_CAP_BITS ( \ + ((uint64_t)CAP_MQES_MASK << CAP_MQES_SHIFT) \ + | ((uint64_t)CAP_CQR_MASK << CAP_CQR_SHIFT) \ + | ((uint64_t)CAP_AMS_MASK << CAP_AMS_SHIFT) \ + | ((uint64_t)CAP_TO_MASK << CAP_TO_SHIFT) \ + | ((uint64_t)CAP_DSTRD_MASK << CAP_DSTRD_SHIFT) \ + | ((uint64_t)CAP_NSSRS_MASK << CAP_NSSRS_SHIFT) \ + | ((uint64_t)CAP_CSS_MASK << CAP_CSS_SHIFT) \ + | ((uint64_t)CAP_MPSMIN_MASK << CAP_MPSMIN_SHIFT) \ + | ((uint64_t)CAP_MPSMAX_MASK << CAP_MPSMAX_SHIFT) \ +) + enum NvmeCapCss { NVME_CAP_CSS_NCSS =3D 1 << 0, NVME_CAP_CSS_IOCSS =3D 1 << 6, --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783378010; cv=none; d=zohomail.com; s=zohoarc; b=KSn3GFEBX+R5mKVRmqSADvJolbb1KgpDJgnhKPJJ4MhBOYzhhtrwxyTf4EHqZB1JlRbsT09gRwKN3ogup+C+MyxZDg5UcNXkCE860OjDoQuDtzhAjJ2PIaFvO7Zevmq6AAuSUjAIccFqkxJio7SymScc4AG5hNvneorJUbF7M5M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783378010; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=C33rOZh3CoainYRieTpjM0jlcmLSHcuqY5ULDaTI+q8=; b=dgz7l/fsTYUM6CxYNxzakkVsn1Oy/slI482iEoZ1p9KkK7u66eMpOG6ixd3Mm2eFwQe8w29j7TBdWmkWOM3QE/LRATRsZ5sD4tPOLTzNCwkzwH+jCyxF7t2zVhpv1vNBp+5Xpj4/7f8HtHk01ZAefWgHQwobB5GEWtziHODSERE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783378010355709.8462753843601; Mon, 6 Jul 2026 15:46:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3h-0001xT-BS; Mon, 06 Jul 2026 18:45:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3e-0001uZ-QA; Mon, 06 Jul 2026 18:44:58 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3d-0000yl-8I; Mon, 06 Jul 2026 18:44:58 -0400 Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.stl.internal (Postfix) with ESMTP id 179707A01DB; Mon, 6 Jul 2026 18:44:56 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Mon, 06 Jul 2026 18:44:56 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:44:54 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377895; x= 1783464295; bh=C33rOZh3CoainYRieTpjM0jlcmLSHcuqY5ULDaTI+q8=; b=j 8wOHw+CbZBtLsdQqsXwXuSz76D5DjasHZKHbkz6B5GietnesVCvxIEkhuWy3TCTL iGbN7gdpQAtq1rubGup6675iboix+0S5vW69igUdcX5hnPajkh81vAGX2q+aZp2R 7RyidXH4I84x/7YYP3EWKS3hW9f5VfjmuhTLPPRw4hub4hEa9SJy/oob8NnA5hbA qJWc/eo7bopZdN7/+rQc/9kLGXeTha/7nO8XfdLteTUKm/T0S3nt5dQ+j26urZCs 3E/A7SBf5eqBzB4cFxX2piVqlbwva51Zm/2dt5KO6I8cGrLSGXhoqXXNfIhfTp2Q //qRTTNgQ9f4AvUXp1CmA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377895; x=1783464295; bh=C 33rOZh3CoainYRieTpjM0jlcmLSHcuqY5ULDaTI+q8=; b=TisbtbdgeFZSBoI3y lXtE7gLvrV/SBHVeVLYFaoyErUxMo3+uQaOgInh/C8+oyDoYj0oIPgyTxrwVOXd1 q31mnupBQhgNtDha3EXIoq0O/HgNi11ttb1Q5IUhLpKBJPYIJdV/SKQghlb0oaSl lygE4NsuaI+joafOXOnDDoGYPtDDed3UFsuCx6TPw66lzhaLbFmVPiHR29RDMNfI PcFTI/PK53M53fvGtBeP1fSIBr6kevFGewSC+/1C7xyiTUWdtjabYZXuRH48BPcv HsbTsQH4JN69TQK9nku6sT6A9hf7SEpZE6Wi77kZEX/oRIovAm5OWbudMPnweumT zva+A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTFj5JQpqUxJDcUJ2sWCsKZKclPuT2bVB0FQRjAKIF/sKDLIeA//H/SeuNaNsjsyI/ GLovIo1aiXSUcVFWzisNrFSjyButVeujuMJk3VZ3IDFeCssXdFFEWUGnPDYbOHrceoCBwt PdKgkdL50wU/Kuc2nXFaj9t58LBXlQ4p+egWMertNslLurP+Gzi2ngjrnO4NmxwVp3q0Gu podh3AKtGUg+x1TtnfBbfnMUe5kA1K2nN9DRBU3fCZO1vv5XOO+7/JlEq0i4CES6GCR0vD hV1pA1aerjW9YEabuc5sq43NfFDlKcLEXtgRzp0jwDe7Yjs1HqLdvCEIdxeBfq8fBzmcsC Hi3yJheuLEcDsUbdy09eYWogjKa5102zF+QeYBF++bdjJgcwXJ5HrORSUj9O6mGTAKphHQ sBQb3NDHwNSa6pVOe6sKNLiHQMQWOiegin8hxNrBvTgB3EI8yYvQjegHYxgXYieCnwLmBL bKAEsn2se6O1ktpcHZmXmdiZAi609ARQA3yxDUPb9cgd0UfzC0VMGCchite5+322mwynVt /6dZjQYQprrdRWPFVrl4g6p+iO6sJNJEvKf6kvdllFkitiX/sr2BRK4heQU1go8WTnb1cD x0n4dz8L5esU+WSaFhDBDwmR+DhjqNyf8F4z3EaNqSDsInUEgT7+MVE8RzfA X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Klaus Jensen , Stefan Hajnoczi , Keith Busch , Klaus Jensen , Jesper Devantier , qemu-block@nongnu.org Subject: [PULL 05/11] hw/nvme: split nvme_init_sq/nvme_init_cq into helpers Date: Tue, 7 Jul 2026 00:44:17 +0200 Message-ID: <20260706224426.14156-6-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.159; envelope-from=its@irrelevant.dk; helo=fhigh-b8-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783378011767158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn We will make a benefit from this split in later patches. Reviewed-by: Klaus Jensen Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 59 +++++++++++++++++++++++++++++++------------------- 1 file changed, 37 insertions(+), 22 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index d2096eb49d97..386a698ba858 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -4869,18 +4869,14 @@ static uint16_t nvme_del_sq(NvmeCtrl *n, NvmeReques= t *req) return NVME_SUCCESS; } =20 -static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, uint64_t dma_addr, - uint16_t sqid, uint16_t cqid, uint16_t size) +static void __nvme_init_sq(NvmeSQueue *sq) { + NvmeCtrl *n =3D sq->ctrl; + uint16_t sqid =3D sq->sqid; + uint16_t cqid =3D sq->cqid; int i; NvmeCQueue *cq; =20 - sq->ctrl =3D n; - sq->dma_addr =3D dma_addr; - sq->sqid =3D sqid; - sq->size =3D size; - sq->cqid =3D cqid; - sq->head =3D sq->tail =3D 0; sq->io_req =3D g_new0(NvmeRequest, sq->size); =20 QTAILQ_INIT(&sq->req_list); @@ -4910,6 +4906,18 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n= , uint64_t dma_addr, n->sq[sqid] =3D sq; } =20 +static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, uint64_t dma_addr, + uint16_t sqid, uint16_t cqid, uint16_t size) +{ + sq->ctrl =3D n; + sq->dma_addr =3D dma_addr; + sq->sqid =3D sqid; + sq->size =3D size; + sq->cqid =3D cqid; + sq->head =3D sq->tail =3D 0; + __nvme_init_sq(sq); +} + static uint16_t nvme_create_sq(NvmeCtrl *n, NvmeRequest *req) { NvmeSQueue *sq; @@ -5570,25 +5578,16 @@ static uint16_t nvme_del_cq(NvmeCtrl *n, NvmeReques= t *req) return NVME_SUCCESS; } =20 -static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr, - uint16_t cqid, uint16_t vector, uint16_t size, - uint16_t irq_enabled) +static void __nvme_init_cq(NvmeCQueue *cq) { + NvmeCtrl *n =3D cq->ctrl; PCIDevice *pci =3D PCI_DEVICE(n); + uint16_t cqid =3D cq->cqid; =20 - if (msix_present(pci) && irq_enabled) { - msix_vector_use(pci, vector); + if (msix_present(pci) && cq->irq_enabled) { + msix_vector_use(pci, cq->vector); } =20 - cq->ctrl =3D n; - cq->cqid =3D cqid; - cq->size =3D size; - cq->dma_addr =3D dma_addr; - cq->phase =3D 1; - cq->irq_enabled =3D irq_enabled; - cq->vector =3D vector; - cq->head =3D cq->tail =3D 0; - QTAILQ_INIT(&cq->req_list); QTAILQ_INIT(&cq->sq_list); if (n->dbbuf_enabled) { cq->db_addr =3D n->dbbuf_dbs + (cqid << 3) + (1 << 2); @@ -5605,6 +5604,22 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n= , uint64_t dma_addr, &DEVICE(cq->ctrl)->mem_reentrancy_guard); } =20 +static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr, + uint16_t cqid, uint16_t vector, uint16_t size, + uint16_t irq_enabled) +{ + cq->ctrl =3D n; + cq->cqid =3D cqid; + cq->size =3D size; + cq->dma_addr =3D dma_addr; + cq->phase =3D 1; + cq->irq_enabled =3D irq_enabled; + cq->vector =3D vector; + cq->head =3D cq->tail =3D 0; + QTAILQ_INIT(&cq->req_list); + __nvme_init_cq(cq); +} + static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeRequest *req) { NvmeCQueue *cq; --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783378000; cv=none; d=zohomail.com; s=zohoarc; b=nxTC1V4E0AZTzoOqgMMk7JpQLq4Yp9hmqKovqnT1ybPfwADbS1Z8e5nGzmObNCni+xixXtYiTxVrKa9uYSAznVkGPh5/TRp0q5RIzVoSiIw+/T/n4eFAC06rD4AnHHuA6r7ETumJRPkUjvPEt/EveOugzwhTzigptYnGCDPDZ30= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783378000; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=68I28SvF6NliEu1oHD4j7MXavEgdixjiWJFe+FDCopk=; b=HEMn3TBKCl5+QDetM5HP/7QSs/EcAD5vcW543thLkBQTIlUmoGx0ZaHZLHI/8usjsZIPi1wr8i7zG5eKAgodai2ZbpKYxJ+LTT6TwPVZxob73woulJa/ObnR6IWtS7wWuxSVLFIz/5eUuJGzEZjRH0xQ+CSKeryeHw1ILHi8O9M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783378000178120.89589498834391; Mon, 6 Jul 2026 15:46:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3i-0001z4-Ma; Mon, 06 Jul 2026 18:45:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3g-0001vd-Iv; Mon, 06 Jul 2026 18:45:00 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3f-00010Q-63; Mon, 06 Jul 2026 18:45:00 -0400 Received: from phl-compute-01.internal (phl-compute-01.internal [10.202.2.41]) by mailfhigh.stl.internal (Postfix) with ESMTP id 0E9EB7A01E4; Mon, 6 Jul 2026 18:44:58 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Mon, 06 Jul 2026 18:44:58 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:44:56 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377897; x= 1783464297; bh=68I28SvF6NliEu1oHD4j7MXavEgdixjiWJFe+FDCopk=; b=K Dda73KgTa8MPQJ1/pfJzy4UoeY1O5DkGkyfPBY6ZZexn2tfYKPOxCdOG4nU7PK1N vkJN3NGFBiCYoIAsRFpOz4xp0Kc4pF101NthevyAXB82VsqyEY/TmumoFMiIYM6d 5Cjv5SPmlAMJzAXLOaqs9ESuDbBNsRpbihNxIsQQOnpbuAry+rWwlR0Gvu/aX9na ap+h2jKPeuqAUXT6un+D2J/Pyb3v3aVF8tE4CYnfWya1oYHuwVMZJh+9F1lLeQUb i8EsoHR2hjbY6FIXeRBrIVY1nw9Id/4vpr2Q8AydC81xwdWBX6fv/XCQ4w1+x+ve 554RuEjRu1EnswPsqAZ4g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377897; x=1783464297; bh=6 8I28SvF6NliEu1oHD4j7MXavEgdixjiWJFe+FDCopk=; b=NNf6IlYTBR+jupXLv naOqkNdpXcBULzpQOV70shL23mc1UIRGHwd7UwSTilDAQmcsGY3cIqj4tNyq2cyD ZLtg3sht/XTuG+405kV9qrmk7/Z7jndzVN2cJ5/lqaZmcJME9wA/Spb981+s5ig/ LWA+hMozPK/BhrIHM91A9OJrcAGbM/4w4jH0386iWimYy+QNDTAVkzKG00vtxune TsUW0lqqVP/wp4fMAn4WRuUt/XmMcKSSM1cCtaXtjoURisqzRyqIggu5jetuKZP5 0UjJ5TqW4eFGwUpEDhsi5MQVC8y7VeS6Zp/AZzJZ3iWd5dzveIlXMBg1JaB0m2oX i6pNA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTEHwWeRLRRHVATDLNRfGQdRW51BWy91UTgjZ9xms8WPZJx8wttvEDTZHq3D5TFnJS JpcAgiFleC6RH6VVkO7NFh7cPeOyvzTMt8kiBIgMvJ4O82Cb5ONjBhLhNCgmFcZiXZbuu8 tK0hyFWfL+mY2UOnglFNd3yTMYCJhcfGnABldkkx3GjyDaCcoIFX8hAst1GxVzxmhbXW0k F8UIcTspuQlxCQXcrxEh4GVVUAX4dzjd0CDDWTl+p5dr6SaWjo/RsLcOrG7kStv357ZQCZ GlA7ExVk+WvZJzn98nWFkIGSyJXQk3XeuIUuoACpyYvPYfpBzamyJfg8xH90ZKXdZL/5kQ e6JB+z8bdnVehlFxKYjqhN43n/Yepf7lb03MFXgjI+TCR4VnK5zDVTR4x8z/z2Rg0w2Zs6 fzyueak0zm6DcC66vLttGwMpcAfQe3vN7+lWEDaDlE5nQxLEeEonCCZRXa262Y08B2eb+y TdxypRWlFKYay+jvVJ9UzTObwyNNA0leOV8BObPHLUnPUelLGtWPNbwcXX+N7okpLybHlK h02OY06+jvMTQOpyFHMJllWTGtp9EOAa2eDgArO69C78EXpj0zz/hkZnbXCazF7QG7lL4K Q9aq3p2RcJ0KnE51h8jKAEPirZL9kvHG7AyAxyPXMyui/Zr2qlkKVYYVhIlg X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Klaus Jensen , Stefan Hajnoczi , Keith Busch , Klaus Jensen , Jesper Devantier , qemu-block@nongnu.org Subject: [PULL 06/11] hw/nvme: set CQE.sq_id earlier in nvme_process_sq Date: Tue, 7 Jul 2026 00:44:18 +0200 Message-ID: <20260706224426.14156-7-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.159; envelope-from=its@irrelevant.dk; helo=fhigh-b8-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783378001707158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Instead of filling req->cqe.sq_id in nvme_post_cqes, let's set it earlier in nvme_process_sq. This shouldn't cause any issues, because req->cqe.sq_id never changes during lifetime of req. This will help us for migration support. Reviewed-by: Klaus Jensen Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 386a698ba858..92825022a4a9 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -1535,7 +1535,6 @@ static void nvme_post_cqes(void *opaque) =20 sq =3D req->sq; req->cqe.status =3D cpu_to_le16((req->status << 1) | cq->phase); - req->cqe.sq_id =3D cpu_to_le16(sq->sqid); req->cqe.sq_head =3D cpu_to_le16(sq->head); addr =3D cq->dma_addr + (cq->tail << NVME_CQES); ret =3D pci_dma_write(PCI_DEVICE(n), addr, (void *)&req->cqe, @@ -7875,6 +7874,7 @@ static void nvme_process_sq(void *opaque) QTAILQ_REMOVE(&sq->req_list, req, entry); QTAILQ_INSERT_TAIL(&sq->out_req_list, req, entry); nvme_req_clear(req); + req->cqe.sq_id =3D cpu_to_le16(sq->sqid); req->cqe.cid =3D cmd.cid; memcpy(&req->cmd, &cmd, sizeof(NvmeCmd)); =20 --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783377997; cv=none; d=zohomail.com; s=zohoarc; b=W5SHZv1kJH87OOJXf4DG0zlE/RBubHYqCcgCc5t4oA0cL2KtrO3OB5PIckhpKdcZUKoh1kS7Hn/d1B+7ub+IOmP3aBClFqWWXApjP7XJuZrmvqFbL6WcDtQJ95hmLDWH7EcA7ifHxTkpblFUgFuolxGonV2TFrQee3lm5EKridA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783377997; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=E1dAo5JLe6EAjfgzmGKUJ5FQZTZEt2TARKRnulPjjSU=; b=N7zGTVfoaVb4p09PDMCD3jD6NnD37cmsc8oPmAAHEBSIHO+D4g9ZrVgvbKYtdJjiwZy0T8jA3Y3hx3qBBnn683yf2osTo2mC/Mw6q6YVmiz2Ney2NO8f0hlHzvVd45UegkTpCrK7NPjg00IKzmwX3mlzwK+YgGivQ8bLUNLRcds= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783377997234661.2653444758239; Mon, 6 Jul 2026 15:46:37 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3l-00021L-64; Mon, 06 Jul 2026 18:45:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3j-0001zP-9P; Mon, 06 Jul 2026 18:45:03 -0400 Received: from fout-b2-smtp.messagingengine.com ([202.12.124.145]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3h-00013G-KX; Mon, 06 Jul 2026 18:45:02 -0400 Received: from phl-compute-02.internal (phl-compute-02.internal [10.202.2.42]) by mailfout.stl.internal (Postfix) with ESMTP id 2CD421D0019A; Mon, 6 Jul 2026 18:45:00 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-02.internal (MEProxy); Mon, 06 Jul 2026 18:45:00 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:44:58 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377900; x= 1783464300; bh=E1dAo5JLe6EAjfgzmGKUJ5FQZTZEt2TARKRnulPjjSU=; b=g y3YXhvM3VCO+Rv5Fk/ba56QW8L6IYpO4Qbe5SOrs057ndHTZ5TVcVmM9ZaDyOZwV BCO3PicWewcHzeVKgNGXStwrmVlce3JpoEDSNFzRoJE3uCmDcZZI7qlBHHUD6Ls/ vMo6T68qVHGYc0G+3jq3xIgH6/Lgs9dOsELcgcazER3FeQghi+bnNKkiM0juwzvi VOu1+As+s8lSFGg88DvAhiiu5u7yCk78Mrn5nFDtjrUwwto5dtzQNdw7aLFAjAMF hiXzTV6JEle7HxnQz+mp+nDGBP+2XMxq9zfRanDNECbNw4Jg5HI7MUX5YLYbYYsx Mom67sS3V6YWcmWsScN3Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377900; x=1783464300; bh=E 1dAo5JLe6EAjfgzmGKUJ5FQZTZEt2TARKRnulPjjSU=; b=DlRcXYZrE5pnlrn14 ULqseZsAuS518VSWA/Llyq4WuT3jsDQzJzw8ZXonTOOaYZYTm7cYQhGyjwXSqLwX 1XZzaXBaACYKVWs/ViUjICbjQrzQmBhvP53zOuHjZrvK8Jh8iDDU07Pqpdd557BW HXa6c1zq7IQ69yHRw6I5N/n4ToeYENQXmq7/YOze1w9mmZ7rkGnXqgJKqZPDaySs B0E7dG5p+l9I5tVO6aQv0VUJoYVxjzlAjvWIcQoL36KjhYnGcB1iokbkJFDFbCye 0VSW0NoCu0C8BfYzbCPdZkrptL/RzCd3yrNiaqLxBpqLc6FUgLlwYZfq30acJI4x JDugA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTEHwWeRLRRHVATDLNRfGQdRW51BWy91UTgjZ9xms8WPZJx8wttvEDTZHq3D5TFnJS JpcAgiFleC6RH6VVkO7NFh7cPeOyvzTMt8kiBIgMvJ4O82Cb5ONjBhLhNCgmFcZiXZbuu8 tK0hyFWfL+mY2UOnglFNd3yTMYCJhcfGnABldkkx3GjyDaCcoIFX8hAst1GxVzxmhbXW0k F8UIcTspuQlxCQXcrxEh4GVVUAX4dzjd0CDDWTl+p5dr6SaWjo/RsLcOrG7kStv357ZQCZ GlA7ExVk+WvZJzn98nWFkIGSyJXQk3XeuIUuoACpyYvPYfpBzamyJfg8xH90ZKXdZL/5t7 gAksx4GFFYGUnie+wQ5yY8Sayg6n0/bDLVVylk5x2nSxAyFch5fh4k9ahashjTXJSR1mT/ zzuaPAZwgKX1yEiXAJPjsznGvOOp0B8S3LOwJ/Iokb2OSmHYzrSqD9HhzoM+r0gsRN3olN tD/aUr1D9WHLF1D+7ur6fc63O5bVsmajYXDw5eNXNpQAxhR1bwj8x1r63tsyrcp661+2z2 QS9QrubAGg3aQURapaihcPp1gkfFJIB4Fd+JPxymgRc8wYnI0o/3Umk+yUhth8rzFbPWzT ev3iGg4QxSHans8p/Zxh1NAOqmjK5hBceFr7/jKcned4GPihxZUK938Py4eg X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Klaus Jensen , Stefan Hajnoczi , Keith Busch , Klaus Jensen , Jesper Devantier , qemu-block@nongnu.org Subject: [PULL 07/11] hw/nvme: unmap req->sg earlier in nvme_enqueue_req_completion Date: Tue, 7 Jul 2026 00:44:19 +0200 Message-ID: <20260706224426.14156-8-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.145; envelope-from=its@irrelevant.dk; helo=fout-b2-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783377997881158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Instead of unmapping req->sg in nvme_post_cqes(), we can do it earlier in nvme_enqueue_req_completion(). When req completion is enqueued we don't need to access req->sg anymore. We only care about req->sq, req->cqe and req->status. Reviewed-by: Klaus Jensen Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 92825022a4a9..cd30735e86d2 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -1549,7 +1549,6 @@ static void nvme_post_cqes(void *opaque) QTAILQ_REMOVE(&cq->req_list, req, entry); =20 nvme_inc_cq_tail(cq); - nvme_sg_unmap(&req->sg); =20 if (QTAILQ_EMPTY(&sq->req_list) && !nvme_sq_empty(sq)) { qemu_bh_schedule(sq->bh); @@ -1579,6 +1578,8 @@ static void nvme_enqueue_req_completion(NvmeCQueue *c= q, NvmeRequest *req) req->status, req->cmd.opcode); } =20 + nvme_sg_unmap(&req->sg); + QTAILQ_REMOVE(&req->sq->out_req_list, req, entry); QTAILQ_INSERT_TAIL(&cq->req_list, req, entry); =20 --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783377918; cv=none; d=zohomail.com; s=zohoarc; b=XjyVJzxwmfplN0Wlq872RQ8/CA1o5DocYbqy4/7pfCsZW0hrTn5noZ779jebGDi2nLqs4FfES40z5rkbyy8yJoNtfudTYZJwOvcfEGXuDIF8aQrsnRgpxb16mgkR9G59DWoccHKKK1sVPqeYnzMJwypm1Vmgwdb5lr4RR1jPyt0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783377918; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WNQXvY4H9ddOejrxd9Cqu1UmGiWUOveKXUjtW6F6vyE=; b=RfYw6Ilt3frIHj7g/SIyZk3zNoD9u0Ebjur2OjfxlydpnILOXsML+d3HBirFvDK3O2qZS14On2XZEyKFk2EoHsgnKEJbhUUfeDMHRJvyTVcq0jcm3PJwtrrMd/9I8lBCKypaQRk+H9AM1bhzSGVbNr+FoWT3ZfM9Bqd0058csVA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178337791868469.79907173113168; Mon, 6 Jul 2026 15:45:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3r-0002CX-Jk; Mon, 06 Jul 2026 18:45:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3q-0002BM-IB; Mon, 06 Jul 2026 18:45:10 -0400 Received: from fout-b2-smtp.messagingengine.com ([202.12.124.145]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3m-0001FZ-H3; Mon, 06 Jul 2026 18:45:10 -0400 Received: from phl-compute-02.internal (phl-compute-02.internal [10.202.2.42]) by mailfout.stl.internal (Postfix) with ESMTP id 422AC1D0019F; Mon, 6 Jul 2026 18:45:05 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-02.internal (MEProxy); Mon, 06 Jul 2026 18:45:05 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:45:00 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377905; x= 1783464305; bh=WNQXvY4H9ddOejrxd9Cqu1UmGiWUOveKXUjtW6F6vyE=; b=b ejjw778Kvc7lh5puxL238w8feCWlMJs4jJkd4pY6JUT5vMhdNpO4uHDIrzYu9yIB +4FM/TSSH6Xtkz1Pd0z83bT4cKIY2uf2wCyosLwiK1fraDuMAX5mwliucchZ9YJQ +ermYvyDxDdwW11nHR8w/kTyA5Rmh97EL4ju9hl6nkH7ptvlV4Uanw2K5SVYODtb GyNi5EX7h3OPhDb2SeIY4mWcf6EIM7nt/DJK7fJEvZPmqwE0PD7sY9Bq3nghq9Mp SRXupEseoQ6tniAD7RE4vl0IowVxKpfWz6jAdaQWvJYgQcb1j0kuZRZtax6cGKkc CkOuJA/KB8MQAEYAND0aA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377905; x=1783464305; bh=W NQXvY4H9ddOejrxd9Cqu1UmGiWUOveKXUjtW6F6vyE=; b=DRBpK3JfYOrvk19fv uZlwmCGLcdyyvF9+aaSDmWoodESD88k2r60MfWMbxISIxH0SXjCAD73u2LVLLGeO 7XKLA83efdrFHE20Gsf3eCCDhVTu8lMQTSuurGboUeNxnbCgw59yDO+WQgvtapE+ RrRRAFKkP3MQ6MOgF0tqEBEq/cI3x0ZaWNSKQPDhqJAGjUNF1KMXNYqucZk7e6u5 GcrKZkHsO7FMreATGGZss9fm3DuPG/umttfu6BvaSDNfl6wTHTtsd7wLMGg2uraZ Ie8yCVStGWdB1lZuP1OUARTRgRaDgAc62GBH3WHZyWtJSQUFGh+7G8vcE3fgkNA+ EEUqA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTGrn9+YLw94kdmNvAdplbQIQOlb8IkhBs6q7ll4jULvFjem/N0AO6WRE0qlj/uT3F ZS3X40xxlZmD0Z35ymPgE7nUI5PksyHECntScirvoEMdJyawdLoWDKXAleD5ACM5Nk0Rdr VeM0fyiLE2/mBBxYjDITCquyvSrdxGs0xsA7HqDEL4FP+VTGfrNMsNtmnFeHTETja3akWG eFthISjb66ZqtqieLXSq/2UeQ1n+qeDPJp+c0vys8OY6FhlH6qB8zZLLpDdVvzt05Ol3gr SmOTv2u/1+9OTq0kbLRlc0tPTTKiGyFt+15NRE8SP+HOvn884lR3MwgbFfPfR0oArpzEHm R94oZXvZ0Ijkgm2+9vZXAnBxEeH4cC39R4aftac9oZv9axgD/VNarQNDRJ7+gUiX5UU164 /vcM4Zle4dQL6Fca87Hd9KUGEjdOh6UGu6L5xxn7S88mMohaiNPCCqGNrK+pHf6xE7Hf60 CwlqaN+BpQT++km8PbnfOeXSfQnLmLDC0Y8CSo8Uhq2qmajjUQ7O0Ab7FUw1hjXqjdGGeB X5sT98Fo1L7VUn2GOnbTI9SDzwPY924LXD67EYvxLpZW96RbvSQPL3xMCMNab1UJw7KM0v cRSppLg7WxWCq0jSgIu8aFBbshjgthQQBB93lXBE9RvQ1467yhQr3mLps7og X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Stefan Hajnoczi , Klaus Jensen , Fabiano Rosas , Keith Busch , Klaus Jensen , Jesper Devantier , qemu-block@nongnu.org Subject: [PULL 08/11] hw/nvme: add basic live migration support Date: Tue, 7 Jul 2026 00:44:20 +0200 Message-ID: <20260706224426.14156-9-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.145; envelope-from=its@irrelevant.dk; helo=fout-b2-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783377919685158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn It has some limitations: - only one NVMe namespace is supported - SMART counters are not preserved - CMB is not supported - PMR is not supported - SPDM is not supported - SR-IOV is not supported Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Reviewed-by: Klaus Jensen Reviewed-by: Fabiano Rosas Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 760 ++++++++++++++++++++++++++++++++++++++++++- hw/nvme/ns.c | 164 ++++++++++ hw/nvme/nvme.h | 9 + hw/nvme/trace-events | 10 + 4 files changed, 934 insertions(+), 9 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index cd30735e86d2..1fa3a24f7635 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -210,6 +210,7 @@ #include "hw/pci/pcie_sriov.h" #include "system/spdm-socket.h" #include "migration/blocker.h" +#include "migration/qemu-file-types.h" #include "migration/vmstate.h" =20 #include "nvme.h" @@ -1533,6 +1534,18 @@ static void nvme_post_cqes(void *opaque) break; } =20 + /* + * Here we take the following fields from NvmeRequest structure + * and write cqe to the guest RAM based on them: + * - req->sq + * - req->status + * - req->cqe + * + * If you change this code and more fields from NvmeRequest are + * used, please make sure that you have handled this in: + * nvme_vmstate_request and nvme_ctrl_pre_save(). + */ + sq =3D req->sq; req->cqe.status =3D cpu_to_le16((req->status << 1) | cq->phase); req->cqe.sq_head =3D cpu_to_le16(sq->head); @@ -4918,6 +4931,25 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n= , uint64_t dma_addr, __nvme_init_sq(sq); } =20 +static void nvme_restore_sq(NvmeSQueue *sq_from) +{ + NvmeCtrl *n =3D sq_from->ctrl; + NvmeSQueue *sq =3D sq_from; + + if (sq_from->sqid =3D=3D 0) { + sq =3D &n->admin_sq; + sq->ctrl =3D n; + sq->dma_addr =3D sq_from->dma_addr; + sq->sqid =3D sq_from->sqid; + sq->size =3D sq_from->size; + sq->cqid =3D sq_from->cqid; + sq->head =3D sq_from->head; + sq->tail =3D sq_from->tail; + } + + __nvme_init_sq(sq); +} + static uint16_t nvme_create_sq(NvmeCtrl *n, NvmeRequest *req) { NvmeSQueue *sq; @@ -5620,6 +5652,39 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n= , uint64_t dma_addr, __nvme_init_cq(cq); } =20 +static void move_cq_req_list(NvmeCQueue *cq_to, NvmeCQueue *cq_from) +{ + NvmeRequest *req, *next; + + QTAILQ_FOREACH_SAFE(req, &cq_from->req_list, entry, next) { + QTAILQ_REMOVE(&cq_from->req_list, req, entry); + QTAILQ_INSERT_TAIL(&cq_to->req_list, req, entry); + } +} + +static void nvme_restore_cq(NvmeCQueue *cq_from) +{ + NvmeCtrl *n =3D cq_from->ctrl; + NvmeCQueue *cq =3D cq_from; + + if (cq_from->cqid =3D=3D 0) { + cq =3D &n->admin_cq; + cq->ctrl =3D n; + cq->cqid =3D cq_from->cqid; + cq->size =3D cq_from->size; + cq->dma_addr =3D cq_from->dma_addr; + cq->phase =3D cq_from->phase; + cq->irq_enabled =3D cq_from->irq_enabled; + cq->vector =3D cq_from->vector; + cq->head =3D cq_from->head; + cq->tail =3D cq_from->tail; + QTAILQ_INIT(&cq->req_list); + move_cq_req_list(cq, cq_from); + } + + __nvme_init_cq(cq); +} + static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeRequest *req) { NvmeCQueue *cq; @@ -7320,7 +7385,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const = NvmeRequest *req) n->dbbuf_eis =3D eis_addr; n->dbbuf_enabled =3D true; =20 - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { NvmeSQueue *sq =3D n->sq[i]; NvmeCQueue *cq =3D n->cq[i]; =20 @@ -7764,7 +7829,7 @@ static int nvme_atomic_write_check(NvmeCtrl *n, NvmeC= md *cmd, /* * Walk the queues to see if there are any atomic conflicts. */ - for (i =3D 1; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 1; i < n->num_queues; i++) { NvmeSQueue *sq; NvmeRequest *req; NvmeRwCmd *req_rw; @@ -7834,6 +7899,12 @@ static void nvme_process_sq(void *opaque) NvmeCmd cmd; NvmeRequest *req; =20 + /* + * We don't want to have a race with nvme_ctrl_pre_save(). + * What implicitly protects us from this is BQL. + */ + assert(bql_locked()); + if (n->dbbuf_enabled) { nvme_update_sq_tail(sq); } @@ -7951,12 +8022,12 @@ static void nvme_ctrl_reset(NvmeCtrl *n, NvmeResetT= ype rst) nvme_ns_drain(ns); } =20 - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { if (n->sq[i] !=3D NULL) { nvme_free_sq(n->sq[i], n); } } - for (i =3D 0; i < n->params.max_ioqpairs + 1; i++) { + for (i =3D 0; i < n->num_queues; i++) { if (n->cq[i] !=3D NULL) { nvme_free_cq(n->cq[i], n); } @@ -8626,6 +8697,8 @@ static bool nvme_check_params(NvmeCtrl *n, Error **er= rp) params->max_ioqpairs =3D params->num_queues - 1; } =20 + n->num_queues =3D params->max_ioqpairs + 1; + if (n->namespace.blkconf.blk && n->subsys) { error_setg(errp, "subsystem support is unavailable with legacy " "namespace ('drive' property)"); @@ -8799,8 +8872,8 @@ static void nvme_init_state(NvmeCtrl *n) n->conf_msix_qsize =3D n->params.msix_qsize; } =20 - n->sq =3D g_new0(NvmeSQueue *, n->params.max_ioqpairs + 1); - n->cq =3D g_new0(NvmeCQueue *, n->params.max_ioqpairs + 1); + n->sq =3D g_new0(NvmeSQueue *, n->num_queues); + n->cq =3D g_new0(NvmeCQueue *, n->num_queues); n->temperature =3D NVME_TEMPERATURE; n->features.temp_thresh_hi =3D NVME_TEMPERATURE_WARNING; n->starttime_ms =3D qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); @@ -9035,7 +9108,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci= _dev, Error **errp) } =20 if (n->params.msix_exclusive_bar && !pci_is_vf(pci_dev)) { - bar_size =3D nvme_mbar_size(n->params.max_ioqpairs + 1, 0, NULL, N= ULL); + bar_size =3D nvme_mbar_size(n->num_queues, 0, NULL, NULL); memory_region_init_io(&n->iomem, OBJECT(n), &nvme_mmio_ops, n, "nv= me", bar_size); pci_register_bar(pci_dev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY | @@ -9047,7 +9120,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci= _dev, Error **errp) /* add one to max_ioqpairs to account for the admin queue pair */ if (!pci_is_vf(pci_dev)) { nr_vectors =3D n->params.msix_qsize; - bar_size =3D nvme_mbar_size(n->params.max_ioqpairs + 1, + bar_size =3D nvme_mbar_size(n->num_queues, nr_vectors, &msix_table_offset, &msix_pba_offset); } else { @@ -9780,9 +9853,678 @@ static uint32_t nvme_pci_read_config(PCIDevice *dev= , uint32_t address, int len) return pci_default_read_config(dev, address, len); } =20 +static const VMStateDescription nvme_vmstate_cqe =3D { + .name =3D "nvme-cqe", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT32(result, NvmeCqe), + VMSTATE_UINT32(dw1, NvmeCqe), + VMSTATE_UINT16(sq_head, NvmeCqe), + VMSTATE_UINT16(sq_id, NvmeCqe), + VMSTATE_UINT16(cid, NvmeCqe), + VMSTATE_UINT16(status, NvmeCqe), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd_dptr_sgl =3D { + .name =3D "nvme-request-cmd-dptr-sgl", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(addr, NvmeSglDescriptor), + VMSTATE_UINT32(len, NvmeSglDescriptor), + VMSTATE_UINT8_ARRAY(rsvd, NvmeSglDescriptor, 3), + VMSTATE_UINT8(type, NvmeSglDescriptor), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd_dptr =3D { + .name =3D "nvme-request-cmd-dptr", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(prp1, NvmeCmdDptr), + VMSTATE_UINT64(prp2, NvmeCmdDptr), + VMSTATE_STRUCT(sgl, NvmeCmdDptr, 0, + nvme_vmstate_cmd_dptr_sgl, NvmeSglDescriptor), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_cmd =3D { + .name =3D "nvme-request-cmd", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(opcode, NvmeCmd), + VMSTATE_UINT8(flags, NvmeCmd), + VMSTATE_UINT16(cid, NvmeCmd), + VMSTATE_UINT32(nsid, NvmeCmd), + VMSTATE_UINT64(res1, NvmeCmd), + VMSTATE_UINT64(mptr, NvmeCmd), + VMSTATE_STRUCT(dptr, NvmeCmd, 0, nvme_vmstate_cmd_dptr, NvmeCmdDpt= r), + VMSTATE_UINT32(cdw10, NvmeCmd), + VMSTATE_UINT32(cdw11, NvmeCmd), + VMSTATE_UINT32(cdw12, NvmeCmd), + VMSTATE_UINT32(cdw13, NvmeCmd), + VMSTATE_UINT32(cdw14, NvmeCmd), + VMSTATE_UINT32(cdw15, NvmeCmd), + VMSTATE_END_OF_LIST() + } +}; + +static bool nvme_req_pre_load(void *opaque, Error **errp) +{ + memset(opaque, 0x0, sizeof(NvmeRequest)); + return true; +} + +static const VMStateDescription nvme_vmstate_request =3D { + .name =3D "nvme-request", + .version_id =3D 1, + .minimum_version_id =3D 1, + .pre_load_errp =3D nvme_req_pre_load, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(status, NvmeRequest), + VMSTATE_STRUCT(cqe, NvmeRequest, 0, nvme_vmstate_cqe, NvmeCqe), + VMSTATE_STRUCT(cmd, NvmeRequest, 0, nvme_vmstate_cmd, NvmeCmd), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_bar =3D { + .name =3D "nvme-bar", + .minimum_version_id =3D 1, + .version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(cap, NvmeBar), + VMSTATE_UINT32(vs, NvmeBar), + VMSTATE_UINT32(intms, NvmeBar), + VMSTATE_UINT32(intmc, NvmeBar), + VMSTATE_UINT32(cc, NvmeBar), + VMSTATE_UINT8_ARRAY(rsvd24, NvmeBar, 4), + VMSTATE_UINT32(csts, NvmeBar), + VMSTATE_UINT32(nssr, NvmeBar), + VMSTATE_UINT32(aqa, NvmeBar), + VMSTATE_UINT64(asq, NvmeBar), + VMSTATE_UINT64(acq, NvmeBar), + VMSTATE_UINT32(cmbloc, NvmeBar), + VMSTATE_UINT32(cmbsz, NvmeBar), + VMSTATE_UINT32(bpinfo, NvmeBar), + VMSTATE_UINT32(bprsel, NvmeBar), + VMSTATE_UINT64(bpmbl, NvmeBar), + VMSTATE_UINT64(cmbmsc, NvmeBar), + VMSTATE_UINT32(cmbsts, NvmeBar), + VMSTATE_UINT8_ARRAY(rsvd92, NvmeBar, 3492), + VMSTATE_UINT32(pmrcap, NvmeBar), + VMSTATE_UINT32(pmrctl, NvmeBar), + VMSTATE_UINT32(pmrsts, NvmeBar), + VMSTATE_UINT32(pmrebs, NvmeBar), + VMSTATE_UINT32(pmrswtp, NvmeBar), + VMSTATE_UINT32(pmrmscl, NvmeBar), + VMSTATE_UINT32(pmrmscu, NvmeBar), + VMSTATE_UINT8_ARRAY(css, NvmeBar, 484), + VMSTATE_END_OF_LIST() + }, +}; + +static bool nvme_cqueue_pre_load(void *opaque, Error **errp) +{ + NvmeCQueue *cq =3D opaque; + + QTAILQ_INIT(&cq->req_list); + return true; +} + +static const VMStateDescription nvme_vmstate_cqueue =3D { + .name =3D "nvme-cq", + .version_id =3D 1, + .minimum_version_id =3D 1, + .pre_load_errp =3D nvme_cqueue_pre_load, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(phase, NvmeCQueue), + VMSTATE_UINT16(cqid, NvmeCQueue), + VMSTATE_UINT16(irq_enabled, NvmeCQueue), + VMSTATE_UINT32(head, NvmeCQueue), + VMSTATE_UINT32(tail, NvmeCQueue), + VMSTATE_UINT32(vector, NvmeCQueue), + VMSTATE_UINT32(size, NvmeCQueue), + VMSTATE_UINT64(dma_addr, NvmeCQueue), + + VMSTATE_QTAILQ_V(req_list, NvmeCQueue, 1, nvme_vmstate_request, + NvmeRequest, entry), + + /* db_addr, ei_addr, etc will be recalculated */ + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_squeue =3D { + .name =3D "nvme-sq", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(sqid, NvmeSQueue), + VMSTATE_UINT16(cqid, NvmeSQueue), + VMSTATE_UINT32(head, NvmeSQueue), + VMSTATE_UINT32(tail, NvmeSQueue), + VMSTATE_UINT32(size, NvmeSQueue), + VMSTATE_UINT64(dma_addr, NvmeSQueue), + /* db_addr, ei_addr, etc will be recalculated */ + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_async_event_result =3D { + .name =3D "nvme-async-event-result", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(event_type, NvmeAerResult), + VMSTATE_UINT8(event_info, NvmeAerResult), + VMSTATE_UINT8(log_page, NvmeAerResult), + VMSTATE_UINT8(resv, NvmeAerResult), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_async_event =3D { + .name =3D "nvme-async-event", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_STRUCT(result, NvmeAsyncEvent, 0, + nvme_vmstate_async_event_result, NvmeAerResult), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_hbs =3D { + .name =3D "nvme-hbs", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(acre, NvmeHostBehaviorSupport), + VMSTATE_UINT8(etdas, NvmeHostBehaviorSupport), + VMSTATE_UINT8(lbafee, NvmeHostBehaviorSupport), + VMSTATE_UINT8(rsvd3, NvmeHostBehaviorSupport), + VMSTATE_UINT16(cdfe, NvmeHostBehaviorSupport), + VMSTATE_UINT8_ARRAY(rsvd6, NvmeHostBehaviorSupport, 506), + VMSTATE_END_OF_LIST() + } +}; + +const VMStateDescription nvme_vmstate_atomic =3D { + .name =3D "nvme-atomic", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT32(atomic_max_write_size, NvmeAtomic), + VMSTATE_UINT64(atomic_boundary, NvmeAtomic), + VMSTATE_UINT64(atomic_nabo, NvmeAtomic), + VMSTATE_BOOL(atomic_writes, NvmeAtomic), + VMSTATE_END_OF_LIST() + } +}; + +static bool pre_save_validate_aer_req(NvmeRequest *req, Error **errp) +{ + /* + * Can't use assert() here, because we don't want + * to just crash QEMU when user requests a migration. + */ + if (!(req->cmd.opcode =3D=3D NVME_ADM_CMD_ASYNC_EV_REQ)) { + error_setg(errp, "req->cmd.opcode (%u) !=3D NVME_ADM_CMD_ASYNC_EV_= REQ", + req->cmd.opcode); + return false; + } + + if (!(req->ns =3D=3D NULL)) { + error_setg(errp, "req->ns !=3D NULL"); + return false; + } + + if (!(req->sq =3D=3D &req->sq->ctrl->admin_sq)) { + error_setg(errp, "req->sq !=3D &req->sq->ctrl->admin_sq"); + return false; + } + + if (!(req->aiocb =3D=3D NULL)) { + error_setg(errp, "req->aiocb !=3D NULL"); + return false; + } + + if (!(req->opaque =3D=3D NULL)) { + error_setg(errp, "req->opaque !=3D NULL"); + return false; + } + + if (!(req->atomic_write =3D=3D false)) { + error_setg(errp, "req->atomic_write !=3D false"); + return false; + } + + if (req->sg.flags & NVME_SG_ALLOC) { + error_setg(errp, "unexpected NVME_SG_ALLOC flag in req->sg.flags"); + return false; + } + + return true; +} + +static bool pre_save_validate_cq_req(NvmeRequest *req, Error **errp) +{ + if (!(req->ns =3D=3D NULL)) { + error_setg(errp, "req->ns !=3D NULL"); + return false; + } + + if (!(req->aiocb =3D=3D NULL)) { + error_setg(errp, "req->aiocb !=3D NULL"); + return false; + } + + if (!(req->opaque =3D=3D NULL)) { + error_setg(errp, "req->opaque !=3D NULL"); + return false; + } + + if (!(req->atomic_write =3D=3D false)) { + error_setg(errp, "req->atomic_write !=3D false"); + return false; + } + + if (req->sg.flags & NVME_SG_ALLOC) { + error_setg(errp, "unexpected NVME_SG_ALLOC flag in req->sg.flags"); + return false; + } + + return true; +} + +static bool nvme_ctrl_pre_save(void *opaque, Error **errp) +{ + NvmeCtrl *n =3D opaque; + int i; + + trace_pci_nvme_pre_save_enter(n); + + /* + * We don't want to have a race with nvme_process_sq(). + * What implicitly protects us from this is BQL. + */ + assert(bql_locked()); + + /* cancel all SQ processing BHs */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) { + continue; + } + + qemu_bh_cancel(sq->bh); + } + + /* drain all IO */ + for (i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns; + + ns =3D nvme_ns(n, i); + if (!ns) { + continue; + } + + trace_pci_nvme_pre_save_ns_drain(n, i); + nvme_ns_drain(ns); + } + + /* + * Now, we should take care of AERs. + * + * 1. Save all queued events (n->aer_queue). + * This is done automatically, see nvme_vmstate VMStateDescription. + * Here we only need to print them for debugging purpose. + * 2. Go over outstanding AER requests (n->aer_reqs) and check they are + * all have expected opcode (NVME_ADM_CMD_ASYNC_EV_REQ) and other f= ields. + * + * We must be really careful here, because in case of further + * QEMU NVMe changes, we may break migration without noticing it, or w= orse, + * introduce silent data corruption during migration. + */ + if (n->aer_queued) { + NvmeAsyncEvent *event; + + QTAILQ_FOREACH(event, &n->aer_queue, entry) { + trace_pci_nvme_pre_save_aer(event->result.event_type, + event->result.event_info, + event->result.log_page); + } + } + + for (i =3D 0; i < n->outstanding_aers; i++) { + NvmeRequest *req =3D n->aer_reqs[i]; + + if (!pre_save_validate_aer_req(req, errp)) { + return false; + } + } + + /* + * Make sure that all in-flight IO requests + * (except NVME_ADM_CMD_ASYNC_EV_REQ) are processed. + */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeRequest *req; + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) { + continue; + } + + trace_pci_nvme_pre_save_sq_out_req_check(n, i, + sq->head, sq->tail, sq->s= ize); + + QTAILQ_FOREACH(req, &sq->out_req_list, entry) { + assert(req->cmd.opcode =3D=3D NVME_ADM_CMD_ASYNC_EV_REQ); + } + } + + /* wait when all IO requests completions are written to guest memory */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeCQueue *cq =3D n->cq[i]; + + if (!cq) { + continue; + } + + qemu_bh_cancel(cq->bh); + /* this should empty cq->req_list unless CQ is full */ + nvme_post_cqes(cq); + + trace_pci_nvme_pre_save_cq_req_check(n, i, + cq->head, cq->tail, cq->size); + + if (!QTAILQ_EMPTY(&cq->req_list)) { + NvmeRequest *req; + + assert(nvme_cq_full(cq)); + + QTAILQ_FOREACH(req, &cq->req_list, entry) { + trace_pci_nvme_pre_save_cq_unposted_cqe( + n, i, nvme_cid(req), + nvme_nsid(req->ns), + le32_to_cpu(req->cqe.result), + le32_to_cpu(req->cqe.dw1), + req->status, req->cmd.opcode); + if (!pre_save_validate_cq_req(req, errp)) { + return false; + } + } + } + } + + for (uint32_t nsid =3D 0; nsid <=3D NVME_MAX_NAMESPACES; nsid++) { + NvmeNamespace *ns =3D n->namespaces[nsid]; + + if (!ns) { + continue; + } + + if (ns !=3D &n->namespace) { + error_setg(errp, + "only one NVMe namespace is supported for migration= "); + return false; + } + } + + return true; +} + +static bool nvme_ctrl_post_load(void *opaque, int version_id, Error **errp) +{ + NvmeCtrl *n =3D opaque; + int i; + + trace_pci_nvme_post_load_enter(n); + + /* restore CQs first */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeCQueue *cq =3D n->cq[i]; + + if (!cq) { + continue; + } + + if (cq->cqid !=3D i) { + error_setg(errp, "inconsistent migration stream (cq->cqid !=3D= i)"); + return false; + } + + cq->ctrl =3D n; + nvme_restore_cq(cq); + trace_pci_nvme_post_load_restore_cq(n, i, cq->head, cq->tail, cq->= size); + + if (i =3D=3D 0) { + /* + * Admin CQ lives in n->admin_cq, we don't need + * memory allocated for it in get_ptrs_array_entry() anymore. + * + * nvme_restore_cq() also takes care of: + * n->cq[0] =3D &n->admin_cq; + * so n->cq[0] remains valid. + */ + g_free(cq); + } + } + + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) { + continue; + } + + if (sq->sqid !=3D i) { + error_setg(errp, "inconsistent migration stream (sq->sqid !=3D= i)"); + return false; + } + + if (!n->cq[sq->cqid]) { + error_setg(errp, + "inconsistent migration stream (n->cq[sq->cqid] is = NULL)"); + return false; + } + + sq->ctrl =3D n; + nvme_restore_sq(sq); + trace_pci_nvme_post_load_restore_sq(n, i, sq->head, sq->tail, sq->= size); + + if (i =3D=3D 0) { + /* same as for CQ */ + g_free(sq); + } + } + + /* restore cq->req_list-s */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeRequest *req_from, *next; + typeof_field(NvmeCQueue, req_list) req_list; + NvmeCQueue *cq =3D n->cq[i]; + + if (!cq || QTAILQ_EMPTY(&cq->req_list)) { + continue; + } + + /* + * We use nvme_vmstate_request VMStateDescription to save/restore + * NvmeRequest structures, but tricky thing here is that + * memory for each cq->req_list item is allocated separately + * during restore. It doesn't work for us. We need to take + * an existing NvmeRequest structure from SQ's req_list pool + * and fill it with data from the newly allocated one (req_from). + * Then, we can safely release allocated memory for it. + */ + + /* make a copy of cq->req_list (QTAILQ head) and clean cq->req_lis= t */ + QTAILQ_INIT(&req_list); + QTAILQ_FOREACH_SAFE(req_from, &cq->req_list, entry, next) { + QTAILQ_REMOVE(&cq->req_list, req_from, entry); + QTAILQ_INSERT_TAIL(&req_list, req_from, entry); + } + QTAILQ_INIT(&cq->req_list); + + QTAILQ_FOREACH_SAFE(req_from, &req_list, entry, next) { + uint16_t sqid =3D le16_to_cpu(req_from->cqe.sq_id); + NvmeRequest *req; + NvmeSQueue *sq; + + assert(!nvme_check_sqid(n, sqid)); + sq =3D n->sq[sqid]; + + req =3D QTAILQ_FIRST(&sq->req_list); + QTAILQ_REMOVE(&sq->req_list, req, entry); + QTAILQ_INSERT_TAIL(&cq->req_list, req, entry); + nvme_req_clear(req); + + /* copy data from the source NvmeRequest */ + req->status =3D req_from->status; + memcpy(&req->cqe, &req_from->cqe, sizeof(NvmeCqe)); + memcpy(&req->cmd, &req_from->cmd, sizeof(NvmeCmd)); + + QTAILQ_REMOVE(&req_list, req_from, entry); + g_free(req_from); + } + + qemu_bh_schedule(cq->bh); + } + + if (n->aer_queued) { + NvmeAsyncEvent *event; + + QTAILQ_FOREACH(event, &n->aer_queue, entry) { + trace_pci_nvme_post_load_aer(event->result.event_type, + event->result.event_info, + event->result.log_page); + } + } + + for (i =3D 0; i < n->outstanding_aers; i++) { + NvmeSQueue *sq =3D &n->admin_sq; + NvmeRequest *req_from =3D n->aer_reqs[i]; + NvmeRequest *req; + + /* Idea here is the same as for "restore cq->req_list-s" step */ + + /* take an NvmeRequest struct from SQ */ + req =3D QTAILQ_FIRST(&sq->req_list); + QTAILQ_REMOVE(&sq->req_list, req, entry); + QTAILQ_INSERT_TAIL(&sq->out_req_list, req, entry); + nvme_req_clear(req); + + /* copy data from the source NvmeRequest */ + req->status =3D req_from->status; + memcpy(&req->cqe, &req_from->cqe, sizeof(NvmeCqe)); + memcpy(&req->cmd, &req_from->cmd, sizeof(NvmeCmd)); + + n->aer_reqs[i] =3D req; + g_free(req_from); + } + + /* + * We need to attach namespaces (currently, only one namespace is + * supported for migration). + * This logic comes from nvme_start_ctrl(). + */ + for (i =3D 1; i <=3D NVME_MAX_NAMESPACES; i++) { + NvmeNamespace *ns =3D nvme_subsys_ns(n->subsys, i); + + if (!ns || (!ns->params.shared && ns->ctrl !=3D n)) { + continue; + } + + if (nvme_csi_supported(n, ns->csi) && !ns->params.detached) { + if (!ns->attached || ns->params.shared) { + nvme_attach_ns(n, ns); + } + } + } + + /* schedule SQ processing */ + for (i =3D 0; i < n->num_queues; i++) { + NvmeSQueue *sq =3D n->sq[i]; + + if (!sq) { + continue; + } + + qemu_bh_schedule(sq->bh); + } + + return true; +} + static const VMStateDescription nvme_vmstate =3D { .name =3D "nvme", - .unmigratable =3D 1, + .minimum_version_id =3D 1, + .version_id =3D 1, + .pre_save_errp =3D nvme_ctrl_pre_save, + .post_load_errp =3D nvme_ctrl_post_load, + .fields =3D (const VMStateField[]) { + VMSTATE_PCI_DEVICE(parent_obj, NvmeCtrl), + VMSTATE_MSIX(parent_obj, NvmeCtrl), + VMSTATE_STRUCT(bar, NvmeCtrl, 0, nvme_vmstate_bar, NvmeBar), + + VMSTATE_BOOL(qs_created, NvmeCtrl), + VMSTATE_UINT32(page_size, NvmeCtrl), + VMSTATE_UINT16(page_bits, NvmeCtrl), + VMSTATE_UINT16(max_prp_ents, NvmeCtrl), + VMSTATE_UINT32(max_q_ents, NvmeCtrl), + VMSTATE_UINT8(outstanding_aers, NvmeCtrl), + VMSTATE_UINT32(irq_status, NvmeCtrl), + VMSTATE_INT32(cq_pending, NvmeCtrl), + + VMSTATE_UINT64(host_timestamp, NvmeCtrl), + VMSTATE_UINT64(timestamp_set_qemu_clock_ms, NvmeCtrl), + VMSTATE_UINT64(starttime_ms, NvmeCtrl), + VMSTATE_UINT16(temperature, NvmeCtrl), + VMSTATE_UINT8(smart_critical_warning, NvmeCtrl), + + VMSTATE_UINT32(conf_msix_qsize, NvmeCtrl), + VMSTATE_UINT32(conf_ioqpairs, NvmeCtrl), + VMSTATE_UINT64(dbbuf_dbs, NvmeCtrl), + VMSTATE_UINT64(dbbuf_eis, NvmeCtrl), + VMSTATE_BOOL(dbbuf_enabled, NvmeCtrl), + + VMSTATE_UINT8(aer_mask, NvmeCtrl), + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT8_ALLOC( + aer_reqs, NvmeCtrl, outstanding_aers, 0, + nvme_vmstate_request, NvmeRequest), + VMSTATE_QTAILQ_V(aer_queue, NvmeCtrl, 1, nvme_vmstate_async_event, + NvmeAsyncEvent, entry), + VMSTATE_INT32(aer_queued, NvmeCtrl), + + VMSTATE_STRUCT(namespace, NvmeCtrl, 0, nvme_vmstate_ns, NvmeNamesp= ace), + + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC( + sq, NvmeCtrl, num_queues, 0, nvme_vmstate_squeue, NvmeSQueue), + VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT32_ALLOC( + cq, NvmeCtrl, num_queues, 0, nvme_vmstate_cqueue, NvmeCQueue), + + VMSTATE_UINT16(features.temp_thresh_hi, NvmeCtrl), + VMSTATE_UINT16(features.temp_thresh_low, NvmeCtrl), + VMSTATE_UINT32(features.async_config, NvmeCtrl), + VMSTATE_STRUCT(features.hbs, NvmeCtrl, 0, + nvme_vmstate_hbs, NvmeHostBehaviorSupport), + + VMSTATE_UINT32(dn, NvmeCtrl), + VMSTATE_STRUCT(atomic, NvmeCtrl, 0, nvme_vmstate_atomic, NvmeAtomi= c), + + VMSTATE_END_OF_LIST() + }, }; =20 static void nvme_class_init(ObjectClass *oc, const void *data) diff --git a/hw/nvme/ns.c b/hw/nvme/ns.c index b0106eaa5c8f..4caab590977f 100644 --- a/hw/nvme/ns.c +++ b/hw/nvme/ns.c @@ -20,6 +20,7 @@ #include "qemu/bitops.h" #include "system/system.h" #include "system/block-backend.h" +#include "migration/vmstate.h" =20 #include "nvme.h" #include "trace.h" @@ -886,6 +887,168 @@ static void nvme_ns_realize(DeviceState *dev, Error *= *errp) } } =20 +static const VMStateDescription nvme_vmstate_lbaf =3D { + .name =3D "nvme_lbaf", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT16(ms, NvmeLBAF), + VMSTATE_UINT8(ds, NvmeLBAF), + VMSTATE_UINT8(rp, NvmeLBAF), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_id_ns =3D { + .name =3D "nvme_id_ns", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(nsze, NvmeIdNs), + VMSTATE_UINT64(ncap, NvmeIdNs), + VMSTATE_UINT64(nuse, NvmeIdNs), + VMSTATE_UINT8(nsfeat, NvmeIdNs), + VMSTATE_UINT8(nlbaf, NvmeIdNs), + VMSTATE_UINT8(flbas, NvmeIdNs), + VMSTATE_UINT8(mc, NvmeIdNs), + VMSTATE_UINT8(dpc, NvmeIdNs), + VMSTATE_UINT8(dps, NvmeIdNs), + VMSTATE_UINT8(nmic, NvmeIdNs), + VMSTATE_UINT8(rescap, NvmeIdNs), + VMSTATE_UINT8(fpi, NvmeIdNs), + VMSTATE_UINT8(dlfeat, NvmeIdNs), + VMSTATE_UINT16(nawun, NvmeIdNs), + VMSTATE_UINT16(nawupf, NvmeIdNs), + VMSTATE_UINT16(nacwu, NvmeIdNs), + VMSTATE_UINT16(nabsn, NvmeIdNs), + VMSTATE_UINT16(nabo, NvmeIdNs), + VMSTATE_UINT16(nabspf, NvmeIdNs), + VMSTATE_UINT16(noiob, NvmeIdNs), + VMSTATE_UINT8_ARRAY(nvmcap, NvmeIdNs, 16), + VMSTATE_UINT16(npwg, NvmeIdNs), + VMSTATE_UINT16(npwa, NvmeIdNs), + VMSTATE_UINT16(npdg, NvmeIdNs), + VMSTATE_UINT16(npda, NvmeIdNs), + VMSTATE_UINT16(nows, NvmeIdNs), + VMSTATE_UINT16(mssrl, NvmeIdNs), + VMSTATE_UINT32(mcl, NvmeIdNs), + VMSTATE_UINT8(msrc, NvmeIdNs), + VMSTATE_UINT8_ARRAY(rsvd81, NvmeIdNs, 18), + VMSTATE_UINT8(nsattr, NvmeIdNs), + VMSTATE_UINT16(nvmsetid, NvmeIdNs), + VMSTATE_UINT16(endgid, NvmeIdNs), + VMSTATE_UINT8_ARRAY(nguid, NvmeIdNs, 16), + VMSTATE_UINT64(eui64, NvmeIdNs), + VMSTATE_STRUCT_ARRAY(lbaf, NvmeIdNs, NVME_MAX_NLBAF, 1, + nvme_vmstate_lbaf, NvmeLBAF), + VMSTATE_UINT8_ARRAY(vs, NvmeIdNs, 3712), + + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_id_ns_nvm =3D { + .name =3D "nvme_id_ns_nvm", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT64(lbstm, NvmeIdNsNvm), + VMSTATE_UINT8(pic, NvmeIdNsNvm), + VMSTATE_UINT8_ARRAY(rsvd9, NvmeIdNsNvm, 3), + VMSTATE_UINT32_ARRAY(elbaf, NvmeIdNsNvm, NVME_MAX_NLBAF), + VMSTATE_UINT32(npdgl, NvmeIdNsNvm), + VMSTATE_UINT32(nprg, NvmeIdNsNvm), + VMSTATE_UINT32(npra, NvmeIdNsNvm), + VMSTATE_UINT32(nors, NvmeIdNsNvm), + VMSTATE_UINT32(npdal, NvmeIdNsNvm), + VMSTATE_UINT8_ARRAY(rsvd288, NvmeIdNsNvm, 3808), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription nvme_vmstate_id_ns_ind =3D { + .name =3D "nvme_id_ns_ind", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_UINT8(nsfeat, NvmeIdNsInd), + VMSTATE_UINT8(nmic, NvmeIdNsInd), + VMSTATE_UINT8(rescap, NvmeIdNsInd), + VMSTATE_UINT8(fpi, NvmeIdNsInd), + VMSTATE_UINT32(anagrpid, NvmeIdNsInd), + VMSTATE_UINT8(nsattr, NvmeIdNsInd), + VMSTATE_UINT8(rsvd9, NvmeIdNsInd), + VMSTATE_UINT16(nvmsetid, NvmeIdNsInd), + VMSTATE_UINT16(endgrpid, NvmeIdNsInd), + VMSTATE_UINT8(nstat, NvmeIdNsInd), + VMSTATE_UINT8_ARRAY(rsvd15, NvmeIdNsInd, 4081), + VMSTATE_END_OF_LIST() + } +}; + +typedef struct TmpNvmeNamespace { + NvmeNamespace *parent; + bool enable_write_cache; +} TmpNvmeNamespace; + +static bool nvme_ns_tmp_pre_save(void *opaque, Error **errp) +{ + struct TmpNvmeNamespace *tns =3D opaque; + + tns->enable_write_cache =3D blk_enable_write_cache(tns->parent->blkcon= f.blk); + + return true; +} + +static bool nvme_ns_tmp_post_load(void *opaque, int version_id, Error **er= rp) +{ + struct TmpNvmeNamespace *tns =3D opaque; + + blk_set_enable_write_cache(tns->parent->blkconf.blk, + tns->enable_write_cache); + + return true; +} + +static const VMStateDescription nvme_vmstate_ns_tmp =3D { + .name =3D "nvme_ns_tmp", + .pre_save_errp =3D nvme_ns_tmp_pre_save, + .post_load_errp =3D nvme_ns_tmp_post_load, + .fields =3D (const VMStateField[]) { + VMSTATE_BOOL(enable_write_cache, TmpNvmeNamespace), + VMSTATE_END_OF_LIST() + } +}; + +const VMStateDescription nvme_vmstate_ns =3D { + .name =3D "nvme_ns", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (const VMStateField[]) { + VMSTATE_WITH_TMP(NvmeNamespace, TmpNvmeNamespace, nvme_vmstate_ns_= tmp), + + VMSTATE_STRUCT(id_ns, NvmeNamespace, 0, nvme_vmstate_id_ns, NvmeId= Ns), + VMSTATE_STRUCT(id_ns_nvm, NvmeNamespace, 0, + nvme_vmstate_id_ns_nvm, NvmeIdNsNvm), + VMSTATE_STRUCT(id_ns_ind, NvmeNamespace, 0, + nvme_vmstate_id_ns_ind, NvmeIdNsInd), + VMSTATE_STRUCT(lbaf, NvmeNamespace, 0, nvme_vmstate_lbaf, NvmeLBAF= ), + VMSTATE_UINT32(nlbaf, NvmeNamespace), + VMSTATE_UINT8(csi, NvmeNamespace), + VMSTATE_UINT16(status, NvmeNamespace), + VMSTATE_UINT8(pif, NvmeNamespace), + + VMSTATE_UINT16(zns.zrwas, NvmeNamespace), + VMSTATE_UINT16(zns.zrwafg, NvmeNamespace), + VMSTATE_UINT32(zns.numzrwa, NvmeNamespace), + + VMSTATE_UINT32(features.err_rec, NvmeNamespace), + VMSTATE_STRUCT(atomic, NvmeNamespace, 0, + nvme_vmstate_atomic, NvmeAtomic), + VMSTATE_END_OF_LIST() + } +}; + static const Property nvme_ns_props[] =3D { DEFINE_BLOCK_PROPERTIES(NvmeNamespace, blkconf), DEFINE_PROP_BOOL("detached", NvmeNamespace, params.detached, false), @@ -937,6 +1100,7 @@ static void nvme_ns_class_init(ObjectClass *oc, const = void *data) dc->bus_type =3D TYPE_NVME_BUS; dc->realize =3D nvme_ns_realize; dc->unrealize =3D nvme_ns_unrealize; + dc->vmsd =3D &nvme_vmstate_ns; device_class_set_props(dc, nvme_ns_props); dc->desc =3D "Virtual NVMe namespace"; } diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 40a9726546dc..43e3c916f73f 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -451,6 +451,11 @@ typedef struct NvmeRequest { NvmeSg sg; bool atomic_write; QTAILQ_ENTRY(NvmeRequest)entry; + /* + * If you add a new field here, please make sure to update + * nvme_vmstate_request, pre_save_validate_aer_req() and + * pre_save_validate_cq_req(). + */ } NvmeRequest; =20 typedef struct NvmeBounceContext { @@ -647,6 +652,7 @@ typedef struct NvmeCtrl { =20 NvmeNamespace namespace; NvmeNamespace *namespaces[NVME_MAX_NAMESPACES + 1]; + uint32_t num_queues; NvmeSQueue **sq; NvmeCQueue **cq; NvmeSQueue admin_sq; @@ -758,4 +764,7 @@ void nvme_atomic_configure_max_write_size(bool dn, uint= 16_t awun, void nvme_ns_atomic_configure_boundary(bool dn, uint16_t nabsn, uint16_t nabspf, NvmeAtomic *atomic= ); =20 +extern const VMStateDescription nvme_vmstate_atomic; +extern const VMStateDescription nvme_vmstate_ns; + #endif /* HW_NVME_NVME_H */ diff --git a/hw/nvme/trace-events b/hw/nvme/trace-events index 6be0bfa1c1fc..f97a6a11f360 100644 --- a/hw/nvme/trace-events +++ b/hw/nvme/trace-events @@ -7,6 +7,16 @@ pci_nvme_dbbuf_config(uint64_t dbs_addr, uint64_t eis_addr= ) "dbs_addr=3D0x%"PRIx64 pci_nvme_map_addr(uint64_t addr, uint64_t len) "addr 0x%"PRIx64" len %"PRI= u64"" pci_nvme_map_addr_cmb(uint64_t addr, uint64_t len) "addr 0x%"PRIx64" len %= "PRIu64"" pci_nvme_map_prp(uint64_t trans_len, uint32_t len, uint64_t prp1, uint64_t= prp2, int num_prps) "trans_len %"PRIu64" len %"PRIu32" prp1 0x%"PRIx64" pr= p2 0x%"PRIx64" num_prps %d" +pci_nvme_pre_save_enter(void *n) "n=3D%p" +pci_nvme_pre_save_ns_drain(void *n, int i) "n=3D%p i=3D%d" +pci_nvme_pre_save_sq_out_req_check(void *n, int i, uint32_t head, uint32_t= tail, uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" = size=3D0x%"PRIx32"" +pci_nvme_pre_save_cq_req_check(void *n, int i, uint32_t head, uint32_t tai= l, uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" size= =3D0x%"PRIx32"" +pci_nvme_pre_save_cq_unposted_cqe(void *n, int i, uint16_t cid, uint32_t n= sid, uint32_t dw0, uint32_t dw1, uint16_t status, uint8_t opc) "n=3D%p i=3D= %d cid %"PRIu16" nsid %"PRIu32" dw0 0x%"PRIx32" dw1 0x%"PRIx32" status 0x%"= PRIx16" opc 0x%"PRIx8"" +pci_nvme_pre_save_aer(uint8_t typ, uint8_t info, uint8_t log_page) "type 0= x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" +pci_nvme_post_load_enter(void *n) "n=3D%p" +pci_nvme_post_load_restore_cq(void *n, int i, uint32_t head, uint32_t tail= , uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" size= =3D0x%"PRIx32"" +pci_nvme_post_load_restore_sq(void *n, int i, uint32_t head, uint32_t tail= , uint32_t size) "n=3D%p i=3D%d head=3D0x%"PRIx32" tail=3D0x%"PRIx32" size= =3D0x%"PRIx32"" +pci_nvme_post_load_aer(uint8_t typ, uint8_t info, uint8_t log_page) "type = 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" pci_nvme_map_sgl(uint8_t typ, uint64_t len) "type 0x%"PRIx8" len %"PRIu64"" pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode= , const char *opname) "cid %"PRIu16" nsid 0x%"PRIx32" sqid %"PRIu16" opc 0x= %"PRIx8" opname '%s'" pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode, const char= *opname) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8" opname '%s'" --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783378008; cv=none; d=zohomail.com; s=zohoarc; b=nCPA7iO6rhCaILzk3jPA6X2AZSYfhrt82k1aAnWFteZCIgBGQmq0TT2vK9h1bbQ615zA268Y6mvpU9kgTYR5pvpq1tqq/p5oFFDGVHkmG1km6JruWJjupWzC6r0MtS3UTk80WwxqLCUmRtkARl3qRUqlVZ0oAuE/O/TVu9x58go= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783378008; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=hWa71vbrRL3wcudsdmqBSlMdacZJYGlOCvuDe0JhTKU=; b=Fmo1b3EGjhkWB9Zb+h2eE/hcNMGvwvHNoU3HwUPrqtMeb0MAId3A1Gd0Jf2nmX5Hie57psdYckUptC0MMG1Nk4hlCMJaSiG3ktBj/7X1J9TTU8Q4/i68wHo7Jh+Yeq1UT1mfLf+Vd8R113txaN2DOMHyklSnKdbCf7NlZpiZFkY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178337800829257.7574897334099; Mon, 6 Jul 2026 15:46:48 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3u-0002G8-Dn; Mon, 06 Jul 2026 18:45:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3s-0002Dg-4W; Mon, 06 Jul 2026 18:45:12 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3p-0001G7-R0; Mon, 06 Jul 2026 18:45:11 -0400 Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.stl.internal (Postfix) with ESMTP id 80A4E7A01E4; Mon, 6 Jul 2026 18:45:08 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Mon, 06 Jul 2026 18:45:08 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:45:05 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377908; x= 1783464308; bh=hWa71vbrRL3wcudsdmqBSlMdacZJYGlOCvuDe0JhTKU=; b=I wqrcxwDXBM6UUPo8nr+WiJaQnU/j0lEocVyq8Y4+jMlBsZFNvhwsLkgM+/2xzFs+ tMRlaakOqzQ1qbg4Gnv0VdFAWuKx4nyIK1hRgI5fuQenB01Twcef9Z3ZDnbZRXu1 ZM7FRi+CutkiQsGuR0UOvd5aghkWqeKB0s0eEFpBUvHraIrOFgfirVacb5vfXh4z +60p2yCJi+na6jZxYVtmMFHxLlOcj3jzAriZNzqmAC1fL6lbD2Fwn3xX9VMRlszg 1bpDY40xcYt+hWX+PzArQO6zjxsQwyDTl3fWKPLitCmcPekdrdVS+o7m+Y1RRQiW 7ZhndAAQvUG/K0iZYH2TQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377908; x=1783464308; bh=h Wa71vbrRL3wcudsdmqBSlMdacZJYGlOCvuDe0JhTKU=; b=YzKQv6/fnoMseDEzV CSCggPajwCL2xFtxbCg7zkoJpXHuy9CFwTUQee2lpr8xWQ+ouowGmZIVpkSTfNBr qyq63fh1N9IucV1MjutNgz9z2YBHQvxTCz/oxICrLULsylJC8VJZ7XSRUhM+v0FS /uwGuVN3oWL1bc2XjmSvLvklMIEp8YB3DOZLzyNjxzSTm0OxSobhEOd/fjKawB7g Vufd2xGdNZYxdfgS5A6pw+cNifgB9XNGjmqOBXfE10FI8BZlHYr58pOmLrMUM+5h kMH4ZhD++h3JYd0Rq65MYZ0jgq76agLlVU5nwChcXlFu0Msh+DZuaOzVxYhYqyxz KKP4w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTEiFTtHt0YOnx1leXJPo4RVyWTN13fcIYGI1lWZMgNLf7wXqeUOLAXBD2zXA/jeli YD45DRJEjXaHCSv2cmn5kqCWcR9Gjq0W1r/X2s3zPlHhk10cPynvjgZmtm2RiaJdIKj8rU VzuIRGO6ubKn2Mgem/ophdM1cCUkseQvW3gtHqaxdehvx8B/vWC/b8CJudpL5m3VRURM/H XP2PWfzrBh/tPglOTuYCZfD8aZrqiMpTwY16/K/Z5G/jEzVnRE9nsWRW4xDZQZ7WWD+9+R HVgwyU31EScXhARPmO/oxvlZY8ttSvEgMGWOm6yhn/MJjVlYYOcJLTWW+sgMeXaZkQNkn5 xHIbMri/gObAcuLl04SaY11IrhNJj9oYCX4pkYYBUsRyGbUtWfTnw43w0BiVb4vgF6AcFL NxpczC2FBp4qq4CCpijIlhHZIbYBVuWrf+Q/Q8UQzW7qhHjJxbDw7dkaipg8SUr7zovyZG 9DqoPqpmYpLvZKSRR5pV3ux/1OZigsJoYy8AIpanMBv2vtwhx7vTEdOdT3IKbxLmb3UQCJ BJ+TprqB0JsfXSbSN/9wH/lCODMpVWrVAjma2TDX1q30iGYV2SdABQAHrCzkWbuXSn5fsI kOZNU1dHajWmdRG4HXCEHi6Wn6fyH83IKYgimNPwcP83wG26EHMsOoJVD1Pg X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Stefan Hajnoczi , Klaus Jensen , Fabiano Rosas , Paolo Bonzini , Zhao Liu , Peter Xu , Keith Busch , Klaus Jensen , Jesper Devantier , qemu-block@nongnu.org Subject: [PULL 09/11] tests/functional/x86_64: add migration test for NVMe device Date: Tue, 7 Jul 2026 00:44:21 +0200 Message-ID: <20260706224426.14156-10-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.159; envelope-from=its@irrelevant.dk; helo=fhigh-b8-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783378009849158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn Introduce a very simple test to ensure that NVMe device migration works fine. Test plan is simple: 1. prepare VM with NVMe device 2. run workload that produces relatively heavy IO on the device 3. migrate VM 4. ensure that workload is alive and finishes without errors Test can be run as simple as: $ meson test 'func-x86_64-nvme_migration' --setup thorough -C build In the future we can extend this approach, and introduce some fio-based tests. And probably, it makes sense to make this test to apply not only to NVMe device, but also virtio-{blk,scsi}, ide, sata and other migratable devices. Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Acked-by: Klaus Jensen Acked-by: Fabiano Rosas Signed-off-by: Klaus Jensen --- MAINTAINERS | 1 + tests/functional/x86_64/meson.build | 1 + .../functional/x86_64/test_nvme_migration.py | 172 ++++++++++++++++++ 3 files changed, 174 insertions(+) create mode 100755 tests/functional/x86_64/test_nvme_migration.py diff --git a/MAINTAINERS b/MAINTAINERS index 97dcc78ded61..24bc14f8fce6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2680,6 +2680,7 @@ S: Supported F: hw/nvme/* F: include/block/nvme.h F: tests/qtest/nvme-test.c +F: tests/functional/x86_64/test_nvme_migration.py F: docs/system/devices/nvme.rst T: git git://git.infradead.org/qemu-nvme.git nvme-next =20 diff --git a/tests/functional/x86_64/meson.build b/tests/functional/x86_64/= meson.build index 1ed10ad6c295..fd77f19d7265 100644 --- a/tests/functional/x86_64/meson.build +++ b/tests/functional/x86_64/meson.build @@ -37,6 +37,7 @@ tests_x86_64_system_thorough =3D [ 'linux_initrd', 'multiprocess', 'netdev_ethtool', + 'nvme_migration', 'replay', 'reverse_debug', 'tuxrun', diff --git a/tests/functional/x86_64/test_nvme_migration.py b/tests/functio= nal/x86_64/test_nvme_migration.py new file mode 100755 index 000000000000..890f0aab6d6e --- /dev/null +++ b/tests/functional/x86_64/test_nvme_migration.py @@ -0,0 +1,172 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# x86_64 NVMe migration test + +from migration import MigrationTest +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern +from qemu_test import exec_command, exec_command_and_wait_for_pattern + + +class X8664NVMeMigrationTest(MigrationTest): + ASSET_KERNEL =3D Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/rele= ases' + '/31/Server/x86_64/os/images/pxeboot/vmlinuz'), + 'd4738d03dbbe083ca610d0821d0a8f1488bebbdccef54ce33e3adb35fda00129') + + ASSET_INITRD =3D Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/rele= ases' + '/31/Server/x86_64/os/images/pxeboot/initrd.img'), + '277cd6c7adf77c7e63d73bbb2cded8ef9e2d3a2f100000e92ff1f8396513cd8b') + + ASSET_DISKIMAGE =3D Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/rele= ases' + '/31/Cloud/x86_64/images/Fedora-Cloud-Base-31-1.9.x86_64.qcow2'), + 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0') + + DEFAULT_KERNEL_PARAMS =3D ('root=3D/dev/nvme0n1p1 console=3DttyS0 net.= ifnames=3D0 ' + 'rd.rescue quiet') + + def wait_for_console_pattern(self, success_message, vm): + wait_for_console_pattern( + self, + success_message, + failure_message=3D"Kernel panic - not syncing", + vm=3Dvm, + ) + + def exec_command_and_check(self, command, vm): + prompt =3D '# ' + exec_command_and_wait_for_pattern(self, + f"{command} && echo OK || echo FAI= L", + 'FAIL', vm=3Dvm) + # Note, that commands we send to the console are echo-ed back, + # so if we have a word "FAIL" in the command itself, we should + # expect to see it once. + wait_for_console_pattern(self, 'OK', failure_message=3D"FAIL", vm= =3Dvm) + self.wait_for_console_pattern(prompt, vm) + + def configure_machine(self, vm): + kernel_path =3D self.ASSET_KERNEL.fetch() + initrd_path =3D self.ASSET_INITRD.fetch() + diskimage_path =3D self.ASSET_DISKIMAGE.fetch() + + vm.set_console() + vm.add_args("-cpu", "max") + vm.add_args("-m", "2G") + vm.add_args("-accel", "kvm") + + vm.add_args('-drive', + f'file=3D{diskimage_path},if=3Dnone,id=3Ddrv0,sna= pshot=3Don') + vm.add_args('-device', 'nvme,bus=3Dpcie.0,' + + 'drive=3Ddrv0,id=3Dnvme-disk0,serial=3Dnvmemigtest,boo= tindex=3D1') + + vm.add_args( + "-kernel", + kernel_path, + "-initrd", + initrd_path, + "-append", + self.DEFAULT_KERNEL_PARAMS + ) + + def launch_source_vm(self, vm): + vm.launch() + + self.wait_for_console_pattern('Entering emergency mode.', vm) + prompt =3D '# ' + self.wait_for_console_pattern(prompt, vm) + + # Synchronize on NVMe driver creating the root device + exec_command_and_wait_for_pattern(self, + "while ! (dmesg -c | grep nvme0n1:) ; do sleep 1 ;= done", + "nvme0n1", vm=3Dvm) + self.wait_for_console_pattern(prompt, vm) + + # prepare system + exec_command_and_wait_for_pattern(self, 'mount /dev/nvme0n1p1 /sys= root', + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, 'chroot /sysroot', + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, 'mount -t proc proc /proc', + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, 'mount -t sysfs sysfs /sys= ', + prompt, vm=3Dvm) + + # Run workload before migration to check if it continues + # to run properly after migration. + # + # Workload is simple: it continuously calculates checksums of + # all files in /usr/bin to generate some I/O load on + # the NVMe disk and at the same time it drops caches to + # make sure that we have some read I/O on the disk as well. + # If there are any issues with the migration of the NVMe device, + # we should see errors in dmesg and consequently in the workload l= og. + exec_command_and_wait_for_pattern(self, + "(while [ ! -f /tmp/test_nvme_mig_workload.stop ]; do \ + rm -f /tmp/test_nvme_mig_workload.iter_finished; \ + echo 3 > /proc/sys/vm/drop_caches; \ + find /usr/bin -type f -exec cksum {} \\;; \ + touch /tmp/test_nvme_mig_workload.iter_finished; \ + done) > /dev/null 2> /tmp/test_nvme_mig_workload.errors &", + prompt, vm=3Dvm) + exec_command_and_wait_for_pattern(self, + 'echo $! > /tmp/test_nvme_mig_workload.pid', + prompt, vm=3Dvm) + + # check if process is alive and running + self.exec_command_and_check( + "kill -0 $(cat /tmp/test_nvme_mig_workload.pid)", vm) + + def assert_dest_vm(self, vm): + prompt =3D '# ' + + # check if process is alive and running after migration, + # if not - fail the test + self.exec_command_and_check( + "kill -0 $(cat /tmp/test_nvme_mig_workload.pid)", vm) + + # signal workload to stop + exec_command_and_wait_for_pattern(self, + 'touch /tmp/test_nvme_mig_workload.stop', + prompt, vm=3Dvm) + + # wait workload to finish, because we want to examine log + # to see if there are any errors + exec_command_and_wait_for_pattern(self, + "while [ ! -f /tmp/test_nvme_mig_workload.iter_finished ]; do \ + sleep 1; \ + done;", + prompt, vm=3Dvm) + + exec_command_and_wait_for_pattern(self, + 'cat /tmp/test_nvme_mig_workload.errors', + prompt, vm=3Dvm) + + # fail the test if non-empty + self.exec_command_and_check( + "[ ! -s /tmp/test_nvme_mig_workload.errors ]", vm) + + def test_migration_with_tcp_localhost(self): + self.set_machine('q35') + self.require_accelerator("kvm") + + self.migration_with_tcp_localhost() + + def test_migration_with_unix(self): + self.set_machine('q35') + self.require_accelerator("kvm") + + self.migration_with_unix() + + def test_migration_with_exec(self): + self.set_machine('q35') + self.require_accelerator("kvm") + + self.migration_with_exec() + + +if __name__ =3D=3D '__main__': + MigrationTest.main() --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783377944; cv=none; d=zohomail.com; s=zohoarc; b=PpOcJ2RlSBeIgNHIkqU6Y8WUeH+rTLZUvuXVw3III/tNHOv15wu+bejEJg/mPgVSfFRbqnzh/b3Y7euLvOeHS0ZCK9f50iEUUutOsOacbzt9w82oq3pV1jwGx1ySIUHadS7fjvnXG6mCx5LqjJ64Y2QhKpf7DJ7JZOZMobLhZo8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783377944; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dkwpCF19oN1zTVemqGjoa4hHaHi7pYuc15wMZDftZss=; b=jN3u491ZiOso2WL0XEBzqJ7QRvbL1Vmth860hzkF4tMe7+lGCJRndjULxVedJfQ4lHQHXsFAbccpwc5GDPJav0uWeTZgRsiMvlB7moZiJVx4PqUMmASS8/+Z5dSvYIShZ/t+vx/iVbJ10qbP5IuqLJD5LHLWIIjcAExyFH7AfQI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783377944859941.6217783283394; Mon, 6 Jul 2026 15:45:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3v-0002IE-Ne; Mon, 06 Jul 2026 18:45:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3u-0002GB-RN; Mon, 06 Jul 2026 18:45:14 -0400 Received: from fout-b2-smtp.messagingengine.com ([202.12.124.145]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3s-0001IR-BT; Mon, 06 Jul 2026 18:45:14 -0400 Received: from phl-compute-01.internal (phl-compute-01.internal [10.202.2.41]) by mailfout.stl.internal (Postfix) with ESMTP id 037C31D001A0; Mon, 6 Jul 2026 18:45:10 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Mon, 06 Jul 2026 18:45:11 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:45:09 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1783377910; x= 1783464310; bh=dkwpCF19oN1zTVemqGjoa4hHaHi7pYuc15wMZDftZss=; b=n 8ZUaubnnfJEpGxHejFNtgjzzoBT21zd/yPFBKDdlMScFQU9e3yhnBFqmvoL0kdgc 6E+dLOA29CguSZxnOX7W8P04KiLvgWYtwFzpR62KeLV+JtDob+1WeupPcz61z+He hCyic2iR5NiEi1howqUtgVx9I/46aGOgX5X3PgMs0fQYg1xiGJymlZaNbqTd9wFh 8QT2UOXJec8HclBo/j26k5q0Q5WfUa3NsGK8lP96dW4uBbiXzaO7i+lTpf4hTXT+ VaJ+67p3ZOai2Q/OHeR3e+4EIJ+ProCTGdwaHAJ9kf0xwpmF7B+S2hQ5J7rHr+ka 8U0GwOHykzLiJMxRIS3Kg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1783377910; x=1783464310; bh=d kwpCF19oN1zTVemqGjoa4hHaHi7pYuc15wMZDftZss=; b=EKQhzVFxeIwfA+miJ K8w6ObT12EZ8eerFOOuzxpsspcqnjtuaIzsvHmhCEVjtSu8cml3rdvqk15cm+cUV hQ5RJjkg2IQ5o/nb2L7ec1WC4HbHWzY+UdESoyV2jTWt6zyJ5b6Hx8u5e/uBi4Tp ByWzWV7ggmu2Fz/m4SZV9xLOXnpxKzX9YcLA58XiQYZdZfEP2o1fHqHqN2LPLhRq QlkMTKGjYp1WbjkvNATmsukzy6wsQKSqBHdihfhVfW7tQjuL3KqjG2KlK6DMFo7M EEL0HzVzS6/d7Cwx89T3k7KZnZHnDoSTeMybPC9SHuWAFl/kEw+cjR80/vkNeLkz dAj6w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTET5nCFT/MuVvE4xAefAzoQyvBHR+RkHY9ixFObea2t0bJM1174F9XqtsnVozRuyR EWxsKAqJK84tStfgTS5bGvHVkH/oh3XdJEnmZ56oHn7QWgM37UKpg+9zZ153tudqGfKeg6 wN1YwB/QsoP9U8TXcYjJRg3HcQCsv5s/2GBHLYvMj9kvs0pP96jQj1braT3PWbJBSkcJqo nmTEwFzT16ZvkfnECKe+AKNhsekQuE0DS1+zLQkhBWmmM4oTRiAdjt37S4aq6K0PqyREkf ojAVDd+PI7ULxjBfmMqPAGVcBfF2fj0mBU+h/YCRWwQbOcAayGdy0fOLhMAJ8NJN1ZTeTW fJe7GnTlNWaaHYghyAKKITk0613wl8sPHHIVK/hvXbjWn6aH/f4/oU7CCscIYPBqT6658K w3n4ShvQoj8Z7U1uQ4dqAso+Dl0SyLR069cNWI4UUkFmM1+eq1jyQWykbsl9iKPcWatiI9 iShMnAhW5CB+zXlXXqEFLFAL1PR96ynu+DFVcajnp+g39MWYDzbQ6F/4OkDH7EFTqVIMW1 /zDBfDUwSx0AMWS9VuSWCeSAk5kx0PfcnpeyWj+MwJK5A8U7JTtrpY9RnAu6JrNmwwOxYr JVG1nYND7Nc1wJllWvvke9jyMRXGXPLeANSXuWZ83Z8UXttkb4YUuTyLfs0w X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , Alexander Mikhalitsyn , Stefan Hajnoczi , Fabiano Rosas , Klaus Jensen , Keith Busch , Klaus Jensen , Jesper Devantier , Laurent Vivier , Paolo Bonzini , qemu-block@nongnu.org Subject: [PULL 10/11] tests/qtest/nvme-test: add migration test with full CQ Date: Tue, 7 Jul 2026 00:44:22 +0200 Message-ID: <20260706224426.14156-11-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.145; envelope-from=its@irrelevant.dk; helo=fout-b2-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783377948174158500 Content-Type: text/plain; charset="utf-8" From: Alexander Mikhalitsyn As suggested by Stefan [1], let's add a migration test to cover rare scenario when CQ is full of non-processed CQEs and migration happens. To run this test: $ meson test -C build 'qtest-x86_64/qos-test' Link: https://lore.kernel.org/qemu-devel/20260408183529.GB319710@fedora/ [1] Suggested-by: Stefan Hajnoczi Acked-by: Stefan Hajnoczi Signed-off-by: Alexander Mikhalitsyn Acked-by: Fabiano Rosas Signed-off-by: Klaus Jensen --- tests/qtest/nvme-test.c | 419 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 419 insertions(+) diff --git a/tests/qtest/nvme-test.c b/tests/qtest/nvme-test.c index 4aec1651e6e2..f9395cc252b2 100644 --- a/tests/qtest/nvme-test.c +++ b/tests/qtest/nvme-test.c @@ -8,9 +8,12 @@ */ =20 #include "qemu/osdep.h" +#include +#include "qemu/bswap.h" #include "qemu/module.h" #include "qemu/units.h" #include "libqtest.h" +#include "libqtest-single.h" #include "libqos/qgraph.h" #include "libqos/pci.h" #include "block/nvme.h" @@ -142,6 +145,420 @@ static void nvmetest_pmr_reg_test(void *obj, void *da= ta, QGuestAllocator *alloc) qpci_iounmap(pdev, pmr_bar); } =20 +typedef struct nvme_ctrl nvme_ctrl; + +typedef struct nvme_queue { + nvme_ctrl *ctrl; + uint64_t doorbell; + uint32_t size; +} nvme_queue; + +typedef struct nvme_cq { + nvme_queue common; + uint64_t phys_cqe; /* NvmeCqe* */ + uint16_t head; + uint8_t phase; +} nvme_cq; + +typedef struct nvme_sq { + nvme_queue common; + uint64_t phys_sqe; /* NvmeCmd* */ + nvme_cq *cq; + uint16_t head; + uint16_t tail; +} nvme_sq; + +struct nvme_ctrl { + QGuestAllocator *alloc; + QPCIDevice *pdev; + QPCIBar bar; + + uint32_t db_stride; + + nvme_sq admin_sq; + nvme_cq admin_cq; +}; + +#define PHYS_ADDR_OF_FIELD(T, base_phys_addr, field) \ + ((uint64_t)&((T *)(base_phys_addr))->field) + +#define PHYS_ADDR_OF(T, base_phys_addr, accessor) \ + ((uint64_t)&((T *)(base_phys_addr))accessor) + +static void nvme_init_queue_common(nvme_ctrl *ctrl, nvme_queue *q, + uint16_t db_idx, uint32_t size) +{ + q->ctrl =3D ctrl; + q->doorbell =3D (sizeof(NvmeBar) + db_idx * ctrl->db_stride); + g_test_message(" q %p db_idx %u doorbell 0x%" PRIx64, q, db_idx, q->do= orbell); + q->size =3D size; +} + +static void nvme_init_sq(nvme_ctrl *ctrl, nvme_sq *sq, uint16_t db_idx, + uint32_t size, nvme_cq *cq) +{ + nvme_init_queue_common(ctrl, &sq->common, db_idx, size); + + sq->phys_sqe =3D guest_alloc(ctrl->alloc, sizeof(NvmeCmd) * size); + g_assert(sq->phys_sqe); + + g_test_message("sq %p db_idx %u sqe 0x%" PRIx64, sq, db_idx, sq->phys_= sqe); + sq->cq =3D cq; + sq->head =3D 0; + sq->tail =3D 0; +} + +static void nvme_init_cq(nvme_ctrl *ctrl, nvme_cq *cq, uint16_t db_idx, + uint32_t size) +{ + nvme_init_queue_common(ctrl, &cq->common, db_idx, size); + + cq->phys_cqe =3D guest_alloc(ctrl->alloc, sizeof(NvmeCqe) * size); + g_assert(cq->phys_cqe); + + g_test_message("cq %p db_idx %u cqe 0x%" PRIx64, cq, db_idx, cq->phys_= cqe); + cq->head =3D 0; + cq->phase =3D 1; +} + +static int nvme_cqe_pending(nvme_cq *cq) +{ + uint16_t status =3D qtest_readw( + cq->common.ctrl->pdev->bus->qts, + PHYS_ADDR_OF(NvmeCqe, cq->phys_cqe, [cq->head].status)); + return (status & 1) =3D=3D cq->phase; +} + +static int nvme_is_cqe_success(NvmeCqe *cqe) +{ + return (le16_to_cpu(cqe->status) >> 1) =3D=3D NVME_SUCCESS; +} + +static NvmeCqe nvme_handle_cqe(nvme_sq *sq) +{ + nvme_cq *cq =3D sq->cq; + uint64_t phys_cqe =3D PHYS_ADDR_OF( + NvmeCqe, cq->phys_cqe, [cq->head]); /* NvmeCqe= * */ + NvmeCqe cqe; + uint16_t cq_next_head; + + g_assert(nvme_cqe_pending(cq)); + + qtest_memread(sq->common.ctrl->pdev->bus->qts, phys_cqe, &cqe, sizeof(= cqe)); + + cq_next_head =3D (cq->head + 1) % cq->common.size; + g_test_message("cq %p head %u -> %u", cq, cq->head, cq_next_head); + if (cq_next_head < cq->head) { + cq->phase ^=3D 1; + } + cq->head =3D cq_next_head; + + if (cqe.sq_head !=3D sq->head) { + sq->head =3D cqe.sq_head; + g_test_message("sq %p head =3D %u", sq, sq->head); + } + + qpci_io_writel(cq->common.ctrl->pdev, cq->common.ctrl->bar, + cq->common.doorbell, cq->head); + + return cqe; +} + +static NvmeCqe nvme_wait(nvme_sq *sq) +{ + int i; + bool ready =3D false; + + for (i =3D 0; i < 10; i++) { + if (nvme_cqe_pending(sq->cq)) { + ready =3D true; + break; + } + + g_usleep(1000); + } + + g_assert(ready); + + return nvme_handle_cqe(sq); +} + +static uint64_t nvme_get_next_sqe(nvme_sq *sq, uint8_t opcode, + uint16_t cid, uint64_t prp1) +{ + uint64_t phys_sqe =3D PHYS_ADDR_OF(NvmeCmd, sq->phys_sqe, [sq->tail]); + + if (((sq->tail + 1) % sq->common.size) =3D=3D sq->head) { + /* no space in SQ */ + g_test_message("%s head %d tail %d", __func__, sq->head, sq->tail); + g_assert_not_reached(); + return 0; + } + + qtest_memset(sq->common.ctrl->pdev->bus->qts, + phys_sqe, 0, sizeof(NvmeCmd)); + + #define GUEST_MEM_WRITE(fn, phys_addr, val) \ + fn(sq->common.ctrl->pdev->bus->qts, phys_addr, (val)) + + GUEST_MEM_WRITE(qtest_writeb, + PHYS_ADDR_OF_FIELD(NvmeCmd, phys_sqe, opcode), opcode); + GUEST_MEM_WRITE(qtest_writew, + PHYS_ADDR_OF_FIELD(NvmeCmd, phys_sqe, cid), cid); + GUEST_MEM_WRITE(qtest_writeq, + PHYS_ADDR_OF_FIELD(NvmeCmd, phys_sqe, dptr.prp1), prp1= ); + + #undef GUEST_MEM_WRITE + + g_test_message("sq %p next_sqe %u sqe 0x%" PRIx64, sq, sq->tail, phys_= sqe); + return phys_sqe; +} + +static void nvme_commit_sqe(nvme_sq *sq) +{ + g_test_message("sq %p commit sqe tail %u", sq, sq->tail); + sq->tail =3D (sq->tail + 1) % sq->common.size; + qpci_io_writel(sq->common.ctrl->pdev, sq->common.ctrl->bar, + sq->common.doorbell, sq->tail); +} + +static uint64_t nvme_admin_identify_ctrl(nvme_ctrl *ctrl, + uint16_t cid, bool no_wait) +{ + uint64_t phys_cmd_identify; /* NvmeCmd* */ + uint64_t phys_identify; /* NvmeIdCtrl* */ + NvmeCqe cqe; + + g_test_message("sending req cid %u no_wait %d", cid, no_wait); + + phys_identify =3D guest_alloc(ctrl->alloc, sizeof(NvmeIdCtrl)); + g_assert(phys_identify); + + phys_cmd_identify =3D nvme_get_next_sqe(&ctrl->admin_sq, + NVME_ADM_CMD_IDENTIFY, cid, + phys_identify); + g_assert(phys_cmd_identify); + + #define GUEST_MEM_WRITE(fn, phys_addr, val) \ + fn(ctrl->pdev->bus->qts, phys_addr, (val)) + + GUEST_MEM_WRITE(qtest_writel, + PHYS_ADDR_OF_FIELD(NvmeCmd, phys_cmd_identify, nsid), = 0); + GUEST_MEM_WRITE(qtest_writel, + PHYS_ADDR_OF_FIELD(NvmeIdentify, phys_cmd_identify, cn= s), + NVME_ID_CNS_CTRL); + + #undef GUEST_MEM_WRITE + + nvme_commit_sqe(&ctrl->admin_sq); + + if (no_wait) { + return phys_identify; + } + + cqe =3D nvme_wait(&ctrl->admin_sq); + g_assert(nvme_is_cqe_success(&cqe)); + g_assert(le16_to_cpu(cqe.cid) =3D=3D cid); + + return phys_identify; +} + +static void nvme_wait_ready(nvme_ctrl *ctrl, int val) +{ + int i; + + for (i =3D 0; i < 10; i++) { + uint32_t csts =3D qpci_io_readl(ctrl->pdev, ctrl->bar, NVME_REG_CS= TS); + g_test_message("%s: csts %x", __func__, csts); + + if (NVME_CSTS_RDY(csts) =3D=3D val) { + return; + } + + g_usleep(1000); + } + + g_assert_not_reached(); +} + +static void test_migrate_setup_nvme_ctrl(nvme_ctrl *ctrl) +{ + uint64_t cap; + + /* disable controller */ + qpci_io_writel(ctrl->pdev, ctrl->bar, NVME_REG_CC, 0); + nvme_wait_ready(ctrl, 0); + + cap =3D qpci_io_readq(ctrl->pdev, ctrl->bar, NVME_REG_CAP); + ctrl->db_stride =3D 4 << NVME_CAP_DSTRD(cap); + + nvme_init_cq(ctrl, &ctrl->admin_cq, 1, 2 /* CQEs num */); + nvme_init_sq(ctrl, &ctrl->admin_sq, 0, 4 /* SQEs num */, &ctrl->admin_= cq); + + qpci_io_writel(ctrl->pdev, ctrl->bar, NVME_REG_AQA, + ((ctrl->admin_cq.common.size - 1) << AQA_ACQS_SHIFT) | + ((ctrl->admin_sq.common.size - 1) << AQA_ASQS_SHIFT) + ); + + qpci_io_writeq(ctrl->pdev, ctrl->bar, + NVME_REG_ASQ, (uint64_t)ctrl->admin_sq.phys_sqe); + qpci_io_writeq(ctrl->pdev, ctrl->bar, + NVME_REG_ACQ, (uint64_t)ctrl->admin_cq.phys_cqe); + + /* enable controller */ + { + uint32_t cc =3D 0; + NVME_SET_CC_EN(cc, 1); + qpci_io_writel(ctrl->pdev, ctrl->bar, NVME_REG_CC, cc); + } + + nvme_wait_ready(ctrl, 1); +} + +typedef struct test_migrate_req { + uint16_t cid; + bool handle_cqe; + uint64_t phys_identify; /* NvmeIdCtrl* */ +} test_migrate_req; + +static void test_migrate_send_nvme_reqs(nvme_ctrl *ctrl, test_migrate_req = *reqs, + int num) +{ + int i; + + for (i =3D 0; i < num; i++) { + reqs[i].phys_identify =3D nvme_admin_identify_ctrl(ctrl, reqs[i].c= id, + !reqs[i].handle_c= qe); + g_assert(reqs[i].phys_identify); + + if (reqs[i].handle_cqe) { + guest_free(ctrl->alloc, reqs[i].phys_identify); + } + } +} + +static void test_migrate_check_nvme(nvme_ctrl *ctrl, + test_migrate_req *reqs, int num) +{ + int i; + + for (i =3D 0; i < num; i++) { + NvmeCqe cqe; + + if (reqs[i].handle_cqe) { + continue; + } + + cqe =3D nvme_wait(&ctrl->admin_sq); + g_assert(nvme_is_cqe_success(&cqe)); + + g_assert_cmpint(le16_to_cpu(cqe.cid), =3D=3D, reqs[i].cid); + + #define GUEST_MEM_READB(phys_addr) \ + qtest_readb(ctrl->pdev->bus->qts, (phys_addr)) + + g_assert_cmpint(GUEST_MEM_READB( + PHYS_ADDR_OF_FIELD(NvmeIdCtrl, reqs[i].phys_identify, ieee[0])= ), + =3D=3D, 0x0); + g_assert_cmpint(GUEST_MEM_READB( + PHYS_ADDR_OF_FIELD(NvmeIdCtrl, reqs[i].phys_identify, ieee[1])= ), + =3D=3D, 0x54); + g_assert_cmpint(GUEST_MEM_READB( + PHYS_ADDR_OF_FIELD(NvmeIdCtrl, reqs[i].phys_identify, ieee[2])= ), + =3D=3D, 0x52); + + #undef GUEST_MEM_READB + + guest_free(ctrl->alloc, reqs[i].phys_identify); + } +} + +static void test_migrate(void *obj, void *data, QGuestAllocator *alloc) +{ + g_autofree gchar *tmpfs =3D NULL; + GError *err =3D NULL; + g_autofree gchar *mig_path =3D NULL; + g_autofree gchar *uri =3D NULL; + GString *dest_cmdline; + QTestState *to; + QDict *rsp; + QNvme *nvme =3D obj; + QPCIDevice *pdev =3D &nvme->dev; + g_autofree nvme_ctrl *ctrl =3D NULL; + test_migrate_req test_reqs[] =3D { + { 123, true }, + { 456, false }, + { 300, false }, + { 333, false } + }; + + if (qpci_check_buggy_msi(pdev)) { + return; + } + + /* create temporary dir and prepare unix socket path for migration */ + tmpfs =3D g_dir_make_tmp("nvme-test-XXXXXX", &err); + if (!tmpfs) { + g_test_message("Can't create temporary directory in %s: %s", + g_get_tmp_dir(), err->message); + g_error_free(err); + } + g_assert(tmpfs); + + mig_path =3D g_strdup_printf("%s/socket.mig", tmpfs); + uri =3D g_strdup_printf("unix:%s", mig_path); + + /* enable NVMe PCI device */ + qpci_device_enable(pdev); + + ctrl =3D g_malloc0(sizeof(*ctrl)); + ctrl->alloc =3D alloc; + ctrl->pdev =3D pdev; + ctrl->bar =3D qpci_iomap(ctrl->pdev, 0, NULL); + g_assert(pdev->bus->qts =3D=3D global_qtest); + + test_migrate_setup_nvme_ctrl(ctrl); + test_migrate_send_nvme_reqs(ctrl, test_reqs, ARRAY_SIZE(test_reqs)); + + qpci_iounmap(ctrl->pdev, ctrl->bar); + + dest_cmdline =3D g_string_new(qos_get_current_command_line()); + g_string_append_printf(dest_cmdline, " -incoming %s", uri); + + /* Create destination VM */ + to =3D qtest_init(dest_cmdline->str); + + /* Get access to PCI device from destination VM */ + nvme =3D qos_allocate_objects(to, &ctrl->alloc); + pdev =3D &nvme->dev; + ctrl->pdev =3D pdev; + ctrl->bar =3D qpci_iomap(ctrl->pdev, 0, NULL); + g_assert(pdev->bus->qts =3D=3D to); + + /* Migrate VM */ + rsp =3D qmp("{ 'execute': 'migrate', 'arguments': { 'uri': %s } }", ur= i); + g_assert(qdict_haskey(rsp, "return")); + qobject_unref(rsp); + + /* Wait when source VM is stopped */ + qmp_eventwait("STOP"); + + /* Copy guest physical memory allocator state */ + migrate_allocator(alloc, ctrl->alloc); + + /* Wait for destination VM to become alive */ + qtest_qmp_eventwait(to, "RESUME"); + + test_migrate_check_nvme(ctrl, test_reqs, ARRAY_SIZE(test_reqs)); + + qpci_iounmap(ctrl->pdev, ctrl->bar); + + qtest_quit(to); + g_unlink(mig_path); + g_rmdir(tmpfs); + g_string_free(dest_cmdline, true); +} + static void nvme_register_nodes(void) { QOSGraphEdgeOptions opts =3D { @@ -168,6 +585,8 @@ static void nvme_register_nodes(void) }); =20 qos_add_test("reg-read", "nvme", nvmetest_reg_read_test, NULL); + + qos_add_test("migrate", "nvme", test_migrate, NULL); } =20 libqos_init(nvme_register_nodes); --=20 2.53.0 From nobody Sun Jul 26 11:05:44 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1783377969; cv=none; d=zohomail.com; s=zohoarc; b=Q+Ns5aAfTJlN5wKZKUe2hlDsxOY3CvqdsWpq33DOO6vWfjRbJh7Z1JweeEjZ7LfAygCA6Ao6RSW6jenN9ZPMLaZaD7gvLayCwqudmkNfNEKSrvVlevIw2tT8ZJYQU1uMSCfiSkOoA6+LnCMwI/gVlBmtqxmx1WfudYuT1BrLu6o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1783377969; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qL09FDFC4S8bD4BCt3Gw8FmiytCd9pm/PII4UutUsRA=; b=JZcWGzaSok42EvUgBrG3XOiUFKnDnCFEusu809pLB3GCEesxdBOrQIFRUozQJyiXAneOrBsMFk8v3ZqupnWKKoisLnfYD4XjIt6/WPoC+TVCPwDmE9cMvn8xNLnMA3zivQbqxqTSRs9gjUoxp/axPM/iNxuOBRvyx+OxTjtApMQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1783377969619663.2292461844977; Mon, 6 Jul 2026 15:46:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wgs3y-0002Ln-Pc; Mon, 06 Jul 2026 18:45:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3w-0002J9-OU; Mon, 06 Jul 2026 18:45:16 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wgs3u-0001M6-P5; Mon, 06 Jul 2026 18:45:16 -0400 Received: from phl-compute-05.internal (phl-compute-05.internal [10.202.2.45]) by mailfhigh.stl.internal (Postfix) with ESMTP id 6D7157A01DD; Mon, 6 Jul 2026 18:45:13 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Mon, 06 Jul 2026 18:45:13 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 6 Jul 2026 18:45:11 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm1; t=1783377913; x=1783464313; bh=qL09FDFC4S8bD4BCt3Gw8FmiytCd9pm/ PII4UutUsRA=; b=Gx4xnGXin7DThpvDfKMkak7QTn8OVN/nEV+WJe6Y7JLpoANa QcwVOGBR45XVVIaRWj+94h9O7SRD//vOeC7LZXCX0rUUHr8sJy0X9szs0zf34aba mP3nTSHeVxxv46RvOU3GoM8GK4mdn55QkHthHhDbdKczov9foyZIXIAFBdPtJ8zR MFBCLVYf73wA8gisSiEtC8y2/bqQ2zZBWsrpRt+HpOPv9N/VONCyuRNVLP1nMjnC dNQLPznWTF11ysyWo9uZIMDJ2UnnNDXVpKsJeYcMFnkzGdj2ILXWdMEJv37p2j7R ieO3dX8tMz/mECLXulk5GxHqAzdNhnqjR7CRGw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1783377913; x= 1783464313; bh=qL09FDFC4S8bD4BCt3Gw8FmiytCd9pm/PII4UutUsRA=; b=K etj4/p6sJ9ECz8eXHPGaJKdueNgCvWJp14qXqkjXHVvl8YSVb0bcFk2xpz+W9LRn KRBhBGI0G4XD9GFVdCPiRFCSX235t7Dp0iRHN8Z9K9KuYJ+5H2ytmsT3dOSICg0M S/c6nxbeqHsrmZLwIOU7OzY2CF9bVSJxESIpnpozQ9GGkvyvrjjiFnMUI3Sk6gH1 tzqrsr0l/99kFa+VVDicVoz+QXpUPUjqEK6FqPv4lUHRH/bNMR15N/zyiO0FZt3u p0jz0qhAmVGxBYCzqtc4aR5FNG1ieoMUVkv9j4ceESkzrBuDi9+/jOoHBi2iSZlk kJw+jQrh+nr/ew0ahuUSw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTFB4Z9PN2gccqOQjjy99tDjdmrkGuGJ4bO0W0ziegtQyHR2uJh95CeZYSnvCtmel7 knQ8BsVUQHaVS4ufmchfCIWK2cmGgh3P8fMGdRFBdVZLDClEBjRBqFu3E1S0x7vizfNsUg cx9CJKSADgLKVf3c0u5NxJEIEJTwlv4XFDNoE8t/zlA26XQUNkWpGCFvVUNUZ05goD7zpX 9j6hGDlOBBgO77WH4VJzhogwIT3i80YnbdQWtedJentL5fTUCSf7yfysxyeIysp0Fa439n W/fWR9Hh4He4FbCnoboeF6+kKjmYzoGy+fFf1xgJxncBs1PYl3CXL5ZWFzimjbgJmQxBSS x561W7VWBtG/9/GDsY5xjI8HweAdThpr8u6cNW2iISqh0nd5Eewo8amsnIByboIRVReMJK BUD5R5gDkIz+yjEtDoPU3kfLDGi90EyZ7pM94gOWbVUoTH9CBxrAVx2tWKHWPij+oq1Szh s4K9JzClxBdbM4nxmvjLvgfnC5DXrqgvtWjyYHOsKE9uho+MnSuuaDWhUO7pim1BZ6SYdP Ef9jaEUXeeIqfcyACiMxNyQ5PY8X7g00Kk6K/TOB+eqyyfpFWV2i0pE+z7BagkKELn26SY X2NyiaC3+OghUv+RcsvypUZoSrvvu57pJkmHvSZIzCJgRNWn/4vFPUVWPGaw X-ME-Proxy: Feedback-ID: idc91472f:Fastmail From: Klaus Jensen To: qemu-devel@nongnu.org Cc: Peter Maydell , mr-083 , Matthieu , Stefan Hajnoczi , Klaus Jensen , Keith Busch , Klaus Jensen , Jesper Devantier , qemu-block@nongnu.org Subject: [PULL 11/11] hw/nvme: add namespace hotplug support Date: Tue, 7 Jul 2026 00:44:23 +0200 Message-ID: <20260706224426.14156-12-its@irrelevant.dk> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260706224426.14156-1-its@irrelevant.dk> References: <20260706224426.14156-1-its@irrelevant.dk> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=202.12.124.159; envelope-from=its@irrelevant.dk; helo=fhigh-b8-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @irrelevant.dk) X-ZM-MESSAGEID: 1783377971852158500 From: mr-083 Add hotplug support for nvme-ns devices on the NvmeBus. This enables NVMe namespace-level hot-add and hot-remove via device_add and device_del with proper Asynchronous Event Notification (AEN), so the guest kernel can react to namespace topology changes. Mark nvme-ns devices as hotpluggable and register the NvmeBus as a hotplug handler with proper plug and unplug callbacks: - plug: attach namespace to all started controllers and send an Asynchronous Event Notification (AEN) with NS_ATTR_CHANGED so the guest kernel rescans namespaces and adds the block device - unplug: drain in-flight I/O, detach from all controllers, send AEN, then unrealize the device. The guest kernel rescans and removes the block device. The plug handler skips controllers that haven't started yet (qs_created =3D=3D false) to avoid interfering with boot-time namespace attachment in nvme_start_ctrl(). The unplug handler drains in-flight I/O via nvme_ns_drain() before detaching the namespace from controllers, so pending requests can complete normally without touching freed state. For symmetry with nvme_ns_realize() which sets subsys->namespaces[nsid], nvme_ns_unrealize() now clears that slot too making the namespace lifecycle complete. Both the controller bus and subsystem bus are configured as hotplug handlers via qbus_set_bus_hotplug_handler() since nvme-ns devices may reparent to the subsystem bus during realize. Example hot-swap sequence using the NVMe subsystem model: # Boot with: -device nvme-subsys,id=3Dsubsys0 # -device nvme,id=3Dctrl0,subsys=3Dsubsys0 # -device nvme-ns,id=3Dns0,drive=3Ddrv0,bus=3Dctrl0,nsid=3D1 device_del ns0 # guest receives AEN, removes /dev/nvme0n1 drive_del drv0 drive_add 0 file=3Ddisk.qcow2,format=3Dqcow2,id=3Ddrv0,if=3Dnone device_add nvme-ns,id=3Dns0,drive=3Ddrv0,bus=3Dctrl0,nsid=3D1 # guest receives AEN, adds /dev/nvme0n1 Tested with Linux 6.1 guest (NVMe driver processes AEN and rescans namespace list automatically). Signed-off-by: Matthieu Reviewed-by: Stefan Hajnoczi Reviewed-by: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++ hw/nvme/ns.c | 8 +++++ hw/nvme/subsys.c | 2 ++ 3 files changed, 97 insertions(+) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 1fa3a24f7635..a67e1598891c 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -9634,6 +9634,7 @@ static void nvme_realize(PCIDevice *pci_dev, Error **= errp) } =20 qbus_init(&n->bus, sizeof(NvmeBus), TYPE_NVME_BUS, dev, dev->id); + qbus_set_bus_hotplug_handler(BUS(&n->bus)); =20 if (nvme_init_subsys(n, errp)) { return; @@ -10571,10 +10572,96 @@ static const TypeInfo nvme_info =3D { }, }; =20 +static void nvme_ns_hot_plug(HotplugHandler *hotplug_dev, DeviceState *dev, + Error **errp) +{ + NvmeNamespace *ns =3D NVME_NS(dev); + NvmeSubsystem *subsys =3D ns->subsys; + uint32_t nsid =3D ns->params.nsid; + int i; + + /* + * Attach to all started controllers and notify via AEN. + * Skip controllers that haven't started yet (boot-time realize) =E2= =80=94 + * nvme_start_ctrl() will attach namespaces during controller init. + */ + for (i =3D 0; i < NVME_MAX_CONTROLLERS; i++) { + NvmeCtrl *ctrl =3D nvme_subsys_ctrl(subsys, i); + if (!ctrl || !ctrl->qs_created) { + continue; + } + + if (nvme_csi_supported(ctrl, ns->csi) && !ns->params.detached) { + nvme_attach_ns(ctrl, ns); + nvme_update_dsm_limits(ctrl, ns); + + if (!test_and_set_bit(nsid, ctrl->changed_nsids)) { + nvme_enqueue_event(ctrl, NVME_AER_TYPE_NOTICE, + NVME_AER_INFO_NOTICE_NS_ATTR_CHANGED, + NVME_LOG_CHANGED_NSLIST); + } + } + } +} + +static void nvme_ns_hot_unplug(HotplugHandler *hotplug_dev, DeviceState *d= ev, + Error **errp) +{ + NvmeNamespace *ns =3D NVME_NS(dev); + NvmeSubsystem *subsys =3D ns->subsys; + uint32_t nsid =3D ns->params.nsid; + int i; + + /* + * Drain in-flight I/O before tearing down the namespace. + * This must happen while the namespace is still attached to the + * controllers so any pending requests can complete normally. + */ + nvme_ns_drain(ns); + + /* + * Detach from all controllers and notify the guest via AEN. + * The guest kernel will rescan namespaces and remove the block device. + */ + for (i =3D 0; i < NVME_MAX_CONTROLLERS; i++) { + NvmeCtrl *ctrl =3D nvme_subsys_ctrl(subsys, i); + if (!ctrl || !nvme_ns(ctrl, nsid)) { + continue; + } + + nvme_detach_ns(ctrl, ns); + nvme_update_dsm_limits(ctrl, NULL); + + if (!test_and_set_bit(nsid, ctrl->changed_nsids)) { + nvme_enqueue_event(ctrl, NVME_AER_TYPE_NOTICE, + NVME_AER_INFO_NOTICE_NS_ATTR_CHANGED, + NVME_LOG_CHANGED_NSLIST); + } + } + + /* + * Unrealize: removes from subsystem (in nvme_ns_unrealize), flushes, + * cleans up structures, and removes from QOM. + */ + qdev_unrealize(dev); +} + +static void nvme_bus_class_init(ObjectClass *klass, const void *data) +{ + HotplugHandlerClass *hc =3D HOTPLUG_HANDLER_CLASS(klass); + hc->plug =3D nvme_ns_hot_plug; + hc->unplug =3D nvme_ns_hot_unplug; +} + static const TypeInfo nvme_bus_info =3D { .name =3D TYPE_NVME_BUS, .parent =3D TYPE_BUS, .instance_size =3D sizeof(NvmeBus), + .class_init =3D nvme_bus_class_init, + .interfaces =3D (const InterfaceInfo[]) { + { TYPE_HOTPLUG_HANDLER }, + { } + }, }; =20 static void nvme_register_types(void) diff --git a/hw/nvme/ns.c b/hw/nvme/ns.c index 4caab590977f..7f0f9ac7662c 100644 --- a/hw/nvme/ns.c +++ b/hw/nvme/ns.c @@ -720,10 +720,17 @@ void nvme_ns_cleanup(NvmeNamespace *ns) static void nvme_ns_unrealize(DeviceState *dev) { NvmeNamespace *ns =3D NVME_NS(dev); + NvmeSubsystem *subsys =3D ns->subsys; + uint32_t nsid =3D ns->params.nsid; =20 nvme_ns_drain(ns); nvme_ns_shutdown(ns); nvme_ns_cleanup(ns); + + /* Symmetric with nvme_ns_realize() which sets subsys->namespaces[nsid= ]. */ + if (subsys && nsid && subsys->namespaces[nsid] =3D=3D ns) { + subsys->namespaces[nsid] =3D NULL; + } } =20 void nvme_ns_atomic_configure_boundary(bool dn, uint16_t nabsn, @@ -1100,6 +1107,7 @@ static void nvme_ns_class_init(ObjectClass *oc, const= void *data) dc->bus_type =3D TYPE_NVME_BUS; dc->realize =3D nvme_ns_realize; dc->unrealize =3D nvme_ns_unrealize; + dc->hotpluggable =3D true; dc->vmsd =3D &nvme_vmstate_ns; device_class_set_props(dc, nvme_ns_props); dc->desc =3D "Virtual NVMe namespace"; diff --git a/hw/nvme/subsys.c b/hw/nvme/subsys.c index 777e1c620fd0..fa35055d3c1e 100644 --- a/hw/nvme/subsys.c +++ b/hw/nvme/subsys.c @@ -9,6 +9,7 @@ #include "qemu/osdep.h" #include "qemu/units.h" #include "qapi/error.h" +#include "hw/core/qdev.h" =20 #include "nvme.h" =20 @@ -205,6 +206,7 @@ static void nvme_subsys_realize(DeviceState *dev, Error= **errp) NvmeSubsystem *subsys =3D NVME_SUBSYS(dev); =20 qbus_init(&subsys->bus, sizeof(NvmeBus), TYPE_NVME_BUS, dev, dev->id); + qbus_set_bus_hotplug_handler(BUS(&subsys->bus)); =20 nvme_subsys_setup(subsys, errp); } --=20 2.53.0