From nobody Tue Feb 10 02:55:20 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail header.i=@wdc.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=wdc.com ARC-Seal: i=1; a=rsa-sha256; t=1604025590; cv=none; d=zohomail.com; s=zohoarc; b=NiiT5p4WMw/sZSLRIvgaHaTNTLP5sfeMYLxWIs4MWvBHSno++Qd34a9iS9KJHbS91/lNMsK3jbhJOfr2XcTRTf6s8asknQL4dSOwyahKyOeugDArCQz7pJZZHLLppIWV3AXXbqBi73XSQqiJORpJJjhrIRzDqTVHpsENEprEe9c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604025590; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UqFHvtsK7axlUlV0pkFzvnBWvZBAya1vaHMg06GODHM=; b=RMuxn33ZtgojWyHLXXuqpKZyci/aOBy61HiixPsv3vgBldTbvxRTApfUU8z0gWnKvFPOLde6LGniIxPx3KTWRJy05ciMVUgzG7dgpZQBuxkx2zMuDKeyhv8QYiJZM1z0ume0qtwbgt1rxFo6BYGkkJedWbvU8zweOrTfN2KMWRY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail header.i=@wdc.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1604025590809177.48471010226615; Thu, 29 Oct 2020 19:39:50 -0700 (PDT) Received: from localhost ([::1]:38362 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kYKKU-0006qJ-0i for importer@patchew.org; Thu, 29 Oct 2020 22:39:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kYKE3-0006Ed-B6; Thu, 29 Oct 2020 22:33:11 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:10998) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kYKDz-0006dn-N9; Thu, 29 Oct 2020 22:33:11 -0400 Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 30 Oct 2020 10:33:06 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2020 19:18:10 -0700 Received: from unknown (HELO redsun50.ssa.fujisawa.hgst.com) ([10.149.66.24]) by uls-op-cesaip01.wdc.com with ESMTP; 29 Oct 2020 19:33:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1604025187; x=1635561187; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f2jhFid1Eer6mVN1Fs9bkQWEt7kkLC8suyNdE1o+BzM=; b=qJ4glgUdQGErde79OjvwexmJbFXkHuE1nQhEYiNeaElv3KySO6wzBGw7 bCOgm/t60ODIL93ycuvYpFrgJw8sKTN481wlnuUzducl4bS5l5RjDoyhx UuvA9ARRsdUFcrGF4a1bM9zbo0jbAbU2BrvkbdoC4FLUh45/4gEjL2Xt5 zn9/VhmyTi5MDhM8cYYYG72g3Xv8QmYFPSQ3pSVd6I2WjizzgYcUMqkBy X8Q7EXYIQI4Q7NOJRgIv21R0g5WW58g41bZIuOesEWDAWCU+wMGL1L28U C8qz/1x1R3iPsNZAqkpUvjy+nR/y4xMTbAek7BdKxGbzDCsCjo5iuNaX2 g==; IronPort-SDR: I3n/DO+TFHOOqvwucmDNzlpq7lbuOBB6v984hgdd7kQ98MjCt9gmXiNmHMVgI5Pli4pqYqN7Dz UBxRp2ET2E3FpGk7uaiENM8xvWnUUZRN/yvhjrQg1Fym2MeqvLTMq2GYBDZYZIdoFjNP8o29gR qTmd6Quy6Kx2YG6Ya9E7RCzG2BfJx09ro6+RJRGhFzkDI4xg8MA7qX7BYvb9JMDecqJSdg7SV3 z+VdS5AfdrvcgnhIfN7hUvPJadnHFAOSvu3W/+4bGgB3DiFnBsx/GB9mCsVPtflF1aEz+9aSHO 7ks= X-IronPort-AV: E=Sophos;i="5.77,431,1596470400"; d="scan'208";a="155748089" IronPort-SDR: qllZtJ0XUDcwWjQ72OGZT9U+FLUQWhqHoTdArQCO4ZT3Dx6KVVqntYERjq2PUGiAByOSurZWr7 Lq/7G+ZOlqCfV0C/t7WepinmlLdvSIhrxKf1Q4bM4iYjiwPyEjQBRSkFx7LbqREoq7guoVQjHk RsqY+HEiq3z6lXqQ0GV8pbIeRTzMiXCu6Eg4ayi+XwqUviqwiV5NPtfOEuR6cHNmjXPu+EKF9y Hdf8zFt86UutNIlA+gJ8MxnfObxqkB0ZwA2n2XVsw4898AxEKxizOQ72/p5oit8ViyRSDKUNXI iaf2tIKCOTF7qp5c6GhZ0uVY IronPort-SDR: oV4aGsJOaeuKtRHjQzWN9jtXcgT7dHUOKkVbxGVJITyCKrkzUHjRX8zpWSIBYxPGereu6PhPsg evecH0VL/Mi/i9Joq4TReFMtCBrFTzwxfRqriFYZW7cLduGvatdlf+QuhOrODdg/qD1it4RDeR KJvoHj3WgRek7IgG96jJK9g/ndE4hkUWwoDVZYqyOfl/lpxym588Plba58VQbSW2GUyAOiBi0+ 4DlMm0Jeu4Isr50tMHPrAOnDDalmsFvYFeL12Wsl9LLtayBDJNZE/t/xYBx+nzs/xpkIkuEMeU XLY= WDCIronportException: Internal From: Dmitry Fomichev To: Keith Busch , Klaus Jensen , Kevin Wolf , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Maxim Levitsky , Fam Zheng Subject: [PATCH v8 08/11] hw/block/nvme: Introduce max active and open zone limits Date: Fri, 30 Oct 2020 11:32:39 +0900 Message-Id: <20201030023242.5204-9-dmitry.fomichev@wdc.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20201030023242.5204-1-dmitry.fomichev@wdc.com> References: <20201030023242.5204-1-dmitry.fomichev@wdc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.71.153.141; envelope-from=prvs=56530b5a8=dmitry.fomichev@wdc.com; helo=esa3.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/29 22:32:49 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Niklas Cassel , Damien Le Moal , qemu-block@nongnu.org, Dmitry Fomichev , qemu-devel@nongnu.org, Alistair Francis , Matias Bjorling Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Add two module properties, "zoned.max_active" and "zoned.max_open" to control the maximum number of zones that can be active or open. Once these variables are set to non-default values, these limits are checked during I/O and Too Many Active or Too Many Open command status is returned if they are exceeded. Signed-off-by: Hans Holmberg Signed-off-by: Dmitry Fomichev Reviewed-by: Niklas Cassel --- hw/block/nvme-ns.c | 30 +++++++++++++- hw/block/nvme-ns.h | 41 +++++++++++++++++++ hw/block/nvme.c | 94 +++++++++++++++++++++++++++++++++++++++++++ hw/block/trace-events | 2 + 4 files changed, 165 insertions(+), 2 deletions(-) diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c index e6db7f7d3b..2e45838c15 100644 --- a/hw/block/nvme-ns.c +++ b/hw/block/nvme-ns.c @@ -119,6 +119,20 @@ static int nvme_calc_zone_geometry(NvmeNamespace *ns, = Error **errp) ns->zone_size_log2 =3D 63 - clz64(ns->zone_size); } =20 + /* Make sure that the values of all ZNS properties are sane */ + if (ns->params.max_open_zones > nz) { + error_setg(errp, + "max_open_zones value %u exceeds the number of zones %u= ", + ns->params.max_open_zones, nz); + return -1; + } + if (ns->params.max_active_zones > nz) { + error_setg(errp, + "max_active_zones value %u exceeds the number of zones = %u", + ns->params.max_active_zones, nz); + return -1; + } + return 0; } =20 @@ -166,8 +180,8 @@ static int nvme_zoned_init_ns(NvmeCtrl *n, NvmeNamespac= e *ns, int lba_index, id_ns_z =3D g_malloc0(sizeof(NvmeIdNsZoned)); =20 /* MAR/MOR are zeroes-based, 0xffffffff means no limit */ - id_ns_z->mar =3D 0xffffffff; - id_ns_z->mor =3D 0xffffffff; + id_ns_z->mar =3D cpu_to_le32(ns->params.max_active_zones - 1); + id_ns_z->mor =3D cpu_to_le32(ns->params.max_open_zones - 1); id_ns_z->zoc =3D 0; id_ns_z->ozcs =3D ns->params.cross_zone_read ? 0x01 : 0x00; =20 @@ -195,6 +209,7 @@ static void nvme_clear_zone(NvmeNamespace *ns, NvmeZone= *zone) trace_pci_nvme_clear_ns_close(state, zone->d.zslba); nvme_set_zone_state(zone, NVME_ZONE_STATE_CLOSED); } + nvme_aor_inc_active(ns); QTAILQ_INSERT_HEAD(&ns->closed_zones, zone, entry); } else { trace_pci_nvme_clear_ns_reset(state, zone->d.zslba); @@ -211,16 +226,23 @@ static void nvme_zoned_ns_shutdown(NvmeNamespace *ns) =20 QTAILQ_FOREACH_SAFE(zone, &ns->closed_zones, entry, next) { QTAILQ_REMOVE(&ns->closed_zones, zone, entry); + nvme_aor_dec_active(ns); nvme_clear_zone(ns, zone); } QTAILQ_FOREACH_SAFE(zone, &ns->imp_open_zones, entry, next) { QTAILQ_REMOVE(&ns->imp_open_zones, zone, entry); + nvme_aor_dec_open(ns); + nvme_aor_dec_active(ns); nvme_clear_zone(ns, zone); } QTAILQ_FOREACH_SAFE(zone, &ns->exp_open_zones, entry, next) { QTAILQ_REMOVE(&ns->exp_open_zones, zone, entry); + nvme_aor_dec_open(ns); + nvme_aor_dec_active(ns); nvme_clear_zone(ns, zone); } + + assert(ns->nr_open_zones =3D=3D 0); } =20 static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp) @@ -306,6 +328,10 @@ static Property nvme_ns_props[] =3D { DEFINE_PROP_SIZE("zoned.zcap", NvmeNamespace, params.zone_cap_bs, 0), DEFINE_PROP_BOOL("zoned.cross_read", NvmeNamespace, params.cross_zone_read, false), + DEFINE_PROP_UINT32("zoned.max_active", NvmeNamespace, + params.max_active_zones, 0), + DEFINE_PROP_UINT32("zoned.max_open", NvmeNamespace, + params.max_open_zones, 0), DEFINE_PROP_END_OF_LIST(), }; =20 diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h index d2631ff5a3..421bab0a57 100644 --- a/hw/block/nvme-ns.h +++ b/hw/block/nvme-ns.h @@ -33,6 +33,8 @@ typedef struct NvmeNamespaceParams { bool cross_zone_read; uint64_t zone_size_bs; uint64_t zone_cap_bs; + uint32_t max_active_zones; + uint32_t max_open_zones; } NvmeNamespaceParams; =20 typedef struct NvmeNamespace { @@ -56,6 +58,8 @@ typedef struct NvmeNamespace { uint64_t zone_capacity; uint64_t zone_array_size; uint32_t zone_size_log2; + int32_t nr_open_zones; + int32_t nr_active_zones; =20 NvmeNamespaceParams params; } NvmeNamespace; @@ -123,6 +127,43 @@ static inline bool nvme_wp_is_valid(NvmeZone *zone) st !=3D NVME_ZONE_STATE_OFFLINE; } =20 +static inline void nvme_aor_inc_open(NvmeNamespace *ns) +{ + assert(ns->nr_open_zones >=3D 0); + if (ns->params.max_open_zones) { + ns->nr_open_zones++; + assert(ns->nr_open_zones <=3D ns->params.max_open_zones); + } +} + +static inline void nvme_aor_dec_open(NvmeNamespace *ns) +{ + if (ns->params.max_open_zones) { + assert(ns->nr_open_zones > 0); + ns->nr_open_zones--; + } + assert(ns->nr_open_zones >=3D 0); +} + +static inline void nvme_aor_inc_active(NvmeNamespace *ns) +{ + assert(ns->nr_active_zones >=3D 0); + if (ns->params.max_active_zones) { + ns->nr_active_zones++; + assert(ns->nr_active_zones <=3D ns->params.max_active_zones); + } +} + +static inline void nvme_aor_dec_active(NvmeNamespace *ns) +{ + if (ns->params.max_active_zones) { + assert(ns->nr_active_zones > 0); + ns->nr_active_zones--; + assert(ns->nr_active_zones >=3D ns->nr_open_zones); + } + assert(ns->nr_active_zones >=3D 0); +} + int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error **errp); void nvme_ns_drain(NvmeNamespace *ns); void nvme_ns_flush(NvmeNamespace *ns); diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 0326e1611c..485ac6fc40 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -199,6 +199,26 @@ static void nvme_assign_zone_state(NvmeNamespace *ns, = NvmeZone *zone, } } =20 +/* + * Check if we can open a zone without exceeding open/active limits. + * AOR stands for "Active and Open Resources" (see TP 4053 section 2.5). + */ +static int nvme_aor_check(NvmeNamespace *ns, uint32_t act, uint32_t opn) +{ + if (ns->params.max_active_zones !=3D 0 && + ns->nr_active_zones + act > ns->params.max_active_zones) { + trace_pci_nvme_err_insuff_active_res(ns->params.max_active_zones); + return NVME_ZONE_TOO_MANY_ACTIVE | NVME_DNR; + } + if (ns->params.max_open_zones !=3D 0 && + ns->nr_open_zones + opn > ns->params.max_open_zones) { + trace_pci_nvme_err_insuff_open_res(ns->params.max_open_zones); + return NVME_ZONE_TOO_MANY_OPEN | NVME_DNR; + } + + return NVME_SUCCESS; +} + static bool nvme_addr_is_cmb(NvmeCtrl *n, hwaddr addr) { hwaddr low =3D n->ctrl_mem.addr; @@ -1203,6 +1223,41 @@ static uint16_t nvme_check_zone_read(NvmeNamespace *= ns, uint64_t slba, return status; } =20 +static void nvme_auto_transition_zone(NvmeNamespace *ns, bool implicit, + bool adding_active) +{ + NvmeZone *zone; + + if (implicit && ns->params.max_open_zones && + ns->nr_open_zones =3D=3D ns->params.max_open_zones) { + zone =3D QTAILQ_FIRST(&ns->imp_open_zones); + if (zone) { + /* + * Automatically close this implicitly open zone. + */ + QTAILQ_REMOVE(&ns->imp_open_zones, zone, entry); + nvme_aor_dec_open(ns); + nvme_assign_zone_state(ns, zone, NVME_ZONE_STATE_CLOSED); + } + } +} + +static uint16_t nvme_auto_open_zone(NvmeNamespace *ns, NvmeZone *zone) +{ + uint16_t status =3D NVME_SUCCESS; + uint8_t zs =3D nvme_get_zone_state(zone); + + if (zs =3D=3D NVME_ZONE_STATE_EMPTY) { + nvme_auto_transition_zone(ns, true, true); + status =3D nvme_aor_check(ns, 1, 1); + } else if (zs =3D=3D NVME_ZONE_STATE_CLOSED) { + nvme_auto_transition_zone(ns, true, false); + status =3D nvme_aor_check(ns, 0, 1); + } + + return status; +} + static bool nvme_finalize_zoned_write(NvmeNamespace *ns, NvmeRequest *req, bool failed) { @@ -1235,7 +1290,11 @@ static bool nvme_finalize_zoned_write(NvmeNamespace = *ns, NvmeRequest *req, switch (nvme_get_zone_state(zone)) { case NVME_ZONE_STATE_IMPLICITLY_OPEN: case NVME_ZONE_STATE_EXPLICITLY_OPEN: + nvme_aor_dec_open(ns); + /* fall through */ case NVME_ZONE_STATE_CLOSED: + nvme_aor_dec_active(ns); + /* fall through */ case NVME_ZONE_STATE_EMPTY: nvme_assign_zone_state(ns, zone, NVME_ZONE_STATE_FULL); /* fall through */ @@ -1264,7 +1323,10 @@ static uint64_t nvme_advance_zone_wp(NvmeNamespace *= ns, NvmeZone *zone, zs =3D nvme_get_zone_state(zone); switch (zs) { case NVME_ZONE_STATE_EMPTY: + nvme_aor_inc_active(ns); + /* fall through */ case NVME_ZONE_STATE_CLOSED: + nvme_aor_inc_open(ns); nvme_assign_zone_state(ns, zone, NVME_ZONE_STATE_IMPLICITLY_OP= EN); } } @@ -1457,6 +1519,11 @@ static uint16_t nvme_write(NvmeCtrl *n, NvmeRequest = *req, bool append, bool wrz) goto invalid; } =20 + status =3D nvme_auto_open_zone(ns, zone); + if (status !=3D NVME_SUCCESS) { + goto invalid; + } + if (append) { slba =3D zone->w_ptr; } @@ -1537,9 +1604,27 @@ enum NvmeZoneProcessingMask { static uint16_t nvme_open_zone(NvmeNamespace *ns, NvmeZone *zone, uint8_t state) { + uint16_t status; + switch (state) { case NVME_ZONE_STATE_EMPTY: + nvme_auto_transition_zone(ns, false, true); + status =3D nvme_aor_check(ns, 1, 0); + if (status !=3D NVME_SUCCESS) { + return status; + } + nvme_aor_inc_active(ns); + /* fall through */ case NVME_ZONE_STATE_CLOSED: + status =3D nvme_aor_check(ns, 0, 1); + if (status !=3D NVME_SUCCESS) { + if (state =3D=3D NVME_ZONE_STATE_EMPTY) { + nvme_aor_dec_active(ns); + } + return status; + } + nvme_aor_inc_open(ns); + /* fall through */ case NVME_ZONE_STATE_IMPLICITLY_OPEN: nvme_assign_zone_state(ns, zone, NVME_ZONE_STATE_EXPLICITLY_OPEN); /* fall through */ @@ -1556,6 +1641,7 @@ static uint16_t nvme_close_zone(NvmeNamespace *ns, Nv= meZone *zone, switch (state) { case NVME_ZONE_STATE_EXPLICITLY_OPEN: case NVME_ZONE_STATE_IMPLICITLY_OPEN: + nvme_aor_dec_open(ns); nvme_assign_zone_state(ns, zone, NVME_ZONE_STATE_CLOSED); /* fall through */ case NVME_ZONE_STATE_CLOSED: @@ -1571,7 +1657,11 @@ static uint16_t nvme_finish_zone(NvmeNamespace *ns, = NvmeZone *zone, switch (state) { case NVME_ZONE_STATE_EXPLICITLY_OPEN: case NVME_ZONE_STATE_IMPLICITLY_OPEN: + nvme_aor_dec_open(ns); + /* fall through */ case NVME_ZONE_STATE_CLOSED: + nvme_aor_dec_active(ns); + /* fall through */ case NVME_ZONE_STATE_EMPTY: zone->w_ptr =3D nvme_zone_wr_boundary(zone); zone->d.wp =3D zone->w_ptr; @@ -1590,7 +1680,11 @@ static uint16_t nvme_reset_zone(NvmeNamespace *ns, N= vmeZone *zone, switch (state) { case NVME_ZONE_STATE_EXPLICITLY_OPEN: case NVME_ZONE_STATE_IMPLICITLY_OPEN: + nvme_aor_dec_open(ns); + /* fall through */ case NVME_ZONE_STATE_CLOSED: + nvme_aor_dec_active(ns); + /* fall through */ case NVME_ZONE_STATE_FULL: zone->w_ptr =3D zone->d.zslba; zone->d.wp =3D zone->w_ptr; diff --git a/hw/block/trace-events b/hw/block/trace-events index 8f221ba73c..91d2d1a1c2 100644 --- a/hw/block/trace-events +++ b/hw/block/trace-events @@ -124,6 +124,8 @@ pci_nvme_err_append_not_at_start(uint64_t slba, uint64_= t zone) "appending at slb pci_nvme_err_zone_write_not_ok(uint64_t slba, uint32_t nlb, uint32_t statu= s) "slba=3D%"PRIu64", nlb=3D%"PRIu32", status=3D0x%"PRIx16"" pci_nvme_err_zone_read_not_ok(uint64_t slba, uint32_t nlb, uint32_t status= ) "slba=3D%"PRIu64", nlb=3D%"PRIu32", status=3D0x%"PRIx16"" pci_nvme_err_append_too_large(uint64_t slba, uint32_t nlb, uint8_t zasl) "= slba=3D%"PRIu64", nlb=3D%"PRIu32", zasl=3D%"PRIu8"" +pci_nvme_err_insuff_active_res(uint32_t max_active) "max_active=3D%"PRIu32= " zone limit exceeded" +pci_nvme_err_insuff_open_res(uint32_t max_open) "max_open=3D%"PRIu32" zone= limit exceeded" pci_nvme_err_invalid_iocsci(uint32_t idx) "unsupported command set combina= tion index %"PRIu32"" pci_nvme_err_invalid_del_sq(uint16_t qid) "invalid submission queue deleti= on, sid=3D%"PRIu16"" pci_nvme_err_invalid_create_sq_cqid(uint16_t cqid) "failed creating submis= sion queue, invalid cqid=3D%"PRIu16"" --=20 2.21.0