From nobody Mon Nov 25 03:37:29 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=samsung.com ARC-Seal: i=1; a=rsa-sha256; t=1718275329; cv=none; d=zohomail.com; s=zohoarc; b=jXEWlGclgVfyHiZO12dleE5AZ3j7eZcKsmpqkqusB2Mk5n4CipwjT2cRnQ82L6psLIJ4qgPisO4F97VogALvublgjem6zOrvLfaC7qCVts56o+hh6qLM8M3YrwJKNMfbKKtpYdX1KhnafsUd7qt0yT2CUFur2puucHSj4ZPT6c4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1718275329; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+MDpyHSoj1eq3DBtYblTsTNr5Yjvtpnjv6AIrthL3/0=; b=CozcjePFkJW/5S5a6U8aNh2lVHSGy/oaq9Eev5/yjeBxvri7oUWQ5PWkn268U3kqIWwGZSVSbh0G7cXw1ZnnnmTAetUQWs1W3cCuP5+WLu3FAQwYgic8oZGr7nLS7NRoIIji/CbdZ2doFAzQa4wKykpz4oZedfLvjers6jSmfnE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1718275329357794.3785595567499; Thu, 13 Jun 2024 03:42:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sHhty-0003W7-3P; Thu, 13 Jun 2024 06:41:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sHhtv-0003Tg-W9 for qemu-devel@nongnu.org; Thu, 13 Jun 2024 06:41:52 -0400 Received: from mailout1.samsung.com ([203.254.224.24]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sHhtn-0003Yp-GX for qemu-devel@nongnu.org; Thu, 13 Jun 2024 06:41:51 -0400 Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20240613104135epoutp01758e488fb6f86a4f8bf51ab32c8c7a9f~YikcOUB4E3249632496epoutp011 for ; Thu, 13 Jun 2024 10:41:35 +0000 (GMT) Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20240613104134epcas5p43ae4593f929b58efeeeb4fbf352c7be2~YikbcvIsA2364123641epcas5p4x; Thu, 13 Jun 2024 10:41:34 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.174]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4W0Jqd3kwLz4x9Pv; Thu, 13 Jun 2024 10:41:33 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 75.B0.09989.4DCCA666; Thu, 13 Jun 2024 19:41:24 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20240613103252epcas5p164032f94c4e84343e88703ec8877383e~Yic1YqeFr1831918319epcas5p1D; Thu, 13 Jun 2024 10:32:52 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240613103252epsmtrp26f830f5a0aba8fcf248d3630d8ef241d~Yic1XKzoE1440314403epsmtrp2s; Thu, 13 Jun 2024 10:32:52 +0000 (GMT) Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 63.D7.08622.4DACA666; Thu, 13 Jun 2024 19:32:52 +0900 (KST) Received: from localhost.sa.corp.samsungelectronics.net (unknown [107.99.41.223]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20240613103251epsmtip1fd9666f4603053a62db792cf7e0f0a70~Yic0ZfPYy1554315543epsmtip1e; Thu, 13 Jun 2024 10:32:51 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20240613104135epoutp01758e488fb6f86a4f8bf51ab32c8c7a9f~YikcOUB4E3249632496epoutp011 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1718275295; bh=+MDpyHSoj1eq3DBtYblTsTNr5Yjvtpnjv6AIrthL3/0=; h=From:To:Cc:Subject:Date:References:From; b=JgRCcjnEncrK+miFpMDi9tcdwX1ni+AH/PSNcT4zm4Zn/Dcfh8tKLfPJACdWfWhuD BlB+y36hLuHGtKn+xqNbO13QjthqQPURfuYxuAKksiBlquiKOKOJ84Bgzo6FPjOqyS 0lMK0xb7FvLGRNOUVJxBIAvPKlz5//uNqF71TDSI= X-AuditID: b6c32a4a-e57f970000002705-6a-666accd4e8b6 From: Arun Kumar To: qemu-devel@nongnu.org Cc: kbusch@kernel.org, its@irrelevant.dk, foss@defmacro.it, Arun Kumar Subject: [PATCH] adding corss namespace copy support (tp-4130) Date: Thu, 13 Jun 2024 10:33:54 +0530 Message-ID: <20240613050354.716451-1-arun.kka@samsung.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrOKsWRmVeSWpSXmKPExsWy7bCmhu6VM1lpBs+Xi1jMuW9hcWJKmMX+ g99YLSYdusZocbx3B4sDq8eUadfYPc7tOM/usWlVJ5vHk2ubmTz6tqxiDGCNyrbJSE1MSS1S SM1Lzk/JzEu3VfIOjneONzUzMNQ1tLQwV1LIS8xNtVVy8QnQdcvMAdqtpFCWmFMKFApILC5W 0rezKcovLUlVyMgvLrFVSi1IySkwKdArTswtLs1L18tLLbEyNDAwMgUqTMjOOPRTteDBbcaK Uxu/sDcw/pnI2MXIySEhYCJx89pH5i5GLg4hgd2MEtuef2SFcD4xSvx4ewLK+cYo0XX+IFAL B1jLnUdQ8b2MEhMX/mQCGSUk0MYk8ficEIjNJqAusXbmDrC4iICkxO+u08wgNrNAvMS3RUtZ QGxhATuJT69/MYPMZBFQlZh4hAMkzCtgJfH43Xd2iOvkJRbvWM4MEReUODnzCQvEGHmJ5q2z wa6WEDjELrFu8RYmiAYXiQn9R5khbGGJV8e3QA2Skvj8bi8bhF0vcWbmHqj6DkaJpzDL7CUu 7vnLBHIPs4CmxPpd+hBhWYmpp9YxQezlk+j9/QSqlVdixzwYW0ni3tafUGMkJOZc+Q4NXQ+J HTf72CDBEytx7/ZjlgmM8rOQvDMLyTuzEDYvYGRexSiZWlCcm55abFpglJdaDo/X5PzcTYzg FKjltYPx4YMPeocYmTgYDzFKcDArifDOWpiVJsSbklhZlVqUH19UmpNafIjRFBjEE5mlRJPz gUk4ryTe0MTSwMTMzMzE0tjMUEmc93Xr3BQhgfTEktTs1NSC1CKYPiYOTqkGppSz/xwflUTs c95z+n/rKwXeY3rzrST5lvtPDlyg/MVnk+xhrvpVVhkeZX/+FCRb9wrd+bRzemiBCP+vlp57 ZTd82/aetkiQTFm7+eqNtV9ZFq+YXBHOnx84RSbtBYfp3bmrg2ufqPW1zy6P7ZtZwJ4fPNun aAP/paTvuaY38zbLLasXS798tD1VYqVIjP+nUC41BolnmismX9fPMuOS3bz1QmEYv2vMxfAn cZFvjqtu/T8t6JnGxSVq7GkX522+9SJ3Qz+384OcFQc2FAnfuBQXFdrE9+FacnRq0eafKveq D5qJHYnNENRgCd3NvtPi0RaOawuLdj9ssjvDVSDSkiGVKRm2e0d7UDJ/qVWxEktxRqKhFnNR cSIADJ91RQoEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrELMWRmVeSWpSXmKPExsWy7bCSnO6VU1lpBr93alnMuW9hcWJKmMX+ g99YLSYdusZocbx3B4sDq8eUadfYPc7tOM/usWlVJ5vHk2ubmTz6tqxiDGCN4rJJSc3JLEst 0rdL4Mo49FO14MFtxopTG7+wNzD+mcjYxcjBISFgInHnEWsXIxeHkMBuRokt5zaxdzFyAsUl JH58+cYIYQtLrPz3nB2iqIVJYumtpawgCTYBdYm1M3cwgdgiApISv7tOM4MMZRZIlnj9wxck LCxgJ/Hp9S+wMIuAqsTEIxwgYV4BK4nH775DrZKXWLxjOTNEXFDi5MwnLCA2M1C8eets5gmM fLOQpGYhSS1gZFrFKJlaUJybnltsWGCUl1quV5yYW1yal66XnJ+7iREcjFpaOxj3rPqgd4iR iYPxEKMEB7OSCO+shVlpQrwpiZVVqUX58UWlOanFhxilOViUxHm/ve5NERJITyxJzU5NLUgt gskycXBKNTDFr/7cHb38wWshP5N+XYZJx/9KPzr8eUmQ6YNoDiad6IBlc6XC/7e7HTr9a5d7 zk+p+Pd1Fl5XZ67bp+n8ULVdLEdtof/RVVwf7vtMe6ui2xP16MviHpb3GtzXj228+Hp/yWcj /ddn23lfXymuT9YWanWL+fphWsUxxtI36UpTmpJ+1F59/zT3LYvSRvZl9SUHjO1j5mrfCHU5 WsbOdOzAjlktsyrjJj66tzvgpKSxWbJdnpRNXtkui30Rs5VXezLW+LI5syZ0S+an6whVvvmx 5NzxGHk7g+MXK385yGfb7DSfXJX1NunR88C9ai+aduqKPbn8sfDK1a/Cc5TCM3xr8hjmlGoH tSbsvJljtVqJpTgj0VCLuag4EQDrnRKYtQIAAA== X-CMS-MailID: 20240613103252epcas5p164032f94c4e84343e88703ec8877383e X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240613103252epcas5p164032f94c4e84343e88703ec8877383e References: Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=203.254.224.24; envelope-from=arun.kka@samsung.com; helo=mailout1.samsung.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_03_06=1.592, DKIMWL_WL_HIGH=-0.145, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @samsung.com) X-ZM-MESSAGEID: 1718275331716100002 Signed-off-by: Arun Kumar --- hw/nvme/ctrl.c | 355 ++++++++++++++++++++++++++++++++----------- include/block/nvme.h | 37 +++-- 2 files changed, 289 insertions(+), 103 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 127c3d2383..c181caae38 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -2696,6 +2696,7 @@ typedef struct NvmeCopyAIOCB { BlockAIOCB common; BlockAIOCB *aiocb; NvmeRequest *req; + NvmeCtrl *n; int ret; =20 void *ranges; @@ -2714,6 +2715,8 @@ typedef struct NvmeCopyAIOCB { uint64_t slba; =20 NvmeZone *zone; + NvmeNamespace *sns; + uint32_t tcl; } NvmeCopyAIOCB; =20 static void nvme_copy_cancel(BlockAIOCB *aiocb) @@ -2760,13 +2763,19 @@ static void nvme_copy_done(NvmeCopyAIOCB *iocb) =20 static void nvme_do_copy(NvmeCopyAIOCB *iocb); =20 -static void nvme_copy_source_range_parse_format0(void *ranges, int idx, - uint64_t *slba, uint32_t = *nlb, - uint16_t *apptag, - uint16_t *appmask, - uint64_t *reftag) +static void nvme_copy_source_range_parse_format0_2(void *ranges, + int idx, uint64_t *slba, + uint32_t *nlb, + uint32_t *snsid, + uint16_t *apptag, + uint16_t *appmask, + uint64_t *reftag) { - NvmeCopySourceRangeFormat0 *_ranges =3D ranges; + NvmeCopySourceRangeFormat0_2 *_ranges =3D ranges; + + if (snsid) { + *snsid =3D le32_to_cpu(_ranges[idx].sparams); + } =20 if (slba) { *slba =3D le64_to_cpu(_ranges[idx].slba); @@ -2789,13 +2798,19 @@ static void nvme_copy_source_range_parse_format0(vo= id *ranges, int idx, } } =20 -static void nvme_copy_source_range_parse_format1(void *ranges, int idx, - uint64_t *slba, uint32_t = *nlb, - uint16_t *apptag, - uint16_t *appmask, - uint64_t *reftag) +static void nvme_copy_source_range_parse_format1_3(void *ranges, int idx, + uint64_t *slba, + uint32_t *nlb, + uint32_t *snsid, + uint16_t *apptag, + uint16_t *appmask, + uint64_t *reftag) { - NvmeCopySourceRangeFormat1 *_ranges =3D ranges; + NvmeCopySourceRangeFormat1_3 *_ranges =3D ranges; + + if (snsid) { + *snsid =3D le32_to_cpu(_ranges[idx].sparams); + } =20 if (slba) { *slba =3D le64_to_cpu(_ranges[idx].slba); @@ -2827,18 +2842,20 @@ static void nvme_copy_source_range_parse_format1(vo= id *ranges, int idx, =20 static void nvme_copy_source_range_parse(void *ranges, int idx, uint8_t fo= rmat, uint64_t *slba, uint32_t *nlb, - uint16_t *apptag, uint16_t *appma= sk, - uint64_t *reftag) + uint32_t *snsid, uint16_t *apptag, + uint16_t *appmask, uint64_t *reft= ag) { switch (format) { case NVME_COPY_FORMAT_0: - nvme_copy_source_range_parse_format0(ranges, idx, slba, nlb, appta= g, - appmask, reftag); + case NVME_COPY_FORMAT_2: + nvme_copy_source_range_parse_format0_2(ranges, idx, slba, nlb, sns= id, + apptag, appmask, reftag); break; =20 case NVME_COPY_FORMAT_1: - nvme_copy_source_range_parse_format1(ranges, idx, slba, nlb, appta= g, - appmask, reftag); + case NVME_COPY_FORMAT_3: + nvme_copy_source_range_parse_format1_3(ranges, idx, slba, nlb, sns= id, + apptag, appmask, reftag); break; =20 default: @@ -2854,10 +2871,10 @@ static inline uint16_t nvme_check_copy_mcl(NvmeName= space *ns, for (int idx =3D 0; idx < nr; idx++) { uint32_t nlb; nvme_copy_source_range_parse(iocb->ranges, idx, iocb->format, NULL, - &nlb, NULL, NULL, NULL); + &nlb, NULL, NULL, NULL, NULL); copy_len +=3D nlb; } - + iocb->tcl =3D copy_len; if (copy_len > ns->id_ns.mcl) { return NVME_CMD_SIZE_LIMIT | NVME_DNR; } @@ -2869,11 +2886,11 @@ static void nvme_copy_out_completed_cb(void *opaque= , int ret) { NvmeCopyAIOCB *iocb =3D opaque; NvmeRequest *req =3D iocb->req; - NvmeNamespace *ns =3D req->ns; + NvmeNamespace *dns =3D req->ns; uint32_t nlb; =20 nvme_copy_source_range_parse(iocb->ranges, iocb->idx, iocb->format, NU= LL, - &nlb, NULL, NULL, NULL); + &nlb, NULL, NULL, NULL, NULL); =20 if (ret < 0) { iocb->ret =3D ret; @@ -2882,8 +2899,8 @@ static void nvme_copy_out_completed_cb(void *opaque, = int ret) goto out; } =20 - if (ns->params.zoned) { - nvme_advance_zone_wp(ns, iocb->zone, nlb); + if (dns->params.zoned) { + nvme_advance_zone_wp(dns, iocb->zone, nlb); } =20 iocb->idx++; @@ -2896,25 +2913,25 @@ static void nvme_copy_out_cb(void *opaque, int ret) { NvmeCopyAIOCB *iocb =3D opaque; NvmeRequest *req =3D iocb->req; - NvmeNamespace *ns =3D req->ns; + NvmeNamespace *dns =3D req->ns; uint32_t nlb; size_t mlen; uint8_t *mbounce; =20 - if (ret < 0 || iocb->ret < 0 || !ns->lbaf.ms) { + if (ret < 0 || iocb->ret < 0 || !dns->lbaf.ms) { goto out; } =20 nvme_copy_source_range_parse(iocb->ranges, iocb->idx, iocb->format, NU= LL, - &nlb, NULL, NULL, NULL); + &nlb, NULL, NULL, NULL, NULL); =20 - mlen =3D nvme_m2b(ns, nlb); - mbounce =3D iocb->bounce + nvme_l2b(ns, nlb); + mlen =3D nvme_m2b(dns, nlb); + mbounce =3D iocb->bounce + nvme_l2b(dns, nlb); =20 qemu_iovec_reset(&iocb->iov); qemu_iovec_add(&iocb->iov, mbounce, mlen); =20 - iocb->aiocb =3D blk_aio_pwritev(ns->blkconf.blk, nvme_moff(ns, iocb->s= lba), + iocb->aiocb =3D blk_aio_pwritev(dns->blkconf.blk, nvme_moff(dns, iocb-= >slba), &iocb->iov, 0, nvme_copy_out_completed_c= b, iocb); =20 @@ -2928,12 +2945,15 @@ static void nvme_copy_in_completed_cb(void *opaque,= int ret) { NvmeCopyAIOCB *iocb =3D opaque; NvmeRequest *req =3D iocb->req; - NvmeNamespace *ns =3D req->ns; + NvmeNamespace *sns =3D iocb->sns; + NvmeNamespace *dns =3D req->ns; + NvmeCopyCmd *copy =3D NULL; + uint8_t *mbounce =3D NULL; uint32_t nlb; uint64_t slba; uint16_t apptag, appmask; uint64_t reftag; - size_t len; + size_t len, mlen; uint16_t status; =20 if (ret < 0) { @@ -2944,43 +2964,51 @@ static void nvme_copy_in_completed_cb(void *opaque,= int ret) } =20 nvme_copy_source_range_parse(iocb->ranges, iocb->idx, iocb->format, &s= lba, - &nlb, &apptag, &appmask, &reftag); - len =3D nvme_l2b(ns, nlb); + &nlb, NULL, &apptag, &appmask, &reftag); =20 trace_pci_nvme_copy_out(iocb->slba, nlb); =20 - if (NVME_ID_NS_DPS_TYPE(ns->id_ns.dps)) { - NvmeCopyCmd *copy =3D (NvmeCopyCmd *)&req->cmd; + len =3D nvme_l2b(sns, nlb); + + if (NVME_ID_NS_DPS_TYPE(sns->id_ns.dps)) { + copy =3D (NvmeCopyCmd *)&req->cmd; =20 uint16_t prinfor =3D ((copy->control[0] >> 4) & 0xf); - uint16_t prinfow =3D ((copy->control[2] >> 2) & 0xf); =20 - size_t mlen =3D nvme_m2b(ns, nlb); - uint8_t *mbounce =3D iocb->bounce + nvme_l2b(ns, nlb); + mlen =3D nvme_m2b(sns, nlb); + mbounce =3D iocb->bounce + nvme_l2b(sns, nlb); =20 - status =3D nvme_dif_mangle_mdata(ns, mbounce, mlen, slba); + status =3D nvme_dif_mangle_mdata(sns, mbounce, mlen, slba); if (status) { goto invalid; } - status =3D nvme_dif_check(ns, iocb->bounce, len, mbounce, mlen, pr= infor, + status =3D nvme_dif_check(sns, iocb->bounce, len, mbounce, mlen, p= rinfor, slba, apptag, appmask, &reftag); if (status) { goto invalid; } + } + + if (NVME_ID_NS_DPS_TYPE(dns->id_ns.dps)) { + copy =3D (NvmeCopyCmd *)&req->cmd; + uint16_t prinfow =3D ((copy->control[2] >> 2) & 0xf); + + mlen =3D nvme_m2b(dns, nlb); + mbounce =3D iocb->bounce + nvme_l2b(dns, nlb); =20 apptag =3D le16_to_cpu(copy->apptag); appmask =3D le16_to_cpu(copy->appmask); =20 if (prinfow & NVME_PRINFO_PRACT) { - status =3D nvme_check_prinfo(ns, prinfow, iocb->slba, iocb->re= ftag); + status =3D nvme_check_prinfo(dns, prinfow, iocb->slba, iocb->r= eftag); if (status) { goto invalid; } =20 - nvme_dif_pract_generate_dif(ns, iocb->bounce, len, mbounce, ml= en, + nvme_dif_pract_generate_dif(dns, iocb->bounce, len, mbounce, m= len, apptag, &iocb->reftag); } else { - status =3D nvme_dif_check(ns, iocb->bounce, len, mbounce, mlen, + status =3D nvme_dif_check(dns, iocb->bounce, len, mbounce, mle= n, prinfow, iocb->slba, apptag, appmask, &iocb->reftag); if (status) { @@ -2989,13 +3017,13 @@ static void nvme_copy_in_completed_cb(void *opaque,= int ret) } } =20 - status =3D nvme_check_bounds(ns, iocb->slba, nlb); + status =3D nvme_check_bounds(dns, iocb->slba, nlb); if (status) { goto invalid; } =20 - if (ns->params.zoned) { - status =3D nvme_check_zone_write(ns, iocb->zone, iocb->slba, nlb); + if (dns->params.zoned) { + status =3D nvme_check_zone_write(dns, iocb->zone, iocb->slba, nlb); if (status) { goto invalid; } @@ -3008,7 +3036,10 @@ static void nvme_copy_in_completed_cb(void *opaque, = int ret) qemu_iovec_reset(&iocb->iov); qemu_iovec_add(&iocb->iov, iocb->bounce, len); =20 - iocb->aiocb =3D blk_aio_pwritev(ns->blkconf.blk, nvme_l2b(ns, iocb->sl= ba), + block_acct_start(blk_get_stats(dns->blkconf.blk), &iocb->acct.write, 0, + BLOCK_ACCT_WRITE); + + iocb->aiocb =3D blk_aio_pwritev(dns->blkconf.blk, nvme_l2b(dns, iocb->= slba), &iocb->iov, 0, nvme_copy_out_cb, iocb); =20 return; @@ -3023,23 +3054,22 @@ out: static void nvme_copy_in_cb(void *opaque, int ret) { NvmeCopyAIOCB *iocb =3D opaque; - NvmeRequest *req =3D iocb->req; - NvmeNamespace *ns =3D req->ns; + NvmeNamespace *sns =3D iocb->sns; uint64_t slba; uint32_t nlb; =20 - if (ret < 0 || iocb->ret < 0 || !ns->lbaf.ms) { + if (ret < 0 || iocb->ret < 0 || !sns->lbaf.ms) { goto out; } =20 nvme_copy_source_range_parse(iocb->ranges, iocb->idx, iocb->format, &s= lba, - &nlb, NULL, NULL, NULL); + &nlb, NULL, NULL, NULL, NULL); =20 qemu_iovec_reset(&iocb->iov); - qemu_iovec_add(&iocb->iov, iocb->bounce + nvme_l2b(ns, nlb), - nvme_m2b(ns, nlb)); + qemu_iovec_add(&iocb->iov, iocb->bounce + nvme_l2b(sns, nlb), + nvme_m2b(sns, nlb)); =20 - iocb->aiocb =3D blk_aio_preadv(ns->blkconf.blk, nvme_moff(ns, slba), + iocb->aiocb =3D blk_aio_preadv(sns->blkconf.blk, nvme_moff(sns, slba), &iocb->iov, 0, nvme_copy_in_completed_cb, iocb); return; @@ -3048,14 +3078,78 @@ out: nvme_copy_in_completed_cb(iocb, ret); } =20 +static inline bool nvme_csi_supports_copy(uint8_t csi) +{ + return csi =3D=3D NVME_CSI_NVM || csi =3D=3D NVME_CSI_ZONED; +} + +static inline bool nvme_copy_ns_format_match(NvmeNamespace *sns, + NvmeNamespace *dns) +{ + return sns->lbaf.ds =3D=3D dns->lbaf.ds && sns->lbaf.ms =3D=3D dns->lb= af.ms; +} + +static bool nvme_copy_matching_ns_format(NvmeNamespace *sns, NvmeNamespace= *dns, + bool pi_enable) +{ + if (!nvme_csi_supports_copy(sns->csi) || + !nvme_csi_supports_copy(dns->csi)) { + return false; + } + + if (!pi_enable && !nvme_copy_ns_format_match(sns, dns)) { + return false; + } + + if (pi_enable && (!nvme_copy_ns_format_match(sns, dns) || + sns->id_ns.dps !=3D dns->id_ns.dps)) { + return false; + } + + return true; +} + +static inline bool nvme_copy_corresp_pi_match(NvmeNamespace *sns, + NvmeNamespace *dns) +{ + return sns->lbaf.ms =3D=3D 0 && + ((dns->lbaf.ms =3D=3D 8 && dns->pif =3D=3D 0) || + (dns->lbaf.ms =3D=3D 16 && dns->pif =3D=3D 1)); +} + +static bool nvme_copy_corresp_pi_format(NvmeNamespace *sns, NvmeNamespace = *dns, + bool sns_pi_en) +{ + if (!nvme_csi_supports_copy(sns->csi) || + !nvme_csi_supports_copy(dns->csi)) { + return false; + } + + if (!sns_pi_en && !nvme_copy_corresp_pi_match(sns, dns)) { + return false; + } + + if (sns_pi_en && !nvme_copy_corresp_pi_match(dns, sns)) { + return false; + } + + return true; +} + static void nvme_do_copy(NvmeCopyAIOCB *iocb) { NvmeRequest *req =3D iocb->req; - NvmeNamespace *ns =3D req->ns; + NvmeNamespace *sns; + NvmeNamespace *dns =3D req->ns; + NvmeCopyCmd *copy =3D (NvmeCopyCmd *)&req->cmd; + uint16_t prinfor =3D ((copy->control[0] >> 4) & 0xf); + uint16_t prinfow =3D ((copy->control[2] >> 2) & 0xf); uint64_t slba; uint32_t nlb; size_t len; uint16_t status; + uint32_t dnsid =3D le32_to_cpu(req->cmd.nsid); + uint32_t snsid =3D dnsid; =20 if (iocb->ret < 0) { goto done; @@ -3065,40 +3159,124 @@ static void nvme_do_copy(NvmeCopyAIOCB *iocb) goto done; } =20 - nvme_copy_source_range_parse(iocb->ranges, iocb->idx, iocb->format, &s= lba, - &nlb, NULL, NULL, NULL); - len =3D nvme_l2b(ns, nlb); + if (iocb->format =3D=3D 2 || iocb->format =3D=3D 3) { + nvme_copy_source_range_parse(iocb->ranges, iocb->idx, iocb->format, + &slba, &nlb, &snsid, NULL, NULL, NULL= ); + if (snsid !=3D dnsid) { + if (snsid =3D=3D NVME_NSID_BROADCAST || + !nvme_nsid_valid(iocb->n, snsid)) { + status =3D NVME_INVALID_NSID | NVME_DNR; + goto invalid; + } + iocb->sns =3D nvme_ns(iocb->n, snsid); + if (unlikely(!iocb->sns)) { + status =3D NVME_INVALID_FIELD | NVME_DNR; + goto invalid; + } + } else { + if (((slba + nlb) > iocb->slba) && + ((slba + nlb) < (iocb->slba + iocb->tcl))) { + status =3D NVME_CMD_OVERLAP_IO_RANGE | NVME_DNR; + goto invalid; + } + } + } else if (iocb->format =3D=3D 0 || iocb->format =3D=3D 1) { + nvme_copy_source_range_parse(iocb->ranges, iocb->idx, iocb->format, + &slba, &nlb, NULL, NULL, NULL, NULL); + } + + sns =3D iocb->sns; + if ((snsid =3D=3D dnsid) && NVME_ID_NS_DPS_TYPE(sns->id_ns.dps) && + ((prinfor & NVME_PRINFO_PRACT) !=3D (prinfow & NVME_PRINFO_PRACT))= ) { + status =3D NVME_INVALID_FIELD | NVME_DNR; + goto invalid; + } else if (snsid !=3D dnsid) { + if (!NVME_ID_NS_DPS_TYPE(sns->id_ns.dps) && + !NVME_ID_NS_DPS_TYPE(dns->id_ns.dps)) { + if (!nvme_copy_matching_ns_format(sns, dns, false)) { + status =3D NVME_CMD_INCOMP_NS_OR_FMT | NVME_DNR; + goto invalid; + } + } + if (NVME_ID_NS_DPS_TYPE(sns->id_ns.dps) && + NVME_ID_NS_DPS_TYPE(dns->id_ns.dps)) { + if ((prinfor & NVME_PRINFO_PRACT) !=3D + (prinfow & NVME_PRINFO_PRACT)) { + status =3D NVME_CMD_INCOMP_NS_OR_FMT | NVME_DNR; + goto invalid; + } else { + if (!nvme_copy_matching_ns_format(sns, dns, true)) { + status =3D NVME_CMD_INCOMP_NS_OR_FMT | NVME_DNR; + goto invalid; + } + } + } + + if (!NVME_ID_NS_DPS_TYPE(sns->id_ns.dps) && + NVME_ID_NS_DPS_TYPE(dns->id_ns.dps)) { + if (!(prinfow & NVME_PRINFO_PRACT)) { + status =3D NVME_CMD_INCOMP_NS_OR_FMT | NVME_DNR; + goto invalid; + } else { + if (!nvme_copy_corresp_pi_format(sns, dns, false)) { + status =3D NVME_CMD_INCOMP_NS_OR_FMT | NVME_DNR; + goto invalid; + } + } + } + + if (NVME_ID_NS_DPS_TYPE(sns->id_ns.dps) && + !NVME_ID_NS_DPS_TYPE(dns->id_ns.dps)) { + if (!(prinfor & NVME_PRINFO_PRACT)) { + status =3D NVME_CMD_INCOMP_NS_OR_FMT | NVME_DNR; + goto invalid; + } else { + if (!nvme_copy_corresp_pi_format(sns, dns, true)) { + status =3D NVME_CMD_INCOMP_NS_OR_FMT | NVME_DNR; + goto invalid; + } + } + } + } + len =3D nvme_l2b(sns, nlb); =20 trace_pci_nvme_copy_source_range(slba, nlb); =20 - if (nlb > le16_to_cpu(ns->id_ns.mssrl)) { + if (nlb > le16_to_cpu(sns->id_ns.mssrl)) { status =3D NVME_CMD_SIZE_LIMIT | NVME_DNR; goto invalid; } =20 - status =3D nvme_check_bounds(ns, slba, nlb); + status =3D nvme_check_bounds(sns, slba, nlb); if (status) { goto invalid; } =20 - if (NVME_ERR_REC_DULBE(ns->features.err_rec)) { - status =3D nvme_check_dulbe(ns, slba, nlb); + if (NVME_ERR_REC_DULBE(sns->features.err_rec)) { + status =3D nvme_check_dulbe(sns, slba, nlb); if (status) { goto invalid; } } =20 - if (ns->params.zoned) { - status =3D nvme_check_zone_read(ns, slba, nlb); + if (sns->params.zoned) { + status =3D nvme_check_zone_read(sns, slba, nlb); if (status) { goto invalid; } } =20 + g_free(iocb->bounce); + iocb->bounce =3D g_malloc_n(le16_to_cpu(sns->id_ns.mssrl), + sns->lbasz + sns->lbaf.ms); + qemu_iovec_reset(&iocb->iov); qemu_iovec_add(&iocb->iov, iocb->bounce, len); =20 - iocb->aiocb =3D blk_aio_preadv(ns->blkconf.blk, nvme_l2b(ns, slba), + block_acct_start(blk_get_stats(sns->blkconf.blk), &iocb->acct.read, 0, + BLOCK_ACCT_READ); + + iocb->aiocb =3D blk_aio_preadv(sns->blkconf.blk, nvme_l2b(sns, slba), &iocb->iov, 0, nvme_copy_in_cb, iocb); return; =20 @@ -3117,9 +3295,7 @@ static uint16_t nvme_copy(NvmeCtrl *n, NvmeRequest *r= eq) nvme_misc_cb, req); uint16_t nr =3D copy->nr + 1; uint8_t format =3D copy->control[0] & 0xf; - uint16_t prinfor =3D ((copy->control[0] >> 4) & 0xf); - uint16_t prinfow =3D ((copy->control[2] >> 2) & 0xf); - size_t len =3D sizeof(NvmeCopySourceRangeFormat0); + size_t len =3D sizeof(NvmeCopySourceRangeFormat0_2); =20 uint16_t status; =20 @@ -3128,13 +3304,9 @@ static uint16_t nvme_copy(NvmeCtrl *n, NvmeRequest *= req) iocb->ranges =3D NULL; iocb->zone =3D NULL; =20 - if (NVME_ID_NS_DPS_TYPE(ns->id_ns.dps) && - ((prinfor & NVME_PRINFO_PRACT) !=3D (prinfow & NVME_PRINFO_PRACT))= ) { - status =3D NVME_INVALID_FIELD | NVME_DNR; - goto invalid; - } - - if (!(n->id_ctrl.ocfs & (1 << format))) { + if (!(n->id_ctrl.ocfs & (1 << format)) || + ((format =3D=3D 2 || format =3D=3D 3) && + !(n->features.hbs.cdfe & (1 << format)))) { trace_pci_nvme_err_copy_invalid_format(format); status =3D NVME_INVALID_FIELD | NVME_DNR; goto invalid; @@ -3145,14 +3317,14 @@ static uint16_t nvme_copy(NvmeCtrl *n, NvmeRequest = *req) goto invalid; } =20 - if ((ns->pif =3D=3D 0x0 && format !=3D 0x0) || - (ns->pif !=3D 0x0 && format !=3D 0x1)) { + if ((ns->pif =3D=3D 0x0 && (format !=3D 0x0 && format !=3D 0x2)) || + (ns->pif !=3D 0x0 && (format !=3D 0x1 && format !=3D 0x3))) { status =3D NVME_INVALID_FORMAT | NVME_DNR; goto invalid; } =20 if (ns->pif) { - len =3D sizeof(NvmeCopySourceRangeFormat1); + len =3D sizeof(NvmeCopySourceRangeFormat1_3); } =20 iocb->format =3D format; @@ -3188,17 +3360,13 @@ static uint16_t nvme_copy(NvmeCtrl *n, NvmeRequest = *req) iocb->idx =3D 0; iocb->reftag =3D le32_to_cpu(copy->reftag); iocb->reftag |=3D (uint64_t)le32_to_cpu(copy->cdw3) << 32; - iocb->bounce =3D g_malloc_n(le16_to_cpu(ns->id_ns.mssrl), - ns->lbasz + ns->lbaf.ms); =20 qemu_iovec_init(&iocb->iov, 1); =20 - block_acct_start(blk_get_stats(ns->blkconf.blk), &iocb->acct.read, 0, - BLOCK_ACCT_READ); - block_acct_start(blk_get_stats(ns->blkconf.blk), &iocb->acct.write, 0, - BLOCK_ACCT_WRITE); - req->aiocb =3D &iocb->common; + iocb->sns =3D req->ns; + iocb->n =3D n; + iocb->bounce =3D NULL; nvme_do_copy(iocb); =20 return NVME_NO_COMPLETE; @@ -4407,10 +4575,6 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest= *req) trace_pci_nvme_io_cmd(nvme_cid(req), nsid, nvme_sqid(req), req->cmd.opcode, nvme_io_opc_str(req->cmd.opcode= )); =20 - if (!nvme_nsid_valid(n, nsid)) { - return NVME_INVALID_NSID | NVME_DNR; - } - /* * In the base NVM command set, Flush may apply to all namespaces * (indicated by NSID being set to FFFFFFFFh). But if that feature is = used @@ -4430,10 +4594,15 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeReques= t *req) * device only supports namespace types that includes the NVM Flush co= mmand * (NVM and Zoned), so always do an NVM Flush. */ + if (req->cmd.opcode =3D=3D NVME_CMD_FLUSH) { return nvme_flush(n, req); } =20 + if (!nvme_nsid_valid(n, nsid) || nsid =3D=3D NVME_NSID_BROADCAST) { + return NVME_INVALID_NSID | NVME_DNR; + } + ns =3D nvme_ns(n, nsid); if (unlikely(!ns)) { return NVME_INVALID_FIELD | NVME_DNR; @@ -8248,7 +8417,8 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pc= i_dev) id->nn =3D cpu_to_le32(NVME_MAX_NAMESPACES); id->oncs =3D cpu_to_le16(NVME_ONCS_WRITE_ZEROES | NVME_ONCS_TIMESTAMP | NVME_ONCS_FEATURES | NVME_ONCS_DSM | - NVME_ONCS_COMPARE | NVME_ONCS_COPY); + NVME_ONCS_COMPARE | NVME_ONCS_COPY | + NVME_ONCS_NVMCSA | NVME_ONCS_NVMAFC); =20 /* * NOTE: If this device ever supports a command set that does NOT use = 0x0 @@ -8259,7 +8429,8 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pc= i_dev) */ id->vwc =3D NVME_VWC_NSID_BROADCAST_SUPPORT | NVME_VWC_PRESENT; =20 - id->ocfs =3D cpu_to_le16(NVME_OCFS_COPY_FORMAT_0 | NVME_OCFS_COPY_FORM= AT_1); + id->ocfs =3D cpu_to_le16(NVME_OCFS_COPY_FORMAT_0 | NVME_OCFS_COPY_FORM= AT_1 | + NVME_OCFS_COPY_FORMAT_2 | NVME_OCFS_COPY_FORMA= T_3); id->sgls =3D cpu_to_le32(NVME_CTRL_SGLS_SUPPORT_NO_ALIGN); =20 nvme_init_subnqn(n); diff --git a/include/block/nvme.h b/include/block/nvme.h index bb231d0b9a..706e32d0f4 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -799,6 +799,8 @@ typedef struct QEMU_PACKED NvmeDsmRange { enum { NVME_COPY_FORMAT_0 =3D 0x0, NVME_COPY_FORMAT_1 =3D 0x1, + NVME_COPY_FORMAT_2 =3D 0x2, + NVME_COPY_FORMAT_3 =3D 0x3, }; =20 typedef struct QEMU_PACKED NvmeCopyCmd { @@ -820,25 +822,30 @@ typedef struct QEMU_PACKED NvmeCopyCmd { uint16_t appmask; } NvmeCopyCmd; =20 -typedef struct QEMU_PACKED NvmeCopySourceRangeFormat0 { - uint8_t rsvd0[8]; +typedef struct QEMU_PACKED NvmeCopySourceRangeFormat0_2 { + uint32_t sparams; + uint8_t rsvd4[4]; uint64_t slba; uint16_t nlb; - uint8_t rsvd18[6]; + uint8_t rsvd18[4]; + uint16_t sopt; uint32_t reftag; uint16_t apptag; uint16_t appmask; -} NvmeCopySourceRangeFormat0; +} NvmeCopySourceRangeFormat0_2; =20 -typedef struct QEMU_PACKED NvmeCopySourceRangeFormat1 { - uint8_t rsvd0[8]; +typedef struct QEMU_PACKED NvmeCopySourceRangeFormat1_3 { + uint32_t sparams; + uint8_t rsvd4[4]; uint64_t slba; uint16_t nlb; - uint8_t rsvd18[8]; + uint8_t rsvd18[4]; + uint16_t sopt; + uint8_t rsvd24[2]; uint8_t sr[10]; uint16_t apptag; uint16_t appmask; -} NvmeCopySourceRangeFormat1; +} NvmeCopySourceRangeFormat1_3; =20 enum NvmeAsyncEventRequest { NVME_AER_TYPE_ERROR =3D 0, @@ -937,6 +944,8 @@ enum NvmeStatusCodes { NVME_INVALID_PROT_INFO =3D 0x0181, NVME_WRITE_TO_RO =3D 0x0182, NVME_CMD_SIZE_LIMIT =3D 0x0183, + NVME_CMD_INCOMP_NS_OR_FMT =3D 0x0185, + NVME_CMD_OVERLAP_IO_RANGE =3D 0x0187, NVME_INVALID_ZONE_OP =3D 0x01b6, NVME_NOZRWA =3D 0x01b7, NVME_ZONE_BOUNDARY_ERROR =3D 0x01b8, @@ -1194,11 +1203,15 @@ enum NvmeIdCtrlOncs { NVME_ONCS_TIMESTAMP =3D 1 << 6, NVME_ONCS_VERIFY =3D 1 << 7, NVME_ONCS_COPY =3D 1 << 8, + NVME_ONCS_NVMCSA =3D 1 << 9, + NVME_ONCS_NVMAFC =3D 1 << 10, }; =20 enum NvmeIdCtrlOcfs { NVME_OCFS_COPY_FORMAT_0 =3D 1 << NVME_COPY_FORMAT_0, NVME_OCFS_COPY_FORMAT_1 =3D 1 << NVME_COPY_FORMAT_1, + NVME_OCFS_COPY_FORMAT_2 =3D 1 << NVME_COPY_FORMAT_2, + NVME_OCFS_COPY_FORMAT_3 =3D 1 << NVME_COPY_FORMAT_3, }; =20 enum NvmeIdctrlVwc { @@ -1332,7 +1345,9 @@ typedef struct NvmeHostBehaviorSupport { uint8_t acre; uint8_t etdas; uint8_t lbafee; - uint8_t rsvd3[509]; + uint8_t rsvd3; + uint16_t cdfe; + uint8_t rsvd6[506]; } NvmeHostBehaviorSupport; =20 typedef struct QEMU_PACKED NvmeLBAF { @@ -1832,8 +1847,8 @@ static inline void _nvme_check_size(void) QEMU_BUILD_BUG_ON(sizeof(NvmeZonedResult) !=3D 8); QEMU_BUILD_BUG_ON(sizeof(NvmeCqe) !=3D 16); QEMU_BUILD_BUG_ON(sizeof(NvmeDsmRange) !=3D 16); - QEMU_BUILD_BUG_ON(sizeof(NvmeCopySourceRangeFormat0) !=3D 32); - QEMU_BUILD_BUG_ON(sizeof(NvmeCopySourceRangeFormat1) !=3D 40); + QEMU_BUILD_BUG_ON(sizeof(NvmeCopySourceRangeFormat0_2) !=3D 32); + QEMU_BUILD_BUG_ON(sizeof(NvmeCopySourceRangeFormat1_3) !=3D 40); QEMU_BUILD_BUG_ON(sizeof(NvmeCmd) !=3D 64); QEMU_BUILD_BUG_ON(sizeof(NvmeDeleteQ) !=3D 64); QEMU_BUILD_BUG_ON(sizeof(NvmeCreateCq) !=3D 64); --=20 2.43.0