From nobody Thu Apr 18 06:20:25 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Authentication-Results: mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail(p=quarantine dis=quarantine) header.from=amazon.com ARC-Seal: i=1; a=rsa-sha256; t=1601500942; cv=none; d=zohomail.com; s=zohoarc; b=GTaQK/yACY0/VIv0xPmY+49vx8nYoDxRDSKrCiv03RJUForkGriG21Ho2i5p26BlaLotFXJBJTtf+JZbHm7D2CIwDbsm2SlkWSTQ/WEqzdIHv9y9Dmsj/VuCu5W3gR6iFEGmYh4l3mUkBbiRkyC/L+wSsjpIuVONgstuM8JFQSU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601500942; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Subject:To; bh=jQ+2uqZB22VDov2h4QQi2B7uTvk9wj5hp9JWIUA6Ek0=; b=RgT00mi1iGvOvdzCPxOc7sy43fndR45d//8E0JlINZrS5bUmu+aTvuW+BzRCLIg/2DWeuUaIQfP0ir0+w1H8yzVFzHWIa7qcALbJ/huxrVNaFbbDc2uVcWcg1dHSKVKZ6tuy7n80x2Ec1Y39ttLbeUWVr9u0EXBUJXfq/1p2aYk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail header.from= (p=quarantine dis=quarantine) header.from= Return-Path: Received: from coreboot.org (coreboot.org [78.46.105.101]) by mx.zohomail.com with SMTPS id 1601500942962586.4809939738014; Wed, 30 Sep 2020 14:22:22 -0700 (PDT) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTPA id A4AC410C0096; Wed, 30 Sep 2020 21:22:18 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTP id 61B4E10C008E for ; Wed, 30 Sep 2020 21:22:04 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP; 30 Sep 2020 21:22:01 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by email-inbound-relay-2a-90c42d1d.us-west-2.amazon.com (Postfix) with ESMTPS id 43576A11FB for ; Wed, 30 Sep 2020 21:11:06 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13MTAUWC001.ant.amazon.com (10.43.162.135) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:05 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13D20UWC001.ant.amazon.com (10.43.162.244) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1601500924; x=1633036924; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WHyNICsRWkvz4Ku4KSw7GPuki67iKxMO7O/DZArkBfs=; b=v6bA/SNkXu57+IaYJzllvuO5mox/gmgEcElRpQ7z3jU/aU/Rg7eEh+s6 btZMEP1ab4LI6nX9kBjGpxnMvJ+J3cnvivpp5MbtISpJYE0JjLcWfTXDa m67u5UgEisEkzpyiSvS6g53gSV1QuUwkuD+9NONNYWGU6zPXhjyDvJ5QV 8=; X-IronPort-AV: E=Sophos;i="5.77,322,1596499200"; d="scan'208";a="57052188" From: Alexander Graf To: Date: Wed, 30 Sep 2020 23:10:53 +0200 Message-ID: <20200930211056.8478-2-graf@amazon.com> In-Reply-To: <20200930211056.8478-1-graf@amazon.com> References: <20200930211056.8478-1-graf@amazon.com> MIME-Version: 1.0 X-ClientProxiedBy: EX13D15UWA002.ant.amazon.com (10.43.160.218) To EX13D20UWC001.ant.amazon.com (10.43.162.244) Message-ID-Hash: 7C35FRJVR65OFKEL77QH3FSPWSQM57AG X-Message-ID-Hash: 7C35FRJVR65OFKEL77QH3FSPWSQM57AG X-MailFrom: prvs=535b814ec=graf@amazon.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-seabios.seabios.org-0; header-match-seabios.seabios.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.3.2b1 Precedence: list Subject: [SeaBIOS] [PATCH v3 1/4] nvme: Record maximum allowed request size List-Id: SeaBIOS mailing list Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable Authentication-Results: coreboot.org; auth=pass smtp.auth=mailman@coreboot.org smtp.mailfrom=seabios-bounces@seabios.org X-Spamd-Bar: ------ X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" NVMe has a limit on how many sectors it can handle at most within a single request. Remember that number, so that in a follow-up patch, we can verify that we don't exceed it. Signed-off-by: Alexander Graf --- v1 -> v2: - fix dprintf v2 -> v3: - Adjust the maximum request size to sector units. The hardware field describes it in pages. --- src/hw/nvme-int.h | 8 +++++++- src/hw/nvme.c | 13 +++++++++++-- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/src/hw/nvme-int.h b/src/hw/nvme-int.h index 9f95dd8..674008a 100644 --- a/src/hw/nvme-int.h +++ b/src/hw/nvme-int.h @@ -117,6 +117,7 @@ struct nvme_namespace { =20 u32 block_size; u32 metadata_size; + u32 max_req_size; =20 /* Page aligned buffer of size NVME_PAGE_SIZE. */ char *dma_buffer; @@ -131,7 +132,12 @@ struct nvme_identify_ctrl { char mn[40]; char fr[8]; =20 - char _boring[516 - 72]; + u8 rab; + u8 ieee[3]; + u8 cmic; + u8 mdts; + + char _boring[516 - 78]; =20 u32 nn; /* number of namespaces */ }; diff --git a/src/hw/nvme.c b/src/hw/nvme.c index 6a01204..5bc2586 100644 --- a/src/hw/nvme.c +++ b/src/hw/nvme.c @@ -238,7 +238,8 @@ nvme_admin_identify_ns(struct nvme_ctrl *ctrl, u32 ns_i= d) } =20 static void -nvme_probe_ns(struct nvme_ctrl *ctrl, struct nvme_namespace *ns, u32 ns_id) +nvme_probe_ns(struct nvme_ctrl *ctrl, struct nvme_namespace *ns, u32 ns_id, + u8 mdts) { ns->ctrl =3D ctrl; ns->ns_id =3D ns_id; @@ -281,6 +282,14 @@ nvme_probe_ns(struct nvme_ctrl *ctrl, struct nvme_name= space *ns, u32 ns_id) ns->drive.blksize =3D ns->block_size; ns->drive.sectors =3D ns->lba_count; =20 + if (mdts) { + ns->max_req_size =3D ((1U << mdts) * NVME_PAGE_SIZE) / ns->block_s= ize; + dprintf(3, "NVME NS %u max request size: %d sectors\n", + ns_id, ns->max_req_size); + } else { + ns->max_req_size =3D -1U; + } + ns->dma_buffer =3D zalloc_page_aligned(&ZoneHigh, NVME_PAGE_SIZE); =20 char *desc =3D znprintf(MAXDESCSIZE, "NVMe NS %u: %llu MiB (%llu %u-by= te " @@ -567,7 +576,7 @@ nvme_controller_enable(struct nvme_ctrl *ctrl) /* Populate namespace IDs */ int ns_idx; for (ns_idx =3D 0; ns_idx < ctrl->ns_count; ns_idx++) { - nvme_probe_ns(ctrl, &ctrl->ns[ns_idx], ns_idx + 1); + nvme_probe_ns(ctrl, &ctrl->ns[ns_idx], ns_idx + 1, identify->mdts); } =20 dprintf(3, "NVMe initialization complete!\n"); --=20 2.16.4 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ SeaBIOS mailing list -- seabios@seabios.org To unsubscribe send an email to seabios-leave@seabios.org From nobody Thu Apr 18 06:20:25 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Authentication-Results: mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail(p=quarantine dis=quarantine) header.from=amazon.com ARC-Seal: i=1; a=rsa-sha256; t=1601500327; cv=none; d=zohomail.com; s=zohoarc; b=LhToA+VTIhhNmsxueeYRR+lYz2Bxz9zCN0gtbMRaqlQOR5HbAZKzXdOhl9WSBO1sY8CpPcW83JTbHPlIORiw/WTw7DgWjoW5I/spqX0xCEOCyOWNR4TFHB9WtTCuo1oprNlIHhoLovQu7Xz5Wgf0CCjdE0L4YftQJ3pcn8HyF2U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601500327; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Subject:To; bh=XcQ9tcQ0ucjtn6KJzTyuxIYE8Datpw3dhFhqvlCuZI4=; b=gOf1eB9gkcfIZTEyUVUH0xICCgw7T63ZdVCMsJNxrQ+FOo/i3n2XbFinGjaEzkYqUz2hoUq6fhMq08KmvlT6Skz4TGCeWloVk+qKZ7JUjQFidEj9wwVAhZ+1K6wremdFlr3FNKS2v2UnSLK8k6EiOwH7NpxIoMSJu6W0xTSPAJ8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail header.from= (p=quarantine dis=quarantine) header.from= Return-Path: Received: from coreboot.org (coreboot.org [78.46.105.101]) by mx.zohomail.com with SMTPS id 1601500327892799.7075362843091; Wed, 30 Sep 2020 14:12:07 -0700 (PDT) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTPA id E4A7210C009E; Wed, 30 Sep 2020 21:12:03 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTP id 308AC10C0084 for ; Wed, 30 Sep 2020 21:11:30 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 30 Sep 2020 21:11:07 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by email-inbound-relay-2c-168cbb73.us-west-2.amazon.com (Postfix) with ESMTPS id 53F33A1FD9 for ; Wed, 30 Sep 2020 21:11:07 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13MTAUWC002.ant.amazon.com (10.43.162.240) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:06 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13D20UWC001.ant.amazon.com (10.43.162.244) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1601500290; x=1633036290; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nyfv60RGr+MkhCdP7OzPi9VDsRVgHEK6GTdL5F6Z0GE=; b=bv5mlSy5AzwjkSAGArqv8FcVgaaxyJU4ybYRyTI9wjD/adXfUnjt31Ns K3MkEdJkZnu0pgVSQsSex3SlwIVjQMg4hMfDp2kNzYdOc+h39NwkGecK3 3xl1mXzuDw7UOTFskymp8kfii9jAcTaRm2Gwdk71sWXmbEmB/hng/3JMQ E=; X-IronPort-AV: E=Sophos;i="5.77,322,1596499200"; d="scan'208";a="57257586" From: Alexander Graf To: Date: Wed, 30 Sep 2020 23:10:54 +0200 Message-ID: <20200930211056.8478-3-graf@amazon.com> In-Reply-To: <20200930211056.8478-1-graf@amazon.com> References: <20200930211056.8478-1-graf@amazon.com> MIME-Version: 1.0 X-ClientProxiedBy: EX13D15UWA002.ant.amazon.com (10.43.160.218) To EX13D20UWC001.ant.amazon.com (10.43.162.244) Message-ID-Hash: XTJCBZOK6UAWFR2OX6ZMFBRR72UHNMN2 X-Message-ID-Hash: XTJCBZOK6UAWFR2OX6ZMFBRR72UHNMN2 X-MailFrom: prvs=535b814ec=graf@amazon.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-seabios.seabios.org-0; header-match-seabios.seabios.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.3.2b1 Precedence: list Subject: [SeaBIOS] [PATCH v3 2/4] nvme: Allow to set PRP2 List-Id: SeaBIOS mailing list Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable Authentication-Results: coreboot.org; auth=pass smtp.auth=mailman@coreboot.org smtp.mailfrom=seabios-bounces@seabios.org X-Spamd-Bar: ------ X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" When creating a PRP based I/O request, we pass in the pointer to operate on. Going forward, we will want to be able to pass additional pointers though for mappings above 4k. This patch adds a parameter to nvme_get_next_sqe() to pass in the PRP2 value of an NVMe I/O request, paving the way for a future patch to implement PRP lists. Signed-off-by: Alexander Graf Reviewed-by: Filippo Sironi --- src/hw/nvme.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/src/hw/nvme.c b/src/hw/nvme.c index 5bc2586..f1b9331 100644 --- a/src/hw/nvme.c +++ b/src/hw/nvme.c @@ -152,7 +152,7 @@ nvme_wait(struct nvme_sq *sq) /* Returns the next submission queue entry (or NULL if the queue is full).= It also fills out Command Dword 0 and clears the rest. */ static struct nvme_sqe * -nvme_get_next_sqe(struct nvme_sq *sq, u8 opc, void *metadata, void *data) +nvme_get_next_sqe(struct nvme_sq *sq, u8 opc, void *metadata, void *data, = void *data2) { if (((sq->head + 1) & sq->common.mask) =3D=3D sq->tail) { dprintf(3, "submission queue is full"); @@ -166,6 +166,7 @@ nvme_get_next_sqe(struct nvme_sq *sq, u8 opc, void *met= adata, void *data) sqe->cdw0 =3D opc | (sq->tail << 16 /* CID */); sqe->mptr =3D (u32)metadata; sqe->dptr_prp1 =3D (u32)data; + sqe->dptr_prp2 =3D (u32)data2; =20 if (sqe->dptr_prp1 & (NVME_PAGE_SIZE - 1)) { /* Data buffer not page aligned. */ @@ -200,7 +201,7 @@ nvme_admin_identify(struct nvme_ctrl *ctrl, u8 cns, u32= nsid) struct nvme_sqe *cmd_identify; cmd_identify =3D nvme_get_next_sqe(&ctrl->admin_sq, NVME_SQE_OPC_ADMIN_IDENTIFY, NULL, - identify_buf); + identify_buf, NULL); =20 if (!cmd_identify) { warn_internalerror(); @@ -338,7 +339,7 @@ nvme_create_io_cq(struct nvme_ctrl *ctrl, struct nvme_c= q *cq, u16 q_idx) =20 cmd_create_cq =3D nvme_get_next_sqe(&ctrl->admin_sq, NVME_SQE_OPC_ADMIN_CREATE_IO_CQ, NUL= L, - cq->cqe); + cq->cqe, NULL); if (!cmd_create_cq) { goto err_destroy_cq; } @@ -382,7 +383,7 @@ nvme_create_io_sq(struct nvme_ctrl *ctrl, struct nvme_s= q *sq, u16 q_idx, struct =20 cmd_create_sq =3D nvme_get_next_sqe(&ctrl->admin_sq, NVME_SQE_OPC_ADMIN_CREATE_IO_SQ, NUL= L, - sq->sqe); + sq->sqe, NULL); if (!cmd_create_sq) { goto err_destroy_sq; } @@ -429,7 +430,7 @@ nvme_io_readwrite(struct nvme_namespace *ns, u64 lba, c= har *buf, u16 count, struct nvme_sqe *io_read =3D nvme_get_next_sqe(&ns->ctrl->io_sq, write ? NVME_SQE_OPC_IO_W= RITE : NVME_SQE_OPC_IO_R= EAD, - NULL, buf); + NULL, buf, NULL); io_read->nsid =3D ns->ns_id; io_read->dword[10] =3D (u32)lba; io_read->dword[11] =3D (u32)(lba >> 32); --=20 2.16.4 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ SeaBIOS mailing list -- seabios@seabios.org To unsubscribe send an email to seabios-leave@seabios.org From nobody Thu Apr 18 06:20:25 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Authentication-Results: mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail(p=quarantine dis=quarantine) header.from=amazon.com ARC-Seal: i=1; a=rsa-sha256; t=1601500305; cv=none; d=zohomail.com; s=zohoarc; b=OfOef9FnEu+KtueELdfSsxmAfKROAi2ozc/cH0VcZcHuKLCY2xKkm/vQthJ/egfMpXbQ3ELOIlfBmRDH6od6AfRMNosnnAWxEi0fZKJmqB9B99l3a7uOuyGWw72g4pfBvaeBf4ZJTbPgGd4RkGYvTLdMa9FB1QTz7+177f3VjLs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601500305; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Subject:To; bh=D34i0rOs7H3wu3VaM3hzgwLS8OhGh+GOZy6pyeuBg8w=; b=PopvuCMAHvtJ3lvUxxW6I6arE/5NK+X1ycVHfs978iHeahI6ohCKAC+zlZluDZSk+PBzIvxCRPY38uXQyVuXmqEuqcBqAx/OyGvMwAz23uyZw0g496KA9YtFcFVflTCXs9mgfoKWsDdD1ycN21/1r+sAvRVTlLZqPGEUTidOrE0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail header.from= (p=quarantine dis=quarantine) header.from= Return-Path: Received: from coreboot.org (coreboot.org [78.46.105.101]) by mx.zohomail.com with SMTPS id 1601500305331809.5604735912675; Wed, 30 Sep 2020 14:11:45 -0700 (PDT) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTPA id 8B91E10C0089; Wed, 30 Sep 2020 21:11:41 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTP id A35D510C0084 for ; Wed, 30 Sep 2020 21:11:28 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 30 Sep 2020 21:11:09 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by email-inbound-relay-2b-4e24fd92.us-west-2.amazon.com (Postfix) with ESMTPS id B4086A06FD for ; Wed, 30 Sep 2020 21:11:08 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13MTAUWC001.ant.amazon.com (10.43.162.135) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:08 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13D20UWC001.ant.amazon.com (10.43.162.244) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1601500289; x=1633036289; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=60j2xZ0wV3xDsoQNpfRngfjNe1EXd7Xu/t8dHI/rkTs=; b=II7GBjnJTvgj4w7dXeho4yLZSRrdNsZCIrx6y+wGHNLSFlb5oRwrxS/n 7eb4Ir51lvnSIZW4YzKflHwPtrdFsRfgmwim8bK27BtPA11wMsorXsVgQ cgX+49o6czgZHkRgei6yxZGlvrwIIzq4Yoj73YzPPsIAtYZnhQcXi2Dvo A=; X-IronPort-AV: E=Sophos;i="5.77,322,1596499200"; d="scan'208";a="57257599" From: Alexander Graf To: Date: Wed, 30 Sep 2020 23:10:55 +0200 Message-ID: <20200930211056.8478-4-graf@amazon.com> In-Reply-To: <20200930211056.8478-1-graf@amazon.com> References: <20200930211056.8478-1-graf@amazon.com> MIME-Version: 1.0 X-ClientProxiedBy: EX13D15UWA002.ant.amazon.com (10.43.160.218) To EX13D20UWC001.ant.amazon.com (10.43.162.244) Message-ID-Hash: DSEJWBRZMVONF5JTGUHC2I4ABHE7N64O X-Message-ID-Hash: DSEJWBRZMVONF5JTGUHC2I4ABHE7N64O X-MailFrom: prvs=535b814ec=graf@amazon.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-seabios.seabios.org-0; header-match-seabios.seabios.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.3.2b1 Precedence: list Subject: [SeaBIOS] [PATCH v3 3/4] nvme: Pass large I/O requests as PRP lists List-Id: SeaBIOS mailing list Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable Authentication-Results: coreboot.org; auth=pass smtp.auth=mailman@coreboot.org smtp.mailfrom=seabios-bounces@seabios.org X-Spamd-Bar: / X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Today, we split every I/O request into at most 4kb chunks and wait for these requests to finish. We encountered issues where the backing storage is netw= ork based, so every I/O request needs to go over the network with associated latency cost. A few ms of latency when loading 100MB initrd in 4kb chunks does add up. NVMe implements a feature to allow I/O requests spanning multiple pages, called PRP lists. This patch takes larger I/O operations and checks if they can be directly passed to the NVMe backing device as PRP list. At least for grub, read operations can always be mapped directly into PRP list items. This reduces the number of I/O operations required during a typical boot path by roughly a factor of 5. Signed-off-by: Alexander Graf --- v1 -> v2: - Fix bounds check on PRP list add logic - Reduce PRP list to 15 entries embedded in the ns struct. This reduces BIOS reserved memory footprint by 4kb. --- src/hw/nvme-int.h | 8 ++++++ src/hw/nvme.c | 84 ++++++++++++++++++++++++++++++++++++++++++++++++---= ---- 2 files changed, 82 insertions(+), 10 deletions(-) diff --git a/src/hw/nvme-int.h b/src/hw/nvme-int.h index 674008a..7947b29 100644 --- a/src/hw/nvme-int.h +++ b/src/hw/nvme-int.h @@ -10,6 +10,8 @@ #include "types.h" // u32 #include "pcidevice.h" // struct pci_device =20 +#define NVME_MAX_PRPL_ENTRIES 15 /* Allows requests up to 64kb */ + /* Data structures */ =20 /* The register file of a NVMe host controller. This struct follows the na= ming @@ -121,6 +123,11 @@ struct nvme_namespace { =20 /* Page aligned buffer of size NVME_PAGE_SIZE. */ char *dma_buffer; + + /* Page List */ + u32 prpl_len; + void *prp1; + u64 prpl[NVME_MAX_PRPL_ENTRIES]; }; =20 /* Data structures for NVMe admin identify commands */ @@ -195,6 +202,7 @@ union nvme_identify { #define NVME_CQE_DW3_P (1U << 16) =20 #define NVME_PAGE_SIZE 4096 +#define NVME_PAGE_MASK ~(NVME_PAGE_SIZE - 1) =20 /* Length for the queue entries. */ #define NVME_SQE_SIZE_LOG 6 diff --git a/src/hw/nvme.c b/src/hw/nvme.c index f1b9331..b92ca52 100644 --- a/src/hw/nvme.c +++ b/src/hw/nvme.c @@ -168,11 +168,6 @@ nvme_get_next_sqe(struct nvme_sq *sq, u8 opc, void *me= tadata, void *data, void * sqe->dptr_prp1 =3D (u32)data; sqe->dptr_prp2 =3D (u32)data2; =20 - if (sqe->dptr_prp1 & (NVME_PAGE_SIZE - 1)) { - /* Data buffer not page aligned. */ - warn_internalerror(); - } - return sqe; } =20 @@ -418,19 +413,29 @@ nvme_io_readwrite(struct nvme_namespace *ns, u64 lba,= char *buf, u16 count, int write) { u32 buf_addr =3D (u32)buf; + void *prp2; =20 - if ((buf_addr & 0x3) || - ((buf_addr & ~(NVME_PAGE_SIZE - 1)) !=3D - ((buf_addr + ns->block_size * count - 1) & ~(NVME_PAGE_SIZE - 1))= )) { - /* Buffer is misaligned or crosses page boundary */ + if (buf_addr & 0x3) { + /* Buffer is misaligned */ warn_internalerror(); return DISK_RET_EBADTRACK; } =20 + if ((ns->block_size * count) > (NVME_PAGE_SIZE * 2)) { + /* We need to describe more than 2 pages, rely on PRP List */ + prp2 =3D ns->prpl; + } else if ((ns->block_size * count) > NVME_PAGE_SIZE) { + /* Directly embed the 2nd page if we only need 2 pages */ + prp2 =3D (void *)(long)ns->prpl[0]; + } else { + /* One page is enough, don't expose anything else */ + prp2 =3D NULL; + } + struct nvme_sqe *io_read =3D nvme_get_next_sqe(&ns->ctrl->io_sq, write ? NVME_SQE_OPC_IO_W= RITE : NVME_SQE_OPC_IO_R= EAD, - NULL, buf, NULL); + NULL, buf, prp2); io_read->nsid =3D ns->ns_id; io_read->dword[10] =3D (u32)lba; io_read->dword[11] =3D (u32)(lba >> 32); @@ -450,6 +455,60 @@ nvme_io_readwrite(struct nvme_namespace *ns, u64 lba, = char *buf, u16 count, return DISK_RET_SUCCESS; } =20 +static void nvme_reset_prpl(struct nvme_namespace *ns) +{ + ns->prpl_len =3D 0; +} + +static int nvme_add_prpl(struct nvme_namespace *ns, u64 base) +{ + if (ns->prpl_len >=3D NVME_MAX_PRPL_ENTRIES) + return -1; + + ns->prpl[ns->prpl_len++] =3D base; + + return 0; +} + +int nvme_build_prpl(struct nvme_namespace *ns, struct disk_op_s *op) +{ + int first_page =3D 1; + u32 base =3D (long)op->buf_fl; + s32 size =3D op->count * ns->block_size; + + if (op->count > ns->max_req_size) + return -1; + + nvme_reset_prpl(ns); + + /* Special case for transfers that fit into PRP1, but are unaligned */ + if (((size + (base & ~NVME_PAGE_MASK)) <=3D NVME_PAGE_SIZE)) { + ns->prp1 =3D op->buf_fl; + return 0; + } + + /* Every request has to be page aligned */ + if (base & ~NVME_PAGE_MASK) + return -1; + + /* Make sure a full block fits into the last chunk */ + if (size & (ns->block_size - 1ULL)) + return -1; + + for (; size > 0; base +=3D NVME_PAGE_SIZE, size -=3D NVME_PAGE_SIZE) { + if (first_page) { + /* First page is special */ + ns->prp1 =3D (void*)base; + first_page =3D 0; + continue; + } + if (nvme_add_prpl(ns, base)) + return -1; + } + + return 0; +} + static int nvme_create_io_queues(struct nvme_ctrl *ctrl) { @@ -668,6 +727,11 @@ nvme_cmd_readwrite(struct nvme_namespace *ns, struct d= isk_op_s *op, int write) u16 const max_blocks =3D NVME_PAGE_SIZE / ns->block_size; u16 i; =20 + if (!nvme_build_prpl(ns, op)) { + /* Request goes via PRP List logic */ + return nvme_io_readwrite(ns, op->lba, ns->prp1, op->count, write); + } + for (i =3D 0; i < op->count && res =3D=3D DISK_RET_SUCCESS;) { u16 blocks_remaining =3D op->count - i; u16 blocks =3D blocks_remaining < max_blocks ? blocks_remaining --=20 2.16.4 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ SeaBIOS mailing list -- seabios@seabios.org To unsubscribe send an email to seabios-leave@seabios.org From nobody Thu Apr 18 06:20:25 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Received-SPF: none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Authentication-Results: mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail(p=quarantine dis=quarantine) header.from=amazon.com ARC-Seal: i=1; a=rsa-sha256; t=1601500338; cv=none; d=zohomail.com; s=zohoarc; b=ckPko1VEYDUK27Ia4hEcv4cz5p5p5VAGYNGIcc/VB0RG6f6QU3+B/KQ+tFxWtk/o6VryYoWh+N7ayS/L7OeBP9fSWlXhGI5qco3cCh2RgSFTmlajubjhejLsy1iu4brXq3FGFUhGCoUNTkEbPwat8jbOKJVxYQE4HQrmjE5ZtcQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601500338; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Subject:To; bh=toQ4Wwax6/9q862mAOev30C2oU9tq4dhlodw5gS+8+Q=; b=UYs/Kti2Mm02ZzSXDyvBbr5DdMAlqGpK87CyZGrntijgxxTMk5m/F8M4WzaoKQuW0OtKHibINSjw1WXdX5m3aIa7aiUPEMzI+umYaHDw17ZUAqY0aVHqxTiJZUBXOL73RtU35CfmlEIHWQppQoaq22xQz5z6cPQb11J0HQjNrIs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail header.i=@amazon.com; spf=none (zohomail.com: 78.46.105.101 is neither permitted nor denied by domain of seabios.org) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=fail header.from= (p=quarantine dis=quarantine) header.from= Return-Path: Received: from coreboot.org (coreboot.org [78.46.105.101]) by mx.zohomail.com with SMTPS id 1601500338588669.9432690082726; Wed, 30 Sep 2020 14:12:18 -0700 (PDT) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTPA id 8DF1710C00A4; Wed, 30 Sep 2020 21:12:14 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTP id 6763510C0089 for ; Wed, 30 Sep 2020 21:11:30 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 30 Sep 2020 21:11:10 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by email-inbound-relay-2a-119b4f96.us-west-2.amazon.com (Postfix) with ESMTPS id B08231A0D1D for ; Wed, 30 Sep 2020 21:11:09 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13MTAUWC002.ant.amazon.com (10.43.162.240) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:09 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13D20UWC001.ant.amazon.com (10.43.162.244) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 30 Sep 2020 21:11:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1601500290; x=1633036290; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=F63TJ7mKpPGimYmibqZcA0p9+Jw3a6RIFzFH+XWuNPk=; b=UtaBKA/S9aW5YblFH6YVRC5rzl7tEcR9apbMmpS45iy9l+DuIhdEVPdh JZY0LbBynq0wrLAii6ude1UGxgZtE3k2nOge0Bq7y6ZYzww38ZieOxH2I y82YidU8F0xG7keFw9exN+RRk9pO4qcZplObsnm4sjguHoxitPHfUWuBA g=; X-IronPort-AV: E=Sophos;i="5.77,322,1596499200"; d="scan'208";a="57257604" From: Alexander Graf To: Date: Wed, 30 Sep 2020 23:10:56 +0200 Message-ID: <20200930211056.8478-5-graf@amazon.com> In-Reply-To: <20200930211056.8478-1-graf@amazon.com> References: <20200930211056.8478-1-graf@amazon.com> MIME-Version: 1.0 X-ClientProxiedBy: EX13D15UWA002.ant.amazon.com (10.43.160.218) To EX13D20UWC001.ant.amazon.com (10.43.162.244) Message-ID-Hash: SU4X27X2L2LCFP3IETMMVBLY4HNPIRGI X-Message-ID-Hash: SU4X27X2L2LCFP3IETMMVBLY4HNPIRGI X-MailFrom: prvs=535b814ec=graf@amazon.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-seabios.seabios.org-0; header-match-seabios.seabios.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.3.2b1 Precedence: list Subject: [SeaBIOS] [PATCH v3 4/4] nvme: Split requests by maximum allowed size List-Id: SeaBIOS mailing list Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable Authentication-Results: coreboot.org; auth=pass smtp.auth=mailman@coreboot.org smtp.mailfrom=seabios-bounces@seabios.org X-Spamd-Bar: ------ X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Some NVMe controllers only support small maximum request sizes, such as the AWS EBS NVMe implementation which only supports NVMe requests of up to 32 pages (256kb) at once. BIOS callers can exceed those request sizes by defining sector counts above this threshold. Currently we fall back to the bounce buffer implementation for those. This is slow. This patch introduces splitting logic to the NVMe I/O request code so that every NVMe I/O request gets handled in a chunk size that is consumable by the NVMe adapter, while maintaining the fast path PRPL logic we just introduced. Signed-off-by: Alexander Graf --- src/hw/nvme.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/hw/nvme.c b/src/hw/nvme.c index b92ca52..cc37bca 100644 --- a/src/hw/nvme.c +++ b/src/hw/nvme.c @@ -727,6 +727,22 @@ nvme_cmd_readwrite(struct nvme_namespace *ns, struct d= isk_op_s *op, int write) u16 const max_blocks =3D NVME_PAGE_SIZE / ns->block_size; u16 i; =20 + /* Split up requests that are larger than the device can handle */ + if (op->count > ns->max_req_size) { + u16 count =3D op->count; + + /* Handle the first max_req_size elements */ + op->count =3D ns->max_req_size; + if (nvme_cmd_readwrite(ns, op, write)) + return res; + + /* Handle the remainder of the request */ + op->count =3D count - ns->max_req_size; + op->lba +=3D ns->max_req_size; + op->buf_fl +=3D (ns->max_req_size * ns->block_size); + return nvme_cmd_readwrite(ns, op, write); + } + if (!nvme_build_prpl(ns, op)) { /* Request goes via PRP List logic */ return nvme_io_readwrite(ns, op->lba, ns->prp1, op->count, write); --=20 2.16.4 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ SeaBIOS mailing list -- seabios@seabios.org To unsubscribe send an email to seabios-leave@seabios.org