From nobody Wed May 8 01:57:34 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of seabios.org designates 78.46.105.101 as permitted sender) client-ip=78.46.105.101; envelope-from=seabios-bounces@seabios.org; helo=coreboot.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of seabios.org designates 78.46.105.101 as permitted sender) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=pass(p=reject dis=none) header.from=seabios.org ARC-Seal: i=1; a=rsa-sha256; t=1642602174; cv=none; d=zohomail.com; s=zohoarc; b=HYpTTyD04dRZvxVBaKx4efyNFfb4FU2wPNreI2diNatiqvhK2FyGWE/PM2ZK09HYmKmGIP4vj5HGgXdTWLCmahltyNqRS1RjAj2nvbCqDrPQRysrJ9NTUexeloZvuSAP7faPVDZYU68GOZzi2YubBaBJjzvQn5eOteyRvbM+tCY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1642602174; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Owner:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Subject:To; bh=A6Sbq+e+Jn6WO2bUL/7kwa+y4ck+5hGLfSK07pELqIA=; b=lhQRAoUteJsoh0bAZkZYvzdeUwoWgE9FSwNHg+zQrBLSG//kBIcrPELJJp8wvZSOBJeaXcQcOzk84kPPpK+HYeAVTnVmuPLK/SzyWjFfwKEpetiH9RaNpNu0DKJkwxmqiY7Hj4ghKy1xldCT26N25Uzh/tQXOtMcRquNMp5i1ek= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of seabios.org designates 78.46.105.101 as permitted sender) smtp.mailfrom=seabios-bounces@seabios.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from coreboot.org (coreboot.org [78.46.105.101]) by mx.zohomail.com with SMTPS id 16426021744891014.8182551939666; Wed, 19 Jan 2022 06:22:54 -0800 (PST) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTPA id 6ECE716E3F6C; Wed, 19 Jan 2022 14:22:47 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by coreboot.org (Postfix) with ESMTP id D21DD16E3D3C for ; Wed, 19 Jan 2022 14:22:11 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP; 19 Jan 2022 14:21:56 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) by email-inbound-relay-pdx-2c-a264e6fe.us-west-2.amazon.com (Postfix) with ESMTPS id C9D0B4188F; Wed, 19 Jan 2022 14:21:55 +0000 (UTC) Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13MTAUWC002.ant.amazon.com (10.43.162.240) with Microsoft SMTP Server (TLS) id 15.0.1497.28; Wed, 19 Jan 2022 14:21:55 +0000 Received: from authenticated-user (PRIMARY_HOSTNAME [PUBLIC_IP]) EX13D20UWC001.ant.amazon.com (10.43.162.244) with Microsoft SMTP Server (TLS) id 15.0.1497.28; Wed, 19 Jan 2022 14:21:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=seabios.org; s=dkim; t=1642602170; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:list-id:list-owner:list-unsubscribe:list-subscribe:list-post; bh=A6Sbq+e+Jn6WO2bUL/7kwa+y4ck+5hGLfSK07pELqIA=; b=WGl5zleL9JZ9X1l0HoVRX29Ae1BQvSa2epsnfUvr3fDoFfeQ3uFPPUdp5GA/QuQu4E6gsI BHI7oitjvfgMv2vImOZJfOf88DEtY7iM1+RGHX4WitiuTzLKuSLj2IikGpeUWbnWT0JAqv 5wIGbbPU3Dmuf3Kj8rMrZCS/jcvOonM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1642602132; x=1674138132; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=akQ0Qhq8k23927+/Cyup72dp5+Zy3/WrJGRmFcMxSRo=; b=SBFW1XcRoMps1ZIqPXsG105u7RFER2ufe9GmMKd2Tvy1rYpTSm58KZI3 CIzcslyjAIT9DfnUxxM2aRpq5ZEYyJpFB1RrSeqrz8uGVtpsPbTmOwNBo kvj2wzAefNwJ+BtABC3ORwn7b49bYgnpPWXzEbtykF75vibDf9ri0Rz1C s=; X-IronPort-AV: E=Sophos;i="5.88,299,1635206400"; d="scan'208";a="56374417" To: "seabios @ seabios . org" Date: Wed, 19 Jan 2022 15:21:46 +0100 Message-ID: <20220119142146.21370-1-graf@amazon.com> MIME-Version: 1.0 X-ClientProxiedBy: EX13D24UWA002.ant.amazon.com (10.43.160.200) To EX13D20UWC001.ant.amazon.com (10.43.162.244) Precedence: Bulk Message-ID-Hash: UKLLF2QOC4UFTAQ4EABLTUPWYV2AD5DC X-Message-ID-Hash: UKLLF2QOC4UFTAQ4EABLTUPWYV2AD5DC X-MailFrom: prvs=011ff3d85=graf@amazon.de X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-seabios.seabios.org-0; header-match-seabios.seabios.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: David Woodhouse , Gerd Hoffmann X-Mailman-Version: 3.3.5rc1 Subject: [SeaBIOS] [PATCH] nvme: Move PRP data to ZoneHigh List-Id: SeaBIOS mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Alexander Graf via SeaBIOS Reply-To: Alexander Graf Content-Transfer-Encoding: quoted-printable Authentication-Results: coreboot.org; auth=pass smtp.auth=mailman@coreboot.org smtp.mailfrom=seabios-bounces@seabios.org X-Spamd-Bar: / X-ZohoMail-DKIM: pass (identity @seabios.org) X-ZM-MESSAGEID: 1642602175167100001 Content-Type: text/plain; charset="utf-8" Commit 01f2736cc905d ("nvme: Pass large I/O requests as PRP lists") introduced multi-page requests using the NVMe PRP mechanism. To store the list and "first page to write to" hints, it added fields to the NVMe namespace struct. Unfortunately, that struct resides in fseg which is read-only at runtime. While KVM ignores the read-only part and allows writes, real hardware and TCG adhere to the semantics and ignore writes to the fseg region. The net effect of that is that reads and writes were always happening on address 0, unless they went through the bounce buffer logic. This patch splits moves all PRP maintenance data from the actual namespace allocation and allocates them in ZoneHigh instead. That way, the PRP list is fully read-write at runtime. Fixes: 01f2736cc905d ("nvme: Pass large I/O requests as PRP lists") Reported-by: Matt DeVillier Signed-off-by: Alexander Graf --- src/hw/nvme-int.h | 12 ++++++++---- src/hw/nvme.c | 24 ++++++++++++++++-------- 2 files changed, 24 insertions(+), 12 deletions(-) diff --git a/src/hw/nvme-int.h b/src/hw/nvme-int.h index a4c1555..ceb2c79 100644 --- a/src/hw/nvme-int.h +++ b/src/hw/nvme-int.h @@ -108,6 +108,13 @@ struct nvme_ctrl { struct nvme_cq io_cq; }; =20 +/* Page List Maintenance Data */ +struct nvme_prp_info { + u32 prpl_len; + void *prp1; + u64 prpl[NVME_MAX_PRPL_ENTRIES]; +}; + struct nvme_namespace { struct drive_s drive; struct nvme_ctrl *ctrl; @@ -123,10 +130,7 @@ struct nvme_namespace { /* Page aligned buffer of size NVME_PAGE_SIZE. */ char *dma_buffer; =20 - /* Page List */ - u32 prpl_len; - void *prp1; - u64 prpl[NVME_MAX_PRPL_ENTRIES]; + struct nvme_prp_info *prp; }; =20 /* Data structures for NVMe admin identify commands */ diff --git a/src/hw/nvme.c b/src/hw/nvme.c index f035fa2..963c31e 100644 --- a/src/hw/nvme.c +++ b/src/hw/nvme.c @@ -267,6 +267,14 @@ nvme_probe_ns(struct nvme_ctrl *ctrl, u32 ns_idx, u8 m= dts) ns->ns_id =3D ns_id; ns->lba_count =3D id->nsze; =20 + ns->prp =3D malloc_high(sizeof(*ns->prp)); + if (!ns->prp) { + warn_noalloc(); + free(ns); + goto free_buffer; + } + memset(ns->prp, 0, sizeof(*ns->prp)); + struct nvme_lba_format *fmt =3D &id->lbaf[current_lba_format]; =20 ns->block_size =3D 1U << fmt->lbads; @@ -431,10 +439,10 @@ nvme_io_readwrite(struct nvme_namespace *ns, u64 lba,= char *buf, u16 count, =20 if ((ns->block_size * count) > (NVME_PAGE_SIZE * 2)) { /* We need to describe more than 2 pages, rely on PRP List */ - prp2 =3D ns->prpl; + prp2 =3D ns->prp->prpl; } else if ((ns->block_size * count) > NVME_PAGE_SIZE) { /* Directly embed the 2nd page if we only need 2 pages */ - prp2 =3D (void *)(long)ns->prpl[0]; + prp2 =3D (void *)(long)ns->prp->prpl[0]; } else { /* One page is enough, don't expose anything else */ prp2 =3D NULL; @@ -465,15 +473,15 @@ nvme_io_readwrite(struct nvme_namespace *ns, u64 lba,= char *buf, u16 count, =20 static void nvme_reset_prpl(struct nvme_namespace *ns) { - ns->prpl_len =3D 0; + ns->prp->prpl_len =3D 0; } =20 static int nvme_add_prpl(struct nvme_namespace *ns, u64 base) { - if (ns->prpl_len >=3D NVME_MAX_PRPL_ENTRIES) + if (ns->prp->prpl_len >=3D NVME_MAX_PRPL_ENTRIES) return -1; =20 - ns->prpl[ns->prpl_len++] =3D base; + ns->prp->prpl[ns->prp->prpl_len++] =3D base; =20 return 0; } @@ -492,7 +500,7 @@ static int nvme_build_prpl(struct nvme_namespace *ns, v= oid *op_buf, u16 count) size =3D count * ns->block_size; /* Special case for transfers that fit into PRP1, but are unaligned */ if (((size + (base & ~NVME_PAGE_MASK)) <=3D NVME_PAGE_SIZE)) { - ns->prp1 =3D op_buf; + ns->prp->prp1 =3D op_buf; return count; } =20 @@ -507,7 +515,7 @@ static int nvme_build_prpl(struct nvme_namespace *ns, v= oid *op_buf, u16 count) for (; size > 0; base +=3D NVME_PAGE_SIZE, size -=3D NVME_PAGE_SIZE) { if (first_page) { /* First page is special */ - ns->prp1 =3D (void*)base; + ns->prp->prp1 =3D (void*)base; first_page =3D 0; continue; } @@ -726,7 +734,7 @@ nvme_cmd_readwrite(struct nvme_namespace *ns, struct di= sk_op_s *op, int write) =20 blocks =3D nvme_build_prpl(ns, op_buf, blocks_remaining); if (blocks) { - res =3D nvme_io_readwrite(ns, op->lba + i, ns->prp1, blocks, w= rite); + res =3D nvme_io_readwrite(ns, op->lba + i, ns->prp->prp1, bloc= ks, write); dprintf(5, "ns %u %s lba %llu+%u: %d\n", ns->ns_id, write ? "w= rite" : "r= ead", op->lba, blocks, res); --=20 2.28.0.394.ge197136389 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ SeaBIOS mailing list -- seabios@seabios.org To unsubscribe send an email to seabios-leave@seabios.org