From nobody Thu Apr 25 14:24:43 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+53035+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+53035+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1578527004; cv=none; d=zohomail.com; s=zohoarc; b=h270hyLGLZWNjIjDzifgw1SLxzyoiaX4dcRRXhSLCg93+0uWTn4chi+ycwBtXnmGS4NBRbE75sSbtkkqpwqGnJsA77h/68cKjNjbJdXcxIT4yKLbUjM1imqG1ji25kZFAmUTVy6GlWypiGgTNqmP4RFasO0V5AILHspPBjp9xcA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1578527004; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=g2LJubW7F01X/DNyJKnulP+1llAyBY9ZcY8fHoQKBb8=; b=bO/jO2tgNRPn7twTErygmtBtqvZ3Gibmqc4B+0yO5sINny1k55lz2+w3Yz7YmaXIWRxXkwFVUYM5SII4KjVtdjLPdIBmwCJZxgw3YLq9jPGJx85w6tjOmCik1z+zMOB8UyzFFuW94PG66LqDbeBLpq+mcngwzs5UTQucZT574tE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+53035+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1578527004493746.1052506500681; Wed, 8 Jan 2020 15:43:24 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id 9RcHYY1788612xIzlXfJnZQF; Wed, 08 Jan 2020 15:43:24 -0800 X-Received: from us-smtp-delivery-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.81]) by mx.groups.io with SMTP id smtpd.web11.1647.1578527003521986060 for ; Wed, 08 Jan 2020 15:43:23 -0800 X-Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-391-SdRPq9GxNfKZIl8CCIq7sw-1; Wed, 08 Jan 2020 18:43:20 -0500 X-Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 540A4DB20; Wed, 8 Jan 2020 23:43:19 +0000 (UTC) X-Received: from lacos-laptop-7.usersys.redhat.com (ovpn-117-37.ams2.redhat.com [10.36.117.37]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B10F1001B28; Wed, 8 Jan 2020 23:43:17 +0000 (UTC) From: "Laszlo Ersek" To: edk2-devel-groups-io Cc: Hao A Wu , Jian J Wang , Ray Ni , Zhichao Gao Subject: [edk2-devel] [PATCH 1/2] MdeModulePkg/UefiBootManagerLib: log reserved mem allocation failure Date: Thu, 9 Jan 2020 00:43:12 +0100 Message-Id: <20200108234313.28510-2-lersek@redhat.com> In-Reply-To: <20200108234313.28510-1-lersek@redhat.com> References: <20200108234313.28510-1-lersek@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: SdRPq9GxNfKZIl8CCIq7sw-1 X-Mimecast-Spam-Score: 0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,lersek@redhat.com X-Gm-Message-State: g5J2q5etA4CJmV2ydLwGBm0Kx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1578527004; bh=cq1CKJu2yhiUH75jYxIALxU53SjpBYmErj4JS8z4nCE=; h=Cc:Content-Type:Date:From:Reply-To:Subject:To; b=upCcSrFT7xeGnyW4+18wXq4VgHVW6FFlDaBqnfouKIXkwmSu/qTnAUl6uMc6hpFtlct nADNjrcZ7dZQLael5CkxstOLDgQMBXmqMvCoTbnhhEVyvKjquY2TY7tVZvTOdnobfBhkS at5rq2ZQivkCy3CIsR3dW9R5oWtCyaTSppY= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" The LoadFile protocol can report such a large buffer size that we cannot allocate enough reserved pages for. This particularly affects HTTP(S) Boot, if the remote file is very large (for example, an ISO image). While the TianoCore wiki mentions this at : > The maximum RAM disk image size depends on how much continuous reserved > memory block the platform could provide. it's hard to remember; so log a DEBUG_ERROR message when the allocation fails. This patch produces error messages such as: > UiApp:BmExpandLoadFile: failed to allocate reserved pages: > BufferSize=3D4501536768 > LoadFile=3D"PciRoot(0x0)/Pci(0x3,0x0)/MAC(5254001B103E,0x1)/ > IPv4(0.0.0.0,TCP,DHCP,192.168.124.106,192.168.124.1,255.255.255.0)/ > Dns(192.168.124.1)/ > Uri(https://ipv4-server/RHEL-7.7-20190723.1-Server-x86_64-dvd1.iso)" > FilePath=3D"" (Manually rewrapped here for keeping PatchCheck.py happy.) Cc: Hao A Wu Cc: Jian J Wang Cc: Ray Ni Cc: Zhichao Gao Signed-off-by: Laszlo Ersek Acked-by: Hao A Wu Reviewed-by: Philippe Mathieu-Daude Reviewed-by: Siyuan Fu --- MdeModulePkg/Library/UefiBootManagerLib/BmBoot.c | 31 ++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/MdeModulePkg/Library/UefiBootManagerLib/BmBoot.c b/MdeModulePk= g/Library/UefiBootManagerLib/BmBoot.c index 62c5b2dc94ab..540d169ec1a6 100644 --- a/MdeModulePkg/Library/UefiBootManagerLib/BmBoot.c +++ b/MdeModulePkg/Library/UefiBootManagerLib/BmBoot.c @@ -1387,6 +1387,37 @@ BmExpandLoadFile ( // FileBuffer =3D AllocateReservedPages (EFI_SIZE_TO_PAGES (BufferSize)); if (FileBuffer =3D=3D NULL) { + DEBUG_CODE ( + EFI_DEVICE_PATH *LoadFilePath; + CHAR16 *LoadFileText; + CHAR16 *FileText; + + LoadFilePath =3D DevicePathFromHandle (LoadFileHandle); + if (LoadFilePath =3D=3D NULL) { + LoadFileText =3D NULL; + } else { + LoadFileText =3D ConvertDevicePathToText (LoadFilePath, FALSE, FAL= SE); + } + FileText =3D ConvertDevicePathToText (FilePath, FALSE, FALSE); + + DEBUG (( + DEBUG_ERROR, + "%a:%a: failed to allocate reserved pages: " + "BufferSize=3D%Lu LoadFile=3D\"%s\" FilePath=3D\"%s\"\n", + gEfiCallerBaseName, + __FUNCTION__, + (UINT64)BufferSize, + LoadFileText, + FileText + )); + + if (FileText !=3D NULL) { + FreePool (FileText); + } + if (LoadFileText !=3D NULL) { + FreePool (LoadFileText); + } + ); return NULL; } =20 --=20 2.19.1.3.g30247aa5d201 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#53035): https://edk2.groups.io/g/devel/message/53035 Mute This Topic: https://groups.io/mt/69550077/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Thu Apr 25 14:24:43 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+53036+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+53036+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1578527008; cv=none; d=zohomail.com; s=zohoarc; b=nM0BNc4mvXUCDEVy7LT7L9VhmJY0hgDCTnykP66qTrXenBQinPk4m8YfDyJV1AQrtMfT1RxLh333GMlqteLVlRu+nNNGdAsEVggvTm0Rl7YPLzRdyCSKeq39t3B5zFLsoGAIqglQMkVc/YyFyCa0yiyukfBTU5VWPK0dj9XC+kc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1578527008; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=i5iV+h+8OVEKQ7vL9PkD7PBXezuV0Ey5p8Dl6FstHy8=; b=b6wmPQtfpOBgYsxM0/pSATyKiRzYK8B51S2Z9M3bFXVJkDMiJYOLJqRAwpPtNsAijBWebQOvP75X4bK4D8zkNUhV+Pgq6dVq1Qvh8OZlsYrQl4cEjtzzkdhnHaMb6D6naB6qWY8xODW9Z+3zu0GUb4VA0gJKjiubEsp6X+QbKrw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+53036+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1578527008185693.8703467490398; Wed, 8 Jan 2020 15:43:28 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id VaDAYY1788612xhT8qimpSct; Wed, 08 Jan 2020 15:43:27 -0800 X-Received: from us-smtp-delivery-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.61]) by mx.groups.io with SMTP id smtpd.web11.1648.1578527006931151424 for ; Wed, 08 Jan 2020 15:43:27 -0800 X-Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-430-ux_dN25ONq-_H9bGXbxvWg-1; Wed, 08 Jan 2020 18:43:21 -0500 X-Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C504E10054E3; Wed, 8 Jan 2020 23:43:20 +0000 (UTC) X-Received: from lacos-laptop-7.usersys.redhat.com (ovpn-117-37.ams2.redhat.com [10.36.117.37]) by smtp.corp.redhat.com (Postfix) with ESMTP id A8F2A10027A6; Wed, 8 Jan 2020 23:43:19 +0000 (UTC) From: "Laszlo Ersek" To: edk2-devel-groups-io Cc: Jiaxin Wu , Maciej Rabeda , Siyuan Fu Subject: [edk2-devel] [PATCH 2/2] NetworkPkg/HttpDxe: fix 32-bit truncation in HTTPS download Date: Thu, 9 Jan 2020 00:43:13 +0100 Message-Id: <20200108234313.28510-3-lersek@redhat.com> In-Reply-To: <20200108234313.28510-1-lersek@redhat.com> References: <20200108234313.28510-1-lersek@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: ux_dN25ONq-_H9bGXbxvWg-1 X-Mimecast-Spam-Score: 0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,lersek@redhat.com X-Gm-Message-State: QJMNNpFwOMOcyQncH2LrFtoyx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1578527007; bh=IKyr7nzKAW+i63BeOU0SvFL+8AeseVk/7Oy8mLwXrIQ=; h=Cc:Content-Type:Date:From:Reply-To:Subject:To; b=Eg6aEhpxTc3WFrnCjwZk0Et3BRuy9EX6sFq095oAuuyq/wLM10GNgJaqLnCWxOGmLy5 69BZphQQliQ//c0FDtdfWG7g+UJxeMM9qBhp7Fon148f2Xe3oUP6siU8+MYlIxk6/1anB IfQ18uppCrvQOnoplQEJ2WJs2M0IzixI3aY= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" When downloading over TLS, each TLS message ("APP packet") is returned as a (decrypted) fragment table by EFI_TLS_PROTOCOL.ProcessPacket(). The TlsProcessMessage() function in "NetworkPkg/HttpDxe/HttpsSupport.c" linearizes the fragment table into a single contiguous data block. The resultant flat data block contains both TLS headers and data. The HttpsReceive() function parses the actual application data -- in this case: decrypted HTTP data -- out of the flattened TLS data block, peeling off the TLS headers. The HttpResponseWorker() function in "NetworkPkg/HttpDxe/HttpImpl.c" propagates this HTTP data outwards, implementing the EFI_HTTP_PROTOCOL.Response() function. Now consider the following documentation for EFI_HTTP_PROTOCOL.Response(), quoted from "MdePkg/Include/Protocol/Http.h": > It is the responsibility of the caller to allocate a buffer for Body and > specify the size in BodyLength. If the remote host provides a response > that contains a content body, up to BodyLength bytes will be copied from > the receive buffer into Body and BodyLength will be updated with the > amount of bytes received and copied to Body. This allows the client to > download a large file in chunks instead of into one contiguous block of > memory. Note that, if the caller-allocated buffer is larger than the server-provided chunk, then the transfer length is limited by the latter. This is in fact the dominant case when downloading a huge file (for which UefiBootManagerLib allocated a huge contiguous RAM Disk buffer) in small TLS messages. For adjusting BodyLength as described above -- i.e., to the application data chunk that has been extracted from the TLS message --, the HttpResponseWorker() function employs the following assignment: HttpMsg->BodyLength =3D MIN (Fragment.Len, (UINT32) HttpMsg->BodyLength= ); The (UINT32) cast is motivated by the MIN() requirement -- in "MdePkg/Include/Base.h" -- that both arguments be of the same type. "Fragment.Len" (NET_FRAGMENT.Len) has type UINT32, and "HttpMsg->BodyLength" (EFI_HTTP_MESSAGE.BodyLength) has type UINTN. Therefore a cast is indeed necessary. Unfortunately, the cast is done in the wrong direction. Consider the following circumstances: - "Fragment.Len" happens to be consistently 16KiB, dictated by the HTTPS Server's TLS stack, - the size of the file to download is 4GiB + N*16KiB, where N is a positive integer. As the download progresses, each received 16KiB application data chunk brings the *next* input value of BodyLength closer down to 4GiB. The cast in MIN() always masks off the high-order bits from the input value of BodyLength, but this is no problem because the low-order bits are nonzero, therefore the MIN() always permits progress. However, once BodyLength reaches 4GiB exactly on input, the MIN() invocation produces a zero value. HttpResponseWorker() adjusts the output value of BodyLength to zero, and then passes it to HttpParseMessageBody(). HttpParseMessageBody() (in "NetworkPkg/Library/DxeHttpLib/DxeHttpLib.c") rejects the zero BodyLength with EFI_INVALID_PARAMETER, which is fully propagated outwards, and aborts the HTTPS download. HttpBootDxe writes the message "Error: Unexpected network error" to the UEFI console. For example, a file with size (4GiB + 197MiB) terminates after downloading just 197MiB. Invert the direction of the cast: widen "Fragment.Len" to UINTN. Cc: Jiaxin Wu Cc: Maciej Rabeda Cc: Siyuan Fu Signed-off-by: Laszlo Ersek Reviewed-by: Maciej Rabeda Reviewed-by: Philippe Mathieu-Daude Reviewed-by: Siyuan Fu --- NetworkPkg/HttpDxe/HttpImpl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/NetworkPkg/HttpDxe/HttpImpl.c b/NetworkPkg/HttpDxe/HttpImpl.c index 6b877314bd57..1acbb60d1014 100644 --- a/NetworkPkg/HttpDxe/HttpImpl.c +++ b/NetworkPkg/HttpDxe/HttpImpl.c @@ -1348,7 +1348,7 @@ HttpResponseWorker ( // // Process the received the body packet. // - HttpMsg->BodyLength =3D MIN (Fragment.Len, (UINT32) HttpMsg->BodyLengt= h); + HttpMsg->BodyLength =3D MIN ((UINTN) Fragment.Len, HttpMsg->BodyLength= ); =20 CopyMem (HttpMsg->Body, Fragment.Bulk, HttpMsg->BodyLength); =20 --=20 2.19.1.3.g30247aa5d201 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#53036): https://edk2.groups.io/g/devel/message/53036 Mute This Topic: https://groups.io/mt/69550085/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-