From nobody Tue May 7 01:33:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+51797+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+51797+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1575571514; cv=none; d=zohomail.com; s=zohoarc; b=XDMZZ3JE62c4BGzPsrZ74y6Lk1XLqeQt7ZQ+R67NmpsQy7X3YrN7yXXfu/c5LLBSLlYZIrspBirzLAim4QqeY12FcjPN9Nch6EMhJVXULa5BA69MChKgpOzimj0RSN6ndmPkulBe5MAaPAt/xf8bS3/eQkrm7/i+w96OI/kIt3w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575571514; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Sender:Subject:To; bh=+0GXzW7vOCtsORZlHeQyjtyCVyIzF3toYEKFlnuTozM=; b=PEQaOC69NYEyk2ROoxXGOIjEnNlZrZ/CJ1HmZ/czPBnYHCNLimjjdzNpmsNlcXBkewg0jLiEVLgQ0lqok0yyF5ugQxTbL4EAzyAuRAG4lwi3Do/NeczMxNp5QWdr21OYF+MbsI+AoZ1rGi+W7u2BWkCNhsmvdfVzi//vGdWJJ64= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+51797+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1575571514679812.3480702891766; Thu, 5 Dec 2019 10:45:14 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id b0sJYY1788612xzcvfmIiZDB; Thu, 05 Dec 2019 10:45:13 -0800 X-Received: from us-smtp-delivery-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.61]) by mx.groups.io with SMTP id smtpd.web12.1665.1575571510141361326 for ; Thu, 05 Dec 2019 10:45:10 -0800 X-Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-434-fddmGAIMNoWdsYVbNs_fdQ-1; Thu, 05 Dec 2019 13:45:05 -0500 X-Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C12A28017DF; Thu, 5 Dec 2019 18:45:04 +0000 (UTC) X-Received: from x1w.redhat.com (ovpn-205-76.brq.redhat.com [10.40.205.76]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0E4815E240; Thu, 5 Dec 2019 18:44:59 +0000 (UTC) From: =?UTF-8?B?UGhpbGlwcGUgTWF0aGlldS1EYXVkw6k=?= To: devel@edk2.groups.io Cc: Zhiju Fan , Philippe Mathieu-Daude , Bob Feng , Liming Gao Subject: [edk2-devel] [PATCH v2] BaseTools: Fix Python3 encoding issue in TestTools Date: Thu, 5 Dec 2019 19:44:56 +0100 Message-Id: <20191205184456.17799-1-philmd@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: fddmGAIMNoWdsYVbNs_fdQ-1 X-Mimecast-Spam-Score: 0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,philmd@redhat.com X-Gm-Message-State: qgFhPonYCdrmyPVXgqhYsXYpx1787277AA= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1575571513; bh=xu+Jcbv7K9rxV5x5uhmvtEKOcFrIfeL3KaCIOpyZAWc=; h=Cc:Content-Type:Date:From:Reply-To:Subject:To; b=faFAINmL6Si3hLL5fEPt3tnTMdil19uytu7pCiEVXYIIyD8OhCbKkXe5wXWQ76BjK38 81XM0egSuCjSx/zqQwOVZBmvNZlT9xMvw9s0SXrvY+62dJWn5evxQoElpHzNo+28kQlaA B/pBFGHogo+wrr5RtCEZq0DcjuctVl0I/9c= X-ZohoMail-DKIM: pass (identity @groups.io) Under Centos 7.7 we get: Build environment: \ Linux-3.10.0-1062.7.1.el7.x86_64-x86_64-with-centos-7.7.1908-Core [...] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ERROR: testRandomDataCycles (TianoCompress.Tests) ---------------------------------------------------------------------- Traceback (most recent call last): File "edk2/BaseTools/Tests/TianoCompress.py", line 60, \ in testRandomDataCycles self.compressionTestCycle(data) File "edk2/BaseTools/Tests/TianoCompress.py", line 46, \ in compressionTestCycle start =3D self.ReadTmpFile('input') File "edk2/BaseTools/Tests/TestTools.py", line 139, in ReadTmpFile data =3D f.read() File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: \ ordinal not in range(128) ---------------------------------------------------------------------- Fix by using the 'io' module (per [1]), and specifying the UTF-8 encoding. [1] https://mail.python.org/pipermail/python-list/2015-March/687124.html Cc: Bob Feng Cc: Liming Gao Signed-off-by: Philippe Mathieu-Daude --- v2: Use io, wrap commit message lines per CheckPatch --- BaseTools/Tests/TestTools.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/BaseTools/Tests/TestTools.py b/BaseTools/Tests/TestTools.py index 1099fd4eeaea..8aaad3609669 100644 --- a/BaseTools/Tests/TestTools.py +++ b/BaseTools/Tests/TestTools.py @@ -11,6 +11,7 @@ from __future__ import print_function # Import Modules # import base64 +import io import os import os.path import random @@ -18,7 +19,6 @@ import shutil import subprocess import sys import unittest -import codecs =20 TestsDir =3D os.path.realpath(os.path.split(sys.argv[0])[0]) BaseToolsDir =3D os.path.realpath(os.path.join(TestsDir, '..')) @@ -135,7 +135,7 @@ class BaseToolsTest(unittest.TestCase): return open(os.path.join(self.testDir, fileName), mode) =20 def ReadTmpFile(self, fileName): - f =3D open(self.GetTmpFilePath(fileName), 'r') + f =3D io.open(self.GetTmpFilePath(fileName), 'r', encoding=3D'utf-= 8') data =3D f.read() f.close() return data @@ -145,7 +145,7 @@ class BaseToolsTest(unittest.TestCase): with open(self.GetTmpFilePath(fileName), 'wb') as f: f.write(data) else: - with codecs.open(self.GetTmpFilePath(fileName), 'w', encoding= =3D'utf-8') as f: + with io.open(self.GetTmpFilePath(fileName), 'w', encoding=3D'u= tf-8') as f: f.write(data) =20 def GenRandomFileData(self, fileName, minlen =3D None, maxlen =3D None= ): --=20 2.21.0 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#51797): https://edk2.groups.io/g/devel/message/51797 Mute This Topic: https://groups.io/mt/67154217/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-