From nobody Wed Feb 11 03:02:10 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529682354492211.25904703568278; Fri, 22 Jun 2018 08:45:54 -0700 (PDT) Received: from localhost ([::1]:34825 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fWOG1-0002SQ-P2 for importer@patchew.org; Fri, 22 Jun 2018 11:45:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56701) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fWO38-0000pj-9q for qemu-devel@nongnu.org; Fri, 22 Jun 2018 11:32:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fWO35-0000J0-OO for qemu-devel@nongnu.org; Fri, 22 Jun 2018 11:32:34 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:59536 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fWO35-0000IJ-JG for qemu-devel@nongnu.org; Fri, 22 Jun 2018 11:32:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2AA8D4023336 for ; Fri, 22 Jun 2018 15:32:31 +0000 (UTC) Received: from blackfin.pond.sub.org (ovpn-117-237.ams2.redhat.com [10.36.117.237]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 07CE81C71D for ; Fri, 22 Jun 2018 15:32:31 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id C9B8D1132D72; Fri, 22 Jun 2018 17:32:28 +0200 (CEST) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Fri, 22 Jun 2018 17:32:25 +0200 Message-Id: <20180622153228.23594-8-armbru@redhat.com> In-Reply-To: <20180622153228.23594-1-armbru@redhat.com> References: <20180622153228.23594-1-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 22 Jun 2018 15:32:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 22 Jun 2018 15:32:31 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'armbru@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 07/10] qapi: Open files with encoding='utf-8' X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Python 2 happily reads UTF-8 files in text mode, but Python 3 requires either UTF-8 locale or an explicit encoding passed to open(). Commit d4e5ec877ca fixed this by setting the en_US.UTF-8 locale. Falls apart when the locale isn't be available. Matthias Maier and Arfrever Frehtes Taifersar Arahesis proposed to use binary mode instead, with manual conversion from bytes to str. Works, but opening with an explicit encoding is simpler, so do that. Since Python 2's open() doesn't support the encoding parameter, we need to suppress it with a version check. Reported-by: Arfrever Frehtes Taifersar Arahesis Reported-by: Matthias Maier Signed-off-by: Markus Armbruster Message-Id: <20180618175958.29073-2-armbru@redhat.com> Reviewed-by: Eduardo Habkost Reviewed-by: Eric Blake --- scripts/qapi/common.py | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py index 4b53f08627..8b6708dbf1 100644 --- a/scripts/qapi/common.py +++ b/scripts/qapi/common.py @@ -16,6 +16,7 @@ import errno import os import re import string +import sys from collections import OrderedDict =20 builtin_types =3D { @@ -340,7 +341,10 @@ class QAPISchemaParser(object): return None =20 try: - fobj =3D open(incl_fname, 'r') + if sys.version_info[0] >=3D 3: + fobj =3D open(incl_fname, 'r', encoding=3D'utf-8') + else: + fobj =3D open(incl_fname, 'r') except IOError as e: raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname)) return QAPISchemaParser(fobj, previously_included, info) @@ -1493,7 +1497,11 @@ class QAPISchemaEvent(QAPISchemaEntity): class QAPISchema(object): def __init__(self, fname): self._fname =3D fname - parser =3D QAPISchemaParser(open(fname, 'r')) + if sys.version_info[0] >=3D 3: + f =3D open(fname, 'r', encoding=3D'utf-8') + else: + f =3D open(fname, 'r') + parser =3D QAPISchemaParser(f) exprs =3D check_exprs(parser.exprs) self.docs =3D parser.docs self._entity_list =3D [] @@ -2007,7 +2015,10 @@ class QAPIGen(object): if e.errno !=3D errno.EEXIST: raise fd =3D os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666) - f =3D os.fdopen(fd, 'r+') + if sys.version_info[0] >=3D 3: + f =3D open(fd, 'r+', encoding=3D'utf-8') + else: + f =3D os.fdopen(fd, 'r+') text =3D (self._top(fname) + self._preamble + self._body + self._bottom(fname)) oldtext =3D f.read(len(text) + 1) --=20 2.17.1