From nobody Mon Feb 9 12:17:17 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of _spf.google.com designates 209.85.221.49 as permitted sender) client-ip=209.85.221.49; envelope-from=philippe.mathieu.daude@gmail.com; helo=mail-wr1-f49.google.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of _spf.google.com designates 209.85.221.49 as permitted sender) smtp.mailfrom=philippe.mathieu.daude@gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1610129396; cv=none; d=zohomail.com; s=zohoarc; b=GHOKb/TrNaJzaNikVc6AUccIGxLhAd8yMuDBmYTCQMkumxJ5+UxMyC9EO9lRmv5Hrm2ARDue5w5Gl5gtl+jOBq4x3zkBEoytf/XEtLp9PsVpyfuKl4xI//At6sExhfQYEjEWTSRfTopExQ0FmGiHe3FRbs+teYIoiBRuVAwiD3s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610129396; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:MIME-Version:Message-ID:Sender:Subject:To; bh=u12OZGwUYSRhqf5Zl/ovMgA/ZyRB7aMBFfpQgY5/rgs=; b=jkfeI3/P0JqQ2YZ56XzO8+eM0cgblqgQd2dVcjwbLCdDbAYila2FTqel8JZs9+iMU3qBv6T5RPZ69aKazjTwVGG9DkXMC7rqsEgnlckbhlA52L162ok9c4j7iuNfwT/pByzdx5hzaU+gLhUuXhkovs0g4uxpcLrvgR7Xv5qIrP8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of _spf.google.com designates 209.85.221.49 as permitted sender) smtp.mailfrom=philippe.mathieu.daude@gmail.com Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by mx.zohomail.com with SMTPS id 1610129396815661.3115673629361; Fri, 8 Jan 2021 10:09:56 -0800 (PST) Received: by mail-wr1-f49.google.com with SMTP id q18so9901460wrn.1 for ; Fri, 08 Jan 2021 10:09:56 -0800 (PST) Return-Path: Return-Path: Received: from x1w.redhat.com (241.red-88-10-103.dynamicip.rima-tde.net. [88.10.103.241]) by smtp.gmail.com with ESMTPSA id o83sm12820962wme.21.2021.01.08.10.09.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 10:09:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=u12OZGwUYSRhqf5Zl/ovMgA/ZyRB7aMBFfpQgY5/rgs=; b=ZW0j5P/7jTbMSvIYOA6zJCcnIJHg1JH4jI7S5hUmGGGebfRLR3vDU1N9xfLvWp1iKm 5Ct9FUIGV0uVXTuT4NNcHjKZxMNTeZfh/uJvuz1hnq8cDTRg6VpfwMXxkBMtR+HmKv1t yWtlTExymIvvwCuHqNNeDYi2OZTfovjUeSGceC47kr1TZ3cKRJI/6Y+CqsKRux7LjRWX /MZHTyygIQChISWmi/kh/byyOVSXz5tN0R8es91T+q2YOQ4cAoZjZRpNiOVsI7wppT/C +kvdlYFQjtE8nEz+/cJS3eozq6Ef9Q+EipJMGM5e0hc+MwnreLRwhUnNDHizfTof8jYt ZVOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :mime-version:content-transfer-encoding; bh=u12OZGwUYSRhqf5Zl/ovMgA/ZyRB7aMBFfpQgY5/rgs=; b=FBJbm7Xqq3GFCZb+/cgRBlBfghb8TZ/CO1kZPdlm0tqaKxtYyOVDJnhooceFtKG/Tp asPvBqbP8itc7Xelg2jpFqQ7CeedtVYqw19bdAPG6Sk3v5WYUlHtHxWbDEHVAxcaId1r /x3iujtU3UYfGNfUdQtE8oIYr1dlhwjoo699dKWS4wMKYZzvB93cP3KMPGIdd4Bu9JG8 weG0XYwgr151AdDIoZ+ZoB+x+f/TmiFl3817UI1fQszA2iPTAdkO8XvkLHn6bNvUZ0Uq NWESaj8rh2a1crVgcFurv54SejtdPKXhX11W+b2mcJn3i4OfcveN+zMFfLf3rrjz4kr3 kLrA== X-Gm-Message-State: AOAM530hQ9w/6aXSgVTO6f3zoTwJEFlxDjYxt4nMxxHZz4thTKngwrtk bizYLIkt6zsEztDZJ6y6Ies= X-Google-Smtp-Source: ABdhPJzzv3DZv/ZKzEcDGOpOPckEkP0BuiKkZxsRGFIqrrJ0f7YhEq+tnqkEl6lz2OE9/l+PE7y71w== X-Received: by 2002:a5d:6ccb:: with SMTP id c11mr4813607wrc.224.1610129395003; Fri, 08 Jan 2021 10:09:55 -0800 (PST) Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Yonggang Luo , Richard Henderson , Cleber Rosa , Eduardo Habkost , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Maydell Subject: [PATCH v2] decodetree: Open files with encoding='utf-8' Date: Fri, 8 Jan 2021 19:09:52 +0100 Message-Id: <20210108180952.358947-1-f4bug@amsat.org> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) When decodetree.py was added in commit 568ae7efae7, QEMU was using Python 2 which happily reads UTF-8 files in text mode. Python 3 requires either UTF-8 locale or an explicit encoding passed to open(). Now that Python 3 is required, explicit UTF-8 encoding for decodetree source files. To avoid further problems with the user locale, also explicit UTF-8 encoding for the generated C files. Explicit both input/output are plain text by using the 't' mode. This fixes: $ /usr/bin/python3 scripts/decodetree.py test.decode Traceback (most recent call last): File "scripts/decodetree.py", line 1397, in main() File "scripts/decodetree.py", line 1308, in main parse_file(f, toppat) File "scripts/decodetree.py", line 994, in parse_file for line in f: File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 80: ordinal not in range(128) Reported-by: Peter Maydell Signed-off-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Eduardo Habkost --- v2: utf-8 output too (Peter) explicit default text mode. --- scripts/decodetree.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/decodetree.py b/scripts/decodetree.py index 47aa9caf6d1..d3857066cfc 100644 --- a/scripts/decodetree.py +++ b/scripts/decodetree.py @@ -1304,7 +1304,7 @@ def main(): =20 for filename in args: input_file =3D filename - f =3D open(filename, 'r') + f =3D open(filename, 'rt', encoding=3D'utf-8') parse_file(f, toppat) f.close() =20 @@ -1324,7 +1324,7 @@ def main(): prop_size(stree) =20 if output_file: - output_fd =3D open(output_file, 'w') + output_fd =3D open(output_file, 'wt', encoding=3D'utf-8') else: output_fd =3D sys.stdout =20 --=20 2.26.2