From nobody Tue Nov 4 15:31:23 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1530066000528521.6256704143284; Tue, 26 Jun 2018 19:20:00 -0700 (PDT) Received: from localhost ([::1]:56145 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fY03r-0001Uw-OZ for importer@patchew.org; Tue, 26 Jun 2018 22:19:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38661) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fXzz3-0006yx-Ma for qemu-devel@nongnu.org; Tue, 26 Jun 2018 22:15:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fXzz2-0002G2-BR for qemu-devel@nongnu.org; Tue, 26 Jun 2018 22:15:01 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52072) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fXzz2-0002Fg-2s for qemu-devel@nongnu.org; Tue, 26 Jun 2018 22:15:00 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6580330820D5; Wed, 27 Jun 2018 02:14:59 +0000 (UTC) Received: from localhost (ovpn-116-35.gru2.redhat.com [10.97.116.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8504B20155E4; Wed, 27 Jun 2018 02:14:53 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Tue, 26 Jun 2018 23:14:20 -0300 Message-Id: <20180627021423.18404-4-ehabkost@redhat.com> In-Reply-To: <20180627021423.18404-1-ehabkost@redhat.com> References: <20180627021423.18404-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.25 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Wed, 27 Jun 2018 02:14:59 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 3/6] docker: Add type annotations X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Cleber Rosa , =?UTF-8?q?Alex=20Benn=C3=A9e?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add type annotations that indicate how the code works today, to make the conversion to Python 3 easier and safer. With these type annotations, "mypy -2" is not reporting any issues, but "mypy" in Python 3 mode reports a few problems: tests/docker/docker.py:233: error: Argument 1 to "_text_checksum" has incom= patible type "str"; expected "bytes" tests/docker/docker.py:358: error: "_Environ[str]" has no attribute "iterit= ems" tests/docker/docker.py:360: error: Argument 3 to "build_image" of "Docker" = has incompatible type "bytes"; expected "str" These problems will be addressed by the following commits. Signed-off-by: Eduardo Habkost --- tests/docker/docker.py | 44 +++++++++++++++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 7 deletions(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index e3bfa1cc9e..db6b463b92 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -29,6 +29,12 @@ from shutil import copy, rmtree from pwd import getpwuid from datetime import datetime,timedelta =20 +try: + from typing import List, Union, Tuple +except ImportError: + # needed only to make type annotations work + pass + =20 FILTERED_ENV_NAMES =3D ['ftp_proxy', 'http_proxy', 'https_proxy'] =20 @@ -37,13 +43,16 @@ DEVNULL =3D open(os.devnull, 'wb') =20 =20 def _text_checksum(text): + # type: (bytes) -> str """Calculate a digest string unique to the text content""" return hashlib.sha1(text).hexdigest() =20 def _file_checksum(filename): + # type: (str) -> str return _text_checksum(open(filename, 'rb').read()) =20 def _guess_docker_command(): + # type: () -> List[str] """ Guess a working docker command or raise exception if not found""" commands =3D [["docker"], ["sudo", "-n", "docker"]] for cmd in commands: @@ -60,6 +69,7 @@ def _guess_docker_command(): commands_txt) =20 def _copy_with_mkdir(src, root_dir, sub_path=3D'.'): + # type: (str, str, str) -> None """Copy src into root_dir, creating sub_path as needed.""" dest_dir =3D os.path.normpath("%s/%s" % (root_dir, sub_path)) try: @@ -73,6 +83,7 @@ def _copy_with_mkdir(src, root_dir, sub_path=3D'.'): =20 =20 def _get_so_libs(executable): + # type: (str) -> List[str] """Return a list of libraries associated with an executable. =20 The paths may be symbolic links which would need to be resolved to @@ -94,6 +105,7 @@ def _get_so_libs(executable): return libs =20 def _copy_binary_with_libs(src, dest_dir): + # type: (str, str) -> None """Copy a binary executable and all its dependant libraries. =20 This does rely on the host file-system being fairly multi-arch @@ -108,11 +120,13 @@ def _copy_binary_with_libs(src, dest_dir): _copy_with_mkdir(l , dest_dir, so_path) =20 def _read_qemu_dockerfile(img_name): + # type: (str) -> str df =3D os.path.join(os.path.dirname(__file__), "dockerfiles", img_name + ".docker") return open(df, "r").read() =20 def _dockerfile_preprocess(df): + # type: (str) -> str out =3D "" for l in df.splitlines(): if len(l.strip()) =3D=3D 0 or l.startswith("#"): @@ -194,11 +208,16 @@ class Docker(object): labels =3D json.loads(resp)[0]["Config"].get("Labels", {}) return labels.get("com.qemu.dockerfile-checksum", "") =20 - def build_image(self, tag, docker_dir, dockerfile, - quiet=3DTrue, user=3DFalse, argv=3DNone, extra_files_c= ksum=3D[]): - if argv =3D=3D None: - argv =3D [] - + def build_image(self, + tag, # type: str + docker_dir, # type: str + dockerfile, # type: str + quiet=3DTrue, # type: bool + user=3DFalse, # type: bool + argv=3D[], # type: List[str] + extra_files_cksum=3D[] # List[Tuple[str, bytes]] + ): + # type(...) -> None tmp_df =3D tempfile.NamedTemporaryFile(dir=3Ddocker_dir, suffix=3D= ".docker") tmp_df.write(dockerfile) =20 @@ -249,7 +268,8 @@ class Docker(object): =20 class SubCommand(object): """A SubCommand template base class""" - name =3D None # Subcommand name + # Subcommand name + name =3D None # type: str def shared_args(self, parser): parser.add_argument("--quiet", action=3D"store_true", help=3D"Run quietly unless an error occured") @@ -258,6 +278,7 @@ class SubCommand(object): """Setup argument parser""" pass def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int """Run command. args: parsed argument by argument parser. argv: remaining arguments from sys.argv. @@ -271,6 +292,7 @@ class RunCommand(SubCommand): parser.add_argument("--keep", action=3D"store_true", help=3D"Don't remove image when command comple= tes") def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int return Docker().run(argv, args.keep, quiet=3Dargs.quiet) =20 class BuildCommand(SubCommand): @@ -294,6 +316,7 @@ class BuildCommand(SubCommand): help=3D"Dockerfile name") =20 def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int dockerfile =3D open(args.dockerfile, "rb").read() tag =3D args.tag =20 @@ -321,7 +344,7 @@ class BuildCommand(SubCommand): =20 # Copy any extra files into the Docker context. These can be # included by the use of the ADD directive in the Dockerfile. - cksum =3D [] + cksum =3D [] # type: List[Tuple[bytes, str]] if args.include_executable: # FIXME: there is no checksum of this executable and the l= inked # libraries, once the image built any change of this execu= table @@ -352,6 +375,7 @@ class UpdateCommand(SubCommand): help=3D"Executable to copy") =20 def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int # Create a temporary tarball with our whole build context and # dockerfile for the update tmp =3D tempfile.NamedTemporaryFile(suffix=3D"dckr.tar.gz") @@ -394,6 +418,7 @@ class CleanCommand(SubCommand): """Clean up docker instances""" name =3D "clean" def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int Docker().clean() return 0 =20 @@ -401,6 +426,7 @@ class ImagesCommand(SubCommand): """Run "docker images" command""" name =3D "images" def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int return Docker().command("images", argv, args.quiet) =20 =20 @@ -409,6 +435,7 @@ class ProbeCommand(SubCommand): name =3D "probe" =20 def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int try: docker =3D Docker() if docker._command[0] =3D=3D "docker": @@ -437,6 +464,7 @@ class CcCommand(SubCommand): reading sources""") =20 def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int if argv and argv[0] =3D=3D "--": argv =3D argv[1:] cwd =3D os.getcwd() @@ -468,6 +496,7 @@ class CheckCommand(SubCommand): help=3D"number of minutes") =20 def run(self, args, argv): + # type: (argparse.Namespace, List[str]) -> int tag =3D args.tag =20 dkr =3D Docker() @@ -506,6 +535,7 @@ class CheckCommand(SubCommand): return 0 =20 def main(): + # type: () -> int parser =3D argparse.ArgumentParser(description=3D"A Docker helper", usage=3D"%s ..." % os.path.basename(sys.argv[0])) subparsers =3D parser.add_subparsers(title=3D"subcommands", help=3DNon= e) --=20 2.18.0.rc1.1.g3f1ff2140