From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327501028452.2123681338303; Mon, 18 Jun 2018 06:11:41 -0700 (PDT) Received: from localhost ([::1]:34730 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtwa-0000qx-6y for importer@patchew.org; Mon, 18 Jun 2018 09:11:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52410) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsJ-0005y3-OD for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsG-0002d8-Tx for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:15 -0400 Received: from mail-wr0-x229.google.com ([2a00:1450:400c:c0c::229]:35100) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsG-0002cj-OM for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:12 -0400 Received: by mail-wr0-x229.google.com with SMTP id l10-v6so16790011wrn.2 for ; Mon, 18 Jun 2018 06:07:12 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id c11-v6sm15253238wrm.65.2018.06.18.06.07.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:10 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 4BD893E05F2; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NfJ5U5jIFnzXeSOLBje2WDYfem2f4jhgHe0dDM3WbUA=; b=WjzsTLA+c4Z6qB/XcnO9EFXJqfwGPFtbKBazBGL+0fGRbl/A3x/LeEDEane7Ga8wGu hg54A76vz0iADoDUJUTQcdB8l3TDnjb4nuwmNawn+wtlYt9ggUQnMlkMwFiCruv5qqCG Rb0HC8KR/aOLD2fYxOZOqQ2JbtyahACFVcqco= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NfJ5U5jIFnzXeSOLBje2WDYfem2f4jhgHe0dDM3WbUA=; b=bQc4Ox7AxRZXgFdzd2SGmkOoCPGp+RsYn3bC+UEK2TVMB088uRRadq3A8D9Pcv00j2 po4nzM1sdsAZZdIhFPzlqh9SOLBmgaMr1S/bT32EV5JW7wpVjObD7c7i1/uIHHqLh6rm aLOzE1odyRxgbHfe1Fdv+7BymaMRmpKsnrXAX4QzUhByIq08nrnYNb3Ve+JPXIHsWb9c ak95GFTBRIM7+Dv2+HH820jph4L8974rxhwHFUVsIZ8DWgi7mfy5EtwW4FVz91S5XiXD C5BxZaxJtsLRm32MLFO5W3dnbUYcjcr4cGfG7L+5aHMm1hC7j/9UO29N5ohcsjhFSdvQ IYgQ== X-Gm-Message-State: APt69E3TGF4ncxfb+KyurwBexjhaO5YBBJ19ZV1PI9f/ThvLH3LOFLRQ 3UkmxNjC06lIcq5gik+UnxOkEg== X-Google-Smtp-Source: ADUXVKIZld2jFkcLp8rEfOeqOWnIlSUHRLeNN1RnHAcPoa7XVgV0y2DQpmKUYMNGhM73Zyda3WCHGQ== X-Received: by 2002:adf:a706:: with SMTP id c6-v6mr10767272wrd.61.1529327231705; Mon, 18 Jun 2018 06:07:11 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:02 +0100 Message-Id: <20180618130709.19730-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::229 Subject: [Qemu-devel] [PATCH v1 1/8] docker: Add "cc" subcommand X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Signed-off-by: Fam Zheng [AJB: add if args.paths check] Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- v2 - add if args.paths check to avoid iterating null argument --- tests/docker/docker.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 306e14cf69..4cd6f9b8ec 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -409,6 +409,31 @@ class ProbeCommand(SubCommand): return =20 =20 +class CcCommand(SubCommand): + """Compile sources with cc in images""" + name =3D "cc" + + def args(self, parser): + parser.add_argument("--image", "-i", required=3DTrue, + help=3D"The docker image in which to run cc") + parser.add_argument("--source-path", "-s", nargs=3D"*", dest=3D"pa= ths", + help=3D"""Extra paths to (ro) mount into conta= iner for + reading sources""") + + def run(self, args, argv): + if argv and argv[0] =3D=3D "--": + argv =3D argv[1:] + cwd =3D os.getcwd() + cmd =3D ["--rm", "-w", cwd, + "-v", "%s:%s:rw" % (cwd, cwd)] + if args.paths: + for p in args.paths: + cmd +=3D ["-v", "%s:%s:ro,z" % (p, p)] + cmd +=3D [args.image, "cc"] + cmd +=3D argv + return Docker().command("run", cmd, args.quiet) + + def main(): parser =3D argparse.ArgumentParser(description=3D"A Docker helper", usage=3D"%s ..." % os.path.basename(sys.argv[0])) --=20 2.17.1 From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327366496463.0055831057963; Mon, 18 Jun 2018 06:09:26 -0700 (PDT) Received: from localhost ([::1]:34715 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtuL-00077W-PS for importer@patchew.org; Mon, 18 Jun 2018 09:09:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52411) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsJ-0005y4-OA for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsH-0002dU-Rb for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:15 -0400 Received: from mail-wr0-x242.google.com ([2a00:1450:400c:c0c::242]:40646) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsH-0002d2-F7 for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:13 -0400 Received: by mail-wr0-x242.google.com with SMTP id l41-v6so16770321wre.7 for ; Mon, 18 Jun 2018 06:07:13 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id i6-v6sm15391392wrr.2.2018.06.18.06.07.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:10 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 624403E0632; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0TZGa3eVh0CeTpRHUcHQmNG5wlIovi0NoSw/xt6gy/U=; b=irzfAmaCUtsINFGqMF0nmF57eF9Wd0WzILPh+fRlWx+duRqIyU6FErI66eisBT/3MO luDTk2eJy2Clvrl9lMSUsWRdEO+QY+gJ908TgXZjBVjP1CRHNlLQ5GkMY3JwhOEv7Xq6 k4zgcybjeJMekBB+E3PUZ8Y8AUR6XcSuMOdyQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0TZGa3eVh0CeTpRHUcHQmNG5wlIovi0NoSw/xt6gy/U=; b=uYiq8N+4HMGc/IQdQe3qiNUOYB5W9j+Yd3LoMTqzy16iDZbpvK45DphdrIRKJ9rYig geNk5o/iurZ5kOcnz4Y0u0cpHzbrnPuZEgKoplNl+jieu+G40HadKt/LcLDkZAFPPtUk xZohZbzwP8uKtLxMNfFqyof0E+LdEDGahownEfWWZvAJJYdIiwE01H7jVRMfgO4W+Y7r B0fww7bFJygMzZy6MpPEv5jYro4UytCHxDfvffmdJCoB0tWaBrfxg82H0S+2RiX556s2 0VpFJGFngu6/QV43RrZM6Y8hdfwA81dsvuabu+G14csBVOtVWcwuL53kCnDd6NKbaoEW +6TA== X-Gm-Message-State: APt69E0FlTDyEvJ3fEArT2D7lf2MLBNSnkmHF4ch5plQ8c1cVKsGCY/r NGsnINwvF9uHpSJaoqYSs9zR0Q== X-Google-Smtp-Source: ADUXVKI1hgqH258YvB0Cl5AQ0Iz5Q8YroeH9Yrkrh90bD6XfwnWG7vJ6RAZWMBLGCHGH5/Rm0VMO6A== X-Received: by 2002:a5d:4392:: with SMTP id i18-v6mr10926092wrq.156.1529327232426; Mon, 18 Jun 2018 06:07:12 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:03 +0100 Message-Id: <20180618130709.19730-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::242 Subject: [Qemu-devel] [PATCH v1 2/8] docker: extend "cc" command to accept compiler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 When calling our cross-compilation images we want to call something other than the default cc. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Fam Zheng --- v2 - use arg.cc default to simplify logic --- tests/docker/docker.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 4cd6f9b8ec..663dfcba4b 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -416,6 +416,8 @@ class CcCommand(SubCommand): def args(self, parser): parser.add_argument("--image", "-i", required=3DTrue, help=3D"The docker image in which to run cc") + parser.add_argument("--cc", default=3D"cc", + help=3D"The compiler executable to call") parser.add_argument("--source-path", "-s", nargs=3D"*", dest=3D"pa= ths", help=3D"""Extra paths to (ro) mount into conta= iner for reading sources""") @@ -429,7 +431,7 @@ class CcCommand(SubCommand): if args.paths: for p in args.paths: cmd +=3D ["-v", "%s:%s:ro,z" % (p, p)] - cmd +=3D [args.image, "cc"] + cmd +=3D [args.image, args.cc] cmd +=3D argv return Docker().command("run", cmd, args.quiet) =20 --=20 2.17.1 From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327512542852.6338810320502; Mon, 18 Jun 2018 06:11:52 -0700 (PDT) Received: from localhost ([::1]:34732 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtwl-0000zP-Qh for importer@patchew.org; Mon, 18 Jun 2018 09:11:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52459) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsN-0005yH-At for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsI-0002dp-Id for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:19 -0400 Received: from mail-wm0-x243.google.com ([2a00:1450:400c:c09::243]:40713) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsI-0002dM-Bs for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:14 -0400 Received: by mail-wm0-x243.google.com with SMTP id n5-v6so15434209wmc.5 for ; Mon, 18 Jun 2018 06:07:14 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id o12-v6sm30214854wrf.30.2018.06.18.06.07.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:10 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 77B0A3E0634; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RSP5eqkQNwptuq4lPD33K39JzkY2J7sMdSmKB/i3gy0=; b=CPEl4JH/QX1/3CMZoDivBnph1aGz0yWD7ZqNS+QyCJEz0cTIj6bQbFuOgTB/YXe3Bt wNIOAXdbv1KLNu3YeRComYzbSeRFGwC58ohGwJ6xPWke2gwp82+8TeZkid1/2njJ0Sfg Udg76V2wvYoA6PaD8DBh5dKiWyWAWp0KJGKkQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RSP5eqkQNwptuq4lPD33K39JzkY2J7sMdSmKB/i3gy0=; b=l5xbFXVmGedvnOAUXwAhlu3r3wfmDfn5EeXWZM+2OUPWwCMp83VwaIy/d2fKdWY4Mf RdZ0fo8X1Ee9CKhM0v5SRfEGiyOUoSQIns8nOSEiub2RmVhtGaRfulvgEpbpjYacf7eK wuMLvgHpNjWLKCczgjzKBk+DYD1+II+ohIO0rucFIchBWgZfeRJ4MaepEOmfFRFHRuef aLIIu0XGtDr9PJ6LSVv1WrRzgiDC7/btDgiP0QXIrB9vkCuCK+g/8NdH4pLEqFMyKMXH RXG97NXOAgxBdg2GroyRJr8IQhGMQCy8Wp9blQondS/Cmwte5T/NAB9hXwZTrgVrrhtp 5jag== X-Gm-Message-State: APt69E057yl1G+XDrnltgwt/EwMgU3wEHrCIJI9G7h29n8vM/b9iW/Wb 0cyZEtDewIL6w+NhhR89aO0TJw== X-Google-Smtp-Source: ADUXVKLqNyEgbnOqCnpzBNr0gUei6G31bJxKSUk5MosqcX3ifP2SZFYseVYhoYmDbHV51xkTX4cmfQ== X-Received: by 2002:a1c:3ac6:: with SMTP id h189-v6mr9424543wma.64.1529327233298; Mon, 18 Jun 2018 06:07:13 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:04 +0100 Message-Id: <20180618130709.19730-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::243 Subject: [Qemu-devel] [PATCH v1 3/8] docker: allow "cc" command to run in user context X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Fam Zheng Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- tests/docker/docker.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 663dfcba4b..c83fd89985 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -418,6 +418,8 @@ class CcCommand(SubCommand): help=3D"The docker image in which to run cc") parser.add_argument("--cc", default=3D"cc", help=3D"The compiler executable to call") + parser.add_argument("--user", + help=3D"The user-id to run under") parser.add_argument("--source-path", "-s", nargs=3D"*", dest=3D"pa= ths", help=3D"""Extra paths to (ro) mount into conta= iner for reading sources""") @@ -431,6 +433,8 @@ class CcCommand(SubCommand): if args.paths: for p in args.paths: cmd +=3D ["-v", "%s:%s:ro,z" % (p, p)] + if args.user: + cmd +=3D ["-u", args.user] cmd +=3D [args.image, args.cc] cmd +=3D argv return Docker().command("run", cmd, args.quiet) --=20 2.17.1 From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327505902638.2701052357867; Mon, 18 Jun 2018 06:11:45 -0700 (PDT) Received: from localhost ([::1]:34731 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtwf-0000tP-7E for importer@patchew.org; Mon, 18 Jun 2018 09:11:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52427) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsK-0005y5-Kn for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsJ-0002eD-GW for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:16 -0400 Received: from mail-wr0-x230.google.com ([2a00:1450:400c:c0c::230]:35107) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsJ-0002dt-9D for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:15 -0400 Received: by mail-wr0-x230.google.com with SMTP id l10-v6so16790189wrn.2 for ; Mon, 18 Jun 2018 06:07:15 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id j131-v6sm8835745wmb.35.2018.06.18.06.07.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:10 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 8FA053E063A; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mQkbtycqgecvtKajhSQdJnVmb+umLT7Ikvt2yrGdEKw=; b=arRpnv/CE6N6xQpwn/Pvym1fpQwYj0GapU3vNlv7m1LJr+HzU0yqlV+fUlHA85jO7h sR5NOBk0bZMe1BakZYZewcrPoLf7ckG15bdcIz8tkD7puDLniyMJE9zQ2IDtQI4IlfHc qrnQSgbQioNrqH9x+rmYfDrpc64FErwm+HO8k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQkbtycqgecvtKajhSQdJnVmb+umLT7Ikvt2yrGdEKw=; b=XkWP5j6KWw952oGazjHiefrbTp0vtDa00gp33QUrVSijUFRWTqY/jikt+mYusZCoA4 pRXquN4sRenoXWJorJUBrW2O09+mSQIPGMXx0rws0oyuxLntgwxS33L1m3qgE3rYEky7 bdxvh9ui5+O63yR/d/MAASqr6YEdzi6nTqHqC1jQLijokT1P/rwnStK9gwQsMmoc4HvI PYoOPwvEHVBKC2bt+ggQBmo+3rvb6ApWrkWxS0BhFdQX4r6fzwsJKW+RoEGgi70pX5z3 pJ2cEt/ba9mGvpT7/R+Xn1d7+RNkI0c9q+CgVbO3SMPpSI0KDkzgEG2IEE40G+/KKYZZ 2bjg== X-Gm-Message-State: APt69E0Rw0Yiqpi5czlvfceD1SIJV+txBdltThuvKWFjoLZdTlvivYPB SEnp+zA5x0bayiNVUC4lOTLB8g== X-Google-Smtp-Source: ADUXVKJlva9UeILKp/e8UodDyCfHo2ThOFZMqOD5K4vDlp2v2sKHI5r2gE53viqBPZEuA3+jJorJ9w== X-Received: by 2002:adf:a581:: with SMTP id g1-v6mr10874863wrc.233.1529327234247; Mon, 18 Jun 2018 06:07:14 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:05 +0100 Message-Id: <20180618130709.19730-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::230 Subject: [Qemu-devel] [PATCH v1 4/8] docker: Makefile.include introduce DOCKER_SCRIPT X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Define this in one place to make it easy to re-use. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/docker/Makefile.include | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 74fd51c22c..8afb383478 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -15,6 +15,8 @@ DOCKER_TESTS :=3D $(notdir $(shell \ =20 DOCKER_TOOLS :=3D travis =20 +DOCKER_SCRIPT=3D$(SRC_PATH)/tests/docker/docker.py + TESTS ?=3D % IMAGES ?=3D % =20 @@ -38,7 +40,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker echo WARNING: EXECUTABLE is not set, debootstrap may fail. 2>&1 ; \ fi $(call quiet-command,\ - $(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \ + $(DOCKER_SCRIPT) build qemu:$* $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ @@ -133,11 +135,11 @@ docker-run: docker-qemu-src fi $(if $(EXECUTABLE), \ $(call quiet-command, \ - $(SRC_PATH)/tests/docker/docker.py update \ + $(DOCKER_SCRIPT) update \ $(IMAGE) $(EXECUTABLE), \ " COPYING $(EXECUTABLE) to $(IMAGE)")) $(call quiet-command, \ - $(SRC_PATH)/tests/docker/docker.py run \ + $(DOCKER_SCRIPT) run \ $(if $(NOUSER),,-u $(shell id -u)) \ --security-opt seccomp=3Dunconfined \ $(if $V,,--rm) \ @@ -167,4 +169,4 @@ docker-run-%: @$(MAKE) docker-run TEST=3D$(CMD) IMAGE=3Dqemu:$(IMAGE) =20 docker-clean: - $(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean) + $(call quiet-command, $(DOCKER_SCRIPT) clean) --=20 2.17.1 From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327365990933.7522362932159; Mon, 18 Jun 2018 06:09:25 -0700 (PDT) Received: from localhost ([::1]:34714 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtuH-00075b-6v for importer@patchew.org; Mon, 18 Jun 2018 09:09:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52449) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsM-0005yF-Ns for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsL-0002fM-G1 for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:18 -0400 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]:51656) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsL-0002er-8D for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:17 -0400 Received: by mail-wm0-x244.google.com with SMTP id r15-v6so13954626wmc.1 for ; Mon, 18 Jun 2018 06:07:17 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id z5-v6sm12504775wrh.10.2018.06.18.06.07.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:14 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id A76593E063B; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ecz+Ilq7CZkxnIqJjCPOjfRhEywCxFiSW4xntMf1J8Q=; b=iwK0cRf6gnW/q3PDSIfYHuaNOBaP3WGyYwQj25AISzv3uTfOEeMnkkxLltnRm5ByOP bmiW+8/f4ACi2uLKRG8TwzwgXtG9A6TvI9Kz+H//k/tSWKNwJMSZ5j+5RoPyclaajVRx I/ddG3gMMuMxu3VZQn1dob6pgb4TsixPOm6dU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ecz+Ilq7CZkxnIqJjCPOjfRhEywCxFiSW4xntMf1J8Q=; b=mZsv4UCAWCEihLOvQAECaJOrrC2TSOs9N4UawV3y3rrnSS/4NDPcuyirNGR9LQW2Yd 5UzN9GBfKtvx+/6/XTJoK9/seAb1S+8Idjtbr+6DuEvJwzm5UqMa65qudop5nOXMskbn 4LQN13UdNaG1pezXSKI564NIBNDsNTkuPNYK4lzhvHH9hbWo5fwIEPaGgJJIRXUBYAbU ZDwJAAgBSjPCYfsQt7ytlQMlwRHUsGuZM2E66E058QGDR7zLhkUcDG5O/fyFVYIof8KP AZJI76HZB3sv2RYKsrH67YRKoouMjqyP1LwC2eGtpdrEaKnMjDfIF/jFSLJ6QPYbr4NF VJxg== X-Gm-Message-State: APt69E1mi7tPie0d1Mu4pw7F8zDuGr699WJsjYKxwyV1HXWfoGTJG2yw MD1Sr025dg7Qvw5Ni7Pl2Dvkvg== X-Google-Smtp-Source: ADUXVKKDQwqPVTyyq+0eZEVJXAEJpbI670Hr3LOp4ufJ3NuTuG0Ul90wxpOE4ijqCBa4cZrXmib2DA== X-Received: by 2002:a1c:8312:: with SMTP id f18-v6mr9029299wmd.127.1529327236178; Mon, 18 Jun 2018 06:07:16 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:06 +0100 Message-Id: <20180618130709.19730-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::244 Subject: [Qemu-devel] [PATCH v1 5/8] docker: docker.py use "ps" to probe usage X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 The "images" command is a fairly heavyweight command to run as it involves searching the whole docker file-system inventory. On a machine with a lot of images this makes start-up fairly expensive. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/docker/docker.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index c83fd89985..3072bf4a31 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -49,7 +49,7 @@ def _guess_docker_command(): commands =3D [["docker"], ["sudo", "-n", "docker"]] for cmd in commands: try: - if subprocess.call(cmd + ["images"], + if subprocess.call(cmd + ["ps"], stdout=3DDEVNULL, stderr=3DDEVNULL) =3D=3D = 0: return cmd except OSError: --=20 2.17.1 From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327368324273.742368863096; Mon, 18 Jun 2018 06:09:28 -0700 (PDT) Received: from localhost ([::1]:34716 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtuR-0007Eh-5R for importer@patchew.org; Mon, 18 Jun 2018 09:09:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52482) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsP-0005zT-Bl for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsK-0002et-LQ for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:21 -0400 Received: from mail-wr0-x242.google.com ([2a00:1450:400c:c0c::242]:43694) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsK-0002eW-E8 for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:16 -0400 Received: by mail-wr0-x242.google.com with SMTP id d2-v6so16751794wrm.10 for ; Mon, 18 Jun 2018 06:07:16 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id q14-v6sm120849wmd.20.2018.06.18.06.07.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:14 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id BCAB53E064A; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4TCZ8aCcNLfGwTb8WsQD6FCOsSdhYeJliOZmFHb1vww=; b=gL3YaT1fLkkPfUX7cJuQBQv5P6+xwGTm3MiHMy8sOH46+jLcFDyjYvXUjOto7SDliK 9/sjcnkmAK5AVqBc+N7CQHW5nFPtddToQFcCuknyX4eQcUA1HL+ghzS1A26JIrj2c9v6 qSXfcVDC/of6aC6OvyH5BMHqqsVfrWy/kHCYQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4TCZ8aCcNLfGwTb8WsQD6FCOsSdhYeJliOZmFHb1vww=; b=ucZqzQICO1wMtqkqJd0FomCfyVRSnnx8x+k8g85phICb/K2ra+BaiYAWDykJ9+fqVU Z942dNEaQMyfHSI5XKcda61YGl1aENsq4SsyTlzFHvi68qMfdur+ZRJfkQZWH356AK2I S+WjrpecAGd4fHmVRf1v9Pqth3LVarucj/HgtSiALAP62lXkfG1WZVyefBlCH/g9pvA0 XudMNi2HNxXig3fItXn6dxQdAaAFV+SAfyFHhVxxSFI+0JbMKmPDwdw9gzFPTUry+Q6h C0p+BZohpMUNbxe2s5mODvRBpnNDPvupYkVhzyP3icEyQPP3f8MLBIPUPDSkV3QVxVW4 2J+Q== X-Gm-Message-State: APt69E0/RVPejgApYTzFgkddpZUGA9ddI7Ry1YqfCMRipGS4610UI5CO Y7h5VlHJmYNyfomzvjzz4vbArQ== X-Google-Smtp-Source: ADUXVKLszgKjAirA52724PJIs4lHvGms27p173M5CaJ9Ce4RO1W+3JO8ACvw4K41GaF5CVzIvjqZsw== X-Received: by 2002:adf:9187:: with SMTP id 7-v6mr11083401wri.69.1529327235497; Mon, 18 Jun 2018 06:07:15 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:07 +0100 Message-Id: <20180618130709.19730-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::242 Subject: [Qemu-devel] [PATCH v1 6/8] docker: docker.py don't conflate checksums for extra_files X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 This just gets confusing especially as the helper function doesn't even take into account any extra files (or the executable). Currently the actual check just ignores them and also passes the result through _dockerfile_preprocess so we fix that too. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/docker/docker.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 3072bf4a31..325915a6f9 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -201,8 +201,10 @@ class Docker(object): =20 tmp_df.write("\n") tmp_df.write("LABEL com.qemu.dockerfile-checksum=3D%s" % - _text_checksum("\n".join([dockerfile] + - extra_files_cksum))) + _text_checksum(_dockerfile_preprocess(dockerfile))) + for f, c in extra_files_cksum: + tmp_df.write("LABEL com.qemu.%s-checksum=3D%s" % (f, c)) + tmp_df.flush() =20 self._do_check(["build", "-t", tag, "-f", tmp_df.name] + argv + \ @@ -317,7 +319,7 @@ class BuildCommand(SubCommand): _copy_binary_with_libs(args.include_executable, docker_dir) for filename in args.extra_files or []: _copy_with_mkdir(filename, docker_dir) - cksum +=3D [_file_checksum(filename)] + cksum +=3D [(filename, _file_checksum(filename))] =20 argv +=3D ["--build-arg=3D" + k.lower() + "=3D" + v for k, v in os.environ.iteritems() --=20 2.17.1 From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327615703665.2760302624412; Mon, 18 Jun 2018 06:13:35 -0700 (PDT) Received: from localhost ([::1]:34739 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtyR-0002N1-1t for importer@patchew.org; Mon, 18 Jun 2018 09:13:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52473) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsO-0005yi-9N for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsN-0002gE-Co for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:20 -0400 Received: from mail-wr0-x22a.google.com ([2a00:1450:400c:c0c::22a]:37870) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsN-0002fl-4Z for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:19 -0400 Received: by mail-wr0-x22a.google.com with SMTP id d8-v6so16781852wro.4 for ; Mon, 18 Jun 2018 06:07:19 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id p38-v6sm34036243wrc.11.2018.06.18.06.07.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:14 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id D2F603E072F; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RTAVjkJnifp60KE2wcnTtu+icpE6wvJBzFNFLwcUhtY=; b=b6svZQ0dj5wXGhmspS+IMhmUl7yB3xlT8O5zRDFcNVrMReYVGUhEEWsesMBQbgGQSO SuT/YVgHvjosxYeQUW1mP9K7/dEaKVWIgOwoqUGVqThWliDwtcxuw2yiuaRLesofze+T UXuy0Hx2WZ/wViiYBxxXyjAYsza1qXqv9I6DE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RTAVjkJnifp60KE2wcnTtu+icpE6wvJBzFNFLwcUhtY=; b=hXWDA7fx8Qmxioi+cFZJAg5xz8bUuartjZIu9rctY1bEjOxPmBdlcbVt9BZgSqII1w aTjBKof6P9oLiD1s/kD36umLhFqDZoQpxfZpXA5T9QaB1qAL6agM0WymKycShp9VUsSr EdNc9nbsuMEmusjCKpT8KT8AaJvbjVI0vxtS3UbJKuR/fx2G+monurBrn8xr4DM3QjfB J9PwPg69PDoTvC2WuIzI208I4Cuq0BX4cuqaHZo0RxFlEg4gvhAfZhGLtM1PGvs7cd2t D1K3i4JtSWnNGjlKaE817cfTWsLIdVp8UmD/w1bK1gJVIfbRxHqv9sNsLgUHyFuIUKEQ 00dw== X-Gm-Message-State: APt69E3qvYcak/HjQZ8D+tgF8od7XzCXyHzYC/ztznT63ydiPW/AYAUK QCiDqHl8nvkT2RjMF/54ibiPjg== X-Google-Smtp-Source: ADUXVKLYbju4YH6kl/aDbQc3HQHR8CQiei+Q8tE8+w7WtBrsBdhaVNS60RIZC1zB2U6txSJok4RiJQ== X-Received: by 2002:adf:ac37:: with SMTP id v52-v6mr10624484wrc.56.1529327238129; Mon, 18 Jun 2018 06:07:18 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:08 +0100 Message-Id: <20180618130709.19730-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::22a Subject: [Qemu-devel] [PATCH v1 7/8] docker: docker.py add check sub-command X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 This command allows you to check if we need to re-build a docker image. If the image isn't in the repository or the checksums don't match then we return false and some text (for processing in makefiles). Signed-off-by: Alex Benn=C3=A9e --- - fix prints to be "modern" - PEP8 fixes - split inspect into function --- tests/docker/docker.py | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 325915a6f9..c157ff826a 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -179,8 +179,14 @@ class Docker(object): stderr=3Dsubprocess.STDOUT, **kwargs) =20 + def inspect_tag(self, tag): + try: + return self._output(["inspect", tag]) + except subprocess.CalledProcessError: + return None + def get_image_dockerfile_checksum(self, tag): - resp =3D self._output(["inspect", tag]) + resp =3D self.inspect_tag(tag) labels =3D json.loads(resp)[0]["Config"].get("Labels", {}) return labels.get("com.qemu.dockerfile-checksum", "") =20 @@ -442,6 +448,36 @@ class CcCommand(SubCommand): return Docker().command("run", cmd, args.quiet) =20 =20 +class CheckCommand(SubCommand): + """Check if we need to re-build a docker image out of a dockerfile. + Arguments: """ + name =3D "check" + + def args(self, parser): + parser.add_argument("tag", + help=3D"Image Tag") + parser.add_argument("dockerfile", + help=3D"Dockerfile name") + + def run(self, args, argv): + dockerfile =3D open(args.dockerfile, "rb").read() + tag =3D args.tag + + dkr =3D Docker() + info =3D dkr.inspect_tag(tag) + if info is None: + print("Image does not exist") + return 1 + + if dkr.image_matches_dockerfile(tag, dockerfile): + if not args.quiet: + print("Image is up to date") + return 0 + else: + print("Image needs updating") + return 1 + + def main(): parser =3D argparse.ArgumentParser(description=3D"A Docker helper", usage=3D"%s ..." % os.path.basename(sys.argv[0])) --=20 2.17.1 From nobody Mon Apr 29 02:11:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529327618140590.1833416200402; Mon, 18 Jun 2018 06:13:38 -0700 (PDT) Received: from localhost ([::1]:34740 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtyT-0002P8-EG for importer@patchew.org; Mon, 18 Jun 2018 09:13:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52464) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUtsN-0005yM-O2 for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUtsM-0002fq-Ik for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:19 -0400 Received: from mail-wr0-x236.google.com ([2a00:1450:400c:c0c::236]:37881) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUtsM-0002fV-Bk for qemu-devel@nongnu.org; Mon, 18 Jun 2018 09:07:18 -0400 Received: by mail-wr0-x236.google.com with SMTP id d8-v6so16781794wro.4 for ; Mon, 18 Jun 2018 06:07:18 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id b80-v6sm9119626wmf.2.2018.06.18.06.07.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Jun 2018 06:07:14 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id E70283E0768; Mon, 18 Jun 2018 14:07:09 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Drk7yosOJoszpUb8YawNCIAsnE+MnRqM7ImKUYir8Cc=; b=OEp5HDc4dEsf6NYqXC5lCRjqihh0OTTcHgDDYIuFAh/K0VcejbTjF1m/Ibgzd/YLM5 drY1Qf5SlupmdTD/RkSrOSLKTVxDTQWxNw6SP77ZMOq3slCWuZou3R7Mws/tJ0B8gwml 5+lOW0rPHLeuhywIXDtIjgZAV4KClXpmK52yI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Drk7yosOJoszpUb8YawNCIAsnE+MnRqM7ImKUYir8Cc=; b=RFAM6sZyqpTB4noQj29TDaGypgOx4FGMSqdqxb21j5yswLdmvcfYJd2lbMIsa0m4Fi r00WDOD0+VgBy+QWAMNpKZuhdp40sjzteCLA9/kFrsRyyVSgB0RvZzTLh8zKnWzqd9e3 Y+rE2f6j2fivt5lJHvPY27PhfTKdf6oq3gQw2pbpQHMc1RG3MWweskxl+449c66EB1ui ZA2/BNrSujpMVBQ7hjUII385sOMkE8Mi6biHWMrcQ0MMOVCJIeDwjtp1NRxvaoGR1A4v QZve6CT6pJI9xZ8DXgwXi8R8JX70vGtdFlP4batwVDM9AlbwNRfWsZJhXnNiBvL1Vau/ CAaw== X-Gm-Message-State: APt69E0tbDRjoos0c9gd6T8WNj/JszweOjp9Nh0aCKHUVex481LCbkDT E6QRU2xfJxhTZVM1Yhra61w8ZQ== X-Google-Smtp-Source: ADUXVKJiGOem77BxErMuSl+0XejgtTE+DaCWA6tLUHa+R5w7sC1qxKrdblCKC0WEZ5UV3RJl+V/hMQ== X-Received: by 2002:adf:f18b:: with SMTP id h11-v6mr11031036wro.214.1529327237315; Mon, 18 Jun 2018 06:07:17 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: famz@redhat.com, cota@braap.org, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Mon, 18 Jun 2018 14:07:09 +0100 Message-Id: <20180618130709.19730-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180618130709.19730-1-alex.bennee@linaro.org> References: <20180618130709.19730-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::236 Subject: [Qemu-devel] [PATCH v1 8/8] docker: docker.py adding age check command X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 This is useful for querying if an image is too old. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- v7 - use cached info --- tests/docker/docker.py | 46 +++++++++++++++++++++++++++++++++--------- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index c157ff826a..da1ed4e879 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -29,6 +29,7 @@ from tarfile import TarFile, TarInfo from StringIO import StringIO from shutil import copy, rmtree from pwd import getpwuid +from datetime import datetime,timedelta =20 =20 FILTERED_ENV_NAMES =3D ['ftp_proxy', 'http_proxy', 'https_proxy'] @@ -185,6 +186,9 @@ class Docker(object): except subprocess.CalledProcessError: return None =20 + def get_image_creation_time(self, info): + return json.loads(info)[0]["Created"] + def get_image_dockerfile_checksum(self, tag): resp =3D self.inspect_tag(tag) labels =3D json.loads(resp)[0]["Config"].get("Labels", {}) @@ -456,11 +460,14 @@ class CheckCommand(SubCommand): def args(self, parser): parser.add_argument("tag", help=3D"Image Tag") - parser.add_argument("dockerfile", - help=3D"Dockerfile name") + parser.add_argument("dockerfile", default=3DNone, + help=3D"Dockerfile name", nargs=3D'?') + parser.add_argument("--checktype", choices=3D["checksum", "age"], + default=3D"checksum", help=3D"check type") + parser.add_argument("--olderthan", default=3D60, type=3Dint, + help=3D"number of minutes") =20 def run(self, args, argv): - dockerfile =3D open(args.dockerfile, "rb").read() tag =3D args.tag =20 dkr =3D Docker() @@ -469,13 +476,32 @@ class CheckCommand(SubCommand): print("Image does not exist") return 1 =20 - if dkr.image_matches_dockerfile(tag, dockerfile): - if not args.quiet: - print("Image is up to date") - return 0 - else: - print("Image needs updating") - return 1 + if args.checktype =3D=3D "checksum": + if not args.dockerfile: + print("Need a dockerfile for tag:%s" % (tag)) + return 1 + + dockerfile =3D open(args.dockerfile, "rb").read() + + if dkr.image_matches_dockerfile(tag, dockerfile): + if not args.quiet: + print("Image is up to date") + return 0 + else: + print("Image needs updating") + return 1 + elif args.checktype =3D=3D "age": + timestr =3D dkr.get_image_creation_time(info).split(".")[0] + created =3D datetime.strptime(timestr, "%Y-%m-%dT%H:%M:%S") + past =3D datetime.now() - timedelta(minutes=3Dargs.olderthan) + if created < past: + print ("Image created @ %s more than %d minutes old" % + (timestr, args.olderthan)) + return 1 + else: + if not args.quiet: + print ("Image less than %d minutes old" % (args.oldert= han)) + return 0 =20 =20 def main(): --=20 2.17.1