From nobody Sat May 18 12:47:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1594652382; cv=none; d=zohomail.com; s=zohoarc; b=hNWWk85Zdwbw+ZdYILHJMGFAE58qzt/lQiIenoZEyVMqJ8bz9vpPQ56UZYbH/Kw5neyyllqbfE9kVOtgyC6jwFM6TQJPjZLI3BZPNt2gRXdtITxI4ZP/BvEt3e3aqasd3SptOiu1ai6iwVhKHsrwOl7cFnkPadX5i3knsfa5vq8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594652382; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=zjzJwG+hzX+Xi4FXzxMFQFE1NjjjSG71gG40wiP+lds=; b=C4BooRxJADIvc1t560l29+Ih0sdyVk+qhaO9gHNXZ5CgR4EHBhnWwZ3s0Gz008Ih+ZbPMaLSqoS6CkWGUtymXFMEAcdjCbca9RSK1qdsEeJFFXc6t81MS4N9P2Z+tVEyl8Fv4VR5U/213LQgbJZpwbrhw6cjj9CmoUGYpzkK/8U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594652382550950.5133750861457; Mon, 13 Jul 2020 07:59:42 -0700 (PDT) Received: from localhost ([::1]:36438 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1juzvh-0002HP-AY for importer@patchew.org; Mon, 13 Jul 2020 10:59:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60950) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1juzuG-00013y-OZ for qemu-devel@nongnu.org; Mon, 13 Jul 2020 10:58:12 -0400 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]:39329) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1juzuE-000724-O3 for qemu-devel@nongnu.org; Mon, 13 Jul 2020 10:58:12 -0400 Received: by mail-wr1-x444.google.com with SMTP id q5so16916143wru.6 for ; Mon, 13 Jul 2020 07:58:10 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id o9sm23694247wrs.1.2020.07.13.07.58.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 07:58:07 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 2B9DE1FF7E; Mon, 13 Jul 2020 15:58:06 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=zjzJwG+hzX+Xi4FXzxMFQFE1NjjjSG71gG40wiP+lds=; b=QAYMMGdX5UCPBnIyngsxzUjKEq4Krs8j1zv8rTzNc15x8nFhqCnyOJ4tmu/filXhjd vg+JopimTl79sBUtYzREFrVxBnutRJi0lK1kMHb/XxS7FyUJ+MqXj6LK9BMbTjcD+eGU hVB/wFumz94CNjAr9N8QRiTkIIPAnLW8TTXqN46BQnEr7dfvqmpLYxr5+rN711JGyUOU sohul8l46CrlWHPjHT9qd1l+ncw3uZ1zmwuP6dJGD2ktza71HcduOfxRbEYMoDH22Gj5 t5DT9YBBe2WgWVPtBBg/uNUgoF0Fnq8pyVOdtpQ1n7YQdJDm1fYUuu8ozw2E6Byd96Gt Q0VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=zjzJwG+hzX+Xi4FXzxMFQFE1NjjjSG71gG40wiP+lds=; b=QBjoF7M9XrV6DrG0tVxCB5z9N5lWmqIMj4HzKA24Pvevk7/340dBhQ/mSPIfY/aWbF yba7EYn2hn+Jqu7FOQwn8cFSRuBTXJCgWon9m9x2jMayW8TXlxCjroyMm79d3JSEfE8s LVh3hPmUPpAF0ycBVl4JChMIgNgSPlwBprXV/g4c7qp+RrsZ4vL1qa+udJNVn8yJG01+ BjMhGUeRUaL5zUdHuiQ8X3j2viFQ9CdQufIiSbA4cDbj682Pr1y+f7f3QZrCBt4Zxr7X zrfPD3JzKjibUGTNIVCf1hQYf0edpk/seAfmtbtkmA00XYi70lHd+CpxSptZb3Nq9HtL 6QiQ== X-Gm-Message-State: AOAM533Zb5cLfBIshazwJaEMWZ1yep7k9tYX1m8c4zdjNnd8Hgn7brl6 qpXaJ5EZyFVNYrzwVMBdbx9APA== X-Google-Smtp-Source: ABdhPJzKuCpjKIajxCw83Taz0lXEJreq7YcP5J0vwG0dNBzavemBr9qF/T4x/RWCtF2JrvqlM1vRTw== X-Received: by 2002:adf:f104:: with SMTP id r4mr83820168wro.90.1594652288900; Mon, 13 Jul 2020 07:58:08 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH] docker.py: fix fetching of FROM layers Date: Mon, 13 Jul 2020 15:58:01 +0100 Message-Id: <20200713145801.29278-1-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::444; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x444.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , peter.maydell@linaro.org, Fam Zheng , berrange@redhat.com, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This worked on a system that was already bootstrapped because the stage 2 images already existed even if they wouldn't be used. What we should have pulled down was the FROM line containers first because building on gitlab doesn't have the advantage of using our build system to build the pre-requisite bits. We still pull the image we want to build just in case we can use the cached data. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Daniel P. Berrang=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- tests/docker/docker.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 2d67bbd15a..c9f20d8d09 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -306,14 +306,18 @@ class Docker(object): checksum =3D _text_checksum(_dockerfile_preprocess(dockerfile)) =20 if registry is not None: - # see if we can fetch a cache copy, may fail... - pull_args =3D ["pull", "%s/%s" % (registry, tag)] - if self._do(pull_args, quiet=3Dquiet) =3D=3D 0: + sources =3D re.findall("FROM qemu\/(.*)", dockerfile) + # Fetch any cache layers we can, may fail + for s in sources: + pull_args =3D ["pull", "%s/qemu/%s" % (registry, s)] + if self._do(pull_args, quiet=3Dquiet) !=3D 0: + registry =3D None + break + # Make substitutions + if registry is not None: dockerfile =3D dockerfile.replace("FROM qemu/", "FROM %s/qemu/" % (registry)) - else: - registry =3D None =20 tmp_df =3D tempfile.NamedTemporaryFile(mode=3D"w+t", encoding=3D'utf-8', @@ -339,6 +343,8 @@ class Docker(object): build_args +=3D ["--build-arg", "BUILDKIT_INLINE_CACHE=3D1"] =20 if registry is not None: + pull_args =3D ["pull", "%s/%s" % (registry, tag)] + self._do(pull_args, quiet=3Dquiet) cache =3D "%s/%s" % (registry, tag) build_args +=3D ["--cache-from", cache] build_args +=3D argv --=20 2.20.1