From nobody Mon Feb 9 09:33:26 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1557485694; cv=none; d=zoho.com; s=zohoarc; b=gXs98+YBguLRUfeH7b0llpCdfb/p8RI5UvWeSV3MX+aOyp2LF6gflv17oIJo6ck5heLTw+jwXEkAryxjduOqceb0uEj1bIWRaRY4A3uYzRDxegmx8lQ6s8JuNK76n8VQtN+SwijxfEpCUtXen8Y64vRwEx8NkwLrX7FMbkhuiWc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557485694; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=IPn4RzgtxTQpaL2YFW6OkckZYZJq3cLSwskFosBwSdM=; b=N9awAziWJCLVAluyoEd/+hHnAK0ETrsvFnl1eceS3cxSNVcA2jjSqUTtgiPs6w9nSbKzOPp1VLRpjkBSntYRK4BHLamAT4IbvuBd31fbUUPdp0o/mDEAPXqD+awftwU/AXcMrI6ZibCUJQJHmnuWHQNpH4BSt2buYoSsLMj+slk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557485694902500.2564897793037; Fri, 10 May 2019 03:54:54 -0700 (PDT) Received: from localhost ([127.0.0.1]:41039 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hP3Ar-0005yZ-Ia for importer@patchew.org; Fri, 10 May 2019 06:54:45 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59724) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hP333-0008Iu-SV for qemu-devel@nongnu.org; Fri, 10 May 2019 06:46:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hP332-00043d-M6 for qemu-devel@nongnu.org; Fri, 10 May 2019 06:46:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41586) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hP332-000436-Ex for qemu-devel@nongnu.org; Fri, 10 May 2019 06:46:40 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BB163D77F2; Fri, 10 May 2019 10:46:39 +0000 (UTC) Received: from sirius.home.kraxel.org (ovpn-117-74.ams2.redhat.com [10.36.117.74]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4209E646BB; Fri, 10 May 2019 10:46:34 +0000 (UTC) Received: by sirius.home.kraxel.org (Postfix, from userid 1000) id 65D16A1E4; Fri, 10 May 2019 12:46:33 +0200 (CEST) From: Gerd Hoffmann To: qemu-devel@nongnu.org Date: Fri, 10 May 2019 12:46:21 +0200 Message-Id: <20190510104633.9428-2-kraxel@redhat.com> In-Reply-To: <20190510104633.9428-1-kraxel@redhat.com> References: <20190510104633.9428-1-kraxel@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 10 May 2019 10:46:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 01/13] scripts: use git archive in archive-source X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Ed Maste , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Kamil Rytarowski , Gerd Hoffmann , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Li-Wen Hsu , Brad Smith Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use git archive to create tarballs of qemu and submodules instead of cloning the repository and the submodules. This is a order of magnitude faster because it doesn't fetch the submodules from the internet each time the script runs. Signed-off-by: Gerd Hoffmann --- scripts/archive-source.sh | 72 +++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 41 deletions(-) diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh index 8b8994826000..93a48d2063f9 100755 --- a/scripts/archive-source.sh +++ b/scripts/archive-source.sh @@ -19,16 +19,25 @@ if test $# -lt 1; then fi =20 tar_file=3D$(realpath "$1") -list_file=3D"${tar_file}.list" -vroot_dir=3D"${tar_file}.vroot" +sub_tdir=3D$(mktemp -d "${tar_file%.tar}.sub.XXXXXXXX") +sub_file=3D"${sub_tdir}/submodule.tar" =20 # We want a predictable list of submodules for builds, that is # independent of what the developer currently has initialized # in their checkout, because the build environment is completely # different to the host OS. submodules=3D"dtc slirp ui/keycodemapdb tests/fp/berkeley-softfloat-3 test= s/fp/berkeley-testfloat-3" +sub_deinit=3D"" =20 -trap "status=3D$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 = 1 2 3 15 +function cleanup() { + local status=3D$? + rm -rf "$sub_tdir" + if test "$sub_deinit" !=3D ""; then + git submodule deinit $sub_deinit + fi + exit $status +} +trap "cleanup" 0 1 2 3 15 =20 if git diff-index --quiet HEAD -- &>/dev/null then @@ -36,45 +45,26 @@ then else HEAD=3D$(git stash create) fi -git clone --shared . "$vroot_dir" -test $? -ne 0 && error "failed to clone into '$vroot_dir'" -for sm in $submodules; do - if test -d "$sm/.git" - then - git clone --shared "$sm" "$vroot_dir/$sm" - test $? -ne 0 && error "failed to clone submodule $sm" - fi -done - -cd "$vroot_dir" -test $? -ne 0 && error "failed to change into '$vroot_dir'" - -git checkout $HEAD -test $? -ne 0 && error "failed to checkout $HEAD revision" =20 +git archive --format tar $HEAD > "$tar_file" +test $? -ne 0 && error "failed to archive qemu" for sm in $submodules; do - git submodule update --init $sm - test $? -ne 0 && error "failed to init submodule $sm" + status=3D"$(git submodule status "$sm")" + smhash=3D"${status#[ +-]}" + smhash=3D"${smhash%% *}" + case "$status" in + -*) + sub_deinit=3D"$sub_deinit $sm" + git submodule update --init "$sm" + test $? -ne 0 && error "failed to update submodule $sm" + ;; + +*) + echo "WARNING: submodule $sm is out of sync" + ;; + esac + (cd $sm; git archive --format tar --prefix "$sm/" $smhash) > "$sub_fil= e" + test $? -ne 0 && error "failed to archive submodule $sm ($smhash)" + tar --concatenate --file "$tar_file" "$sub_file" + test $? -ne 0 && error "failed append submodule $sm to $tar_file" done - -if test -n "$submodules"; then - { - git ls-files || error "git ls-files failed" - for sm in $submodules; do - (cd $sm; git ls-files) | sed "s:^:$sm/:" - if test "${PIPESTATUS[*]}" !=3D "0 0"; then - error "git ls-files in submodule $sm failed" - fi - done - } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$li= st_file" -else - git ls-files > "$list_file" -fi - -if test $? -ne 0; then - error "failed to generate list file" -fi - -tar -cf "$tar_file" -T "$list_file" || error "failed to create tar file" - exit 0 --=20 2.18.1