scripts/archive-source.sh | 58 ++++++++++++++++++----------------------------- 1 file changed, 22 insertions(+), 36 deletions(-)
Use git archive to create tarballs of qemu and submodules instead of
cloning the repository and the submodules. This is a order of magnitude
faster because it doesn't fetch the submodules from the internet each
time the script runs.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
Notes:
v2: don't run git submodule update (except for uninitialized submodules)
scripts/archive-source.sh | 58 ++++++++++++++++++-----------------------------
1 file changed, 22 insertions(+), 36 deletions(-)
diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
index 6eed2a29bd..b9711059f9 100755
--- a/scripts/archive-source.sh
+++ b/scripts/archive-source.sh
@@ -19,8 +19,7 @@ if test $# -lt 1; then
fi
tar_file=$(realpath "$1")
-list_file="${tar_file}.list"
-vroot_dir="${tar_file}.vroot"
+sub_file=$(mktemp "${tar_file%.tar}.sub.XXXXXXXX.tar")
# We want a predictable list of submodules for builds, that is
# independent of what the developer currently has initialized
@@ -28,7 +27,7 @@ vroot_dir="${tar_file}.vroot"
# different to the host OS.
submodules="dtc ui/keycodemapdb tests/fp/berkeley-softfloat-3 tests/fp/berkeley-testfloat-3"
-trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15
+trap "status=$?; rm -rf \"$sub_file\" ; exit \$status" 0 1 2 3 15
if git diff-index --quiet HEAD -- &>/dev/null
then
@@ -36,38 +35,25 @@ then
else
HEAD=$(git stash create)
fi
-git clone --shared . "$vroot_dir"
-test $? -ne 0 && error "failed to clone into '$vroot_dir'"
-
-cd "$vroot_dir"
-test $? -ne 0 && error "failed to change into '$vroot_dir'"
-
-git checkout $HEAD
-test $? -ne 0 && error "failed to checkout $HEAD revision"
-
+git archive --format tar $HEAD > "$tar_file"
+test $? -ne 0 && error "failed to archive qemu"
for sm in $submodules; do
- git submodule update --init $sm
- test $? -ne 0 && error "failed to init submodule $sm"
+ status="$(git submodule status "$sm")"
+ smhash="${status# }"
+ smhash="${smhash#+}"
+ smhash="${smhash#-}"
+ smhash="${smhash%% *}"
+ case "$status" in
+ -*)
+ echo "NOTICE: fetching submodule $sm"
+ git submodule update --init "$sm"
+ ;;
+ +*)
+ echo "WARNING: submodule $sm is out of sync"
+ ;;
+ esac
+ (cd $sm; git archive --format tar --prefix "$sm/" $smhash) > "$sub_file"
+ test $? -ne 0 && error "failed to archive submodule $sm ($smhash)"
+ tar --concatenate --file "$tar_file" "$sub_file"
+ test $? -ne 0 && error "failed append submodule $sm to $tar_file"
done
-
-if test -n "$submodules"; then
- {
- git ls-files || error "git ls-files failed"
- for sm in $submodules; do
- (cd $sm; git ls-files) | sed "s:^:$sm/:"
- if test "${PIPESTATUS[*]}" != "0 0"; then
- error "git ls-files in submodule $sm failed"
- fi
- done
- } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file"
-else
- git ls-files > "$list_file"
-fi
-
-if test $? -ne 0; then
- error "failed to generate list file"
-fi
-
-tar -cf "$tar_file" -T "$list_file" || error "failed to create tar file"
-
-exit 0
--
2.9.3
Patchew URL: https://patchew.org/QEMU/20190131075132.17706-1-kraxel@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [Qemu-devel] [PATCH v2] scripts: use git archive in archive-source Message-id: 20190131075132.17706-1-kraxel@redhat.com Type: series === TEST SCRIPT BEGIN === #!/bin/bash git config --local diff.renamelimit 0 git config --local diff.renames True git config --local diff.algorithm histogram ./scripts/checkpatch.pl --mailback base.. === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Submodule 'capstone' (https://git.qemu.org/git/capstone.git) registered for path 'capstone' Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path 'dtc' Submodule 'roms/QemuMacDrivers' (https://git.qemu.org/git/QemuMacDrivers.git) registered for path 'roms/QemuMacDrivers' Submodule 'roms/SLOF' (https://git.qemu.org/git/SLOF.git) registered for path 'roms/SLOF' Submodule 'roms/ipxe' (https://git.qemu.org/git/ipxe.git) registered for path 'roms/ipxe' Submodule 'roms/openbios' (https://git.qemu.org/git/openbios.git) registered for path 'roms/openbios' Submodule 'roms/openhackware' (https://git.qemu.org/git/openhackware.git) registered for path 'roms/openhackware' Submodule 'roms/qemu-palcode' (https://git.qemu.org/git/qemu-palcode.git) registered for path 'roms/qemu-palcode' Submodule 'roms/seabios' (https://git.qemu.org/git/seabios.git/) registered for path 'roms/seabios' Submodule 'roms/seabios-hppa' (https://github.com/hdeller/seabios-hppa.git) registered for path 'roms/seabios-hppa' Submodule 'roms/sgabios' (https://git.qemu.org/git/sgabios.git) registered for path 'roms/sgabios' Submodule 'roms/skiboot' (https://git.qemu.org/git/skiboot.git) registered for path 'roms/skiboot' Submodule 'roms/u-boot' (https://git.qemu.org/git/u-boot.git) registered for path 'roms/u-boot' Submodule 'roms/u-boot-sam460ex' (https://git.qemu.org/git/u-boot-sam460ex.git) registered for path 'roms/u-boot-sam460ex' Submodule 'tests/fp/berkeley-softfloat-3' (https://github.com/cota/berkeley-softfloat-3) registered for path 'tests/fp/berkeley-softfloat-3' Submodule 'tests/fp/berkeley-testfloat-3' (https://github.com/cota/berkeley-testfloat-3) registered for path 'tests/fp/berkeley-testfloat-3' Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into 'capstone'... Submodule path 'capstone': checked out '22ead3e0bfdb87516656453336160e0a37b066bf' Cloning into 'dtc'... Submodule path 'dtc': checked out '88f18909db731a627456f26d779445f84e449536' Cloning into 'roms/QemuMacDrivers'... Submodule path 'roms/QemuMacDrivers': checked out 'd4e7d7ac663fcb55f1b93575445fcbca372f17a7' Cloning into 'roms/SLOF'... Submodule path 'roms/SLOF': checked out '9b7ab2fa020341dee8bf9df6c9cf40003e0136df' Cloning into 'roms/ipxe'... Submodule path 'roms/ipxe': checked out 'de4565cbe76ea9f7913a01f331be3ee901bb6e17' Cloning into 'roms/openbios'... Submodule path 'roms/openbios': checked out '441a84d3a642a10b948369c63f32367e8ff6395b' Cloning into 'roms/openhackware'... Submodule path 'roms/openhackware': checked out 'c559da7c8eec5e45ef1f67978827af6f0b9546f5' Cloning into 'roms/qemu-palcode'... Submodule path 'roms/qemu-palcode': checked out '51c237d7e20d05100eacadee2f61abc17e6bc097' Cloning into 'roms/seabios'... Submodule path 'roms/seabios': checked out 'a698c8995ffb2838296ec284fe3c4ad33dfca307' Cloning into 'roms/seabios-hppa'... Submodule path 'roms/seabios-hppa': checked out '1ef99a01572c2581c30e16e6fe69e9ea2ef92ce0' Cloning into 'roms/sgabios'... Submodule path 'roms/sgabios': checked out 'cbaee52287e5f32373181cff50a00b6c4ac9015a' Cloning into 'roms/skiboot'... Submodule path 'roms/skiboot': checked out 'e0ee24c27a172bcf482f6f2bc905e6211c134bcc' Cloning into 'roms/u-boot'... Submodule path 'roms/u-boot': checked out 'd85ca029f257b53a96da6c2fb421e78a003a9943' Cloning into 'roms/u-boot-sam460ex'... Submodule path 'roms/u-boot-sam460ex': checked out '60b3916f33e617a815973c5a6df77055b2e3a588' Cloning into 'tests/fp/berkeley-softfloat-3'... Submodule path 'tests/fp/berkeley-softfloat-3': checked out 'b64af41c3276f97f0e181920400ee056b9c88037' Cloning into 'tests/fp/berkeley-testfloat-3'... Submodule path 'tests/fp/berkeley-testfloat-3': checked out '5a59dcec19327396a011a17fd924aed4fec416b3' Cloning into 'ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce' Switched to a new branch 'test' abfc980 scripts: use git archive in archive-source === OUTPUT BEGIN === ERROR: code indent should never use tabs #56: FILE: scripts/archive-source.sh:41: +^Istatus="$(git submodule status "$sm")"$ ERROR: code indent should never use tabs #57: FILE: scripts/archive-source.sh:42: +^Ismhash="${status# }"$ ERROR: code indent should never use tabs #58: FILE: scripts/archive-source.sh:43: +^Ismhash="${smhash#+}"$ ERROR: code indent should never use tabs #59: FILE: scripts/archive-source.sh:44: +^Ismhash="${smhash#-}"$ ERROR: code indent should never use tabs #60: FILE: scripts/archive-source.sh:45: +^Ismhash="${smhash%% *}"$ ERROR: code indent should never use tabs #61: FILE: scripts/archive-source.sh:46: +^Icase "$status" in$ ERROR: code indent should never use tabs #62: FILE: scripts/archive-source.sh:47: +^I -*)$ ERROR: code indent should never use tabs #63: FILE: scripts/archive-source.sh:48: +^I^Iecho "NOTICE: fetching submodule $sm"$ ERROR: code indent should never use tabs #64: FILE: scripts/archive-source.sh:49: +^I^Igit submodule update --init "$sm"$ ERROR: code indent should never use tabs #65: FILE: scripts/archive-source.sh:50: +^I^I;;$ ERROR: code indent should never use tabs #66: FILE: scripts/archive-source.sh:51: +^I +*)$ ERROR: code indent should never use tabs #67: FILE: scripts/archive-source.sh:52: +^I^Iecho "WARNING: submodule $sm is out of sync"$ ERROR: code indent should never use tabs #68: FILE: scripts/archive-source.sh:53: +^I^I;;$ ERROR: code indent should never use tabs #69: FILE: scripts/archive-source.sh:54: +^Iesac$ ERROR: code indent should never use tabs #70: FILE: scripts/archive-source.sh:55: +^I(cd $sm; git archive --format tar --prefix "$sm/" $smhash) > "$sub_file"$ ERROR: code indent should never use tabs #71: FILE: scripts/archive-source.sh:56: +^Itest $? -ne 0 && error "failed to archive submodule $sm ($smhash)"$ ERROR: code indent should never use tabs #72: FILE: scripts/archive-source.sh:57: +^Itar --concatenate --file "$tar_file" "$sub_file"$ ERROR: code indent should never use tabs #73: FILE: scripts/archive-source.sh:58: +^Itest $? -ne 0 && error "failed append submodule $sm to $tar_file"$ total: 18 errors, 0 warnings, 53 lines checked Commit abfc980abb39 (scripts: use git archive in archive-source) has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. === OUTPUT END === Test command exited with code: 1 The full log is available at http://patchew.org/logs/20190131075132.17706-1-kraxel@redhat.com/testing.checkpatch/?type=message. --- Email generated automatically by Patchew [http://patchew.org/]. Please send your feedback to patchew-devel@redhat.com
© 2016 - 2024 Red Hat, Inc.