./configure and make now take care of submodules, we only need to clone
the top tree.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 scripts/archive-source.sh | 47 ++++-------------------------------------------
 1 file changed, 4 insertions(+), 43 deletions(-)

diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
index 4e63774f9a..d53774d507 100755
--- a/scripts/archive-source.sh
+++ b/scripts/archive-source.sh
@@ -18,51 +18,12 @@ if test $# -lt 1; then
     error "Usage: $0 <output tarball>"
 fi
 
-tar_file=`realpath "$1"`
-list_file="${tar_file}.list"
-vroot_dir="${tar_file}.vroot"
+tar_file="$1"
+list_file="$1.list"
 
-# We want a predictable list of submodules for builds, that is
-# independent of what the developer currently has initialized
-# in their checkout, because the build environment is completely
-# different to the host OS.
-submodules="dtc ui/keycodemapdb"
+trap "status=$?; rm -f \"$list_file\"; exit \$status" 0 1 2 3 15
 
-trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15
-
-if git diff-index --quiet HEAD -- &>/dev/null
-then
-    HEAD=HEAD
-else
-    HEAD=`git stash create`
-fi
-git clone --shared . "$vroot_dir"
-test $? -ne 0 && error "failed to clone into '$vroot_dir'"
-
-cd "$vroot_dir"
-test $? -ne 0 && error "failed to change into '$vroot_dir'"
-
-git checkout $HEAD
-test $? -ne 0 && error "failed to checkout $HEAD revision"
-
-for sm in $submodules; do
-    git submodule update --init $sm
-    test $? -ne 0 && error "failed to init submodule $sm"
-done
-
-if test -n "$submodules"; then
-    {
-        git ls-files || error "git ls-files failed"
-        for sm in $submodules; do
-            (cd $sm; git ls-files) | sed "s:^:$sm/:"
-            if test "${PIPESTATUS[*]}" != "0 0"; then
-                error "git ls-files in submodule $sm failed"
-            fi
-        done
-    } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file"
-else
-    git ls-files > "$list_file"
-fi
+( git ls-files && echo '.git' ) > "$list_file"
 
 if test $? -ne 0; then
     error "failed to generate list file"
-- 
2.14.3

On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> ./configure and make now take care of submodules, we only need to clone
> the top tree.

If you don't include the submodules in the tar.gz archive, then the
test system needs to be able to clone them from qemu.org, which
requires networking.

The VM tests have networking by default, but the docker tests do
not have networking - you need to opt-in by setting NETWORK=1
make variable. So AFAICT, this will break docker tests.

> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  scripts/archive-source.sh | 47 ++++-------------------------------------------
>  1 file changed, 4 insertions(+), 43 deletions(-)
> 
> diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
> index 4e63774f9a..d53774d507 100755
> --- a/scripts/archive-source.sh
> +++ b/scripts/archive-source.sh
> @@ -18,51 +18,12 @@ if test $# -lt 1; then
>      error "Usage: $0 <output tarball>"
>  fi
>  
> -tar_file=`realpath "$1"`
> -list_file="${tar_file}.list"
> -vroot_dir="${tar_file}.vroot"
> +tar_file="$1"
> +list_file="$1.list"
>  
> -# We want a predictable list of submodules for builds, that is
> -# independent of what the developer currently has initialized
> -# in their checkout, because the build environment is completely
> -# different to the host OS.
> -submodules="dtc ui/keycodemapdb"
> +trap "status=$?; rm -f \"$list_file\"; exit \$status" 0 1 2 3 15
>  
> -trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15
> -
> -if git diff-index --quiet HEAD -- &>/dev/null
> -then
> -    HEAD=HEAD
> -else
> -    HEAD=`git stash create`
> -fi
> -git clone --shared . "$vroot_dir"
> -test $? -ne 0 && error "failed to clone into '$vroot_dir'"
> -
> -cd "$vroot_dir"
> -test $? -ne 0 && error "failed to change into '$vroot_dir'"
> -
> -git checkout $HEAD
> -test $? -ne 0 && error "failed to checkout $HEAD revision"
> -
> -for sm in $submodules; do
> -    git submodule update --init $sm
> -    test $? -ne 0 && error "failed to init submodule $sm"
> -done
> -
> -if test -n "$submodules"; then
> -    {
> -        git ls-files || error "git ls-files failed"
> -        for sm in $submodules; do
> -            (cd $sm; git ls-files) | sed "s:^:$sm/:"
> -            if test "${PIPESTATUS[*]}" != "0 0"; then
> -                error "git ls-files in submodule $sm failed"
> -            fi
> -        done
> -    } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file"
> -else
> -    git ls-files > "$list_file"
> -fi
> +( git ls-files && echo '.git' ) > "$list_file"
>  
>  if test $? -ne 0; then
>      error "failed to generate list file"
> -- 
> 2.14.3
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Wed, 04/04 09:23, Daniel P. Berrangé wrote:
> On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> > ./configure and make now take care of submodules, we only need to clone
> > the top tree.
> 
> If you don't include the submodules in the tar.gz archive, then the
> test system needs to be able to clone them from qemu.org, which
> requires networking.
> 
> The VM tests have networking by default, but the docker tests do
> not have networking - you need to opt-in by setting NETWORK=1
> make variable. So AFAICT, this will break docker tests.

Yes, I'll revise the patch.

Fam

On Wed, Apr 04, 2018 at 05:07:56PM +0800, Fam Zheng wrote:
> On Wed, 04/04 09:23, Daniel P. Berrangé wrote:
> > On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> > > ./configure and make now take care of submodules, we only need to clone
> > > the top tree.
> > 
> > If you don't include the submodules in the tar.gz archive, then the
> > test system needs to be able to clone them from qemu.org, which
> > requires networking.
> > 
> > The VM tests have networking by default, but the docker tests do
> > not have networking - you need to opt-in by setting NETWORK=1
> > make variable. So AFAICT, this will break docker tests.
> 
> Yes, I'll revise the patch.

BTW, I can't help feeling that scripts/archive-source.sh should not even
exist. We already have scripts/make-release that has significant overlap
of functionality and is used by 'make dist'. Ideally imho we should be
able to either invoke 'make dist' for testing, or call make-release.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Wed, 04/04 10:24, Daniel P. Berrangé wrote:
> On Wed, Apr 04, 2018 at 05:07:56PM +0800, Fam Zheng wrote:
> > On Wed, 04/04 09:23, Daniel P. Berrangé wrote:
> > > On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> > > > ./configure and make now take care of submodules, we only need to clone
> > > > the top tree.
> > > 
> > > If you don't include the submodules in the tar.gz archive, then the
> > > test system needs to be able to clone them from qemu.org, which
> > > requires networking.
> > > 
> > > The VM tests have networking by default, but the docker tests do
> > > not have networking - you need to opt-in by setting NETWORK=1
> > > make variable. So AFAICT, this will break docker tests.
> > 
> > Yes, I'll revise the patch.
> 
> BTW, I can't help feeling that scripts/archive-source.sh should not even
> exist. We already have scripts/make-release that has significant overlap
> of functionality and is used by 'make dist'. Ideally imho we should be
> able to either invoke 'make dist' for testing, or call make-release.

But it has the advantage that it, as test code, never messes with the process to
release QEMU.  :)

Fam