[v6] scripts: Convert qemu-version.sh to qemu-version.py

[PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

Posted by Yonggang Luo 3 years, 6 months ago

The sh script are harder to maintain for compatible different
xsh environment so convert it to python script
Also incorporate the fixes in
https://patchew.org/QEMU/20200929143654.518157-1-marcandre.lureau@redhat.com/

The quote of v* should preserve, otherwise git command would fail

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Message-Id: <20201006112139.700-1-luoyonggang@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build             |  2 +-
 scripts/qemu-version.py | 34 ++++++++++++++++++++++++++++++++++
 scripts/qemu-version.sh | 25 -------------------------
 3 files changed, 35 insertions(+), 26 deletions(-)
 create mode 100644 scripts/qemu-version.py
 delete mode 100755 scripts/qemu-version.sh

diff --git a/meson.build b/meson.build
index 26230614ba..1d3bb25bc6 100644
--- a/meson.build
+++ b/meson.build
@@ -1132,7 +1132,7 @@ tracetool = [
    '--backend=' + config_host['TRACE_BACKENDS']
 ]
 
-qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
+qemu_version_cmd = [find_program('scripts/qemu-version.py'),
                     meson.current_source_dir(),
                     config_host['PKGVERSION'], meson.project_version()]
 qemu_version = custom_target('qemu-version.h',
diff --git a/scripts/qemu-version.py b/scripts/qemu-version.py
new file mode 100644
index 0000000000..019911b270
--- /dev/null
+++ b/scripts/qemu-version.py
@@ -0,0 +1,34 @@
+#!/usr/bin/env python3
+
+#
+# Script for retrieve qemu git version information
+#
+# Authors:
+#  Yonggang Luo <luoyonggang@gmail.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or, at your option, any later version.  See the COPYING file in
+# the top-level directory.
+
+import sys
+import subprocess
+import os, os.path
+
+def main(_program, dir, pkgversion, version, *unused):
+    os.chdir(dir)
+    if not pkgversion and os.path.exists('.git'):
+        # The quote of v* should preserve, otherwise git command would fail
+        pc = subprocess.run(['git', 'describe', '--match', "'v*'", '--dirty', '--always'],
+                            stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, encoding='utf8')
+        if pc.returncode == 0:
+            pkgversion = pc.stdout.strip()
+
+    fullversion = version
+    if pkgversion:
+        fullversion = "{} ({})".format(version, pkgversion)
+
+    print('#define QEMU_PKGVERSION "%s"' % pkgversion)
+    print('#define QEMU_FULL_VERSION "%s"' % fullversion)
+
+if __name__ == "__main__":
+    main(*sys.argv)
diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
deleted file mode 100755
index 03128c56a2..0000000000
--- a/scripts/qemu-version.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-set -eu
-
-dir="$1"
-pkgversion="$2"
-version="$3"
-
-if [ -z "$pkgversion" ]; then
-    cd "$dir"
-    if [ -e .git ]; then
-        pkgversion=$(git describe --match 'v*' --dirty | echo "")
-    fi
-fi
-
-if [ -n "$pkgversion" ]; then
-    fullversion="$version ($pkgversion)"
-else
-    fullversion="$version"
-fi
-
-cat <<EOF
-#define QEMU_PKGVERSION "$pkgversion"
-#define QEMU_FULL_VERSION "$fullversion"
-EOF
-- 
2.28.0.windows.1

Re: [PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

Posted by 罗勇刚 (Yonggang Luo) 3 years, 6 months ago

Hi Paolo, the qemu-version script you queued are mis removed the quote of
"'v*'", that's
would cause the building of qemu failed.
please queue this instead.
I also revised the commit message

On Thu, Oct 8, 2020 at 12:21 AM Yonggang Luo <luoyonggang@gmail.com> wrote:
>
> The sh script are harder to maintain for compatible different
> xsh environment so convert it to python script
> Also incorporate the fixes in
>
https://patchew.org/QEMU/20200929143654.518157-1-marcandre.lureau@redhat.com/
>
> The quote of v* should preserve, otherwise git command would fail
>
> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> Message-Id: <20201006112139.700-1-luoyonggang@gmail.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  meson.build             |  2 +-
>  scripts/qemu-version.py | 34 ++++++++++++++++++++++++++++++++++
>  scripts/qemu-version.sh | 25 -------------------------
>  3 files changed, 35 insertions(+), 26 deletions(-)
>  create mode 100644 scripts/qemu-version.py
>  delete mode 100755 scripts/qemu-version.sh
>
> diff --git a/meson.build b/meson.build
> index 26230614ba..1d3bb25bc6 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1132,7 +1132,7 @@ tracetool = [
>     '--backend=' + config_host['TRACE_BACKENDS']
>  ]
>
> -qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
> +qemu_version_cmd = [find_program('scripts/qemu-version.py'),
>                      meson.current_source_dir(),
>                      config_host['PKGVERSION'], meson.project_version()]
>  qemu_version = custom_target('qemu-version.h',
> diff --git a/scripts/qemu-version.py b/scripts/qemu-version.py
> new file mode 100644
> index 0000000000..019911b270
> --- /dev/null
> +++ b/scripts/qemu-version.py
> @@ -0,0 +1,34 @@
> +#!/usr/bin/env python3
> +
> +#
> +# Script for retrieve qemu git version information
> +#
> +# Authors:
> +#  Yonggang Luo <luoyonggang@gmail.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or, at your option, any later version.  See the COPYING file in
> +# the top-level directory.
> +
> +import sys
> +import subprocess
> +import os, os.path
> +
> +def main(_program, dir, pkgversion, version, *unused):
> +    os.chdir(dir)
> +    if not pkgversion and os.path.exists('.git'):
> +        # The quote of v* should preserve, otherwise git command would
fail
> +        pc = subprocess.run(['git', 'describe', '--match', "'v*'",
'--dirty', '--always'],
> +                            stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, encoding='utf8')
> +        if pc.returncode == 0:
> +            pkgversion = pc.stdout.strip()
> +
> +    fullversion = version
> +    if pkgversion:
> +        fullversion = "{} ({})".format(version, pkgversion)
> +
> +    print('#define QEMU_PKGVERSION "%s"' % pkgversion)
> +    print('#define QEMU_FULL_VERSION "%s"' % fullversion)
> +
> +if __name__ == "__main__":
> +    main(*sys.argv)
> diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
> deleted file mode 100755
> index 03128c56a2..0000000000
> --- a/scripts/qemu-version.sh
> +++ /dev/null
> @@ -1,25 +0,0 @@
> -#!/bin/sh
> -
> -set -eu
> -
> -dir="$1"
> -pkgversion="$2"
> -version="$3"
> -
> -if [ -z "$pkgversion" ]; then
> -    cd "$dir"
> -    if [ -e .git ]; then
> -        pkgversion=$(git describe --match 'v*' --dirty | echo "")
> -    fi
> -fi
> -
> -if [ -n "$pkgversion" ]; then
> -    fullversion="$version ($pkgversion)"
> -else
> -    fullversion="$version"
> -fi
> -
> -cat <<EOF
> -#define QEMU_PKGVERSION "$pkgversion"
> -#define QEMU_FULL_VERSION "$fullversion"
> -EOF
> --
> 2.28.0.windows.1
>


--
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: [PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

Posted by Paolo Bonzini 3 years, 6 months ago

On 07/10/20 18:23, 罗勇刚(Yonggang Luo) wrote:
>> +    if not pkgversion and os.path.exists('.git'):
>> +        # The quote of v* should preserve, otherwise git command
> would fail
>> +        pc = subprocess.run(['git', 'describe', '--match', "'v*'",
> '--dirty', '--always'],
>> +                            stdout=subprocess.PIPE,
> stderr=subprocess.DEVNULL, encoding='utf8')

It's a total mess.

On Linux, if you use 'v*' git is invoked with an argument that looks
like 'v*', that is as if you used \'v\*\' on bash.

On Windows, wildcard expansion is done by the program so you need to
pass a quoted value to stop the wildcard expansion.

I have no idea how to fix it.  Probably it can be made to work using a
string argument and "shell=True", but at this point it makes more sense
to keep the shell script version.

Paolo

Re: [PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

Posted by 罗勇刚 (Yonggang Luo) 3 years, 6 months ago

On Thu, Oct 8, 2020 at 12:43 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 07/10/20 18:23, 罗勇刚(Yonggang Luo) wrote:
> >> +    if not pkgversion and os.path.exists('.git'):
> >> +        # The quote of v* should preserve, otherwise git command
> > would fail
> >> +        pc = subprocess.run(['git', 'describe', '--match', "'v*'",
> > '--dirty', '--always'],
> >> +                            stdout=subprocess.PIPE,
> > stderr=subprocess.DEVNULL, encoding='utf8')
>
> It's a total mess.
>
> On Linux, if you use 'v*' git is invoked with an argument that looks
> like 'v*', that is as if you used \'v\*\' on bash.
>
> On Windows, wildcard expansion is done by the program so you need to
> pass a quoted value to stop the wildcard expansion.
>
> I have no idea how to fix it.  Probably it can be made to work using a
> string argument and "shell=True", but at this point it makes more sense
> to keep the shell script version.
Even though the shell script version are work, but it's not working in good
shape, when I running
the shell script, it's executed very slow, and so with python shell=True, I
guess it's because
the v* character cause the program doing a lot IO operation. So i still
think
'--match=v*' is a better solution, filename with --match=v123 are rare,
even sart with -- are rare.
so this is acceptable, and also I've creating a pull request on python to
resolve this issue completely
https://github.com/python/cpython/pull/22590
>
> Paolo
>


--
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: [PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

Posted by Peter Maydell 3 years, 6 months ago

On Wed, 7 Oct 2020 at 17:43, Paolo Bonzini <pbonzini@redhat.com> wrote:
> I have no idea how to fix it.  Probably it can be made to work using a
> string argument and "shell=True", but at this point it makes more sense
> to keep the shell script version.

If you're keeping the shell script version could you queue
the bugfix patch to it?

https://patchew.org/QEMU/20200929143654.518157-1-marcandre.lureau@redhat.com/

thanks
-- PMM

Re: [PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

Posted by Paolo Bonzini 3 years, 6 months ago

Il mer 7 ott 2020, 21:18 Peter Maydell <peter.maydell@linaro.org> ha
scritto:

> > I have no idea how to fix it.  Probably it can be made to work using a
> > string argument and "shell=True", but at this point it makes more sense
> > to keep the shell script version.
>
> If you're keeping the shell script version could you queue
> the bugfix patch to it?
>

Yes, I did.

Thanks,

Paolo


>
> https://patchew.org/QEMU/20200929143654.518157-1-marcandre.lureau@redhat.com/
>
> thanks
> -- PMM
>
>

Re: [PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

Posted by no-reply@patchew.org 3 years, 6 months ago

Patchew URL: https://patchew.org/QEMU/20201007162138.425-1-luoyonggang@gmail.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20201007162138.425-1-luoyonggang@gmail.com
Subject: [PATCH v6] scripts: Convert qemu-version.sh to qemu-version.py

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20201007162138.425-1-luoyonggang@gmail.com -> patchew/20201007162138.425-1-luoyonggang@gmail.com
Switched to a new branch 'test'
c1c6c9c scripts: Convert qemu-version.sh to qemu-version.py

=== OUTPUT BEGIN ===
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#33: 
new file mode 100644

WARNING: line over 80 characters
#58: FILE: scripts/qemu-version.py:21:
+        pc = subprocess.run(['git', 'describe', '--match', "'v*'", '--dirty', '--always'],

ERROR: line over 90 characters
#59: FILE: scripts/qemu-version.py:22:
+                            stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, encoding='utf8')

total: 1 errors, 2 warnings, 42 lines checked

Commit c1c6c9c04349 (scripts: Convert qemu-version.sh to qemu-version.py) has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20201007162138.425-1-luoyonggang@gmail.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com