Gitlab also provides runners with Windows, we can use them to
test compilation with MSYS2, in both, 64-bit and 32-bit.

However, it takes quite a long time to set up the VM, so to
stay in the 1h time frame, we can only compile and check one
target here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 "make check" recently broke on MSYS2, and nobody noticed since apparently
 hardly anybody looks at the cirrus-CI output ... so here's another try
 to get some more test coverage in this area in the gitlab-CI instead.
 Patch needs to be applied after the "tests/unit/test-smp-parse:
 Make an unified name for the tested machine" patch to get "make check"
 fixed first.

 RFC -> v1:
 - Use cache to speed up installation a little bit
 - Add a 32-bit builder, too

 .gitlab-ci.d/qemu-project.yml |  1 +
 .gitlab-ci.d/windows.yml      | 98 +++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)
 create mode 100644 .gitlab-ci.d/windows.yml

diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index b3d79bc429..871262fe0e 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -11,3 +11,4 @@ include:
   - local: '/.gitlab-ci.d/static_checks.yml'
   - local: '/.gitlab-ci.d/custom-runners.yml'
   - local: '/.gitlab-ci.d/cirrus.yml'
+  - local: '/.gitlab-ci.d/windows.yml'
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
new file mode 100644
index 0000000000..309f7e7fb8
--- /dev/null
+++ b/.gitlab-ci.d/windows.yml
@@ -0,0 +1,98 @@
+.shared_msys2_builder:
+  tags:
+  - shared-windows
+  - windows
+  - windows-1809
+  cache:
+    key: "${CI_JOB_NAME}-cache"
+    paths:
+      - ${CI_PROJECT_DIR}/msys64/var/cache
+  needs: []
+  stage: build
+  timeout: 70m
+  before_script:
+  - If ( !(Test-Path -Path msys64\var\cache ) ) {
+      mkdir msys64\var\cache
+    }
+  - If ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) {
+      Invoke-WebRequest
+      "https://github.com/msys2/msys2-installer/releases/download/2021-07-25/msys2-base-x86_64-20210725.sfx.exe"
+      -outfile "msys64\var\cache\msys2.exe"
+    }
+  - msys64\var\cache\msys2.exe -y
+  - ((Get-Content -path .\msys64\etc\\post-install\\07-pacman-key.post -Raw)
+      -replace '--refresh-keys', '--version') |
+     Set-Content -Path ${CI_PROJECT_DIR}\msys64\etc\\post-install\\07-pacman-key.post
+  - .\msys64\usr\bin\bash -lc "sed -i 's/^CheckSpace/#CheckSpace/g' /etc/pacman.conf"
+  - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu'  # Core update
+  - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu'  # Normal update
+  - taskkill /F /FI "MODULES eq msys-2.0.dll"
+
+msys2-64bit:
+  extends: .shared_msys2_builder
+  script:
+  - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed
+      diffutils git grep make sed
+      mingw-w64-x86_64-capstone
+      mingw-w64-x86_64-curl
+      mingw-w64-x86_64-cyrus-sasl
+      mingw-w64-x86_64-gcc
+      mingw-w64-x86_64-glib2
+      mingw-w64-x86_64-gnutls
+      mingw-w64-x86_64-libnfs
+      mingw-w64-x86_64-libpng
+      mingw-w64-x86_64-libssh
+      mingw-w64-x86_64-libtasn1
+      mingw-w64-x86_64-libusb
+      mingw-w64-x86_64-libxml2
+      mingw-w64-x86_64-nettle
+      mingw-w64-x86_64-ninja
+      mingw-w64-x86_64-pixman
+      mingw-w64-x86_64-pkgconf
+      mingw-w64-x86_64-python
+      mingw-w64-x86_64-SDL2
+      mingw-w64-x86_64-SDL2_image
+      mingw-w64-x86_64-snappy
+      mingw-w64-x86_64-usbredir
+      mingw-w64-x86_64-zstd "
+  - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
+  - $env:MSYSTEM = 'MINGW64'     # Start a 64 bit Mingw environment
+  - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
+      --enable-capstone=system'
+  - .\msys64\usr\bin\bash -lc "sed -i '/^ROMS=/d' build/config-host.mak"
+  - .\msys64\usr\bin\bash -lc 'make -j2'
+  - .\msys64\usr\bin\bash -lc 'make check'
+
+msys2-32bit:
+  extends: .shared_msys2_builder
+  script:
+  - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed
+      diffutils git grep make sed
+      mingw-w64-i686-capstone
+      mingw-w64-i686-curl
+      mingw-w64-i686-cyrus-sasl
+      mingw-w64-i686-gcc
+      mingw-w64-i686-glib2
+      mingw-w64-i686-gnutls
+      mingw-w64-i686-gtk3
+      mingw-w64-i686-libgcrypt
+      mingw-w64-i686-libjpeg-turbo
+      mingw-w64-i686-libssh
+      mingw-w64-i686-libtasn1
+      mingw-w64-i686-libusb
+      mingw-w64-i686-libxml2
+      mingw-w64-i686-lzo2
+      mingw-w64-i686-ninja
+      mingw-w64-i686-pixman
+      mingw-w64-i686-pkgconf
+      mingw-w64-i686-python
+      mingw-w64-i686-snappy
+      mingw-w64-i686-usbredir "
+  - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
+  - $env:MSYSTEM = 'MINGW32'     # Start a 32-bit MinG environment
+  - mkdir output
+  - cd output
+  - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu
+      --enable-capstone=system"
+  - ..\msys64\usr\bin\bash -lc 'make -j2'
+  - ..\msys64\usr\bin\bash -lc 'make check'
-- 
2.27.0

On 11/15/21 15:06, Thomas Huth wrote:
> Gitlab also provides runners with Windows, we can use them to
> test compilation with MSYS2, in both, 64-bit and 32-bit.
> 
> However, it takes quite a long time to set up the VM, so to
> stay in the 1h time frame, we can only compile and check one
> target here.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  "make check" recently broke on MSYS2, and nobody noticed since apparently
>  hardly anybody looks at the cirrus-CI output ... so here's another try
>  to get some more test coverage in this area in the gitlab-CI instead.
>  Patch needs to be applied after the "tests/unit/test-smp-parse:
>  Make an unified name for the tested machine" patch to get "make check"
>  fixed first.
> 
>  RFC -> v1:
>  - Use cache to speed up installation a little bit
>  - Add a 32-bit builder, too
> 
>  .gitlab-ci.d/qemu-project.yml |  1 +
>  .gitlab-ci.d/windows.yml      | 98 +++++++++++++++++++++++++++++++++++
>  2 files changed, 99 insertions(+)
>  create mode 100644 .gitlab-ci.d/windows.yml
> 
> diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
> index b3d79bc429..871262fe0e 100644
> --- a/.gitlab-ci.d/qemu-project.yml
> +++ b/.gitlab-ci.d/qemu-project.yml
> @@ -11,3 +11,4 @@ include:
>    - local: '/.gitlab-ci.d/static_checks.yml'
>    - local: '/.gitlab-ci.d/custom-runners.yml'
>    - local: '/.gitlab-ci.d/cirrus.yml'
> +  - local: '/.gitlab-ci.d/windows.yml'
> diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> new file mode 100644
> index 0000000000..309f7e7fb8
> --- /dev/null
> +++ b/.gitlab-ci.d/windows.yml
> @@ -0,0 +1,98 @@
> +.shared_msys2_builder:
> +  tags:
> +  - shared-windows
> +  - windows
> +  - windows-1809
> +  cache:
> +    key: "${CI_JOB_NAME}-cache"

It would be nice to cache the shared 'before_script' part,
but it doesn't seems trivial; meanwhile this patch works and
is KISS, so:

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

> +    paths:
> +      - ${CI_PROJECT_DIR}/msys64/var/cache
> +  needs: []
> +  stage: build
> +  timeout: 70m
> +  before_script:
> +  - If ( !(Test-Path -Path msys64\var\cache ) ) {
> +      mkdir msys64\var\cache
> +    }
> +  - If ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) {
> +      Invoke-WebRequest
> +      "https://github.com/msys2/msys2-installer/releases/download/2021-07-25/msys2-base-x86_64-20210725.sfx.exe"
> +      -outfile "msys64\var\cache\msys2.exe"
> +    }
> +  - msys64\var\cache\msys2.exe -y
> +  - ((Get-Content -path .\msys64\etc\\post-install\\07-pacman-key.post -Raw)
> +      -replace '--refresh-keys', '--version') |
> +     Set-Content -Path ${CI_PROJECT_DIR}\msys64\etc\\post-install\\07-pacman-key.post
> +  - .\msys64\usr\bin\bash -lc "sed -i 's/^CheckSpace/#CheckSpace/g' /etc/pacman.conf"
> +  - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu'  # Core update
> +  - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu'  # Normal update
> +  - taskkill /F /FI "MODULES eq msys-2.0.dll"
> +
> +msys2-64bit:
> +  extends: .shared_msys2_builder
> +  script:
> +  - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed
> +      diffutils git grep make sed
> +      mingw-w64-x86_64-capstone
[...]

Hi

On Mon, Nov 15, 2021 at 6:31 PM Philippe Mathieu-Daudé <f4bug@amsat.org>
wrote:

> On 11/15/21 15:06, Thomas Huth wrote:
> > Gitlab also provides runners with Windows, we can use them to
> > test compilation with MSYS2, in both, 64-bit and 32-bit.
> >
> > However, it takes quite a long time to set up the VM, so to
> > stay in the 1h time frame, we can only compile and check one
> > target here.
>

I wonder why gitlab does not offer the docker executor. On the freedesktop
gitlab instance, they have windows docker executor, which speeds up the
build time. Maybe we could also have our own Windows runner for qemu?

>
> > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > ---
> >  "make check" recently broke on MSYS2, and nobody noticed since
> apparently
> >  hardly anybody looks at the cirrus-CI output ... so here's another try
> >  to get some more test coverage in this area in the gitlab-CI instead.
> >  Patch needs to be applied after the "tests/unit/test-smp-parse:
> >  Make an unified name for the tested machine" patch to get "make check"
> >  fixed first.
> >
> >  RFC -> v1:
> >  - Use cache to speed up installation a little bit
> >  - Add a 32-bit builder, too
> >
> >  .gitlab-ci.d/qemu-project.yml |  1 +
> >  .gitlab-ci.d/windows.yml      | 98 +++++++++++++++++++++++++++++++++++
> >  2 files changed, 99 insertions(+)
> >  create mode 100644 .gitlab-ci.d/windows.yml
> >
> > diff --git a/.gitlab-ci.d/qemu-project.yml
> b/.gitlab-ci.d/qemu-project.yml
> > index b3d79bc429..871262fe0e 100644
> > --- a/.gitlab-ci.d/qemu-project.yml
> > +++ b/.gitlab-ci.d/qemu-project.yml
> > @@ -11,3 +11,4 @@ include:
> >    - local: '/.gitlab-ci.d/static_checks.yml'
> >    - local: '/.gitlab-ci.d/custom-runners.yml'
> >    - local: '/.gitlab-ci.d/cirrus.yml'
> > +  - local: '/.gitlab-ci.d/windows.yml'
> > diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> > new file mode 100644
> > index 0000000000..309f7e7fb8
> > --- /dev/null
> > +++ b/.gitlab-ci.d/windows.yml
> > @@ -0,0 +1,98 @@
> > +.shared_msys2_builder:
> > +  tags:
> > +  - shared-windows
> > +  - windows
> > +  - windows-1809
> > +  cache:
> > +    key: "${CI_JOB_NAME}-cache"
>
> It would be nice to cache the shared 'before_script' part,
> but it doesn't seems trivial; meanwhile this patch works and
> is KISS, so:
>
> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> > +    paths:
> > +      - ${CI_PROJECT_DIR}/msys64/var/cache
> > +  needs: []
> > +  stage: build
> > +  timeout: 70m
> > +  before_script:
> > +  - If ( !(Test-Path -Path msys64\var\cache ) ) {
> > +      mkdir msys64\var\cache
> > +    }
> > +  - If ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) {
> > +      Invoke-WebRequest
> > +      "
> https://github.com/msys2/msys2-installer/releases/download/2021-07-25/msys2-base-x86_64-20210725.sfx.exe
> "
> > +      -outfile "msys64\var\cache\msys2.exe"
> > +    }
> > +  - msys64\var\cache\msys2.exe -y
> > +  - ((Get-Content -path .\msys64\etc\\post-install\\07-pacman-key.post
> -Raw)
> > +      -replace '--refresh-keys', '--version') |
> > +     Set-Content -Path ${CI_PROJECT_DIR}\msys64\etc\\post-install\\
> 07-pacman-key.post
> > +  - .\msys64\usr\bin\bash -lc "sed -i 's/^CheckSpace/#CheckSpace/g'
> /etc/pacman.conf"
> > +  - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu'  # Core update
> > +  - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu'  # Normal
> update
> > +  - taskkill /F /FI "MODULES eq msys-2.0.dll"
> > +
> > +msys2-64bit:
> > +  extends: .shared_msys2_builder
> > +  script:
> > +  - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed
> > +      diffutils git grep make sed
> > +      mingw-w64-x86_64-capstone
> [...]
>
>

-- 
Marc-André Lureau

On 11/16/21 08:05, Marc-André Lureau wrote:
> Hi
> 
> On Mon, Nov 15, 2021 at 6:31 PM Philippe Mathieu-Daudé <f4bug@amsat.org
> <mailto:f4bug@amsat.org>> wrote:
> 
>     On 11/15/21 15:06, Thomas Huth wrote:
>     > Gitlab also provides runners with Windows, we can use them to
>     > test compilation with MSYS2, in both, 64-bit and 32-bit.
>     >
>     > However, it takes quite a long time to set up the VM, so to
>     > stay in the 1h time frame, we can only compile and check one
>     > target here.
> 
> 
> I wonder why gitlab does not offer the docker executor. On the
> freedesktop gitlab instance, they have windows docker executor, which
> speeds up the build time. Maybe we could also have our own Windows
> runner for qemu?

We could, foss.org provides the QEMU project with x86 VMs resources
we are not using. What we miss is a sysadmin willing to setup &
maintain a such runner.

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 11/16/21 08:05, Marc-André Lureau wrote:
>> Hi
>> 
>> On Mon, Nov 15, 2021 at 6:31 PM Philippe Mathieu-Daudé <f4bug@amsat.org
>> <mailto:f4bug@amsat.org>> wrote:
>> 
>>     On 11/15/21 15:06, Thomas Huth wrote:
>>     > Gitlab also provides runners with Windows, we can use them to
>>     > test compilation with MSYS2, in both, 64-bit and 32-bit.
>>     >
>>     > However, it takes quite a long time to set up the VM, so to
>>     > stay in the 1h time frame, we can only compile and check one
>>     > target here.
>> 
>> 
>> I wonder why gitlab does not offer the docker executor. On the
>> freedesktop gitlab instance, they have windows docker executor, which
>> speeds up the build time. Maybe we could also have our own Windows
>> runner for qemu?
>
> We could, foss.org provides the QEMU project with x86 VMs resources
> we are not using. What we miss is a sysadmin willing to setup &
> maintain a such runner.

I think we might also have Azure credits from MS, but the same issues
about admin and setup probably exist.

-- 
Alex Bennée