From nobody Sat May 18 19:24:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1603072451; cv=none; d=zohomail.com; s=zohoarc; b=FZiiKcY1xVreZNrM0BKXdcxUKtJcunJ5vEYCGAUqVeLUH/1Mfsuik48tdG4jQ3fmsdLkqsZqZVnkyNLZTDkkwDTMmcldrdP0tbE3FnNNcpuzQF17GWbwkrf0saSIYtDBqI03vsHg+5NigTa7r4Ez6lMC5Or6vBhMdrT+L0fXpvI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1603072451; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=27t77hnH3Ov0M62MYQ1su/k0+zUMdrC4ifQ4bD7cDRI=; b=mXHgKrGndi5aRkib9jzqy3+YyJzQucMicEDV4GOTWlfOGkVCtlYSvbVAiYo+qLVyJVJH5L+l2HQSh3uYwtqFyLSKxn7IIv0jWrRmk7coPvKJj4gy2mlLeTQQDFyJr3Z6cxxBQDTUE1giuoqUCsd2pc3IHZSJnvpF0imy+NRNg4g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1603072451132957.9605688949748; Sun, 18 Oct 2020 18:54:11 -0700 (PDT) Received: from localhost ([::1]:40346 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kUKNG-0006of-4N for importer@patchew.org; Sun, 18 Oct 2020 21:54:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51794) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUKJe-0001fj-Gl for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:47483) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kUKJY-0001Ff-Hm for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:26 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-258-qYpS3yxQNzCa-p7WQT1Ekw-1; Sun, 18 Oct 2020 21:50:15 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 22ECC1DDFF; Mon, 19 Oct 2020 01:50:14 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-119-55.rdu2.redhat.com [10.10.119.55]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C295D6EF50; Mon, 19 Oct 2020 01:50:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603072219; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=27t77hnH3Ov0M62MYQ1su/k0+zUMdrC4ifQ4bD7cDRI=; b=e9UG1UpTq9GoJZ/Z8AvrSNpeZJg8zxa0vOjFInlUkb8zbNxsQsOp4Xg8faY44xJD+bVjVP MioWYs07f5FC4GNQeq2/1WThR3nOrB3plskNnLfKOSnY3WkzTerxHUhBW0W18JxXmUehGp UazwTwmvPpjRYbYAcqCes9B5hlitesA= X-MC-Unique: qYpS3yxQNzCa-p7WQT1Ekw-1 From: Cleber Rosa To: qemu-devel@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Peter Maydell Subject: [PATCH v4 1/4] Jobs based on custom runners: documentation and configuration placeholder Date: Sun, 18 Oct 2020 21:50:00 -0400 Message-Id: <20201019015003.1527746-2-crosa@redhat.com> In-Reply-To: <20201019015003.1527746-1-crosa@redhat.com> References: <20201019015003.1527746-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/18 21:50:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , Erik Skultety , Stefan Hajnoczi , Andrea Bolognani , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) As described in the included documentation, the "custom runner" jobs extend the GitLab CI jobs already in place. Those jobs are intended to run on hardware and/or Operating Systems not provided by GitLab's shared runners. Signed-off-by: Cleber Rosa Reviewed-by: Daniel P. Berrang=C3=A9 --- .gitlab-ci.d/custom-runners.yml | 14 +++++++++ .gitlab-ci.yml | 1 + docs/devel/ci.rst | 54 +++++++++++++++++++++++++++++++++ docs/devel/index.rst | 1 + 4 files changed, 70 insertions(+) create mode 100644 .gitlab-ci.d/custom-runners.yml create mode 100644 docs/devel/ci.rst diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.= yml new file mode 100644 index 0000000000..3004da2bda --- /dev/null +++ b/.gitlab-ci.d/custom-runners.yml @@ -0,0 +1,14 @@ +# The CI jobs defined here require GitLab runners installed and +# registered on machines that match their operating system names, +# versions and architectures. This is in contrast to the other CI +# jobs that are intended to run on GitLab's "shared" runners. + +# Different than the default approach on "shared" runners, based on +# containers, the custom runners have no such *requirement*, as those +# jobs should be capable of running on operating systems with no +# compatible container implementation, or no support from +# gitlab-runner. To avoid problems that gitlab-runner can cause while +# reusing the GIT repository, let's enable the recursive submodule +# strategy. +variables: + GIT_SUBMODULE_STRATEGY: recursive diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 8ffd415ca5..b33c433fd7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -18,6 +18,7 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' - local: '/.gitlab-ci.d/crossbuilds.yml' + - local: '/.gitlab-ci.d/custom-runners.yml' =20 .native_build_job_template: &native_build_job_definition stage: build diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst new file mode 100644 index 0000000000..41a4bbddad --- /dev/null +++ b/docs/devel/ci.rst @@ -0,0 +1,54 @@ +=3D=3D +CI +=3D=3D + +QEMU has configurations enabled for a number of different CI services. +The most up to date information about them and their status can be +found at:: + + https://wiki.qemu.org/Testing/CI + +Jobs on Custom Runners +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Besides the jobs run under the various CI systems listed before, there +are a number additional jobs that will run before an actual merge. +These use the same GitLab CI's service/framework already used for all +other GitLab based CI jobs, but rely on additional systems, not the +ones provided by GitLab as "shared runners". + +The architecture of GitLab's CI service allows different machines to +be set up with GitLab's "agent", called gitlab-runner, which will take +care of running jobs created by events such as a push to a branch. +Here, the combination of a machine, properly configured with GitLab's +gitlab-runner, is called a "custom runner" here. + +The GitLab CI jobs definition for the custom runners are located under:: + + .gitlab-ci.d/custom-runners.yml + +Current Jobs +------------ + +The current CI jobs based on custom runners have the primary goal of +catching and preventing regressions on a wider number of host systems +than the ones provided by GitLab's shared runners. + +Also, the mechanics of reliability, capacity and overall maintanance +of the machines provided by the QEMU project itself for those jobs +will be evaluated. + +Future Plans and Jobs +--------------------- + +Once the CI Jobs based on custom runners have been proved mature with +the initial set of jobs run on machines from the QEMU project, other +members in the community should be able provide their own machine +configuration documentation/scripts, and accompanying job definitions. + +As a general rule, those newly added contributed jobs should run as +"non-gating", until their reliability is verified. + +The precise minimum requirements and exact rules for machine +configuration documentation/scripts, and the success rate of jobs are +still to be defined. diff --git a/docs/devel/index.rst b/docs/devel/index.rst index 77baae5c77..2fdd36e751 100644 --- a/docs/devel/index.rst +++ b/docs/devel/index.rst @@ -21,6 +21,7 @@ Contents: atomics stable-process testing + ci qtest decodetree secure-coding-practices --=20 2.25.4 From nobody Sat May 18 19:24:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1603072318; cv=none; d=zohomail.com; s=zohoarc; b=S5tyHBUt8z2mNsF9gwMLuPZQ+DpjR+47OLyniy75IB5yECJUx/rBuBxD3sOCoWUur6sb4c+OyYU9sNSCt+I2h75sgNDtNFAGN4lGKlGxCnDcKraUWYE2jVzh0LWY08jNzvXl0asKKEvFTEVoyp7yfK+ke+KuDa5gAfB/bUm7h+g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1603072318; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vQUWv6g5MWCDHehGrOsttrmyoSLaHsUoEVyKO5KtHAI=; b=U5NvB7gbvcGSA7WP8NlvT8Z7adf8QgHPmhRqHj2NwwDWbVWLjJZa/DPerJLHjfN80Ct2b3eDwZ5ZVvoq/zF12NxcNAhlDGBLMTvP/SdKfM6+fxVPz+OJhxKyYV9T03dAisKllm2rkXBg9lXAz4/wbihsJCM0gFo77ghg3648n/8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1603072318524807.4537849857179; Sun, 18 Oct 2020 18:51:58 -0700 (PDT) Received: from localhost ([::1]:59430 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kUKL7-00037h-Cm for importer@patchew.org; Sun, 18 Oct 2020 21:51:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUKJg-0001ir-Hq for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:28215) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kUKJa-0001Gv-6c for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-296-icD7BgBTNLGspYgYr_qrog-1; Sun, 18 Oct 2020 21:50:16 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A36A3879511; Mon, 19 Oct 2020 01:50:15 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-119-55.rdu2.redhat.com [10.10.119.55]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 448F46EF50; Mon, 19 Oct 2020 01:50:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603072220; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vQUWv6g5MWCDHehGrOsttrmyoSLaHsUoEVyKO5KtHAI=; b=E+xqxobl0pT4NWX7p4LRnQWAQY9xNqvNg4jb77DMA2sBxQpoftvnSOg+j0WVI8KZB4TCPM BZD2W8doKQV0nNQjwXgsg8mAyP40XAu27sLlJ+KgVu1Wzsr9FrJ37ji76COgmeefEYxeTt mMcysiK8GXaDa4FM/iJz8tScVkTXyIo= X-MC-Unique: icD7BgBTNLGspYgYr_qrog-1 From: Cleber Rosa To: qemu-devel@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Peter Maydell Subject: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook Date: Sun, 18 Oct 2020 21:50:01 -0400 Message-Id: <20201019015003.1527746-3-crosa@redhat.com> In-Reply-To: <20201019015003.1527746-1-crosa@redhat.com> References: <20201019015003.1527746-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/18 21:50:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , Erik Skultety , Stefan Hajnoczi , Andrea Bolognani , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed. The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment. Signed-off-by: Cleber Rosa --- docs/devel/ci.rst | 32 ++++ scripts/ci/setup/build-environment.yml | 233 +++++++++++++++++++++++++ scripts/ci/setup/inventory | 2 + 3 files changed, 267 insertions(+) create mode 100644 scripts/ci/setup/build-environment.yml create mode 100644 scripts/ci/setup/inventory diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst index 41a4bbddad..208b5e399b 100644 --- a/docs/devel/ci.rst +++ b/docs/devel/ci.rst @@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs sh= ould run as The precise minimum requirements and exact rules for machine configuration documentation/scripts, and the success rate of jobs are still to be defined. + +Machine Setup Howto +------------------- + +For all Linux based systems, the setup can be mostly automated by the +execution of two Ansible playbooks. Start by adding your machines to +the ``inventory`` file under ``scripts/ci/setup``, such as this:: + + [local] + fully.qualified.domain + other.machine.hostname + +You may need to set some variables in the inventory file itself. One +very common need is to tell Ansible to use a Python 3 interpreter on +those hosts. This would look like:: + + [local] + fully.qualified.domain ansible_python_interpreter=3D/usr/bin/python3 + other.machine.hostname ansible_python_interpreter=3D/usr/bin/python3 + +Build environment +~~~~~~~~~~~~~~~~~ + +The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will +set up machines with the environment needed to perform builds and run +QEMU tests. It covers a number of different Linux distributions and +FreeBSD. + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory build-environment.yml diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/buil= d-environment.yml new file mode 100644 index 0000000000..074aaca927 --- /dev/null +++ b/scripts/ci/setup/build-environment.yml @@ -0,0 +1,233 @@ +--- +- name: Installation of basic packages to build QEMU + hosts: all + tasks: + - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04 + apt: + update_cache: yes + # Originally from tests/docker/dockerfiles/ubuntu1804.docker + pkg: + - ccache + - clang + - gcc + - gettext + - git + - glusterfs-common + - libaio-dev + - libattr1-dev + - libbrlapi-dev + - libbz2-dev + - libcacard-dev + - libcap-ng-dev + - libcurl4-gnutls-dev + - libdrm-dev + - libepoxy-dev + - libfdt-dev + - libgbm-dev + - libgtk-3-dev + - libibverbs-dev + - libiscsi-dev + - libjemalloc-dev + - libjpeg-turbo8-dev + - liblzo2-dev + - libncurses5-dev + - libncursesw5-dev + - libnfs-dev + - libnss3-dev + - libnuma-dev + - libpixman-1-dev + - librados-dev + - librbd-dev + - librdmacm-dev + - libsasl2-dev + - libsdl2-dev + - libseccomp-dev + - libsnappy-dev + - libspice-protocol-dev + - libssh-dev + - libusb-1.0-0-dev + - libusbredirhost-dev + - libvdeplug-dev + - libvte-2.91-dev + - libzstd-dev + - make + - ninja-build + - python3-yaml + - python3-sphinx + - sparse + - xfslibs-dev + state: present + when: "ansible_facts['distribution'] =3D=3D 'Ubuntu'" + + - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s3= 90x + apt: + update_cache: yes + pkg: + - libspice-server-dev + - libxen-dev + state: present + when: + - "ansible_facts['distribution'] =3D=3D 'Ubuntu'" + - "ansible_facts['architecture'] !=3D 's390x'" + + - name: Install basic packages to build QEMU on FreeBSD 12.x + pkgng: + # Originally from packages on .cirrus.yml under the freebsd_12_task + name: bash,curl,cyrus-sasl,git,glib,gmake,gnutls,gsed,nettle,ninja= ,perl5,pixman,pkgconf,png,usbredir + state: present + when: "ansible_facts['os_family'] =3D=3D 'FreeBSD'" + + - name: Install basic packages to build QEMU on Fedora 30, 31 and 32 + dnf: + # Originally from tests/docker/dockerfiles/fedora.docker + name: + - SDL2-devel + - bc + - brlapi-devel + - bzip2 + - bzip2-devel + - ccache + - clang + - cyrus-sasl-devel + - dbus-daemon + - device-mapper-multipath-devel + - diffutils + - findutils + - gcc + - gcc-c++ + - genisoimage + - gettext + - git + - glib2-devel + - glusterfs-api-devel + - gnutls-devel + - gtk3-devel + - hostname + - libaio-devel + - libasan + - libattr-devel + - libblockdev-mpath-devel + - libcap-ng-devel + - libcurl-devel + - libepoxy-devel + - libfdt-devel + - libiscsi-devel + - libjpeg-devel + - libpmem-devel + - libpng-devel + - librbd-devel + - libseccomp-devel + - libssh-devel + - libubsan + - libudev-devel + - libusbx-devel + - libxml2-devel + - libzstd-devel + - llvm + - lzo-devel + - make + - mingw32-SDL2 + - mingw32-bzip2 + - mingw32-curl + - mingw32-glib2 + - mingw32-gmp + - mingw32-gnutls + - mingw32-gtk3 + - mingw32-libjpeg-turbo + - mingw32-libpng + - mingw32-libtasn1 + - mingw32-nettle + - mingw32-nsis + - mingw32-pixman + - mingw32-pkg-config + - mingw64-SDL2 + - mingw64-bzip2 + - mingw64-curl + - mingw64-glib2 + - mingw64-gmp + - mingw64-gnutls + - mingw64-gtk3 + - mingw64-libjpeg-turbo + - mingw64-libpng + - mingw64-libtasn1 + - mingw64-nettle + - mingw64-pixman + - mingw64-pkg-config + - ncurses-devel + - nettle-devel + - ninja-build + - nss-devel + - numactl-devel + - perl + - perl-Test-Harness + - pixman-devel + - python3 + - python3-PyYAML + - python3-numpy + - python3-opencv + - python3-pillow + - python3-pip + - python3-sphinx + - python3-virtualenv + - rdma-core-devel + - snappy-devel + - sparse + - spice-server-devel + - systemd-devel + - systemtap-sdt-devel + - tar + - tesseract + - tesseract-langpack-eng + - usbredir-devel + - virglrenderer-devel + - vte291-devel + - which + - xen-devel + - zlib-devel + state: present + when: "ansible_facts['distribution'] =3D=3D 'Fedora'" + + - name: Enable PowerTools repo on CentOS 8 + ini_file: + path: /etc/yum.repos.d/CentOS-PowerTools.repo + section: PowerTools + option: enabled + value: "1" + when: + - "ansible_facts['distribution'] =3D=3D 'CentOS'" + - "ansible_facts['distribution_major_version'] =3D=3D '8'" + + - name: Install basic packages to build QEMU on CentOS 8 + dnf: + # Originally from tests/docker/dockerfiles/centos8.docker + name: + - SDL-devel + - bzip2 + - bzip2-devel + - dbus-daemon + - gcc + - gcc-c++ + - genisoimage + - gettext + - git + - glib2-devel + - libaio-devel + - libepoxy-devel + - libgcrypt-devel + - lzo-devel + - make + - mesa-libEGL-devel + - nettle-devel + - ninja-build + - perl-Test-Harness + - pixman-devel + - python36 + - rdma-core-devel + - spice-glib-devel + - spice-server + - tar + - zlib-devel + state: present + when: + - "ansible_facts['distribution'] =3D=3D 'CentOS'" + - "ansible_facts['distribution_major_version'] =3D=3D '8'" diff --git a/scripts/ci/setup/inventory b/scripts/ci/setup/inventory new file mode 100644 index 0000000000..8bb7ba6b33 --- /dev/null +++ b/scripts/ci/setup/inventory @@ -0,0 +1,2 @@ +[local] +localhost --=20 2.25.4 From nobody Sat May 18 19:24:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1603072438; cv=none; d=zohomail.com; s=zohoarc; b=iP7E9ZOu/KAQRGREV6DHaUkAI+OstV3goc6Bhph9MOJ/G+3vPaurrwbdCxrzceR6h9MY82YRfvr/uBMi7LH+wOgAfjDiPxOeFCrcVan1ACtgXdJd1CoTKgNY85aI8rmJI4jexh2gwnkzXoHtkNC73rCecz85najALVqgTItkKXg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1603072438; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Q09udQSpOqJryDUc6T1JF6tDvIft0oQYAmRB4pZ755M=; b=cfvGaE2JP2Bp4Gy51kqU31AxLBdzAWJrG6QUmsZgZsDwNqDLelawQx6E1+39xSlRixVUI7mn6gGmuCndVL56pyvv0iSfLsmAm4BUpsrYvjmBMadOaEw7PKc1o/e6COKcpo1WgKFKghShO6V+PYRhUzjyoaFgqw7yMn/z4g8vlhg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1603072438314863.7758374964112; Sun, 18 Oct 2020 18:53:58 -0700 (PDT) Received: from localhost ([::1]:38924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kUKN2-0006Fh-5x for importer@patchew.org; Sun, 18 Oct 2020 21:53:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51810) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUKJf-0001h1-BM for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:22722) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kUKJZ-0001H8-W2 for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-323-bQoBwzP-NH-QUf8D1TAg7A-1; Sun, 18 Oct 2020 21:50:18 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 27CEF802B69; Mon, 19 Oct 2020 01:50:17 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-119-55.rdu2.redhat.com [10.10.119.55]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C9FD76EF45; Mon, 19 Oct 2020 01:50:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603072221; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q09udQSpOqJryDUc6T1JF6tDvIft0oQYAmRB4pZ755M=; b=K+6r9tYeOwjje+nT0noYiQbf6E8oJaaUZdp+2udWNXA5nk9gBepxZjG5Mq7J/D/yMop3Ce tS6YDlioPpixLKxFqOm5gXVycR817+4EU0D7f3EdfAUMZdSrq2EetJPqmQwQGz7bz+5Qsz p+42N7EakU8dAdReC2X8c+4nHYWmeEE= X-MC-Unique: bQoBwzP-NH-QUf8D1TAg7A-1 From: Cleber Rosa To: qemu-devel@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Peter Maydell Subject: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Date: Sun, 18 Oct 2020 21:50:02 -0400 Message-Id: <20201019015003.1527746-4-crosa@redhat.com> In-Reply-To: <20201019015003.1527746-1-crosa@redhat.com> References: <20201019015003.1527746-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/18 21:44:00 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , Erik Skultety , Stefan Hajnoczi , Andrea Bolognani , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps. The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment. Signed-off-by: Cleber Rosa Reviewed-by: Daniel P. Berrang=C3=A9 --- docs/devel/ci.rst | 63 ++++++++++++++++++++++++++ scripts/ci/setup/.gitignore | 1 + scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++ scripts/ci/setup/vars.yml.template | 13 ++++++ 4 files changed, 149 insertions(+) create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/vars.yml.template diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst index 208b5e399b..a234a5e24c 100644 --- a/docs/devel/ci.rst +++ b/docs/devel/ci.rst @@ -84,3 +84,66 @@ To run the playbook, execute:: =20 cd scripts/ci/setup ansible-playbook -i inventory build-environment.yml + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "Use the following registration token during setup" + +Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +.. note:: gitlab-runner is not available from the standard location + for all OS and architectures combinations. For some systems, + a custom build may be necessary. Some builds are avaiable + at https://cleber.fedorapeople.org/gitlab-runner/ and this + URI may be used as a value on ``vars.yml`` + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory gitlab-runner.yml + +.. note:: there are currently limitations to gitlab-runner itself when + setting up a service under FreeBSD systems. You will need to + perform steps 4 to 10 manually, as described at + https://docs.gitlab.com/runner/install/freebsd.html + +Following the registration, it's necessary to configure the runner tags, +and optionally other configurations on the GitLab UI. Navigate to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * "Runners activated for this project", then + * Click on the "Edit" icon (next to the "Lock" Icon) + +Under tags, add values matching the jobs a runner should run. For a +FreeBSD 12.1 x86_64 system, the tags should be set as:: + + freebsd12.1,x86_64 + +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` +would contain:: + + freebsd-12.1-x86_64-all: + tags: + - freebsd_12.1 + - x86_64 + +It's also recommended to: + + * increase the "Maximum job timeout" to something like ``2h`` + * uncheck the "Run untagged jobs" check box + * give it a better Description diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore new file mode 100644 index 0000000000..f112d05dd0 --- /dev/null +++ b/scripts/ci/setup/.gitignore @@ -0,0 +1 @@ +vars.yml \ No newline at end of file diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-r= unner.yml new file mode 100644 index 0000000000..c2f52dad10 --- /dev/null +++ b/scripts/ci/setup/gitlab-runner.yml @@ -0,0 +1,72 @@ +--- +- name: Installation of gitlab-runner + hosts: all + vars_files: + - vars.yml + tasks: + - debug: + msg: 'Checking for a valid GitLab registration token' + failed_when: "gitlab_runner_registration_token =3D=3D 'PLEASE_PROVID= E_A_VALID_TOKEN'" + + - name: Checks the availability of official gitlab-runner builds in th= e archive + uri: + url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_r= unner_version }}/binaries/gitlab-runner-linux-386 + method: HEAD + status_code: + - 200 + - 403 + register: gitlab_runner_available_archive + + - name: Update base url + set_fact: + gitlab_runner_base_url: https://s3.amazonaws.com/gitlab-runner-dow= nloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner- + when: gitlab_runner_available_archive.status =3D=3D 200 + - debug: + msg: Base gitlab-runner url is {{ gitlab_runner_base_url }} + + - name: Set OS name (FreeBSD) + set_fact: + gitlab_runner_os: freebsd + when: "ansible_facts['system'] =3D=3D 'FreeBSD'" + + - name: Create a group for the gitlab-runner service + group: + name: gitlab-runner + + - name: Create a user for the gitlab-runner service + user: + user: gitlab-runner + group: gitlab-runner + comment: GitLab Runner + home: /home/gitlab-runner + shell: /bin/bash + + - name: Remove the .bash_logout file when on Ubuntu systems + file: + path: /home/gitlab-runner/.bash_logout + state: absent + when: "ansible_facts['distribution'] =3D=3D 'Ubuntu'" + + - name: Downloads the matching gitlab-runner + get_url: + dest: /usr/local/bin/gitlab-runner + url: "{{ gitlab_runner_base_url }}{{ gitlab_runner_os }}-{{ gitlab= _runner_arch }}" + owner: gitlab-runner + group: gitlab-runner + mode: u=3Drwx,g=3Drwx,o=3Drx + + - name: Register the gitlab-runner + command: "/usr/local/bin/gitlab-runner register --non-interactive --= url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_re= gistration_token }} --executor shell --description '{{ ansible_facts[\"dis= tribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_fa= cts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + + - name: Install the gitlab-runner service using its own functionality + command: /usr/local/bin/gitlab-runner install --user gitlab-runner -= -working-directory /home/gitlab-runner + register: gitlab_runner_install_service_result + failed_when: "gitlab_runner_install_service_result.rc !=3D 0 and \"a= lready exists\" not in gitlab_runner_install_service_result.stderr" + when: "ansible_facts['os_family'] !=3D 'FreeBSD'" + + - name: Enable the gitlab-runner service + service: + name: gitlab-runner + state: started + enabled: yes + when: "ansible_facts['os_family'] !=3D 'FreeBSD'" diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml= .template new file mode 100644 index 0000000000..621435d030 --- /dev/null +++ b/scripts/ci/setup/vars.yml.template @@ -0,0 +1,13 @@ +# The version of the gitlab-runner to use +gitlab_runner_version: 13.1.1 +# The base location of gitlab-runner binaries, this will be suffixed by $O= S-$ARCH +gitlab_runner_base_url: https://gitlab-runner-downloads.s3.amazonaws.com/l= atest/binaries/gitlab-runner- +# The URL of the gitlab server to use, usually https://gitlab.com unless y= ou're +# using a private GitLab instance +gitlab_runner_server_url: https://gitlab.com +# Defaults to linux, checks can be used to change this +gitlab_runner_os: linux +# Defaults to amd64 (x86_64), checks can be used to change this +gitlab_runner_arch: amd64 +# A unique token made available by GitLab to your project for registering = runners +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN --=20 2.25.4 From nobody Sat May 18 19:24:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1603072547; cv=none; d=zohomail.com; s=zohoarc; b=Mn6Q2FOIfgTndPRY6D25Mx2jmtaT1X9rkxQnPulMBX3bRZvlVGTIZ0TK5tlUi48UyuhKLfWBQGErUdh55H4/1FZ0KtWLnpyl6rlS5a59V1u7t+mrTR7WJeE63kPCv9f6il74Ud4rXXT7BXHgZkzw0ZCkCpkHX2/YTBe2aaat8dM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1603072547; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Ves9dhQ7uA40qroFqiDykwJd40sbkpK3awlCdW3aOiE=; b=n68HMC/AAK5EcMWSgEPM60hC6jWMcl3Hob8cFV3wWjzvSR+0D14sj0sYrzCFhLdlFAWl8x6aC05MXzoL7ai1MMm160UI9TdzK3UQfq/k5ZhIwxjS8S21QvtMz/5qbruttUX+wi2s+hP29nW//Z9Fid5c4xSaRYk8qNQa068W0jQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1603072547912482.44384434209303; Sun, 18 Oct 2020 18:55:47 -0700 (PDT) Received: from localhost ([::1]:44168 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kUKOo-0008Qa-JU for importer@patchew.org; Sun, 18 Oct 2020 21:55:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUKJh-0001k7-Bb for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:59204) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kUKJe-0001HY-67 for qemu-devel@nongnu.org; Sun, 18 Oct 2020 21:50:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-372-pCJqHEF3MUOO1-ZKoL4zAA-1; Sun, 18 Oct 2020 21:50:19 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 962D21DE0F; Mon, 19 Oct 2020 01:50:18 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-119-55.rdu2.redhat.com [10.10.119.55]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 49FEC6EF45; Mon, 19 Oct 2020 01:50:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603072221; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ves9dhQ7uA40qroFqiDykwJd40sbkpK3awlCdW3aOiE=; b=X47BeEeCOh7ZesDXIAg4jCRMME9GUVGbFxzkNuBkDfUlDTxFh785uw3PZq+cP5kVfEs43g yq7AVx8OL25GEdJIfi/9B8ghiVXscZ41igrviggUJZZvELKTRfd2dZABfRdK5696qMbnVp xSK8pIqVpDBvB3XyeV5PBud8LVQjf08= X-MC-Unique: pCJqHEF3MUOO1-ZKoL4zAA-1 From: Cleber Rosa To: qemu-devel@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Peter Maydell Subject: [PATCH v4 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Date: Sun, 18 Oct 2020 21:50:03 -0400 Message-Id: <20201019015003.1527746-5-crosa@redhat.com> In-Reply-To: <20201019015003.1527746-1-crosa@redhat.com> References: <20201019015003.1527746-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/18 21:50:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , Erik Skultety , Stefan Hajnoczi , Andrea Bolognani , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The QEMU project has two machines (aarch64 and s390) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose. Signed-off-by: Cleber Rosa Reviewed-by: Daniel P. Berrang=C3=A9 --- .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+) diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.= yml index 3004da2bda..5b51d1b336 100644 --- a/.gitlab-ci.d/custom-runners.yml +++ b/.gitlab-ci.d/custom-runners.yml @@ -12,3 +12,195 @@ # strategy. variables: GIT_SUBMODULE_STRATEGY: recursive + +# All ubuntu-18.04 jobs should run successfully in an environment +# setup by the scripts/ci/setup/build-environment.yml task +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04" +ubuntu-18.04-s390x-all-linux-static: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+= bug/1838763 + # --disable-glusterfs is needed because there's no static version of thos= e libs in distro supplied packages + - mkdir build + - cd build + - ../configure --enable-debug --static --disable-system --disable-gluster= fs --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + - make --output-sync -j`nproc` check-tcg V=3D1 + +ubuntu-18.04-s390x-all: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + +ubuntu-18.04-s390x-alldbg: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --enable-debug --disable-libssh + - make clean + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + +ubuntu-18.04-s390x-clang: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --cc=3Dclang --cxx=3Dclang++ --enable-san= itizers + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + +ubuntu-18.04-s390x-tci: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --enable-tcg-interpreter + - make --output-sync -j`nproc` + +ubuntu-18.04-s390x-notcg: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --disable-tcg + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + +# All ubuntu-20.04 jobs should run successfully in an environment +# setup by the scripts/ci/setup/qemu/build-environment.yml task +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04" +ubuntu-20.04-aarch64-all-linux-static: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+= bug/1838763 + # --disable-glusterfs is needed because there's no static version of thos= e libs in distro supplied packages + - mkdir build + - cd build + - ../configure --enable-debug --static --disable-system --disable-gluster= fs --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + - make --output-sync -j`nproc` check-tcg V=3D1 + +ubuntu-20.04-aarch64-all: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + +ubuntu-20.04-aarch64-alldbg: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --enable-debug --disable-libssh + - make clean + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + +ubuntu-20.04-aarch64-clang: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --cc=3Dclang --cxx=3Dclang++ --enable-san= itizers + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 + +ubuntu-20.04-aarch64-tci: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --enable-tcg-interpreter + - make --output-sync -j`nproc` + +ubuntu-20.04-aarch64-notcg: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =3D~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --disable-tcg + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=3D1 --=20 2.25.4