From nobody Fri Mar 29 14:07:57 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652345480; cv=none; d=zohomail.com; s=zohoarc; b=HSuAxZoFbLM+AOyXemtYWM7Sq7gutDj7koltV3ZNDLZcx9Pc+msMacJ8fXE2gPMxAAgtC4dQKxPkYjzhxnJcdQtL6aIwEUIjQ1WnIs337VXT1WB2isbAqWcjzwS0JhM6Sn5n9uotgJMiQxt5SgU1YRKz5nD+H7RhOMbRwHK5S8c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652345480; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Jrp9qMf1yt67FDDynaX7SZ8EqQmCV1wWXwdCoOmsT8k=; b=nhU1mxNBiEIZ/1SNWtcsvHfIpR1NkL7yKg1V9xtWvKcpqNgPPjp9xyHJuEebjFMPkG/hrSZs+AmqRydKze1c/U4L4J+ZmxYP+bz6GN34jIeuQ7j6IFEvQ/IF1HUQWpby+SZpIyqyUVRHopChdykMvp5gQwUJfsZ+8B7ozz4VfsU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1652345480917268.61393879453556; Thu, 12 May 2022 01:51:20 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-424-Pv03-iZbPwmjPBbS53XbHA-1; Thu, 12 May 2022 04:51:16 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F2F1690E982; Thu, 12 May 2022 08:51:13 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id DFF7941617C; Thu, 12 May 2022 08:51:13 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id AE7AC193220C; Thu, 12 May 2022 08:51:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id CD4371932224 for ; Thu, 12 May 2022 08:51:11 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B1C2D112131E; Thu, 12 May 2022 08:51:11 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.193.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id 30B761121314; Thu, 12 May 2022 08:51:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652345479; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Jrp9qMf1yt67FDDynaX7SZ8EqQmCV1wWXwdCoOmsT8k=; b=UKQd/xQsHtYm8A6xhjkr5NgzN6R3FMKIcY+Bl1FKXKjYXudqhOnctqdorOK3Ge4Z1NAwqF T5nJo1MGknVT6Ra7jeiFcbohYxDeCC+xcz5xETHyLPaysns14hIQJruDlNHlMn9ZF/TXvv S6a32Ci+I+w09hI9jFdrJ0TxaRxmjLc= X-MC-Unique: Pv03-iZbPwmjPBbS53XbHA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH v2 2/5] ci: Break off the integration_tests template into more templates Date: Thu, 12 May 2022 10:50:56 +0200 Message-Id: <159f7d34898bf07e571f8fa322eb8c52ef911766.1652345345.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eskultet@redhat.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652345481465100005 There's quite a lot happening in the .integration_tests template already even without adding upstream QEMU build into the mix. Let's break the template into more pieces which can then reference in the .integration_tests template when putting all the pieces back together using YAML anchors. Signed-off-by: Erik Skultety Reviewed-by: Michal Privoznik Reviewed-by: Daniel P. Berrang=C3=A9 --- ci/integration-template.yml | 76 +++++++++++++++++++++++-------------- 1 file changed, 47 insertions(+), 29 deletions(-) diff --git a/ci/integration-template.yml b/ci/integration-template.yml index c4201df6db..195e9cfa5f 100644 --- a/ci/integration-template.yml +++ b/ci/integration-template.yml @@ -1,44 +1,62 @@ + + +.install-deps: &install-deps + - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/* + - sudo pip3 install --prefix=3D/usr avocado-framework + + +.enable-core-dumps: &enable-core-dumps + - sudo sh -c "echo DefaultLimitCORE=3Dinfinity >> /etc/systemd/system.co= nf" # Explicitly allow storing cores globally + - sudo systemctl daemon-reexec # need to reexec systemd after changing c= onfig + + +.enable-libvirt-debugging: &enable-libvirt-debugging + - source /etc/os-release # in order to query the vendor-provided variab= les + - if test "$ID" =3D "centos" && test "$VERSION_ID" -lt 9 || + test "$ID" =3D "fedora" && test "$VERSION_ID" -lt 35; + then + DAEMONS=3D"libvirtd virtlogd virtlockd"; + else + DAEMONS=3D"virtproxyd virtqemud virtinterfaced virtsecretd virtstora= ged virtnwfilterd virtnodedevd virtlogd virtlockd"; + fi + - for daemon in $DAEMONS; + do + LOG_OUTPUTS=3D"1:file:/var/log/libvirt/${daemon}.log"; + LOG_FILTERS=3D"3:remote 4:event 3:util.json 3:util.object 3:util.dbu= s 3:util.netlink 3:node_device 3:rpc 3:access 1:*"; + sudo augtool set /files/etc/libvirt/${daemon}.conf/log_filters "$LOG= _FILTERS" &>/dev/null; + sudo augtool set /files/etc/libvirt/${daemon}.conf/log_outputs "$LOG= _OUTPUTS" &>/dev/null; + sudo systemctl --quiet stop ${daemon}.service; + sudo systemctl restart ${daemon}.socket; + done + + +.collect-logs: &collect-logs + - mkdir logs + - test -e "$SCRATCH_DIR"/avocado && sudo mv "$SCRATCH_DIR"/avocado/lates= t/test-results logs/avocado; + - sudo coredumpctl info --no-pager > logs/coredumpctl.txt + - sudo mv /var/log/libvirt logs/libvirt + - sudo chown -R $(whoami):$(whoami) logs + # rename all Avocado stderr/stdout logs to *.log so that GitLab's web = UI doesn't mangle the MIME type + - find logs/avocado/ -type f ! -name "*.log" -exec + sh -c 'DIR=3D$(dirname {}); NAME=3D$(basename {}); mv $DIR/$NAME{,.l= og}' \; + + .integration_tests: stage: integration_tests before_script: - mkdir "$SCRATCH_DIR" - - sudo sh -c "echo DefaultLimitCORE=3Dinfinity >> /etc/systemd/system.= conf" # Explicitly allow storing cores globally - - sudo systemctl daemon-reexec # need to reexec systemd after changing= config - - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/* - - sudo pip3 install --prefix=3D/usr avocado-framework - - source /etc/os-release # in order to query the vendor-provided vari= ables - - if test "$ID" =3D "centos" && test "$VERSION_ID" -lt 9 || - test "$ID" =3D "fedora" && test "$VERSION_ID" -lt 35; - then - DAEMONS=3D"libvirtd virtlogd virtlockd"; - else - DAEMONS=3D"virtproxyd virtqemud virtinterfaced virtsecretd virtsto= raged virtnwfilterd virtnodedevd virtlogd virtlockd"; - fi - - for daemon in $DAEMONS; - do - LOG_OUTPUTS=3D"1:file:/var/log/libvirt/${daemon}.log"; - LOG_FILTERS=3D"3:remote 4:event 3:util.json 3:util.object 3:util.d= bus 3:util.netlink 3:node_device 3:rpc 3:access 1:*"; - sudo augtool set /files/etc/libvirt/${daemon}.conf/log_filters "$L= OG_FILTERS" &>/dev/null; - sudo augtool set /files/etc/libvirt/${daemon}.conf/log_outputs "$L= OG_OUTPUTS" &>/dev/null; - sudo systemctl --quiet stop ${daemon}.service; - sudo systemctl restart ${daemon}.socket; - done + - *install-deps + - *enable-core-dumps + - *enable-libvirt-debugging - sudo virsh net-start default &>/dev/null || true; script: - - mkdir logs - cd "$SCRATCH_DIR" - git clone --depth 1 https://gitlab.com/libvirt/libvirt-tck.git - cd libvirt-tck - sudo avocado --config avocado.config run --job-results-dir "$SCRATCH= _DIR"/avocado after_script: - test "$CI_JOB_STATUS" =3D "success" && exit 0; - - test -e "$SCRATCH_DIR"/avocado && sudo mv "$SCRATCH_DIR"/avocado/lat= est/test-results logs/avocado; - - sudo coredumpctl info --no-pager > logs/coredumpctl.txt - - sudo mv /var/log/libvirt logs/libvirt - - sudo chown -R $(whoami):$(whoami) logs - # rename all Avocado stderr/stdout logs to *.log so that GitLab's we= b UI doesn't mangle the MIME type - - find logs/avocado/ -type f ! -name "*.log" -exec - sh -c 'DIR=3D$(dirname {}); NAME=3D$(basename {}); mv $DIR/$NAME{,= .log}' \; + - *collect-logs variables: SCRATCH_DIR: "/tmp/scratch" artifacts: --=20 2.34.3