From nobody Mon Feb 9 16:19:26 2026 Delivered-To: importer@patchew.org Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1556819682; cv=none; d=zoho.com; s=zohoarc; b=NVpI664F3apuc2INIegzVfQwVhrSeoxILKXgXrjzG9saK6M7Y7opJgavc7lzPn3CICpZsmsU0siNFa/DnR1x8WsHG1GzKGnfqmexxNkQQKMPmnjlYseDkB93gSw1/lk2pZ0hebjRasV2WaHQO978H8RNuY2CslBFOtxLxh8qAKU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1556819682; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=dDbOYZ8xMFl6vDRnhQfpTLt8wCk683IglgGRiaT3cOg=; b=Mbfm9LkiaPW7CkJNbDs3pG1rii8g7/5xol6ZLsb7gjMQUpAN228z9YQaPB+ErM0cR/xDVW29VvRW9QHWT706bmZ4O+olbRRhWvq9heP9VVmsC4XtbwMCSRTGWBy7yO55L0PkuYxvOL/FG9ORudhBqHmZnyZYbCdvnWqTFV9yBm0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1556819681981586.0678902235815; Thu, 2 May 2019 10:54:41 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMFtU-0006i1-3c; Thu, 02 May 2019 17:53:16 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMFtS-0006hT-Ao for xen-devel@lists.xenproject.org; Thu, 02 May 2019 17:53:14 +0000 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 27ab44f2-6d03-11e9-843c-bc764e045a96; Thu, 02 May 2019 17:53:13 +0000 (UTC) X-Inumbo-ID: 27ab44f2-6d03-11e9-843c-bc764e045a96 X-IronPort-AV: E=Sophos;i="5.60,422,1549929600"; d="scan'208";a="84983442" From: Ian Jackson To: Date: Thu, 2 May 2019 18:42:38 +0100 Message-ID: <20190502174238.23848-10-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190502174238.23848-1-ian.jackson@eu.citrix.com> References: <20190502174238.23848-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 9/9] mg-repro-flight: Provide --rebuild to make variant build jobs too X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This allows a single command to repro a particular job with a variety of different source code. The implementation technique is: - run the build job in a separate flight, so that it can run with a separate task which gives its host up after the build - do much of the heavy lifting of runvar fiddling etc. in a new helper routine in cs-adjust-flight Signed-off-by: Ian Jackson --- cs-adjust-flight | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ mg-repro-setup | 84 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 192 insertions(+), 1 deletion(-) diff --git a/cs-adjust-flight b/cs-adjust-flight index cc1684b4..5be5af38 100755 --- a/cs-adjust-flight +++ b/cs-adjust-flight @@ -469,6 +469,115 @@ END $runvarq->execute($newjob, $dstflight, $oldjob); }) } + +sub change__repro_buildjobs { + # helper for mg-repro-setup + # arguments to this change are + # EXAMPLE-FLIGHT EXAMPLE-JOB THING... + # where THING is + # .BUILDJOB =3DBUILDJOBRUNVAR +TREE=3DURL#REVISION + # (we eat all until it doesn't start with . =3D +) + # stdout output is + # list of runvar settings for repro job + die unless @changes >=3D 2; + my $eflight =3D shift @changes; + my $ejob =3D shift @changes; + my @refspecs; + my @varspecs; + my %treespecs; + while (@changes && $changes[0] =3D~ m/^[.=3D+]/) { + local $_ =3D shift @changes; + if (m/^[.=3D]/) { + push @refspecs, $_; + } elsif (m/^\+(.*?)\=3D(.*)\#(.*)$/) { + die $1 if $treespecs{$1}; + $treespecs{$1} =3D [$2,$3]; + } elsif (m/^-r/) { + push @varspecs, $_; + } else { + die "bad .BUILDJOB / =3DBUILDJOBRUNVAR / +TREE=3DURL#REVISION): $_\n"; + } + } + my $testq =3D db_prepare(<execute($eflight, $ejob); + while (my ($refvar, $bjobref) =3D $testq->fetchrow_array()) { + my ($bflight, $bjob) =3D flight_otherjob($eflight,$bjobref); + if (@refspecs) { + my $y =3D 0; + foreach my $rs (@refspecs) { + if ($rs eq ".$bjob" || $rs eq "=3D$refvar") { + $y =3D 1; + last; + } + } + next unless $y; + } + $buildq->execute($bflight,$bjob,@buildq_xvars); + my %got; + while (my ($treevar) =3D $buildq->fetchrow_array()) { + $treevar =3D~ m/^tree_/ or die "$treevar ?"; + $got{$'} =3D 1; + next unless $treespecs{$'}; + $treespecs{$'}[2]++; + } + next unless %got || @refspecs; + + $bjobs{$bjob} //=3D { Template =3D> $bflight }; + $bjobs{$bjob}{Template} eq $bflight or + die "inconsistent $bjob: $bjobs{$bjob}{Template} !=3D $bflight"; + push @{ $bjobs{$bjob}{Refs} }, $refvar; + $bjobs{$bjob}{Trees}{$_} =3D 1 foreach keys %got; + } + + foreach my $tree (sort keys %treespecs) { + die "unused tree/revision adjustment $tree" + .(@refspecs + ? " (no tree_$tree var in any of " + .(join ' ', map { "$bjobs{$_}{Template}.$_" } + sort keys %bjobs).")" + : '') + unless $treespecs{$tree}[2]; + } + + my @copy_jobs_qs =3D copy_jobs_qs(); + foreach my $bjob (sort keys %bjobs) { + copy_jobs_do(\@copy_jobs_qs, $bjobs{$bjob}{Template}, $bjob); + foreach my $tree (sort keys %{ $bjobs{$bjob}{Trees} }) { + runvar_set($bjob, "tree_$tree", $treespecs{$tree}[0]); + runvar_set($bjob, "revision_$tree", $treespecs{$tree}[1]); + } + foreach (@varspecs) { + if (m/^-r!|^-r^/) { + $runvar_rm_q->execute($dstflight, $bjob, $'); + } elsif (m/^-r(.*?)=3D/) { + runvar_set($bjob, $1, $'); + } else { + die "bad -r $_ ?"; + } + } + foreach my $refvar (@{ $bjobs{$bjob}{Refs} }) { + die "$bjob $refvar $& ?" if + "$bjob.$refvar" =3D~ m{[^+=3D_./,:0-9a-z-]}i; + print "runvar-set . $refvar $dstflight.$bjob\n" or die $!; + } + } +} + sub changes () { debug("CHANGES...\n"); =20 diff --git a/mg-repro-setup b/mg-repro-setup index 2e1d3b88..abff2ef7 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -19,7 +19,7 @@ =20 usage () { cat <] [-r...] + [.BUILDJOB | =3DBUILDJOBRUNVAR...] + +TREE=3DURL#REVISION... + Also use a different build. Specifically, use URL and REVISION + for certain builds which mention TREE and which are referenced=20 + (directly) in the job JOB in EXAMPLE-FLIGHT. The affected builds + are those referenced by any of the runvars BUILDJOBRUNVAR (in + which case only that job reference is edited) or any whose job + name is any of the BUILDJOB (in which case all references to that + job are adjusted). If no .BUILDJOB and no =3DBUILDJOBRUNVAR are + specified, then all jobs referred to from the example JOB + which mention TREE are affected. + + For example, + --rebuild =3Dxenbuildjob \ + +xen=3Dgit://xenbits.xen.org/people/alice/xen.git#fixes + would look for xenbuild and (say) find that it referred to + build-amd64; it would then copy and use the build-amd64 job + that EXAMPLE-FLIGHT.JOB used, adjusting \`tree_xen' and + \`revision_xen' as specified, and use that for \`xenbuildjob' in + the repro (but not for \`buildjob') + + Whereas + --rebuild .build-amd64 \ + +linux=3Dgit://xenbits.xen.org/people/alice/linux.git#fix= es + would replace all references to any job named build-amd64 + with a new build-amd64 job. + + And + --rebuild \ + +xen=3Dgit://xenbits.xen.org/people/alice/linux.git#fixes + would replace all builds mentioning xen, including for example + build-amd64 (for xenbuildjob) and build-i386 (for buildjob) + in an x86 32-on-64 flight. + + Host allocation for the build is done "normally" (ie, the host + is thrown away after the build is complete), with a default + blessing of \`adhoc'. + + --rebuild is affected by -E or -P but not general -B or -f. + END =20 } @@ -122,10 +165,37 @@ progress "logging to $logfile" savelog "$logfile" exec 3>"$logfile" =20 +rebuilds_flight=3D'' + while [ $# -ne 0 ]; do arg=3D$1; shift =20 case "$arg" in + + --rebuild) + rebuilds_blessing=3Dadhoc + rebuild_specs=3D() + while true; do + case "$1" in + [.=3D+]*|-r) rebuild_specs+=3D("$1"); shift ;; + -B?*) rebuilds_blessing=3D${1#-B}; shift ;; + -*) badusage ': bad --rebuild option' ;; + *) break ;; + esac + done + if [ x$rebuilds_flight =3D x ]; then + rebuilds_flight=3D$( + ./cs-adjust-flight new:$rebuilds_blessing + ) + fi + + adjusts+=3D($( + ./cs-adjust-flight $rebuilds_flight \ + repro-buildjobs $example_flight $job \ + "${rebuild_specs[@]}" + )) + ;; + none:) # provided so we can repro a job with no hosts ;; @@ -170,6 +240,18 @@ done flight=3D$(./cs-adjust-flight new:$blessing) progress "new flight is $flight" =20 +if [ "x$rebuilds_flight" !=3D x ]; then + progress "running build(s) $rebuilds_flight" + ./mg-execute-flight -B$rebuilds_blessing --progress-fd=3D2 \ + "${mgexecflags[@]}" $rebuilds_flight + + mro=3Dtmp/$rebuilds_flight.mro + if ! egrep '^perfect' >/dev/null $mro; then + echo >&2 "build(s) failed (no 'perfect' in $mro)" + exit 1 + fi +fi + OSSTEST_TASK=3D$(perl -e ' use Osstest; use Osstest::Executive; --=20 2.11.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel