From nobody Mon Feb 9 20:46:18 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=eu.citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1573239089; cv=none; d=zoho.com; s=zohoarc; b=EmJCyEAsNcM2eNxoffq/y7jcW5yVEuiL3m7BrMocuH9RgPFdG8SsT+hdPuMWpaKhIKxEvnat44V4yHsBYCIS7bOaNciNtPVjBJITRFd72pQ/ayRSAjgs0Uua0TRPrcvSUUl2ZRkIrR0k/jErUJ/e26fwHTtEdKjhgr9FEDBgn0k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573239089; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pOdAIx4BOXoPp8MbiKfYF61Bn4TB9z0qhOXVmNkbrt8=; b=bmzgAPZ7IuQqrObOcTS/Hrl0uza1TwRh6E/PY6SxfBjvIByJgA7ZiELxSbGTf0qZLe/4RouETHFPaCz/z2uAInIqM6R3puTAqzOB+3FSJgFfel75uoQ8Q0l5d4AGBSq6IelOsval6Jy4okH/MWFVhqf4EBPCTUTt9iXEDV1KdPM= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1573239089867327.9032543920697; Fri, 8 Nov 2019 10:51:29 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iT9L2-0007rV-GJ; Fri, 08 Nov 2019 18:50:28 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iT9L0-0007qi-H5 for xen-devel@lists.xenproject.org; Fri, 08 Nov 2019 18:50:26 +0000 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 985fefc2-0258-11ea-adbe-bc764e2007e4; Fri, 08 Nov 2019 18:50:13 +0000 (UTC) X-Inumbo-ID: 985fefc2-0258-11ea-adbe-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1573239013; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=/1BUrxLf7Hu33d8hmx2S8NiK/Eg5xK2ZvgQwMUqqH3c=; b=Z2vCB02RM9TMbydqTBpYjP8tyZB9jL28oF9ZLvZJXSBGPJFLKi0LisiG Glaxp97u8E0Dxbor0WLC64VMSx0Xni1U2ml0M12H8Scx+byMCo9dKI92I BOmbZFm2nUqKOgAOZe2DJ0fFs/iT4knn+UdQKrhBuiA/gqLFuHEl1kq2u Y=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=ian.jackson@eu.citrix.com; spf=Pass smtp.mailfrom=Ian.Jackson@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of ian.jackson@eu.citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Ian.Jackson@citrix.com"; x-sender="ian.jackson@eu.citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of Ian.Jackson@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Ian.Jackson@citrix.com"; x-sender="Ian.Jackson@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Ian.Jackson@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: VJ/yjvYsLCAWisoLUxuc5rNBOyWwMd7I4cDzFHpGGwV4L9kOnZIFAaFGd6pWFxVDiZVOOKpWCK wdHLuBXSUFodI6bFvqDOuKvam6VDsgNM3+Q2QwMhx8UJkvV/o/7K5OPR3l53DJ5HX/Zn3iZGxa RILbzu0VLKsOJyuMurduSv5mnH4hF2jDlYDNncKYPG9/jXfp74K6X7kbvgUp6Fwg5lgjNIyG5U oe2iThxnAMkEx27SMBUROOMnIW4C05bVWP47/XHmv6IpuL6gwLT41oDU1EUvqlc0Iq6aywjbyJ C+E= X-SBRS: 2.7 X-MesageID: 8185732 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.68,282,1569297600"; d="scan'208";a="8185732" From: Ian Jackson To: Date: Fri, 8 Nov 2019 18:49:58 +0000 Message-ID: <20191108185001.3319-11-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20191108185001.3319-1-ian.jackson@eu.citrix.com> References: <20191108185001.3319-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 10/13] sg-report-host-history: Read cache entries X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Signed-off-by: Ian Jackson --- sg-report-host-history | 57 ++++++++++++++++++++++++++++++++++++++++++++++= +++- 1 file changed, 56 insertions(+), 1 deletion(-) diff --git a/sg-report-host-history b/sg-report-host-history index 7dcfac9a..e67c7346 100755 --- a/sg-report-host-history +++ b/sg-report-host-history @@ -31,6 +31,7 @@ use Osstest::Executive qw(:DEFAULT :colours); our $limit=3D 200; our $flightlimit; our $htmlout =3D "."; +our $read_existing=3D1; our $doinstall=3D1; our @blessings; =20 @@ -52,6 +53,8 @@ while (@ARGV && $ARGV[0] =3D~ m/^-/) { push @blessings, split ',', $1; } elsif (m/^--html-dir=3D(.*)$/) { $htmlout=3D $1; + } elsif (m/^--regenerate$/) { + $read_existing=3D 0; } elsif (m/^--no-install$/) { $doinstall=3D 0; } elsif (m/^--debug/) { @@ -69,6 +72,41 @@ our $restrictflight_cond =3D restrictflight_cond(); our $flightcond; our $minflight; =20 +our %hcaches; + +sub read_existing_logs ($) { + my ($hostname) =3D @_; + return unless $read_existing; + my $html_file =3D "$htmlout/$hostname.html"; + if (!open H, $html_file) { + return if $!=3D=3DENOENT; + die "failed to open $html_file: $!"; + } + my $tcache =3D { }; + $hcaches{$hostname} =3D $tcache; + for (;;) { + $_ =3D // last; + next unless m{^\<\!-- osstest-report-reuseable (.*)--\>$}; + my $jr =3D {}; + my $ch =3D $jr; + foreach (split / /, $1) { + if (m{^\w+$}) { + $ch =3D { }; + $jr->{'%'.$&} =3D $ch; + next; + } + s{^(\w+)=3D}{} or die; + my $k =3D $1; + s{\%([0-9a-f]{2})}{ chr hex $1 }ge; + $ch->{$k} =3D $_; + print DEBUG "GOTCACHE $hostname $k\n"; + } + print DEBUG "GOTCACHE $hostname \@ $jr->{flight} $jr->{job} $jr->{status}= ,$jr->{name}\n"; + $tcache->{$jr->{flight},$jr->{job},$jr->{status},$jr->{name}} =3D $jr; + } + close H; +} + sub computeflightsrange () { if (!$flightlimit) { my $flagscond =3D @@ -225,16 +263,26 @@ END my $inrows =3D $hosts{$hostname}; print DEBUG "FOUND ", (scalar @$inrows), " ROWS for $hostname\n"; =20 + my $tcache =3D $hcaches{$hostname}; + # Each entry in @$inrows is a $jr, which is a hash # It has keys for the result columns in mainquery # It also has keys '%' (yes, with a literal '%') # which are the results of per-job queries. - # The contents of $jr for each job is cached across runs. (TODO) + # The contents of $jr for each job is cached across runs. =20 my @rows; + my $cachehits =3D 0; foreach my $jr (@$inrows) { print DEBUG "JOB $jr->{flight}.$jr->{job} "; =20 + my $cacherow =3D + $tcache->{$jr->{flight},$jr->{job},$jr->{status},$jr->{name}}; + if ($cacherow) { + $jr =3D $cacherow; + $cachehits++; + } + my $endedrow =3D jobquery($endedq, $jr, 'e'); if (!$endedrow) { print DEBUG "no-finished\n"; @@ -246,6 +294,9 @@ END push @rows, { %$jr, %$endedrow }; } =20 + print DEBUG "CACHE $hostname $cachehits / ".(scalar @rows) + ." of ".(scalar %$tcache)."\n"; + my $write_cache_entry =3D sub { my ($jr) =3D @_; print H "