From nobody Tue Feb 10 02:49:45 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=eu.citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1595356979; cv=none; d=zohomail.com; s=zohoarc; b=QEUlxRtTkpQPSIEA+iWVipTWehNC3qIusV+9Ie+XdmIGuQ15RURw/cTdOWO1iSnAYaP5erdxlDvOdWhhsUUxTDG6R3YXJ2gDG7OexWKTeBrAHa35BuEtJCKvET9qfrWTyjKjeC9PwF5D+gpS7VmUpI62aFDyOruJ+yJeSZLIMXg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595356979; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c95XezDVRe5W2bK8IajUL61YmK9eImbaYj/bwDB1CCk=; b=lJ5LltOC3x8XgNetknqZArceQDbQeCYWHioeumh3u5BEc2kHHuUMeTmznh+Y+Cv2AIm03NGPqbPWS/VglNnDAzps1wN4Z3UQAI7LbW+FZQE3jxaH+lMbX99DLvMfaDvZzZrrJJy1h2WUzGaApRM+0wEYKWLy6NFZ9DwGkTZ/8pU= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1595356979332847.2883333731872; Tue, 21 Jul 2020 11:42:59 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jxxDu-00024h-PK; Tue, 21 Jul 2020 18:42:42 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jxxDt-0001xV-Mo for xen-devel@lists.xenproject.org; Tue, 21 Jul 2020 18:42:41 +0000 Received: from chiark.greenend.org.uk (unknown [2001:ba8:1e3::]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e7f1f12a-cb81-11ea-85a2-bc764e2007e4; Tue, 21 Jul 2020 18:42:21 +0000 (UTC) Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk) by chiark.greenend.org.uk (Debian Exim 4.84_2 #1) with esmtp (return-path ijackson@chiark.greenend.org.uk) id 1jxxDY-0001u7-F0; Tue, 21 Jul 2020 19:42:20 +0100 X-Inumbo-ID: e7f1f12a-cb81-11ea-85a2-bc764e2007e4 From: Ian Jackson To: xen-devel@lists.xenproject.org Subject: [OSSTEST PATCH 05/14] sg-report-flight: Use WITH to use best index use for $flightsq Date: Tue, 21 Jul 2020 19:41:56 +0100 Message-Id: <20200721184205.15232-6-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200721184205.15232-1-ian.jackson@eu.citrix.com> References: <20200721184205.15232-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Content-Type: text/plain; charset="utf-8" While we're here, convert this EXISTS subquery to a JOIN. Perf: runtime of my test case now ~200-300s. Example query before (from the Perl DBI trace): SELECT * FROM ( SELECT DISTINCT flight, blessing FROM flights JOIN runvars r1 USING (flight) WHERE (branch=3D'xen-unstable') AND ( (TRUE AND flight <=3D 151903) AND (blessing=3D'real') ) AND EXISTS (SELECT 1 FROM jobs WHERE jobs.flight =3D flights.flight AND jobs.job =3D ?) AND r1.name LIKE 'built_revision_%' AND r1.name =3D ? AND r1.val=3D ? ORDER BY flight DESC LIMIT 1000 ) AS sub ORDER BY blessing ASC, flight DESC With bind variables: "test-armhf-armhf-libvirt" 'built_revision_xen' '165f3afbfc3db70fcfdccad07085cde0a03c858b' After: WITH sub AS ( SELECT DISTINCT flight, blessing FROM flights JOIN runvars r1 USING (flight) WHERE (branch=3D'xen-unstable') AND ( (TRUE AND flight <=3D 151903) AND (blessing=3D'real') ) AND r1.name LIKE 'built_revision_%' AND r1.name =3D ? AND r1.val=3D ? ORDER BY flight DESC LIMIT 1000 ) SELECT * FROM sub JOIN jobs USING (flight) WHERE (1=3D1) AND jobs.job =3D ? ORDER BY blessing ASC, flight DESC With bind variables: 'built_revision_xen' '165f3afbfc3db70fcfdccad07085cde0a03c858b' "test-armhf-armhf-libvirt" Diff to the query: - SELECT * FROM ( + WITH sub AS ( SELECT DISTINCT flight, blessing FROM flights JOIN runvars r1 USING (flight) WHERE (branch=3D'xen-unstable') AND ( (TRUE AND flight <=3D 151903) AND (blessing=3D'real') ) - AND EXISTS (SELECT 1 - FROM jobs - WHERE jobs.flight =3D flights.flight - AND jobs.job =3D ?) - AND r1.name LIKE 'built_revision_%' AND r1.name =3D ? AND r1.val=3D ? ORDER BY flight DESC LIMIT 1000 - ) AS sub + ) + SELECT * + FROM sub + JOIN jobs USING (flight) + + WHERE (1=3D1) + AND jobs.job =3D ? + ORDER BY blessing ASC, flight DESC CC: George Dunlap Signed-off-by: Ian Jackson --- sg-report-flight | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/sg-report-flight b/sg-report-flight index 61aec7a8..b5398573 100755 --- a/sg-report-flight +++ b/sg-report-flight @@ -180,18 +180,6 @@ END return undef; } =20 - my @flightsq_params; - my $flightsq_jobcond=3D'(1=3D1)'; - if (defined $job) { - push @flightsq_params, $job; - $flightsq_jobcond =3D <