From nobody Mon Feb 9 09:34:19 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=eu.citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1596195565; cv=none; d=zohomail.com; s=zohoarc; b=LrNp6xPQuMzEgL/MW5oqGxQK66e6xOJaIXXXZI3De/YNI5HNfTRv3dHZgPCkTaWjm6Zh1fKFvmzOQ6ZpQVqITPWv8Y1BCnKFUjRD7Zcw4Lp3qxyDtw+M/+SxREzImPeE6naXTwgG51hKOvFGwBJ9Juq+T+YqWCDVdNu7Wkc/Aws= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1596195565; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lzvuwsnB2oy/o6KxuMQEWH6fug3TeH0YJ6LgsKPMQ+4=; b=fifvzje2JzCMdGTHIvXgrqsyMtPXGgQQMl0PaXdmSjXw5eU9ArYs7MKKSeYWVNdajinUBGaIl+pTwsnwFrM26Qy7/ZqPwL8yR85SxC53BW/oFWFJnGRuHWPOPp5KdFadY1+rPF+QN8bMNNHc47zDu2+c7nnstmcvW3HtntWM+bg= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1596195565795866.217322798193; Fri, 31 Jul 2020 04:39:25 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1TNX-00065Q-AR; Fri, 31 Jul 2020 11:39:11 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1TNX-0005kZ-0j for xen-devel@lists.xenproject.org; Fri, 31 Jul 2020 11:39:11 +0000 Received: from chiark.greenend.org.uk (unknown [2001:ba8:1e3::]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 5aa0cf16-d322-11ea-8e26-bc764e2007e4; Fri, 31 Jul 2020 11:38:30 +0000 (UTC) Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk) by chiark.greenend.org.uk (Debian Exim 4.84_2 #1) with esmtp (return-path ijackson@chiark.greenend.org.uk) id 1k1TMr-0001W4-KU; Fri, 31 Jul 2020 12:38:29 +0100 X-Inumbo-ID: 5aa0cf16-d322-11ea-8e26-bc764e2007e4 From: Ian Jackson To: xen-devel@lists.xenproject.org Subject: [OSSTEST PATCH v2 09/41] sg-report-flight: Use WITH to use best index use for $flightsq Date: Fri, 31 Jul 2020 12:37:48 +0100 Message-Id: <20200731113820.5765-10-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200731113820.5765-1-ian.jackson@eu.citrix.com> References: <20200731113820.5765-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Content-Type: text/plain; charset="utf-8" While we're here, convert this EXISTS subquery to a JOIN. Perf: runtime of my test case now ~200-300s. Example query before (from the Perl DBI trace): SELECT * FROM ( SELECT DISTINCT flight, blessing FROM flights JOIN runvars r1 USING (flight) WHERE (branch=3D'xen-unstable') AND ( (TRUE AND flight <=3D 151903) AND (blessing=3D'real') ) AND EXISTS (SELECT 1 FROM jobs WHERE jobs.flight =3D flights.flight AND jobs.job =3D ?) AND r1.name LIKE 'built_revision_%' AND r1.name =3D ? AND r1.val=3D ? ORDER BY flight DESC LIMIT 1000 ) AS sub ORDER BY blessing ASC, flight DESC With bind variables: "test-armhf-armhf-libvirt" 'built_revision_xen' '165f3afbfc3db70fcfdccad07085cde0a03c858b' After: WITH sub AS ( SELECT DISTINCT flight, blessing FROM flights JOIN runvars r1 USING (flight) WHERE (branch=3D'xen-unstable') AND ( (TRUE AND flight <=3D 151903) AND (blessing=3D'real') ) AND r1.name LIKE 'built_revision_%' AND r1.name =3D ? AND r1.val=3D ? ORDER BY flight DESC LIMIT 1000 ) SELECT * FROM sub JOIN jobs USING (flight) WHERE (1=3D1) AND jobs.job =3D ? ORDER BY blessing ASC, flight DESC With bind variables: 'built_revision_xen' '165f3afbfc3db70fcfdccad07085cde0a03c858b' "test-armhf-armhf-libvirt" Diff to the query: - SELECT * FROM ( + WITH sub AS ( SELECT DISTINCT flight, blessing FROM flights JOIN runvars r1 USING (flight) WHERE (branch=3D'xen-unstable') AND ( (TRUE AND flight <=3D 151903) AND (blessing=3D'real') ) - AND EXISTS (SELECT 1 - FROM jobs - WHERE jobs.flight =3D flights.flight - AND jobs.job =3D ?) - AND r1.name LIKE 'built_revision_%' AND r1.name =3D ? AND r1.val=3D ? ORDER BY flight DESC LIMIT 1000 - ) AS sub + ) + SELECT * + FROM sub + JOIN jobs USING (flight) + + WHERE (1=3D1) + AND jobs.job =3D ? + ORDER BY blessing ASC, flight DESC Reviewed-by: George Dunlap Signed-off-by: Ian Jackson --- sg-report-flight | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/sg-report-flight b/sg-report-flight index 10127582..d06be292 100755 --- a/sg-report-flight +++ b/sg-report-flight @@ -180,18 +180,6 @@ END return undef; } =20 - my @flightsq_params; - my $flightsq_jobcond=3D'(1=3D1)'; - if (defined $job) { - push @flightsq_params, $job; - $flightsq_jobcond =3D <