This is a combination of two series and some new work:
* [OSSTEST PATCH 00/14] Flight report performance improvements
* [OSSTEST PATCH 00/11] Improve performance of sg-report-host-history
* New work to improve the performance of cs-bisection-step
* Fixes to usage of SQL LIKE
Thanks to George for reviews of some of the most critical patches to
the regression analyser in sg-report-flight. Where necessary I have
rebased those reviewed patches over the SQL LIKE fixes, and in those
cases I retained the reviewed-by. I hope that's OK.
Outstanding in my perf programme is sg-report-job-history.
Ian Jackson (41):
Add cperl-indent-level to .dir-locals.el
SQL: Use "LIKE" rather than "like", etc.
SQL: Fix incorrect LIKE pattern syntax (literals)
SQL: Fix incorrect LIKE pattern syntax (program variables)
sg-report-flight: Add a comment re same-flight search narrowing
sg-report-flight: Sort failures by job name as last resort
schema: Provide indices for sg-report-flight
sg-report-flight: Ask the db for flights of interest
sg-report-flight: Use WITH to use best index use for $flightsq
sg-report-flight: Use WITH clause to use index for $anypassq
sg-report-flight: Use the job row from the intitial query
Executive: Use index for report__find_test
duration_estimator: Ignore truncated jobs unless we know the step
duration_estimator: Introduce some _qtxt variables
duration_estimator: Explicitly provide null in general host q
duration_estimator: Return job column in first query
duration_estimator: Move $uptincl_testid to separate @x_params
duration_estimator: Move duration query loop into database
Executive: Drop redundant AND clause
schema: Add index for quick lookup by host
sg-report-host-history: Find flight limit by flight start date
sg-report-host-history: Drop per-job debug etc.
Executive: Export opendb_tests
sg-report-host-history: Add a debug print after sorting jobs
sg-report-host-history: Do the main query per host
sg-report-host-history: Rerganisation: Make mainquery per-host
sg-report-host-history: Rerganisation: Read old logs later
sg-report-host-history: Rerganisation: Change loops
sg-report-host-history: Drop a redundznt AND clause
sg-report-host-history: Fork
schema: Add index to help cs-bisection-step
adhoc-revtuple-generator: Fix an undef warning in a debug print
cs-bisection-step: Generalise qtxt_common_rev_ok
cs-bisection-step: Move an AND
cs-bisection-step: Break out qtxt_common_ok
cs-bisection-step: Use db_prepare a few times instead of ->do
cs-bisection-step: temporary table: Insert only rows we care about
SQL: Change LIKE E'...\\_...' to LIKE '...\_...'
cs-bisection-step: Add a debug print when we run dot(1)
cs-bisection-step: Lay out the revision tuple graph once
duration_estimator: Clarify recentflights query a bit
.dir-locals.el | 3 +-
Osstest.pm | 8 +-
Osstest/Executive.pm | 79 +++++++++------
Osstest/JobDB/Executive.pm | 2 +-
adhoc-revtuple-generator | 2 +-
cr-ensure-disk-space | 4 +-
cs-adjust-flight | 2 +-
cs-bisection-step | 51 +++++++---
mg-force-push | 2 +-
mg-report-host-usage-collect | 10 +-
ms-planner | 2 +-
schema/runvars-built-index.sql | 7 ++
schema/runvars-host-index.sql | 8 ++
schema/runvars-revision-index.sql | 7 ++
schema/steps-broken-index.sql | 7 ++
schema/steps-job-index.sql | 7 ++
sg-report-flight | 129 ++++++++++++++++++++----
sg-report-host-history | 161 +++++++++++++++++-------------
sg-report-job-history | 4 +-
ts-logs-capture | 2 +-
20 files changed, 344 insertions(+), 153 deletions(-)
create mode 100644 schema/runvars-built-index.sql
create mode 100644 schema/runvars-host-index.sql
create mode 100644 schema/runvars-revision-index.sql
create mode 100644 schema/steps-broken-index.sql
create mode 100644 schema/steps-job-index.sql
--
2.20.1