From nobody Mon Sep 16 18:52:01 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=patchew-devel-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1542637536400180.28005323012246; Mon, 19 Nov 2018 06:25:36 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9F4643082B71; Mon, 19 Nov 2018 14:25:34 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9041F5D75D; Mon, 19 Nov 2018 14:25:34 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 82999181B9EA; Mon, 19 Nov 2018 14:25:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wAJDgjrI026472 for ; Mon, 19 Nov 2018 08:42:45 -0500 Received: by smtp.corp.redhat.com (Postfix) id 00C9E5C223; Mon, 19 Nov 2018 13:42:45 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-112-56.ams2.redhat.com [10.36.112.56]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D5B85C8A1 for ; Mon, 19 Nov 2018 13:42:43 +0000 (UTC) From: Paolo Bonzini To: patchew-devel@redhat.com Date: Mon, 19 Nov 2018 14:42:36 +0100 Message-Id: <20181119134236.11260-3-pbonzini@redhat.com> In-Reply-To: <20181119134236.11260-1-pbonzini@redhat.com> References: <20181119134236.11260-1-pbonzini@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: patchew-devel@redhat.com Subject: [Patchew-devel] [PATCH 2/2] work around bug in postgresql full text search X-BeenThere: patchew-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Patchew development and discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: patchew-devel-bounces@redhat.com Errors-To: patchew-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Mon, 19 Nov 2018 14:25:35 +0000 (UTC) Content-Type: text/plain; charset="utf-8" The GIN index is not used when searching for keywords. Hack around this bug in Postgres. Signed-off-by: Paolo Bonzini --- api/search.py | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/api/search.py b/api/search.py index 488afa1..66f78b9 100644 --- a/api/search.py +++ b/api/search.py @@ -14,7 +14,7 @@ from functools import reduce from django.db import connection from django.db.models import Q =20 -from django.contrib.postgres.search import SearchQuery, SearchVector +from django.contrib.postgres.search import SearchQuery, SearchVector, Sear= chVectorField from django.db.models import Lookup, lookups from django.db.models.fields import Field =20 @@ -33,6 +33,20 @@ class NotEqual(Lookup): class InvalidSearchTerm(Exception): pass =20 +# Hack alert: Django wraps each argument to to_tsvector with a COALESCE fu= nction, +# and that causes postgres not to use the index. Monkeypatch the construc= tor +# to skip that step, which we do not need since the subject field is not n= ullable. +class NonNullSearchVector(SearchVector): + function =3D 'to_tsvector' + arg_joiner =3D " || ' ' || " + _output_field =3D SearchVectorField() + config =3D None + + def __init__(self, *expressions, **extra): + super(SearchVector, self).__init__(*expressions, **extra) + self.config =3D self.extra.get('config', self.config) + self.weight =3D None + =20 class SearchEngine(object): """ @@ -346,7 +360,7 @@ Search text keyword in the email message. Example: queryset =3D Message.objects.series_heads() if self._last_keywords: if connection.vendor =3D=3D 'postgresql': - queryset =3D queryset.annotate(subjsearch=3DSearchVector('= subject', config=3D'english')) + queryset =3D queryset.annotate(subjsearch=3DNonNullSearchV= ector('subject', config=3D'english')) searchq =3D reduce(lambda x, y: x & SearchQuery(y,config= =3D'english'), self._last_keywords) q =3D q & Q(subjsearch=3Dsearchq) --=20 2.19.1 _______________________________________________ Patchew-devel mailing list Patchew-devel@redhat.com https://www.redhat.com/mailman/listinfo/patchew-devel