From nobody Tue Feb 10 10:59:19 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=patchew-devel-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1579187362; cv=none; d=zohomail.com; s=zohoarc; b=I9VkuvtvyfnKUiFzhaWJMvDdUMBHRY3fStrvB6i3As8FnldTX3mkJmfpAWPxW8jngpTRcwjlGCTNZ/HQGztsSQeIVvwgdbq8KHYb4YOwmXmMmxyhydOPYYQGqmUL53RFYn7Ng2UfMle9XHhwNYuF2QVvqmpK/VV+RlFcpD6l4RM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579187362; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ONX87NW5GsjNnn3eIv4U3wFn4ZNp2hSfszTEHEXewig=; b=Xmi/bb2Fadv4jrWtY3GJ4VPVJQ00HzQmb5HxzEGFTZZxbTunhYdd7o1iaty48FRPTmldsl7zhF8iNgEaXR+PSQJy+Lltg0jvLbSjLilxNCapLOaCfhLshbjIJegDu33Hv0O3ex4Eg+TpyeqyzitJDICLM5XbrKN0PLdygl5WBr0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 15791873622191013.8167316887954; Thu, 16 Jan 2020 07:09:22 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-407-Rp2yOsulNMuqtQL98vnujw-1; Thu, 16 Jan 2020 10:09:20 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E50111800D48; Thu, 16 Jan 2020 15:09:18 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D95208889D; Thu, 16 Jan 2020 15:09:18 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id CC29C18034EA; Thu, 16 Jan 2020 15:09:18 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 00GF9Go0024600 for ; Thu, 16 Jan 2020 10:09:16 -0500 Received: by smtp.corp.redhat.com (Postfix) id ECB2A842BA; Thu, 16 Jan 2020 15:09:16 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-112-61.ams2.redhat.com [10.36.112.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5448A80890 for ; Thu, 16 Jan 2020 15:09:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579187361; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ONX87NW5GsjNnn3eIv4U3wFn4ZNp2hSfszTEHEXewig=; b=HWGqkgEqqqpiL6Q6W+HXVkJd6xXCT4jOr9n36Ti21+HGer0BGt7b/Jq50509Vu3WxxaxPi NpY7U0EnoJ5NaNErAWg05QVCjZVz3s4AFPovvkJIthhO6oqMXMWjh3ZBLBs7gDc1nfE9xd g2ji/rwMZjKl9oD9BzBKmYf+DRk0Hb8= From: Paolo Bonzini To: patchew-devel@redhat.com Date: Thu, 16 Jan 2020 16:09:03 +0100 Message-Id: <20200116150909.21388-4-pbonzini@redhat.com> In-Reply-To: <20200116150909.21388-1-pbonzini@redhat.com> References: <20200116150909.21388-1-pbonzini@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: patchew-devel@redhat.com Subject: [Patchew-devel] [PATCH 3/9] add topic model X-BeenThere: patchew-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Patchew development and discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: patchew-devel-bounces@redhat.com Errors-To: patchew-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: Rp2yOsulNMuqtQL98vnujw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This will be used to allow grouping together even series whose subject has changed. It will also replace the is_series_head field. To migrate the database, create a new topic for every stripped_subject. Signed-off-by: Paolo Bonzini --- api/migrations/0053_auto_20200116_0955.py | 27 +++++++++++++ api/migrations/0054_populate_topic.py | 29 ++++++++++++++ api/models.py | 46 ++++++++++++++++++++--- tests/test_message.py | 30 +++++++++++++++ 4 files changed, 126 insertions(+), 6 deletions(-) create mode 100644 api/migrations/0053_auto_20200116_0955.py create mode 100644 api/migrations/0054_populate_topic.py diff --git a/api/migrations/0053_auto_20200116_0955.py b/api/migrations/005= 3_auto_20200116_0955.py new file mode 100644 index 0000000..3fae555 --- /dev/null +++ b/api/migrations/0053_auto_20200116_0955.py @@ -0,0 +1,27 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.11.20 on 2020-01-16 09:55 +from __future__ import unicode_literals + +from django.db import migrations, models +import django.db.models.deletion + + +class Migration(migrations.Migration): + + dependencies =3D [ + ('api', '0052_auto_20190418_1357'), + ] + + operations =3D [ + migrations.CreateModel( + name=3D'Topic', + fields=3D[ + ('id', models.AutoField(auto_created=3DTrue, primary_key= =3DTrue, serialize=3DFalse, verbose_name=3D'ID')), + ], + ), + migrations.AddField( + model_name=3D'message', + name=3D'topic', + field=3Dmodels.ForeignKey(null=3DTrue, on_delete=3Ddjango.db.m= odels.deletion.CASCADE, to=3D'api.Topic'), + ), + ] diff --git a/api/migrations/0054_populate_topic.py b/api/migrations/0054_po= pulate_topic.py new file mode 100644 index 0000000..359f00e --- /dev/null +++ b/api/migrations/0054_populate_topic.py @@ -0,0 +1,29 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.conf import settings +from django.db import migrations + + +def topic_fill(apps, schema_editor): + Message =3D apps.get_model("api", "Message") + Topic =3D apps.get_model("api", "Topic") + for record in ( + Message.objects.filter(is_series_head=3DTrue) + .values("stripped_subject") + .distinct() + ): + topic =3D Topic() + topic.save() + topic_subject =3D record["stripped_subject"] + Message.objects.filter( + is_series_head=3DTrue, stripped_subject=3Dtopic_subject + ).update(topic=3Dtopic) + + +class Migration(migrations.Migration): + dependencies =3D [("api", "0053_auto_20200116_0955")] + + operations =3D [ + migrations.RunPython(topic_fill, reverse_code=3Dmigrations.RunPyth= on.noop) + ] diff --git a/api/models.py b/api/models.py index 81bc167..6f4ea4b 100644 --- a/api/models.py +++ b/api/models.py @@ -457,7 +457,11 @@ class MessageManager(models.Manager): msg.stripped_subject =3D m.get_subject(strip_tags=3DTrue) msg.version =3D m.get_version() msg.prefixes =3D m.get_prefixes() - msg.is_series_head =3D m.is_series_head() + msg.is_series_head =3D False + if m.is_series_head(): + msg.is_series_head =3D True + msg.topic =3D Topic.objects.for_stripped_subject(msg.stripped_= subject) + msg.is_patch =3D m.is_patch() msg.patch_num =3D m.get_num()[0] msg.project =3D project @@ -477,18 +481,25 @@ class MessageManager(models.Manager): projects =3D [Project.object.get(name=3Dproject_name)] else: projects =3D find_message_projects(m) + stripped_subject =3D m.get_subject(strip_tags=3DTrue) + is_series_head =3D m.is_series_head() for p in projects: msg =3D Message( message_id=3Dmsgid, in_reply_to=3Dm.get_in_reply_to() or "", date=3Dm.get_date(), subject=3Dm.get_subject(), - stripped_subject=3Dm.get_subject(strip_tags=3DTrue), + stripped_subject=3Dstripped_subject, version=3Dm.get_version(), sender=3Dm.get_from(), recipients=3Dm.get_to() + m.get_cc(), prefixes=3Dm.get_prefixes(), - is_series_head=3Dm.is_series_head(), + is_series_head=3Dis_series_head, + topic=3D( + Topic.objects.for_stripped_subject(stripped_subject) + if is_series_head + else None + ), is_patch=3Dm.is_patch(), patch_num=3Dm.get_num()[0], ) @@ -520,6 +531,26 @@ class QueuedSeries(models.Model): index_together =3D [("user", "message")] =20 =20 +class TopicManager(models.Manager): + def for_stripped_subject(self, stripped_subject): + q =3D ( + Message.objects.filter(stripped_subject=3Dstripped_subject) + .order_by("date") + .reverse()[:1] + .values("topic") + ) + if q: + topic =3D self.get(pk=3Dq[0]["topic"]) + else: + topic =3D Topic() + topic.save() + return topic + + +class Topic(models.Model): + objects =3D TopicManager() + + class Message(models.Model): """ Patch email message """ =20 @@ -555,6 +586,11 @@ class Message(models.Model): is_tested =3D models.BooleanField(default=3DFalse) is_reviewed =3D models.BooleanField(default=3DFalse) =20 + # is series head if not Null + topic =3D models.ForeignKey( + "Topic", on_delete=3Dmodels.CASCADE, null=3DTrue, db_index=3DTrue + ) + # patch index number if is_patch patch_num =3D models.PositiveSmallIntegerField(null=3DTrue, blank=3DTr= ue) =20 @@ -854,9 +890,7 @@ class Message(models.Model): =20 def get_alternative_revisions(self): assert self.is_series_head - return Message.objects.series_heads().filter( - project=3Dself.project, stripped_subject=3Dself.stripped_subje= ct - ) + return Message.objects.filter(project=3Dself.project, topic=3Dself= .topic) =20 def set_complete(self): if self.is_complete: diff --git a/tests/test_message.py b/tests/test_message.py index bc0a98d..3999948 100755 --- a/tests/test_message.py +++ b/tests/test_message.py @@ -20,6 +20,7 @@ from api.models import Message class MessageTest(PatchewTestCase): def setUp(self): self.create_superuser() + self.p =3D self.add_project("QEMU", "qemu-devel@nongnu.org") =20 def test_0_second(self): message =3D Message() @@ -54,6 +55,35 @@ class MessageTest(PatchewTestCase): asctime =3D message.get_asctime() self.assertEqual(asctime, "Sat Oct 22 9:06:04 2016") =20 + def test_topic_on_series_head(self): + self.cli_login() + self.cli_import("0004-multiple-patch-reviewed.mbox.gz") + m1 =3D Message.objects.get( + message_id=3D"1469192015-16487-1-git-send-email-berrange@redha= t.com" + ) + assert m1.is_series_head + assert m1.topic is not None + m2 =3D Message.objects.get( + message_id=3D"1469192015-16487-2-git-send-email-berrange@redha= t.com" + ) + assert not m2.is_series_head + assert m2.topic is None + + def test_topic_assignment(self): + self.cli_login() + self.cli_import("0004-multiple-patch-reviewed.mbox.gz") + self.cli_import("0009-obsolete-by.mbox.gz") + + m1 =3D Message.objects.get(message_id=3D"20160628014747.20971-1-fa= mz@redhat.com") + m2 =3D Message.objects.get(message_id=3D"20160628014747.20971-2-fa= mz@redhat.com") + m3 =3D Message.objects.get(message_id=3D"20160628014747.20971-3-fa= mz@redhat.com") + self.assertEqual(m1.topic, m2.topic) + self.assertEqual(m1.topic, m3.topic) + n =3D Message.objects.get( + message_id=3D"1469192015-16487-1-git-send-email-berrange@redha= t.com" + ) + self.assertNotEqual(m1.topic, n.topic) + =20 if __name__ =3D=3D "__main__": main() --=20 2.21.0 _______________________________________________ Patchew-devel mailing list Patchew-devel@redhat.com https://www.redhat.com/mailman/listinfo/patchew-devel