From nobody Thu Apr 18 22:48:35 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=patchew-devel-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1556795905; cv=none; d=zoho.com; s=zohoarc; b=keeT+6WYcL7ZYjaE+QlrA2lZWxXIceoHzY9ssHQXbac9SeDsCYDLzxVzH94Gp/6IJnF+kuULcrYFC2EBykGXLCF9GFvtpD71P7yO1VaD7rEa8IAdNKO+aLk4fTb3ubPlmDs355zK1RN199DmKDg7UrhA0X8eTiYYh34nujE9gAk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1556795905; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=eOijn/I8+LlrEpnIqMYDtb8eHhiwC4kfcJ1zyLIQkl8=; b=dc+t7B7HgzF5k4ijom9vrl/qH8PRVMQC/xxuVKDbWrFJxBHo8GDy/e1HTqahrPKDZrRQd3jtuNsVI8dSV8b/d4Aj8tRs11KvR5ebCkyVpsdL8uBRKjct9w6ylPOPoTSocB/y51Bvd+i7X4/nHhM0N2hS6zaM2Z8pxrDNKLCDa/I= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=patchew-devel-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1556795905689206.61590541160865; Thu, 2 May 2019 04:18:25 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6E35B30820E6; Thu, 2 May 2019 11:18:24 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6368A827B6; Thu, 2 May 2019 11:18:24 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 598803FB11; Thu, 2 May 2019 11:18:24 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x42BILZ7009052 for ; Thu, 2 May 2019 07:18:21 -0400 Received: by smtp.corp.redhat.com (Postfix) id 395AA7EE9D; Thu, 2 May 2019 11:18:21 +0000 (UTC) Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 31CC37EE8B for ; Thu, 2 May 2019 11:18:21 +0000 (UTC) Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com [209.85.210.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 573E78830A for ; Thu, 2 May 2019 11:18:20 +0000 (UTC) Received: by mail-ot1-f43.google.com with SMTP id n15so1703679ota.1 for ; Thu, 02 May 2019 04:18:20 -0700 (PDT) Received: from localhost.localdomain ([198.59.53.9]) by smtp.gmail.com with ESMTPSA id o1sm19095697otj.11.2019.05.02.04.18.18 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 02 May 2019 04:18:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PyBx+aUzNS6sZ2XP9cPfcyxAYa7BXpEvfryRiVISjME=; b=HPspldjL6tiZlzHcB4gHZEWF2v/jojPW08H/Z0nBQxngdf735Ox4zIIv69dTPkdXU4 9hSGPSu9uymVFqQoXZAPQdm/Y2fbb61CRmycUl6fg1uHTHNzkRaANPvGrP3o2UNhlmgU SldS5h9EpUCOZ/fKAK9gPp4clIpfTe9e74tl3HUBmJ1iveWUK/cgscWe5JyHCWpo3OaW QrxXgmqZ+XNuSZ/WfRpdmnuu+XipPIVQRpLUf4pe2LZHRpieAhX9s3Wj6hJwAG/XB+tD rHToRKDTRb2Ex2w7VrWZrXY+ZyjGselxXfbpx1bZoiXgjSRBLKJyvIHPgDz9+LjEOZ1Y umEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=PyBx+aUzNS6sZ2XP9cPfcyxAYa7BXpEvfryRiVISjME=; b=PBjpZuTnUi5sWdSNamm2Hx53WOkX2onHcOHZmNEj9Oh+pu8zt60ZavvdXemoW3edUf JAPi0WY6NL+kYerNvnZ1udCDRx4d3UGNpeTBLIyMpzz6qFd0u1ukzUcizJB3ga4iizky JjCAk7MwWKkbDOZWJz3IfTuinTJS63AxXr0smuGpRsbIBaOwgovlCs6WJjQyugxQ3xWP szUWTSUbcDzeNShMfrJAJYF929/HIBTGhq2nvnD5Z9FE7aTQSPMXamRJ2YuDSLsI5ZAs mj6Jcjp30u8mkZorSZVfAv6POKbKK25d0dAnvlUCd4M+QLFMFLhaSTh1QrMlYGpxKCYs fg5A== X-Gm-Message-State: APjAAAXCS9+5dbkxlRAWyez8QqNQM9skPFWRVVLRzw0Mo56sb3E8KS+l sfu+bZLSO6Tvmqk6QsamvOXl0t5dz+I= X-Google-Smtp-Source: APXvYqy85kkRui9l08eOQZdnAU7I25xXjWmU9tjKzWVpfqiFxODUUSMgabPgYAhAzu0gx0Hcr/Grtg== X-Received: by 2002:a9d:4e1a:: with SMTP id p26mr2006249otf.116.1556795899529; Thu, 02 May 2019 04:18:19 -0700 (PDT) From: Paolo Bonzini To: patchew-devel@redhat.com Date: Thu, 2 May 2019 05:18:03 -0600 Message-Id: <20190502111804.15843-17-pbonzini@redhat.com> In-Reply-To: <20190502111804.15843-1-pbonzini@redhat.com> References: <20190502111804.15843-1-pbonzini@redhat.com> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 02 May 2019 11:18:20 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 02 May 2019 11:18:20 +0000 (UTC) for IP:'209.85.210.43' DOMAIN:'mail-ot1-f43.google.com' HELO:'mail-ot1-f43.google.com' FROM:'paolo.bonzini@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.125 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS) 209.85.210.43 mail-ot1-f43.google.com 209.85.210.43 mail-ot1-f43.google.com X-RedHat-Possible-Forgery: Paolo Bonzini X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: patchew-devel@redhat.com Subject: [Patchew-devel] [PATCH 16/17] models: switch from property tables to field X-BeenThere: patchew-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Patchew development and discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: patchew-devel-bounces@redhat.com Errors-To: patchew-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Thu, 02 May 2019 11:18:24 +0000 (UTC) X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Fortunately, most accesses to the property tables were hidden behind accessors. Therefore, apart from the usual ugly migration (which in this case is more or less a copy of the config migration) it is almost enough to change the accessors to look into the JSON dictionary. We can also use the new API to access a whole subset of the properties, which simplifies the access to testing.check_in.*. --- .../0052_populate_property_fields.py | 67 +++++++++++ api/models.py | 111 ++++++++++-------- mods/testing.py | 6 +- 3 files changed, 130 insertions(+), 54 deletions(-) create mode 100644 api/migrations/0052_populate_property_fields.py diff --git a/api/migrations/0052_populate_property_fields.py b/api/migratio= ns/0052_populate_property_fields.py new file mode 100644 index 0000000..432e809 --- /dev/null +++ b/api/migrations/0052_populate_property_fields.py @@ -0,0 +1,67 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.conf import settings +from django.db import migrations +from django.db.models import Q + +def do_properties_to_field(obj, propset): + properties =3D {} + props =3D propset.all() + for p in props: + *path, last =3D p.name.split('.') + parent =3D properties + for item in path: + parent =3D parent.setdefault(item, {}) + parent[last] =3D p.value + obj.properties =3D properties + #print(obj, properties) + obj.save() + props.delete() + +def properties_to_field(apps, schema_editor): + Project =3D apps.get_model('api', 'Project') + for po in Project.objects.all(): + do_properties_to_field(po, po.projectproperty_set) + Message =3D apps.get_model('api', 'Message') + for po in Message.objects.all(): + do_properties_to_field(po, po.messageproperty_set) + +def flatten_properties(source, prefix, result=3DNone): + if result is None: + result =3D {} + for k, v in source.items(): + if isinstance(v, dict): + flatten_properties(v, prefix + k + '.', result) + else: + result[prefix + k] =3D v + return result + +def do_field_to_properties(source, propclass, **kwargs): + props =3D flatten_properties(source, '') + for k, v in props.items(): + #print(k, v) + new_prop =3D propclass(name=3Dk, value=3Dv, **kwargs) + new_prop.save() + +def field_to_properties(apps, schema_editor): + Project =3D apps.get_model('api', 'Project') + ProjectProperty =3D apps.get_model('api', 'ProjectProperty') + for po in Project.objects.all(): + do_field_to_properties(po.properties, ProjectProperty, project=3Dp= o) + Message =3D apps.get_model('api', 'Message') + MessageProperty =3D apps.get_model('api', 'MessageProperty') + for m in Message.objects.all(): + do_field_to_properties(m.properties, MessageProperty, message=3Dm) + + +class Migration(migrations.Migration): + + dependencies =3D [ + ('api', '0051_auto_20190418_1346'), + ] + + operations =3D [ + migrations.RunPython(properties_to_field, + reverse_code=3Dfield_to_properties), + ] diff --git a/api/models.py b/api/models.py index dd385ad..9573672 100644 --- a/api/models.py +++ b/api/models.py @@ -189,30 +189,40 @@ class Project(models.Model): emit_event("SetProjectConfig", obj=3Dself) =20 def get_property(self, prop, default=3DNone): - a =3D ProjectProperty.objects.filter(project=3Dself, name=3Dprop).= first() - if a: - return a.value - else: - return default - - def get_properties(self): - r =3D {} - for m in ProjectProperty.objects.filter(project=3Dself): - r[m.name] =3D m.value - return r - - def _do_set_property(self, prop, value): - if value is None: - ProjectProperty.objects.filter(project=3Dself, name=3Dprop).de= lete() + x =3D self.properties + *path, last =3D prop.split('.') + for item in path: + if not item in x: + return default + x =3D x[item] + return x.get(last, default) + + def delete_property(self, prop): + x =3D self.properties + *path, last =3D prop.split('.') + for item in path: + if not item in x: + return + x =3D x[item] + if not last in x: return - pp, created =3D ProjectProperty.objects.get_or_create(project=3Dse= lf, - name=3Dprop) - pp.value =3D value - pp.save() + old_val =3D x[last] + del x[last] + self.save() + emit_event("SetProperty", obj=3Dself, name=3Dprop, value=3DNone, + old_value=3Dold_val) =20 def set_property(self, prop, value): - old_val =3D self.get_property(prop) - self._do_set_property(prop, value) + if value is None: + self.delete_property(prop) + return + x =3D self.properties + *path, last =3D prop.split('.') + for item in path: + x =3D x.setdefault(item, {}) + old_val =3D x.get(last) + x[last] =3D value + self.save() emit_event("SetProperty", obj=3Dself, name=3Dprop, value=3Dvalue, old_value=3Dold_val) =20 @@ -623,37 +633,40 @@ class Message(models.Model): self.save() =20 def get_property(self, prop, default=3DNone): - return self.get_properties().get(prop, default) - - def get_properties(self): - if hasattr(self, '_properties'): - if self._properties is not None: - return self._properties - else: - # The prefetch cache is invalidated, query again - all_props =3D MessageProperty.objects.filter(message=3Dsel= f) - else: - all_props =3D self.messageproperty_set.all() - r =3D {} - for m in all_props: - r[m.name] =3D m.value - self._properties =3D r - return r - - def _do_set_property(self, prop, value): - if value is None: - MessageProperty.objects.filter(message=3Dself, name=3Dprop).de= lete() + x =3D self.properties + *path, last =3D prop.split('.') + for item in path: + if not item in x: + return default + x =3D x[item] + return x.get(last, default) + + def delete_property(self, prop): + x =3D self.properties + *path, last =3D prop.split('.') + for item in path: + if not item in x: + return + x =3D x[item] + if not last in x: return - mp, created =3D MessageProperty.objects.get_or_create(message=3Dse= lf, - name=3Dprop) - mp.value =3D value - mp.save() - # Invalidate cache - self._properties =3D None + old_val =3D x[last] + del x[last] + self.save() + emit_event("SetProperty", obj=3Dself, name=3Dprop, value=3DNone, + old_value=3Dold_val) =20 def set_property(self, prop, value): - old_val =3D self.get_property(prop) - self._do_set_property(prop, value) + if value is None: + self.delete_property(prop) + return + x =3D self.properties + *path, last =3D prop.split('.') + for item in path: + x =3D x.setdefault(item, {}) + old_val =3D x.get(last) + x[last] =3D value + self.save() emit_event("SetProperty", obj=3Dself, name=3Dprop, value=3Dvalue, old_value=3Dold_val) =20 diff --git a/mods/testing.py b/mods/testing.py index 969ce84..e5562ba 100644 --- a/mods/testing.py +++ b/mods/testing.py @@ -369,14 +369,10 @@ class TestingModule(PatchewModule): =20 def check_active_testers(self, project): at =3D [] - for k, v in project.get_properties().items(): - prefix =3D "testing.check_in." - if not k.startswith(prefix): - continue + for tn, v in project.get_property('testing.check_in', {}).items(): age =3D time.time() - v if age > 10 * 60: continue - tn =3D k[len(prefix):] at.append("%s (%dmin)" % (tn, math.ceil(age / 60))) if not at: return --=20 2.21.0 _______________________________________________ Patchew-devel mailing list Patchew-devel@redhat.com https://www.redhat.com/mailman/listinfo/patchew-devel