Add script to automatically commit tree-wide changes per-subsystem.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---

CC: kwolf@redhat.com
CC: mreitz@redhat.com
CC: jsnow@redhat.com
CC: fam@euphon.net
CC: sw@weilnetz.de
CC: codyprime@gmail.com
CC: marcandre.lureau@redhat.com
CC: pbonzini@redhat.com
CC: groug@kaod.org
CC: sundeep.lkml@gmail.com
CC: peter.maydell@linaro.org
CC: stefanha@redhat.com
CC: pburton@wavecomp.com
CC: arikalo@wavecomp.com
CC: berrange@redhat.com
CC: ehabkost@redhat.com
CC: david@gibson.dropbear.id.au
CC: clg@kaod.org
CC: mst@redhat.com
CC: marcel.apfelbaum@gmail.com
CC: mark.cave-ayland@ilande.co.uk
CC: yuval.shaia@oracle.com
CC: cohuck@redhat.com
CC: farman@linux.ibm.com
CC: rth@twiddle.net
CC: david@redhat.com
CC: pasic@linux.ibm.com
CC: borntraeger@de.ibm.com
CC: kraxel@redhat.com
CC: alex.williamson@redhat.com
CC: andrew@aj.id.au
CC: joel@jms.id.au
CC: eblake@redhat.com
CC: armbru@redhat.com
CC: mdroth@linux.vnet.ibm.com
CC: quintela@redhat.com
CC: dgilbert@redhat.com
CC: jasowang@redhat.com
CC: qemu-block@nongnu.org
CC: integration@gluster.org
CC: qemu-arm@nongnu.org
CC: qemu-ppc@nongnu.org
CC: qemu-s390x@nongnu.org


 python/commit-per-subsystem.py | 69 ++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100755 python/commit-per-subsystem.py

diff --git a/python/commit-per-subsystem.py b/python/commit-per-subsystem.py
new file mode 100755
index 0000000000..d8442d9ea3
--- /dev/null
+++ b/python/commit-per-subsystem.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+#
+# Copyright (c) 2019 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import subprocess
+import sys
+
+
+def git_add(pattern):
+    subprocess.run(['git', 'add', pattern])
+
+
+def git_commit(msg):
+    subprocess.run(['git', 'commit', '-m', msg], capture_output=True)
+
+
+maintainers = sys.argv[1]
+message = sys.argv[2].strip()
+
+subsystem = None
+
+shortnames = {
+    'Block layer core': 'block',
+    'ARM cores': 'arm',
+    'Network Block Device (NBD)': 'nbd',
+    'Command line option argument parsing': 'cmdline',
+    'Character device backends': 'chardev',
+    'S390 general architecture support': 's390'
+}
+
+
+def commit():
+    if subsystem:
+        msg = subsystem
+        if msg in shortnames:
+            msg = shortnames[msg]
+        msg += ': ' + message
+        git_commit(msg)
+
+
+with open(maintainers) as f:
+    for line in f:
+        line = line.rstrip()
+        if not line:
+            continue
+        if len(line) >= 2 and line[1] == ':':
+            if line[0] == 'F' and line[3:] not in ['*', '*/']:
+                git_add(line[3:])
+        else:
+            # new subsystem start
+            commit()
+
+            subsystem = line
+
+commit()
-- 
2.21.0

On Tue,  1 Oct 2019 18:52:54 +0300
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:

> Add script to automatically commit tree-wide changes per-subsystem.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> 
> CC: kwolf@redhat.com
> CC: mreitz@redhat.com
> CC: jsnow@redhat.com
> CC: fam@euphon.net
> CC: sw@weilnetz.de
> CC: codyprime@gmail.com
> CC: marcandre.lureau@redhat.com
> CC: pbonzini@redhat.com
> CC: groug@kaod.org
> CC: sundeep.lkml@gmail.com
> CC: peter.maydell@linaro.org
> CC: stefanha@redhat.com
> CC: pburton@wavecomp.com
> CC: arikalo@wavecomp.com
> CC: berrange@redhat.com
> CC: ehabkost@redhat.com
> CC: david@gibson.dropbear.id.au
> CC: clg@kaod.org
> CC: mst@redhat.com
> CC: marcel.apfelbaum@gmail.com
> CC: mark.cave-ayland@ilande.co.uk
> CC: yuval.shaia@oracle.com
> CC: cohuck@redhat.com
> CC: farman@linux.ibm.com
> CC: rth@twiddle.net
> CC: david@redhat.com
> CC: pasic@linux.ibm.com
> CC: borntraeger@de.ibm.com
> CC: kraxel@redhat.com
> CC: alex.williamson@redhat.com
> CC: andrew@aj.id.au
> CC: joel@jms.id.au
> CC: eblake@redhat.com
> CC: armbru@redhat.com
> CC: mdroth@linux.vnet.ibm.com
> CC: quintela@redhat.com
> CC: dgilbert@redhat.com
> CC: jasowang@redhat.com
> CC: qemu-block@nongnu.org
> CC: integration@gluster.org
> CC: qemu-arm@nongnu.org
> CC: qemu-ppc@nongnu.org
> CC: qemu-s390x@nongnu.org
> 
> 
>  python/commit-per-subsystem.py | 69 ++++++++++++++++++++++++++++++++++
>  1 file changed, 69 insertions(+)
>  create mode 100755 python/commit-per-subsystem.py
> 
> diff --git a/python/commit-per-subsystem.py b/python/commit-per-subsystem.py
> new file mode 100755
> index 0000000000..d8442d9ea3
> --- /dev/null
> +++ b/python/commit-per-subsystem.py
> @@ -0,0 +1,69 @@
> +#!/usr/bin/env python3
> +#
> +# Copyright (c) 2019 Virtuozzo International GmbH
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +#
> +
> +import subprocess
> +import sys
> +
> +
> +def git_add(pattern):
> +    subprocess.run(['git', 'add', pattern])
> +
> +
> +def git_commit(msg):
> +    subprocess.run(['git', 'commit', '-m', msg], capture_output=True)
> +
> +
> +maintainers = sys.argv[1]
> +message = sys.argv[2].strip()
> +
> +subsystem = None
> +
> +shortnames = {
> +    'Block layer core': 'block',
> +    'ARM cores': 'arm',
> +    'Network Block Device (NBD)': 'nbd',
> +    'Command line option argument parsing': 'cmdline',
> +    'Character device backends': 'chardev',
> +    'S390 general architecture support': 's390'
> +}
> +
> +
> +def commit():
> +    if subsystem:
> +        msg = subsystem
> +        if msg in shortnames:
> +            msg = shortnames[msg]
> +        msg += ': ' + message
> +        git_commit(msg)
> +
> +
> +with open(maintainers) as f:
> +    for line in f:
> +        line = line.rstrip()
> +        if not line:
> +            continue
> +        if len(line) >= 2 and line[1] == ':':
> +            if line[0] == 'F' and line[3:] not in ['*', '*/']:
> +                git_add(line[3:])
> +        else:
> +            # new subsystem start
> +            commit()
> +
> +            subsystem = line
> +
> +commit()

Hm... I'm not sure about the purpose of this script (and my python is
rather weak)... is this supposed to collect all changes covered by a
subsystem F: pattern into one patch? If so, what happens to files
covered by multiple sections?

07.10.2019 18:55, Cornelia Huck wrote:
> On Tue,  1 Oct 2019 18:52:54 +0300
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:
> 
>> Add script to automatically commit tree-wide changes per-subsystem.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>
>> CC: kwolf@redhat.com
>> CC: mreitz@redhat.com
>> CC: jsnow@redhat.com
>> CC: fam@euphon.net
>> CC: sw@weilnetz.de
>> CC: codyprime@gmail.com
>> CC: marcandre.lureau@redhat.com
>> CC: pbonzini@redhat.com
>> CC: groug@kaod.org
>> CC: sundeep.lkml@gmail.com
>> CC: peter.maydell@linaro.org
>> CC: stefanha@redhat.com
>> CC: pburton@wavecomp.com
>> CC: arikalo@wavecomp.com
>> CC: berrange@redhat.com
>> CC: ehabkost@redhat.com
>> CC: david@gibson.dropbear.id.au
>> CC: clg@kaod.org
>> CC: mst@redhat.com
>> CC: marcel.apfelbaum@gmail.com
>> CC: mark.cave-ayland@ilande.co.uk
>> CC: yuval.shaia@oracle.com
>> CC: cohuck@redhat.com
>> CC: farman@linux.ibm.com
>> CC: rth@twiddle.net
>> CC: david@redhat.com
>> CC: pasic@linux.ibm.com
>> CC: borntraeger@de.ibm.com
>> CC: kraxel@redhat.com
>> CC: alex.williamson@redhat.com
>> CC: andrew@aj.id.au
>> CC: joel@jms.id.au
>> CC: eblake@redhat.com
>> CC: armbru@redhat.com
>> CC: mdroth@linux.vnet.ibm.com
>> CC: quintela@redhat.com
>> CC: dgilbert@redhat.com
>> CC: jasowang@redhat.com
>> CC: qemu-block@nongnu.org
>> CC: integration@gluster.org
>> CC: qemu-arm@nongnu.org
>> CC: qemu-ppc@nongnu.org
>> CC: qemu-s390x@nongnu.org
>>
>>
>>   python/commit-per-subsystem.py | 69 ++++++++++++++++++++++++++++++++++
>>   1 file changed, 69 insertions(+)
>>   create mode 100755 python/commit-per-subsystem.py
>>
>> diff --git a/python/commit-per-subsystem.py b/python/commit-per-subsystem.py
>> new file mode 100755
>> index 0000000000..d8442d9ea3
>> --- /dev/null
>> +++ b/python/commit-per-subsystem.py
>> @@ -0,0 +1,69 @@
>> +#!/usr/bin/env python3
>> +#
>> +# Copyright (c) 2019 Virtuozzo International GmbH
>> +#
>> +# This program is free software; you can redistribute it and/or modify
>> +# it under the terms of the GNU General Public License as published by
>> +# the Free Software Foundation; either version 2 of the License, or
>> +# (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
>> +#
>> +
>> +import subprocess
>> +import sys
>> +
>> +
>> +def git_add(pattern):
>> +    subprocess.run(['git', 'add', pattern])
>> +
>> +
>> +def git_commit(msg):
>> +    subprocess.run(['git', 'commit', '-m', msg], capture_output=True)
>> +
>> +
>> +maintainers = sys.argv[1]
>> +message = sys.argv[2].strip()
>> +
>> +subsystem = None
>> +
>> +shortnames = {
>> +    'Block layer core': 'block',
>> +    'ARM cores': 'arm',
>> +    'Network Block Device (NBD)': 'nbd',
>> +    'Command line option argument parsing': 'cmdline',
>> +    'Character device backends': 'chardev',
>> +    'S390 general architecture support': 's390'
>> +}
>> +
>> +
>> +def commit():
>> +    if subsystem:
>> +        msg = subsystem
>> +        if msg in shortnames:
>> +            msg = shortnames[msg]
>> +        msg += ': ' + message
>> +        git_commit(msg)
>> +
>> +
>> +with open(maintainers) as f:
>> +    for line in f:
>> +        line = line.rstrip()
>> +        if not line:
>> +            continue
>> +        if len(line) >= 2 and line[1] == ':':
>> +            if line[0] == 'F' and line[3:] not in ['*', '*/']:
>> +                git_add(line[3:])
>> +        else:
>> +            # new subsystem start
>> +            commit()
>> +
>> +            subsystem = line
>> +
>> +commit()
> 
> Hm... I'm not sure about the purpose of this script (and my python is
> rather weak)... is this supposed to collect all changes covered by a
> subsystem F: pattern into one patch?

Yes

> If so, what happens to files
> covered by multiple sections?
> 

Hmm, they just go to the first of these sections, mentioned in MAINTAINERS.
Is it bad I don't know, but I tried to automate it somehow. Anyway, I myself
can't have better idea about how to organize patches to the subsystems which
I don't know.

-- 
Best regards,
Vladimir

On Mon, 7 Oct 2019 16:10:02 +0000
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:

> 07.10.2019 18:55, Cornelia Huck wrote:
> > On Tue,  1 Oct 2019 18:52:54 +0300
> > Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:

> >> +def git_add(pattern):
> >> +    subprocess.run(['git', 'add', pattern])
> >> +
> >> +
> >> +def git_commit(msg):
> >> +    subprocess.run(['git', 'commit', '-m', msg], capture_output=True)
> >> +
> >> +
> >> +maintainers = sys.argv[1]
> >> +message = sys.argv[2].strip()
> >> +
> >> +subsystem = None
> >> +
> >> +shortnames = {
> >> +    'Block layer core': 'block',
> >> +    'ARM cores': 'arm',
> >> +    'Network Block Device (NBD)': 'nbd',
> >> +    'Command line option argument parsing': 'cmdline',
> >> +    'Character device backends': 'chardev',
> >> +    'S390 general architecture support': 's390'
> >> +}
> >> +
> >> +
> >> +def commit():
> >> +    if subsystem:
> >> +        msg = subsystem
> >> +        if msg in shortnames:
> >> +            msg = shortnames[msg]
> >> +        msg += ': ' + message
> >> +        git_commit(msg)
> >> +
> >> +
> >> +with open(maintainers) as f:
> >> +    for line in f:
> >> +        line = line.rstrip()
> >> +        if not line:
> >> +            continue
> >> +        if len(line) >= 2 and line[1] == ':':
> >> +            if line[0] == 'F' and line[3:] not in ['*', '*/']:
> >> +                git_add(line[3:])
> >> +        else:
> >> +            # new subsystem start
> >> +            commit()
> >> +
> >> +            subsystem = line
> >> +
> >> +commit()  
> > 
> > Hm... I'm not sure about the purpose of this script (and my python is
> > rather weak)... is this supposed to collect all changes covered by a
> > subsystem F: pattern into one patch?  
> 
> Yes
> 
> > If so, what happens to files
> > covered by multiple sections?
> >   
> 
> Hmm, they just go to the first of these sections, mentioned in MAINTAINERS.
> Is it bad I don't know, but I tried to automate it somehow. Anyway, I myself
> can't have better idea about how to organize patches to the subsystems which
> I don't know.
> 

Yeah, that is a problem I don't have a solution for, either.

But the script should probably get at least a comment about its
intended purpose and limitations? We don't really want people to start
using it blindly.

On Mon, Oct 07, 2019 at 06:16:51PM +0200, Cornelia Huck wrote:
> On Mon, 7 Oct 2019 16:10:02 +0000
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:
> 
> > 07.10.2019 18:55, Cornelia Huck wrote:
> > > On Tue,  1 Oct 2019 18:52:54 +0300
> > > Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:
> 
> > >> +def git_add(pattern):
> > >> +    subprocess.run(['git', 'add', pattern])
> > >> +
> > >> +
> > >> +def git_commit(msg):
> > >> +    subprocess.run(['git', 'commit', '-m', msg], capture_output=True)
> > >> +
> > >> +
> > >> +maintainers = sys.argv[1]
> > >> +message = sys.argv[2].strip()
> > >> +
> > >> +subsystem = None
> > >> +
> > >> +shortnames = {
> > >> +    'Block layer core': 'block',
> > >> +    'ARM cores': 'arm',
> > >> +    'Network Block Device (NBD)': 'nbd',
> > >> +    'Command line option argument parsing': 'cmdline',
> > >> +    'Character device backends': 'chardev',
> > >> +    'S390 general architecture support': 's390'
> > >> +}
> > >> +
> > >> +
> > >> +def commit():
> > >> +    if subsystem:
> > >> +        msg = subsystem
> > >> +        if msg in shortnames:
> > >> +            msg = shortnames[msg]
> > >> +        msg += ': ' + message
> > >> +        git_commit(msg)
> > >> +
> > >> +
> > >> +with open(maintainers) as f:
> > >> +    for line in f:
> > >> +        line = line.rstrip()
> > >> +        if not line:
> > >> +            continue
> > >> +        if len(line) >= 2 and line[1] == ':':
> > >> +            if line[0] == 'F' and line[3:] not in ['*', '*/']:
> > >> +                git_add(line[3:])
> > >> +        else:
> > >> +            # new subsystem start
> > >> +            commit()
> > >> +
> > >> +            subsystem = line
> > >> +
> > >> +commit()  
> > > 
> > > Hm... I'm not sure about the purpose of this script (and my python is
> > > rather weak)... is this supposed to collect all changes covered by a
> > > subsystem F: pattern into one patch?  
> > 
> > Yes
> > 
> > > If so, what happens to files
> > > covered by multiple sections?
> > >   
> > 
> > Hmm, they just go to the first of these sections, mentioned in MAINTAINERS.
> > Is it bad I don't know, but I tried to automate it somehow. Anyway, I myself
> > can't have better idea about how to organize patches to the subsystems which
> > I don't know.
> > 
> 
> Yeah, that is a problem I don't have a solution for, either.
> 
> But the script should probably get at least a comment about its
> intended purpose and limitations? We don't really want people to start
> using it blindly.

Is this really a common enough problem to even justify having the
script to start with ?

It looks like its only really usable in the case where the changes
to each subsystem are totally self-contained, otherwise you'll get
git bisect failures. The user still has to go back and edit each
commit here to fill in a useful commit messages. It doesn't seem
to save much effort over 'git add -u sub/dir/ && git commit -s'
which is what I'd typically do for grouping changes that are
spread across the tree.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

07.10.2019 19:21, Daniel P. Berrangé wrote:
> On Mon, Oct 07, 2019 at 06:16:51PM +0200, Cornelia Huck wrote:
>> On Mon, 7 Oct 2019 16:10:02 +0000
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:
>>
>>> 07.10.2019 18:55, Cornelia Huck wrote:
>>>> On Tue,  1 Oct 2019 18:52:54 +0300
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:
>>
>>>>> +def git_add(pattern):
>>>>> +    subprocess.run(['git', 'add', pattern])
>>>>> +
>>>>> +
>>>>> +def git_commit(msg):
>>>>> +    subprocess.run(['git', 'commit', '-m', msg], capture_output=True)
>>>>> +
>>>>> +
>>>>> +maintainers = sys.argv[1]
>>>>> +message = sys.argv[2].strip()
>>>>> +
>>>>> +subsystem = None
>>>>> +
>>>>> +shortnames = {
>>>>> +    'Block layer core': 'block',
>>>>> +    'ARM cores': 'arm',
>>>>> +    'Network Block Device (NBD)': 'nbd',
>>>>> +    'Command line option argument parsing': 'cmdline',
>>>>> +    'Character device backends': 'chardev',
>>>>> +    'S390 general architecture support': 's390'
>>>>> +}
>>>>> +
>>>>> +
>>>>> +def commit():
>>>>> +    if subsystem:
>>>>> +        msg = subsystem
>>>>> +        if msg in shortnames:
>>>>> +            msg = shortnames[msg]
>>>>> +        msg += ': ' + message
>>>>> +        git_commit(msg)
>>>>> +
>>>>> +
>>>>> +with open(maintainers) as f:
>>>>> +    for line in f:
>>>>> +        line = line.rstrip()
>>>>> +        if not line:
>>>>> +            continue
>>>>> +        if len(line) >= 2 and line[1] == ':':
>>>>> +            if line[0] == 'F' and line[3:] not in ['*', '*/']:
>>>>> +                git_add(line[3:])
>>>>> +        else:
>>>>> +            # new subsystem start
>>>>> +            commit()
>>>>> +
>>>>> +            subsystem = line
>>>>> +
>>>>> +commit()
>>>>
>>>> Hm... I'm not sure about the purpose of this script (and my python is
>>>> rather weak)... is this supposed to collect all changes covered by a
>>>> subsystem F: pattern into one patch?
>>>
>>> Yes
>>>
>>>> If so, what happens to files
>>>> covered by multiple sections?
>>>>    
>>>
>>> Hmm, they just go to the first of these sections, mentioned in MAINTAINERS.
>>> Is it bad I don't know, but I tried to automate it somehow. Anyway, I myself
>>> can't have better idea about how to organize patches to the subsystems which
>>> I don't know.
>>>
>>
>> Yeah, that is a problem I don't have a solution for, either.
>>
>> But the script should probably get at least a comment about its
>> intended purpose and limitations? We don't really want people to start
>> using it blindly.
> 
> Is this really a common enough problem to even justify having the
> script to start with ?
> 
> It looks like its only really usable in the case where the changes
> to each subsystem are totally self-contained, otherwise you'll get
> git bisect failures. The user still has to go back and edit each
> commit here to fill in a useful commit messages. It doesn't seem
> to save much effort over 'git add -u sub/dir/ && git commit -s'
> which is what I'd typically do for grouping changes that are
> spread across the tree.
> 


Commit message is the same except "subsystem: ", I don't think it shoud
be edited.
Still, we can just not commit this script to Qemu if we don't like it. It just
helped me to create patches, and we can't take them without a review anyway.


-- 
Best regards,
Vladimir