Since I claimed that supporting py2 & py3 in parallel would be easy
for QEMU, I figured I ought to actually give it a try to backup that
assertion.

This small patch series is the result of that effort. I tested this
series on Fedora 26 using 2.7.13 and Python 3.6.2.

To test with py3, I hacked config-host.mak to change the PYTHON
variable to point to 'python3' binary, then compared the following
generated content for the files:

   qmp-commands.h qapi-types.h  qapi-visit.h  qapi-event.h
   qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
   qmp-introspect.c qmp-introspect.h

with that generated under py2 to see they are identical.

It is possible there's still more bugs hiding that could impact
on 2.6 or earlier versions of 3.x or 2.7.x, so this probably
needs a bit wider testing, but I think the series illustrates
the broad scope of the changes we can expect. Only the need
to adapt to different module import locations adds to the
line count, and that's fairly minimal.

Daniel P. Berrange (5):
  qapi: convert to use python print function instead of statement
  qapi: use items()/values() intead of iteritems()/itervalues()
  qapi: Use OrderedDict from standard library if available
  qapi: adapt to moved location of StringIO module in py3
  qapi: Adapt to moved location of 'maketrans' function in py3

 scripts/qapi.py      | 41 +++++++++++++++++++++++++----------------
 scripts/qapi2texi.py | 11 ++++++-----
 2 files changed, 31 insertions(+), 21 deletions(-)

-- 
2.13.5

"Daniel P. Berrange" <berrange@redhat.com> writes:

> Since I claimed that supporting py2 & py3 in parallel would be easy
> for QEMU, I figured I ought to actually give it a try to backup that
> assertion.
>
> This small patch series is the result of that effort. I tested this
> series on Fedora 26 using 2.7.13 and Python 3.6.2.
>
> To test with py3, I hacked config-host.mak to change the PYTHON
> variable to point to 'python3' binary, then compared the following
> generated content for the files:
>
>    qmp-commands.h qapi-types.h  qapi-visit.h  qapi-event.h
>    qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
>    qmp-introspect.c qmp-introspect.h
>
> with that generated under py2 to see they are identical.
>
> It is possible there's still more bugs hiding that could impact
> on 2.6 or earlier versions of 3.x or 2.7.x, so this probably
> needs a bit wider testing, but I think the series illustrates
> the broad scope of the changes we can expect. Only the need
> to adapt to different module import locations adds to the
> line count, and that's fairly minimal.

This hasn't made it to the front of my review queue, but I got a quick
question meanwhile.

I guess this was triggered by the discussion of David's "[PATCH]
scripts: Support building with Python 3".  How does it differ from
David's patch?  Is it derived from it?  Or is it an independently
developed replacement?

On Fri, Sep 08, 2017 at 11:33:01AM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
> 
> > Since I claimed that supporting py2 & py3 in parallel would be easy
> > for QEMU, I figured I ought to actually give it a try to backup that
> > assertion.
> >
> > This small patch series is the result of that effort. I tested this
> > series on Fedora 26 using 2.7.13 and Python 3.6.2.
> >
> > To test with py3, I hacked config-host.mak to change the PYTHON
> > variable to point to 'python3' binary, then compared the following
> > generated content for the files:
> >
> >    qmp-commands.h qapi-types.h  qapi-visit.h  qapi-event.h
> >    qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
> >    qmp-introspect.c qmp-introspect.h
> >
> > with that generated under py2 to see they are identical.
> >
> > It is possible there's still more bugs hiding that could impact
> > on 2.6 or earlier versions of 3.x or 2.7.x, so this probably
> > needs a bit wider testing, but I think the series illustrates
> > the broad scope of the changes we can expect. Only the need
> > to adapt to different module import locations adds to the
> > line count, and that's fairly minimal.
> 
> This hasn't made it to the front of my review queue, but I got a quick
> question meanwhile.
> 
> I guess this was triggered by the discussion of David's "[PATCH]
> scripts: Support building with Python 3".  How does it differ from
> David's patch?  Is it derived from it?  Or is it an independently
> developed replacement?

I didn't notice David's existed. In terms of the qapi file changes,
his patch appears equivelent to combining my patches 2->5.

He didn't seem to convert the print statement to a print function
though (my patch 1), so I'm surprised his changes actually work
with py3...


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Fri, Sep 8, 2017 at 2:40 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
> On Fri, Sep 08, 2017 at 11:33:01AM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > Since I claimed that supporting py2 & py3 in parallel would be easy
>> > for QEMU, I figured I ought to actually give it a try to backup that
>> > assertion.
>> >
>> > This small patch series is the result of that effort. I tested this
>> > series on Fedora 26 using 2.7.13 and Python 3.6.2.
>> >
>> > To test with py3, I hacked config-host.mak to change the PYTHON
>> > variable to point to 'python3' binary, then compared the following
>> > generated content for the files:
>> >
>> >    qmp-commands.h qapi-types.h  qapi-visit.h  qapi-event.h
>> >    qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
>> >    qmp-introspect.c qmp-introspect.h
>> >
>> > with that generated under py2 to see they are identical.
>> >
>> > It is possible there's still more bugs hiding that could impact
>> > on 2.6 or earlier versions of 3.x or 2.7.x, so this probably
>> > needs a bit wider testing, but I think the series illustrates
>> > the broad scope of the changes we can expect. Only the need
>> > to adapt to different module import locations adds to the
>> > line count, and that's fairly minimal.
>>
>> This hasn't made it to the front of my review queue, but I got a quick
>> question meanwhile.
>>
>> I guess this was triggered by the discussion of David's "[PATCH]
>> scripts: Support building with Python 3".  How does it differ from
>> David's patch?  Is it derived from it?  Or is it an independently
>> developed replacement?
>
> I didn't notice David's existed. In terms of the qapi file changes,
> his patch appears equivelent to combining my patches 2->5.
>
> He didn't seem to convert the print statement to a print function
> though (my patch 1), so I'm surprised his changes actually work
> with py3...

It looks like all of those qapi.py print statements are for handling
errors, so my builds must not have hit any of them.  I had skimmed the
output of pylint for errors, and it seems the print statements are
only listed as warnings when printing to stderr, which I had
overlooked.

    W:1945, 8: Expression "((print) >> (sys.stderr), ('%s: %s') %
((sys.argv[0], str(err))))" is assigned to nothing
(expression-not-assigned

And instead of importing the print function, I just switched to
sys.stderr.write etc. in qapi2texi.py.

Thanks.

David