[Qemu-devel] [PATCH 0/4] qtest: fix "device_del" out-of-order events

Peter Xu posted 4 patches 6 years, 7 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/1505295366-25295-1-git-send-email-peterx@redhat.com
Test checkpatch passed
Test docker passed
Test s390x passed
tests/libqos/pci.c        | 15 +++--------
tests/libqos/usb.c        | 28 +++++----------------
tests/libqtest.c          | 63 +++++++++++++++++++++++++++++++++++++++++++++++
tests/libqtest.h          | 17 +++++++++++++
tests/usb-hcd-uhci-test.c | 28 +++------------------
tests/usb-hcd-xhci-test.c | 52 ++++----------------------------------
6 files changed, 97 insertions(+), 106 deletions(-)
[Qemu-devel] [PATCH 0/4] qtest: fix "device_del" out-of-order events
Posted by Peter Xu 6 years, 7 months ago
It starts from a "make check" failure on one of my private tree. The
problem is that when we do "device_del" we normally looking for two
things: one response (which is mostly empty), and a REMOVE event.  The
tricky point is the event can either be there before/after the empty
response.  So I added qmp_device_del() to make sure the order does not
matter, then use it where proper.

Since I'm at it, I also added the sister helper qmp_device_add(), it
helps to remove LOCs.

I still don't 100% sure why my private tree can trigger this error,
while the master cannot. Anyway, I think this is something we should
have, no matter what.

Please review.  Thanks.

Peter Xu (4):
  libqtest: add qmp_device_del()
  tests: use qmp_device_del() where proper
  libqtest: add qmp_device_add()
  tests: use qmp_device_add() where proper

 tests/libqos/pci.c        | 15 +++--------
 tests/libqos/usb.c        | 28 +++++----------------
 tests/libqtest.c          | 63 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/libqtest.h          | 17 +++++++++++++
 tests/usb-hcd-uhci-test.c | 28 +++------------------
 tests/usb-hcd-xhci-test.c | 52 ++++----------------------------------
 6 files changed, 97 insertions(+), 106 deletions(-)

-- 
2.7.4


Re: [Qemu-devel] [PATCH 0/4] qtest: fix "device_del" out-of-order events
Posted by Thomas Huth 6 years, 7 months ago
On 13.09.2017 11:36, Peter Xu wrote:
> It starts from a "make check" failure on one of my private tree. The
> problem is that when we do "device_del" we normally looking for two
> things: one response (which is mostly empty), and a REMOVE event.  The
> tricky point is the event can either be there before/after the empty
> response.  So I added qmp_device_del() to make sure the order does not
> matter, then use it where proper.
> 
> Since I'm at it, I also added the sister helper qmp_device_add(), it
> helps to remove LOCs.

I've had a similar idea a couple of weeks ago, see here:

http://patchwork.ozlabs.org/patch/801487/

> I still don't 100% sure why my private tree can trigger this error,
> while the master cannot. Anyway, I think this is something we should
> have, no matter what.

Did you maybe touch the USB tests in your private tree?
As far as I know, some test currently use QMP in a bad way, for example
usb_test_hotplug() only checks for the DEVICE_DELETED at the end, but
forgets to read back the final return value. That return value is then
presented to the next part of the code that uses QMP instead ... it
currently only works more or less by accident, but as soon as you try to
add new code inbetween, it certainly will fail.
==> We really got to clean this up (either with my patch or your patch
series).

 Thomas

Re: [Qemu-devel] [PATCH 0/4] qtest: fix "device_del" out-of-order events
Posted by Peter Xu 6 years, 7 months ago
On Wed, Sep 13, 2017 at 11:53:16AM +0200, Thomas Huth wrote:
> On 13.09.2017 11:36, Peter Xu wrote:
> > It starts from a "make check" failure on one of my private tree. The
> > problem is that when we do "device_del" we normally looking for two
> > things: one response (which is mostly empty), and a REMOVE event.  The
> > tricky point is the event can either be there before/after the empty
> > response.  So I added qmp_device_del() to make sure the order does not
> > matter, then use it where proper.
> > 
> > Since I'm at it, I also added the sister helper qmp_device_add(), it
> > helps to remove LOCs.
> 
> I've had a similar idea a couple of weeks ago, see here:
> 
> http://patchwork.ozlabs.org/patch/801487/

Good to know!

> 
> > I still don't 100% sure why my private tree can trigger this error,
> > while the master cannot. Anyway, I think this is something we should
> > have, no matter what.
> 
> Did you maybe touch the USB tests in your private tree?

No.  But I changed some internals of QMP there, I guess that's the
reason that caused the misorder to happen more easily.

> As far as I know, some test currently use QMP in a bad way, for example
> usb_test_hotplug() only checks for the DEVICE_DELETED at the end, but
> forgets to read back the final return value. That return value is then
> presented to the next part of the code that uses QMP instead ... it
> currently only works more or less by accident, but as soon as you try to
> add new code inbetween, it certainly will fail.
> ==> We really got to clean this up (either with my patch or your patch
> series).

Agree.

I think your patch is nicer on the interface (as you have mentioned in
the other reply), I can try to review it later.

However it seems that your patch didn't really solve the problem I
encountered (mis-ordered message arrivals).  It would be good if you
want to solve it together, or I can draft patch upon yours.

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 0/4] qtest: fix "device_del" out-of-order events
Posted by Thomas Huth 6 years, 7 months ago
On 13.09.2017 12:28, Peter Xu wrote:
> On Wed, Sep 13, 2017 at 11:53:16AM +0200, Thomas Huth wrote:
>> On 13.09.2017 11:36, Peter Xu wrote:
>>> It starts from a "make check" failure on one of my private tree. The
>>> problem is that when we do "device_del" we normally looking for two
>>> things: one response (which is mostly empty), and a REMOVE event.  The
>>> tricky point is the event can either be there before/after the empty
>>> response.  So I added qmp_device_del() to make sure the order does not
>>> matter, then use it where proper.
>>>
>>> Since I'm at it, I also added the sister helper qmp_device_add(), it
>>> helps to remove LOCs.
>>
>> I've had a similar idea a couple of weeks ago, see here:
>>
>> http://patchwork.ozlabs.org/patch/801487/
> 
> Good to know!
> 
>>
>>> I still don't 100% sure why my private tree can trigger this error,
>>> while the master cannot. Anyway, I think this is something we should
>>> have, no matter what.
>>
>> Did you maybe touch the USB tests in your private tree?
> 
> No.  But I changed some internals of QMP there, I guess that's the
> reason that caused the misorder to happen more easily.
> 
>> As far as I know, some test currently use QMP in a bad way, for example
>> usb_test_hotplug() only checks for the DEVICE_DELETED at the end, but
>> forgets to read back the final return value. That return value is then
>> presented to the next part of the code that uses QMP instead ... it
>> currently only works more or less by accident, but as soon as you try to
>> add new code inbetween, it certainly will fail.
>> ==> We really got to clean this up (either with my patch or your patch
>> series).
> 
> Agree.
> 
> I think your patch is nicer on the interface (as you have mentioned in
> the other reply), I can try to review it later.
> 
> However it seems that your patch didn't really solve the problem I
> encountered (mis-ordered message arrivals).  It would be good if you
> want to solve it together, or I can draft patch upon yours.

True, I'll try to respin my patch, including the ideas from your patch...

 Thomas

Re: [Qemu-devel] [PATCH 0/4] qtest: fix "device_del" out-of-order events
Posted by Peter Xu 6 years, 7 months ago
On Wed, Sep 13, 2017 at 12:35:17PM +0200, Thomas Huth wrote:

[...]

> >> As far as I know, some test currently use QMP in a bad way, for example
> >> usb_test_hotplug() only checks for the DEVICE_DELETED at the end, but
> >> forgets to read back the final return value. That return value is then
> >> presented to the next part of the code that uses QMP instead ... it
> >> currently only works more or less by accident, but as soon as you try to
> >> add new code inbetween, it certainly will fail.
> >> ==> We really got to clean this up (either with my patch or your patch
> >> series).
> > 
> > Agree.
> > 
> > I think your patch is nicer on the interface (as you have mentioned in
> > the other reply), I can try to review it later.
> > 
> > However it seems that your patch didn't really solve the problem I
> > encountered (mis-ordered message arrivals).  It would be good if you
> > want to solve it together, or I can draft patch upon yours.
> 
> True, I'll try to respin my patch, including the ideas from your patch...

That would be great.  I've left some comments there already.  Thanks!

-- 
Peter Xu