Start to use dedicate IO thread for QMP monitors that are not using
MUXed chardev.

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
 monitor.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/monitor.c b/monitor.c
index c6de5f123e..9e8ee2ed14 100644
--- a/monitor.c
+++ b/monitor.c
@@ -36,6 +36,7 @@
 #include "net/slirp.h"
 #include "chardev/char-fe.h"
 #include "chardev/char-io.h"
+#include "chardev/char-mux.h"
 #include "ui/qemu-spice.h"
 #include "sysemu/numa.h"
 #include "monitor/monitor.h"
@@ -4533,8 +4534,10 @@ static void monitor_qmp_setup_handlers_bh(void *opaque)
 void monitor_init(Chardev *chr, int flags)
 {
     Monitor *mon = g_malloc(sizeof(*mon));
+    /* Enable IOThread for QMPs that are not using MUX chardev backends. */
+    bool use_io_thr = (!CHARDEV_IS_MUX(chr)) && (flags & MONITOR_USE_CONTROL);
 
-    monitor_data_init(mon, false, false);
+    monitor_data_init(mon, false, use_io_thr);
 
     qemu_chr_fe_init(&mon->chr, chr, &error_abort);
     mon->flags = flags;
-- 
2.14.3

As Max Reitz said, this breaks several qemu iotests. I can reproduce that on s390
e.g. with ./check -qcow2 030 in the qemu-iotest.

Why was this merged?


On 03/09/2018 10:00 AM, Peter Xu wrote:
> Start to use dedicate IO thread for QMP monitors that are not using
> MUXed chardev.
> 
> Reviewed-by: Fam Zheng <famz@redhat.com>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  monitor.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/monitor.c b/monitor.c
> index c6de5f123e..9e8ee2ed14 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -36,6 +36,7 @@
>  #include "net/slirp.h"
>  #include "chardev/char-fe.h"
>  #include "chardev/char-io.h"
> +#include "chardev/char-mux.h"
>  #include "ui/qemu-spice.h"
>  #include "sysemu/numa.h"
>  #include "monitor/monitor.h"
> @@ -4533,8 +4534,10 @@ static void monitor_qmp_setup_handlers_bh(void *opaque)
>  void monitor_init(Chardev *chr, int flags)
>  {
>      Monitor *mon = g_malloc(sizeof(*mon));
> +    /* Enable IOThread for QMPs that are not using MUX chardev backends. */
> +    bool use_io_thr = (!CHARDEV_IS_MUX(chr)) && (flags & MONITOR_USE_CONTROL);
> 
> -    monitor_data_init(mon, false, false);
> +    monitor_data_init(mon, false, use_io_thr);
> 
>      qemu_chr_fe_init(&mon->chr, chr, &error_abort);
>      mon->flags = flags;
> 

On Fri, Mar 23, 2018 at 01:10:51PM +0100, Christian Borntraeger wrote:
> As Max Reitz said, this breaks several qemu iotests. I can reproduce that on s390
> e.g. with ./check -qcow2 030 in the qemu-iotest.
> 
> Why was this merged?

My fault.  Sorry.  I should have done iotests before submission.

Now I'm trying to fix those iotest issues.

I'm not extremely familiar with block, so the progress is a bit slow,
but I'll do it asap (though I'll possibly need some help from the
block team, since there are some not-easy ones for me).

-- 
Peter Xu

On 03/23/2018 01:25 PM, Peter Xu wrote:
> On Fri, Mar 23, 2018 at 01:10:51PM +0100, Christian Borntraeger wrote:
>> As Max Reitz said, this breaks several qemu iotests. I can reproduce that on s390
>> e.g. with ./check -qcow2 030 in the qemu-iotest.
>>
>> Why was this merged?
> 
> My fault.  Sorry.  I should have done iotests before submission.
> 
> Now I'm trying to fix those iotest issues.
> 
> I'm not extremely familiar with block, so the progress is a bit slow,
> but I'll do it asap (though I'll possibly need some help from the
> block team, since there are some not-easy ones for me).

As this patch also seems to trigger something else for Eric, can we revert this
patch for the time being and re-apply as soon as we have solutions for the issues?

After all a hard hang in the iotest can break CI environments (as you run into timeouts).

On Fri, Mar 23, 2018 at 01:44:54PM +0100, Christian Borntraeger wrote:
> 
> 
> On 03/23/2018 01:25 PM, Peter Xu wrote:
> > On Fri, Mar 23, 2018 at 01:10:51PM +0100, Christian Borntraeger wrote:
> >> As Max Reitz said, this breaks several qemu iotests. I can reproduce that on s390
> >> e.g. with ./check -qcow2 030 in the qemu-iotest.
> >>
> >> Why was this merged?
> > 
> > My fault.  Sorry.  I should have done iotests before submission.
> > 
> > Now I'm trying to fix those iotest issues.
> > 
> > I'm not extremely familiar with block, so the progress is a bit slow,
> > but I'll do it asap (though I'll possibly need some help from the
> > block team, since there are some not-easy ones for me).
> 
> As this patch also seems to trigger something else for Eric, can we revert this
> patch for the time being and re-apply as soon as we have solutions for the issues?
> 
> After all a hard hang in the iotest can break CI environments (as you run into timeouts).

I'm fine with it if it helps.

Then I can work in background to solved existing problems, then we
turn it on again in 2.13 when ready (but still it might break things -
at least it can be during dev cycle rather than softfreeze).

-- 
Peter Xu

On 23 March 2018 at 13:01, Peter Xu <peterx@redhat.com> wrote:
> On Fri, Mar 23, 2018 at 01:44:54PM +0100, Christian Borntraeger wrote:
>>
>>
>> On 03/23/2018 01:25 PM, Peter Xu wrote:
>> > On Fri, Mar 23, 2018 at 01:10:51PM +0100, Christian Borntraeger wrote:
>> >> As Max Reitz said, this breaks several qemu iotests. I can reproduce that on s390
>> >> e.g. with ./check -qcow2 030 in the qemu-iotest.
>> >>
>> >> Why was this merged?
>> >
>> > My fault.  Sorry.  I should have done iotests before submission.
>> >
>> > Now I'm trying to fix those iotest issues.
>> >
>> > I'm not extremely familiar with block, so the progress is a bit slow,
>> > but I'll do it asap (though I'll possibly need some help from the
>> > block team, since there are some not-easy ones for me).
>>
>> As this patch also seems to trigger something else for Eric, can we revert this
>> patch for the time being and re-apply as soon as we have solutions for the issues?
>>
>> After all a hard hang in the iotest can break CI environments (as you run into timeouts).
>
> I'm fine with it if it helps.

OK. I'm running a revert of commit 3fd2457d18edf5736f713dfe1ada9c87a9badab1
through my buildtests and will push it to master assuming it passes.

thanks
-- PMM

On 03/23/2018 02:01 PM, Peter Xu wrote:
> On Fri, Mar 23, 2018 at 01:44:54PM +0100, Christian Borntraeger wrote:
>>
>>
>> On 03/23/2018 01:25 PM, Peter Xu wrote:
>>> On Fri, Mar 23, 2018 at 01:10:51PM +0100, Christian Borntraeger wrote:
>>>> As Max Reitz said, this breaks several qemu iotests. I can reproduce that on s390
>>>> e.g. with ./check -qcow2 030 in the qemu-iotest.
>>>>
>>>> Why was this merged?
>>>
>>> My fault.  Sorry.  I should have done iotests before submission.
>>>
>>> Now I'm trying to fix those iotest issues.
>>>
>>> I'm not extremely familiar with block, so the progress is a bit slow,
>>> but I'll do it asap (though I'll possibly need some help from the
>>> block team, since there are some not-easy ones for me).
>>
>> As this patch also seems to trigger something else for Eric, can we revert this
>> patch for the time being and re-apply as soon as we have solutions for the issues?
>>
>> After all a hard hang in the iotest can break CI environments (as you run into timeouts).
> 
> I'm fine with it if it helps.
> 
> Then I can work in background to solved existing problems, then we
> turn it on again in 2.13 when ready (but still it might break things -
> at least it can be during dev cycle rather than softfreeze).

Strangely enough, reverting 3fd2457d18edf5736f713dfe1
now causes make check errors... :-/


  GTESTER check-qtest-s390x
  GTESTER check-qtest-i386
**
ERROR:/home/cborntra/REPOS/qemu/tests/qmp-test.c:97:test_qmp_protocol: assertion failed: (entry)
GTester: last random seed: R02S1cbb7e297cc349b33ae39f10dd92764a
**
ERROR:/home/cborntra/REPOS/qemu/tests/qmp-test.c:190:test_qmp_oob: assertion failed: (qdict_haskey(resp, "return"))
GTester: last random seed: R02S7afc5ab63f6f29401d702dceb1f8030b
make: *** [/home/cborntra/REPOS/qemu/tests/Makefile.include:880: check-qtest-s390x] Error 1
make: *** Waiting for unfinished jobs....
^Cmake: *** [/home/cborntra/REPOS/qemu/tests/Makefile.include:880: check-qtest-i386] Interrupt


Do we have patches depending on this change?

On Fri, Mar 23, 2018 at 02:23:27PM +0100, Christian Borntraeger wrote:
> 
> 
> On 03/23/2018 02:01 PM, Peter Xu wrote:
> > On Fri, Mar 23, 2018 at 01:44:54PM +0100, Christian Borntraeger wrote:
> >>
> >>
> >> On 03/23/2018 01:25 PM, Peter Xu wrote:
> >>> On Fri, Mar 23, 2018 at 01:10:51PM +0100, Christian Borntraeger wrote:
> >>>> As Max Reitz said, this breaks several qemu iotests. I can reproduce that on s390
> >>>> e.g. with ./check -qcow2 030 in the qemu-iotest.
> >>>>
> >>>> Why was this merged?
> >>>
> >>> My fault.  Sorry.  I should have done iotests before submission.
> >>>
> >>> Now I'm trying to fix those iotest issues.
> >>>
> >>> I'm not extremely familiar with block, so the progress is a bit slow,
> >>> but I'll do it asap (though I'll possibly need some help from the
> >>> block team, since there are some not-easy ones for me).
> >>
> >> As this patch also seems to trigger something else for Eric, can we revert this
> >> patch for the time being and re-apply as soon as we have solutions for the issues?
> >>
> >> After all a hard hang in the iotest can break CI environments (as you run into timeouts).
> > 
> > I'm fine with it if it helps.
> > 
> > Then I can work in background to solved existing problems, then we
> > turn it on again in 2.13 when ready (but still it might break things -
> > at least it can be during dev cycle rather than softfreeze).
> 
> Strangely enough, reverting 3fd2457d18edf5736f713dfe1
> now causes make check errors... :-/
> 
> 
>   GTESTER check-qtest-s390x
>   GTESTER check-qtest-i386
> **
> ERROR:/home/cborntra/REPOS/qemu/tests/qmp-test.c:97:test_qmp_protocol: assertion failed: (entry)
> GTester: last random seed: R02S1cbb7e297cc349b33ae39f10dd92764a
> **
> ERROR:/home/cborntra/REPOS/qemu/tests/qmp-test.c:190:test_qmp_oob: assertion failed: (qdict_haskey(resp, "return"))
> GTester: last random seed: R02S7afc5ab63f6f29401d702dceb1f8030b
> make: *** [/home/cborntra/REPOS/qemu/tests/Makefile.include:880: check-qtest-s390x] Error 1
> make: *** Waiting for unfinished jobs....
> ^Cmake: *** [/home/cborntra/REPOS/qemu/tests/Makefile.include:880: check-qtest-i386] Interrupt
> 
> 
> Do we have patches depending on this change?

It's the new OOB tests that's failing.  I'm preparing a patchset that
reverts the enablement patch, along with those tests.

Now I'm testing that branch with more platforms and also all
raw+qcow2 iotests.  After that's solid, I'll post that out soon if it
goes smoothly.

Hopefully that would fix all problems so far.

Thanks,

-- 
Peter Xu