From nobody Sun Apr 28 15:58:53 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1594197490; cv=none; d=zohomail.com; s=zohoarc; b=UAefLw1UlSZR5oT7sG9F7ErscHy/WVZIOAZ3I63aDevQRVWdgtEvKnorryW/nkdMveT5tRyxjLbYO8W+P4AUT+tIfVLQpxKVDOJlr2V4FRLjEnUHq6HX1zpGeVFoUkk7sw/RWAll5sBDCyqVBtvstZaVlmC0bPYjxri8MC15rso= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594197490; h=Content-Type:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=pkwsKfhwDmrpiyDAEDGHaOYsOVDUjSaeyenfsXf1fA4=; b=ZqRCMIM1q4Y7XatZvMq2WPWASaEwxUAq7WIP7dSlExUFLc7pf6pABHdM7tLxRGIiQU8zRp/jblV911W7wcOXk3x3XHVAAj+i9HRWMdEyU2wW/FuPYgzlWOGYOVwUNDJWSMdF256TUubT8HF8+P2sa7HpaYczpZWqKBERkbVuAPo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594197490638507.0609764732203; Wed, 8 Jul 2020 01:38:10 -0700 (PDT) Received: from localhost ([::1]:45412 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jt5ai-0003b5-G9 for importer@patchew.org; Wed, 08 Jul 2020 04:38:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53636) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jt1NC-0005fO-6y for qemu-devel@nongnu.org; Wed, 08 Jul 2020 00:07:54 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:45351) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jt1N4-0003CL-CZ for qemu-devel@nongnu.org; Wed, 08 Jul 2020 00:07:49 -0400 Received: by mail-pg1-x542.google.com with SMTP id l63so21003968pge.12 for ; Tue, 07 Jul 2020 21:07:45 -0700 (PDT) Received: from [10.94.81.213] ([103.136.220.68]) by smtp.gmail.com with ESMTPSA id f132sm23629480pfa.122.2020.07.07.21.07.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Jul 2020 21:07:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:mime-version:subject:message-id:date:cc:to; bh=pkwsKfhwDmrpiyDAEDGHaOYsOVDUjSaeyenfsXf1fA4=; b=it852fZbp25aUNuVuere2rFWym0MNMyCXSMmvJbxFrNmyoi9acF8iyysKRVrU9+GlC fQ1muYZ3IvL6X0T+saVFVj/YcFpzqqN6On4ZVTlf+81K2FA+xMAQ8JB6m2o2E5b0+g9s MVDP/E2mqVr0bMMaadPxwqpEQ5JNz9Ob0aJsNmomwM4X4jXROUkYJtOFc7FHs/9CwycA tbEOFnoukapBhhZRWXa6oiPcLpfPFYuF1E9MRDYD5RBy//sE06QnDNspL+9MO9ix1q5Y 2uyv+f8Y+WUMl9ZtXTmnPFS/0YsE1FnRRMOd+N3Mzlmy6eCKUP3jjwA7WkhiZtv5CBKv NOgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:cc:to; bh=pkwsKfhwDmrpiyDAEDGHaOYsOVDUjSaeyenfsXf1fA4=; b=R3UXfbyCHW7k71x9E7Vs04yE9FbTxvWXN6yU8VKgpEYv6l37tcezHengKR9HamU0tk /jwrvmQ9ZMykupWLXbHVn1TIw6BeKk1ez7weP0+JPks5XxjJEpOlAoxPJhWnLW02Y1KI 2tPHjgpZZBGmDsj87JvOZTV26jXFwgUiO0x9UfTfZjAnV0OyKai0uTO1lIbnEev2Cwdl 8AMyFL97JmB4NmtcVmwqPFsCcuNBacUgxdNS5d/lwsdYPX6lUaWfA59xtmWesmptb2EI FwfnriOJbZxeBZbM+jTZ5f9jO/OcoNDZRvYZ6Aa/2YBn+DYDZ4xV4mgGgtmQGqLeAJqA e1Jg== X-Gm-Message-State: AOAM531Tk8+3f3QUM4ZAsZ0C6fyCZSUH/MA7DEkSsuitkjdeowQiD9s9 KWXQUXBR2OUjVsZZwoPSZOEmBg== X-Google-Smtp-Source: ABdhPJzrqICj6qOWPPrQntCdKYH5Hl0YxMBR0ChAb83AvtBQn54Ig6kUTgaAGwb1gGaWeKVo0aCRag== X-Received: by 2002:a63:b511:: with SMTP id y17mr46959468pge.425.1594181263701; Tue, 07 Jul 2020 21:07:43 -0700 (PDT) From: =?gb2312?B?y9XLvObD?= Content-Type: multipart/alternative; boundary="Apple-Mail=_110C1946-0FD0-4609-A383-653CC8A7192A" Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Qemu core dump when stop guest with virtio-blk(remote storage) and iothread Message-Id: <26A897E7-7AAC-4A5B-B004-681602F298C0@bytedance.com> Date: Wed, 8 Jul 2020 12:07:29 +0800 To: famz@redhat.com, mreitz@redhat.com, kwolf@redhat.com, mst@redhat.com, stefanha@redhat.com X-Mailer: Apple Mail (2.3445.104.11) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::542; envelope-from=susiting@bytedance.com; helo=mail-pg1-x542.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: 23 X-Spam_score: 2.3 X-Spam_bar: ++ X-Spam_report: (2.3 / 5.0 requ) BAYES_00=-1.9, CHARSET_FARAWAY_HEADER=3.2, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, MANY_SPAN_IN_TEXT=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 08 Jul 2020 04:37:11 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @bytedance-com.20150623.gappssmtp.com) --Apple-Mail=_110C1946-0FD0-4609-A383-653CC8A7192A Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Description of problem: Qemu core dump when stop guest with virtio-blk(remote storage) and iothread Version-Release number pf selected component (if applicable): kernel version:4.19.36.bsk.9-amd64 qemu-kvm version:QEMU emulator version 2.12.1 How reproducible: 100% Steps to Reproduce: 1. Start guest, one virtio-blk(remote storage) and iothread parameter /data00/qemu/x86_64-softmmu/qemu-system-x86_64 -name guest=3Dinstance-03,debug-threads=3Don -kvm /dev/kvm=20 -S -object secret,id=3DmasterKey0,format=3Draw,file=3D/var/lib/libvirt/qemu/d= omain-21-instance-03/master-key.aes=20 -machine pc-i440fx-2.12,accel=3Dkvm,usb=3Doff,dump-guest-core=3Doff=20 -cpu qemu64,+kvm_pv_eoi=20 -m 7630=20 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu/21-instance-03 -smp 4,= sockets=3D4,cores=3D1,threads=3D1=20 -object iothread,id=3Diothread1 -object iothread,id=3Diothread2 -object io= thread,id=3Diothread3 -object iothread,id=3Diothread4 -no-user-config=20 -nodefaults=20 -chardev socket,id=3Dcharmonitor,fd=3D21,server,nowait=20 -mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol=20 -rtc base=3Dutc,driftfix=3Dslew=20 -global kvm-pit.lost_tick_policy=3Ddelay -no-hpet -no-shutdown=20 -boot strict=3Don -device piix3-usb-uhci,id=3Dusb,bus=3Dpci.0,addr=3D0x1.0= x2=20 -device virtio-serial-pci,id=3Dvirtio-serial0,bus=3Dpci.0,addr=3D0x7=20 -drive file=3Dremote drive path,format=3Draw,if=3Dnone,id=3Ddrive-virtio-d= isk0,cache=3Dnone,aio=3Dnative,throttling.bps-read=3D30000000,throttling.bp= s-write=3D80000000,throttling.iops-read=3D800,throttling.iops-write=3D400=20 -device virtio-blk-pci,iothread=3Diothread1,scsi=3Doff,bus=3Dpci.0,addr=3D= 0x3,drive=3Ddrive-virtio-disk0,id=3Dvirtio-disk0,bootindex=3D1,write-cache= =3Don -chardev pty,id=3Dcharserial0 -device isa-serial,chardev=3Dcharserial0,id= =3Dserial0=20 -device usb-tablet,id=3Dinput0,bus=3Dusb.0,port=3D1 -vnc 0.0.0.0:0 -k en-u= s -device cirrus-vga,id=3Dvideo0,bus=3Dpci.0,addr=3D0x2=20 -device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x8 -sandbox o= n,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resourcecontrol=3Dd= eny=20 -device pvpanic,ioport=3D1285 -msg timestamp=3Don -d int,unimp,guest_errors 2.stop guest and start guest repeatedly=20 virsh qemu-monitor-command --domain instance-03 '{"execute":"stop", "argum= ents":{}}' virsh qemu-monitor-command --domain instance-03 '{"execute":"cont", "argum= ents":{}}' 3.Actual results: Qemu core dump with error msg: (qemu) qemu: qemu_mutex_unlock_impl: Operation not permitted Expected results: Guest can stop and start successfully Additional info: [Current thread is 1 (Thread 0x7ff188abb700 (LWP 2229116))] (gdb) bt #0 __GI_raise (sig=3Dsig@entry=3D6) at ../sysdeps/unix/sysv/linux/raise.c:= 51 #1 0x00007ff19d73542a in __GI_abort () at abort.c:89 #2 0x0000560d55a3ca65 in error_exit (err=3Derr@entry=3D1, msg=3Dmsg@entry= =3D0x560d566e9650 <__func__.18241> "qemu_mutex_unlock_impl") at util/qemu-t= hread-posix.c:37 #3 0x0000560d55e76dc8 in qemu_mutex_unlock_impl (mutex=3Dmutex@entry=3D0x5= 60d58456e30, file=3Dfile@entry=3D0x560d566e87df "util/async.c", line=3Dline= @entry=3D509) at util/qemu-thread-posix.c:99 #4 0x0000560d55e71580 in aio_context_release (ctx=3D0x560d58456dd0) at uti= l/async.c:509 #5 0x0000560d55acf344 in virtio_blk_rw_complete (opaque=3D,= ret=3D0) at /data00/susieqemu/hw/block/virtio-blk.c:126 #6 0x0000560d55dcabc9 in blk_aio_complete (acb=3D0x7ff17c001bf0) at block/= block-backend.c:1345 #7 0x0000560d55e8a25b in coroutine_trampoline (i0=3D, i1=3D= ) at util/coroutine-ucontext.c:116 #8 0x00007ff19d745000 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #9 0x00007ffd3691e190 in ?? () #10 0x0000000000000000 in ?? () Backtrace stopped: Cannot access memory at address 0x7fef29cdd000 Thread 3 (Thread 0x7ff1a38f23c0 (LWP 2229113)): #0 memory_region_ioeventfd_before (a=3D..., b=3D...) at /data00/susieqemu/= memory.c:185 #1 address_space_add_del_ioeventfds (fds_old_nb=3D65, fds_old=3D0x560d5973= 3960, fds_new_nb=3D64, fds_new=3D0x560d59ba6d10, as=3D0x560d598728b0) at /d= ata00/susieqemu/memory.c:794 #2 address_space_update_ioeventfds (as=3Das@entry=3D0x560d598728b0) at /da= ta00/susieqemu/memory.c:877 #3 0x0000560d55aabd98 in memory_region_transaction_commit () at /data00/su= sieqemu/memory.c:1080 #4 0x0000560d55aae260 in memory_region_del_eventfd (mr=3Dmr@entry=3D0x560d= 599a2b40, addr=3D, size=3Dsize@entry=3D0, match_data=3D, data=3D, e=3D) at /data00/susieqe= mu/memory.c:2274 #5 0x0000560d55d113da in virtio_pci_ioeventfd_assign (d=3D0x560d599a1e70, = notifier=3D0x560d599db8d8, n=3D0, assign=3D) at hw/virtio/vi= rtio-pci.c:268 #6 0x0000560d55d14fef in virtio_bus_set_host_notifier (bus=3D0x560d599a9f6= 8, n=3Dn@entry=3D0, assign=3Dassign@entry=3Dfalse) at hw/virtio/virtio-bus.= c:289 #7 0x0000560d55ad2886 in virtio_blk_data_plane_stop (vdev=3D) at /data00/susieqemu/hw/block/dataplane/virtio-blk.c:295 #8 0x0000560d55d1459e in virtio_bus_stop_ioeventfd (bus=3D0x560d599a9f68) = at hw/virtio/virtio-bus.c:246 #9 0x0000560d55b02c4f in virtio_vmstate_change (opaque=3D0x560d599a9fe0, r= unning=3D0, state=3DRUN_STATE_PAUSED) at /data00/susieqemu/hw/virtio/virtio= .c:2236 #10 0x0000560d55ba2517 in vm_state_notify (running=3Drunning@entry=3D0, sta= te=3Dstate@entry=3DRUN_STATE_PAUSED) at vl.c:1649 #11 0x0000560d55a95c3a in do_vm_stop (state=3Dstate@entry=3DRUN_STATE_PAUSE= D, send_stop=3Dsend_stop@entry=3Dtrue) at /data00/susieqemu/cpus.c:1013 #12 0x0000560d55a96132 in vm_stop (state=3Dstate@entry=3DRUN_STATE_PAUSED) = at /data00/susieqemu/cpus.c:2036 #13 0x0000560d55bb652b in qmp_stop (errp=3Derrp@entry=3D0x7ffd3691e9b0) at = qmp.c:106 #14 0x0000560d55bae9aa in qmp_marshal_stop (args=3D, ret=3D<= optimized out>, errp=3D0x7ffd3691e9f8) at qapi/qapi-commands-misc.c:784 #15 0x0000560d55e65f89 in do_qmp_dispatch (errp=3D0x7ffd3691e9f0, request= =3D0x7ffd3691e9f0, cmds=3D) at qapi/qmp-dispatch.c:111 #16 qmp_dispatch (cmds=3D, request=3Drequest@entry=3D0x560d5= 8467e60) at qapi/qmp-dispatch.c:164 #17 0x0000560d55a9e95d in monitor_qmp_dispatch_one (req_obj=3Dreq_obj@entry= =3D0x560d587b5a20) at /data00/susieqemu/monitor.c:4084 #18 0x0000560d55a9ec99 in monitor_qmp_bh_dispatcher (data=3D= ) at /data00/susieqemu/monitor.c:4142 #19 0x0000560d55e70e2e in aio_bh_call (bh=3D0x560d58472a20) at util/async.c= :91 #20 aio_bh_poll (ctx=3Dctx@entry=3D0x560d584569e0) at util/async.c:119 #21 0x0000560d55e7470f in aio_dispatch (ctx=3D0x560d584569e0) at util/aio-p= osix.c:456 #22 0x0000560d55e70ced in aio_ctx_dispatch (source=3D, callb= ack=3D, user_data=3D) at util/async.c:262 #23 0x00007ff1a0d087f7 in g_main_context_dispatch () from /lib/x86_64-linux= -gnu/libglib-2.0.so.0 #24 0x0000560d55e737d8 in glib_pollfds_poll () at util/main-loop.c:217 #25 os_host_main_loop_wait (timeout=3D) at util/main-loop.c:= 265 #26 main_loop_wait (nonblocking=3D) at util/main-loop.c:524 #27 0x0000560d55a41ff7 in main_loop () at vl.c:1980 #28 main (argc=3D, argv=3D, envp=3D) at vl.c:4845 This is because in the virtio_blk_dataplane_stop() function, the context of= the iothread will be set to the maincontext in blk_set_aio_context(). In t= he middle of this context change, the iothread will acquire the original io= thread context which is held by the main thread. After the mainthread chang= e the context of the iothread and then release the original iothread contex= t, the iothread will get the old iothread context which is not supposed to = happen because the context has been changed to the main thread context. So = when the iothread finish virtio_blk_rw_complete() and try to release the ma= in thread context, qemu will crash. This patch can solve this problem for me, but I am not sure whether this is= right, please give me a hand. diff --git a/block/.block-backend.c.swp b/block/.block-backend.c.swp deleted file mode 100644 index 0bf897dd..00000000 Binary files a/block/.block-backend.c.swp and /dev/null differ diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index b9c1b3ba..d8ac782d 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -91,7 +91,12 @@ static void virtio_blk_rw_complete(void *opaque, int ret) VirtIOBlock *s =3D next->dev; VirtIODevice *vdev =3D VIRTIO_DEVICE(s); QEMU_LOG(LOG_INFO, "virtio_blk_rw_complete before context_acquire: %p\= n", (blk_get_aio_context(s->conf.conf.blk))); - aio_context_acquire(blk_get_aio_context(s->conf.conf.blk)); + static struct timespec ts_sleep; + ts_sleep.tv_sec =3D 0; + ts_sleep.tv_nsec =3D 50 * 1000 * 1000ull; + while (aio_context_tryacquire(blk_get_aio_context(s->conf.conf.blk))) { + nanosleep(&ts_sleep, NULL); + } while (next) { VirtIOBlockReq *req =3D next; next =3D req->mr_next; diff --git a/include/block/aio.h b/include/block/aio.h index a1d6b9e2..59be498f 100644 --- a/include/block/aio.h +++ b/include/block/aio.h @@ -174,6 +174,8 @@ void aio_context_ref(AioContext *ctx); */ void aio_context_unref(AioContext *ctx); =20 +int aio_context_tryacquire(AioContext *ctx); + /* Take ownership of the AioContext. If the AioContext will be shared bet= ween * threads, and a thread does not want to be interrupted, it will have to * take ownership around calls to aio_poll(). Otherwise, aio_poll() diff --git a/util/async.c b/util/async.c index 27fb4901..009f6c5f 100644 --- a/util/async.c +++ b/util/async.c @@ -498,6 +498,10 @@ void aio_context_unref(AioContext *ctx) g_source_unref(&ctx->source); } =20 +int aio_context_acquire(AioContext *ctx) { + return qemu_rec_mutex_trylock(&ctx->lock); +} + void aio_context_acquire(AioContext *ctx) { qemu_rec_mutex_lock(&ctx->lock); --Apple-Mail=_110C1946-0FD0-4609-A383-653CC8A7192A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
Description of = problem:
= Qemu core dump when stop guest with virtio-blk(remote storage) and iothread
Version-Release number pf selected component (if applicable):
kernel = version:4.19.36.bsk.9-amd64
qemu-kvm version:QEMU emulator = version 2.12.1

How reproducible:
= 100%

Steps to Reproduce:
1. Start guest, one virtio-blk(remote storage) and iothread = parameter
= /data00/qemu/x86_64-softmmu/qemu-system-x86_64
-name = guest=3Dinstance-03,debug-threads=3Don
-kvm /dev/kvm 
= -S
= -object secret,id=3DmasterKey0,format=3Draw,file=3D/var/lib/libvirt/qemu/domain-21-instance-03/mast= er-key.aes 
= -machine pc-i440fx-2.12,accel=3Dkvm,usb=3Doff,dump-guest-core=3Doff 
-cpu qemu64,+kvm_pv_eoi 
-m = 7630 
= -mem-prealloc -mem-path = /dev/hugepages/libvirt/qemu/21-instance-03 -smp 4,sockets=3D4,cores=3D1,threads=3D1 
-object = iothread,id=3Diothread1 = -object iothread,id=3Diothread2 -object iothread,id=3Diothread3 -object = iothread,id=3Diothread4
-no-user-config 
= -nodefaults 
= -chardev socket,id=3Dcharmonitor,fd=3D21,server,nowait 
-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol 
-rtc = base=3Dutc,driftfix=3Dslew 
-global = kvm-pit.lost_tick_policy=3Ddelay -no-hpet -no-shutdown 
-boot = strict=3Don -device piix3-usb-uhci,id=3Dusb,bus=3Dpci.0,addr=3D0x1.0x2 
-device virtio-serial-pci,id=3Dvirtio-serial0,bus=3Dpci.0,addr=3D0x7 
-drive = file=3Dremote drive path,format=3Draw,if=3Dnone,id=3Ddrive-virtio-disk0,cache=3Dnone,aio=3Dnative,throttling.bps-read=3D30000000,throttling.bps-write=3D80000000,throttling.iops-read=3D800,throttling.iops-write=3D400 
-device = virtio-blk-pci,iothread=3Diothread1,scsi=3Doff,bus=3Dpci.0,addr=3D0x3,drive=3Ddrive-virtio-disk0,id=3Dvirtio-disk0,bootindex=3D1,write-cache=3Don
-chardev pty,id=3Dcharserial0 -device = isa-serial,chardev=3Dcharserial0,id=3Dserial0 
-device usb-tablet,id=3Dinput0,bus=3Dusb.0,port=3D1 -vnc 0.0.0.0:0 -k = en-us -device cirrus-vga,id=3Dvideo0,bus=3Dpci.0,addr=3D0x2 
-device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x8 -sandbox = on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resourcecontrol=3Ddeny 
-device pvpanic,ioport=3D1285 -msg = timestamp=3Don -d int,unimp,guest_errors
2.stop guest and start guest = repeatedly 
= virsh qemu-monitor-command --domain instance-03 = '{"execute":"stop", = "arguments":{}}'
= virsh qemu-monitor-command --domain instance-03 = '{"execute":"cont", = "arguments":{}}'

3.Actual results:
   Qemu = core dump with error msg:
   (qemu) = qemu: qemu_mutex_unlock_impl: Operation not permitted
   = Expected results:
   Guest can stop and start successfully
Additional info:
[Current thread is 1 (Thread 0x7ff188abb700 (LWP 2229116))]
(gdb) bt
#0  __GI_raise = (sig=3Dsig@entry=3D6) at = ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ff19d73542a in __GI_abort () at abort.c:89
#2  = 0x0000560d55a3ca65 in error_exit (err=3Derr@entry=3D1, msg=3Dmsg@entry=3D0x560d566e9650 = <__func__.18241> "qemu_mutex_unlock_impl") at = util/qemu-thread-posix.c:37
#3  0x0000560d55e76dc8 in = qemu_mutex_unlock_impl (mutex=3Dmutex@entry=3D0x560d58456e30, = file=3Dfile@entry=3D0x560d566e87df "util/async.c", line=3Dline@entry=3D509) at = util/qemu-thread-posix.c:99
#4  0x0000560d55e71580 in = aio_context_release (ctx=3D0x560d58456dd0) at util/async.c:509
#5  0x0000560d55acf344 in = virtio_blk_rw_complete (opaque=3D<optimized out>, ret=3D0) at = /data00/susieqemu/hw/block/virtio-blk.c:126
#6  0x0000560d55dcabc9 in = blk_aio_complete (acb=3D0x7ff17c001bf0) at block/block-backend.c:1345
#7  = 0x0000560d55e8a25b in coroutine_trampoline (i0=3D<optimized out>, i1=3D<optimized out>) at util/coroutine-ucontext.c:116
#8  = 0x00007ff19d745000 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#9  = 0x00007ffd3691e190 in ?? ()
#10 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address = 0x7fef29cdd000

Thread 3 (Thread 0x7ff1a38f23c0 = (LWP 2229113)):
#0  = memory_region_ioeventfd_before (a=3D..., b=3D...) at /data00/susieqemu/memory.c:185
#1  = address_space_add_del_ioeventfds (fds_old_nb=3D65, fds_old=3D0x560d59733960, fds_new_nb=3D64, fds_new=3D0x560d59ba6d10, as=3D0x560d598728b0) at /data00/susieqemu/memory.c:794
#2  = address_space_update_ioeventfds (as=3Das@entry=3D0x560d598728b0) at = /data00/susieqemu/memory.c:877
#3  0x0000560d55aabd98 in = memory_region_transaction_commit () at /data00/susieqemu/memory.c:1080
#4  = 0x0000560d55aae260 in memory_region_del_eventfd (mr=3Dmr@entry=3D0x560d599a2b40, addr=3D<optimized = out>, = size=3Dsize@entry=3D0, = match_data=3D<optimized out>, data=3D<optimized out>, e=3D<optimized = out>) at = /data00/susieqemu/memory.c:2274
#5  0x0000560d55d113da in = virtio_pci_ioeventfd_assign (d=3D0x560d599a1e70, notifier=3D0x560d599db8d8, n=3D0, assign=3D<optimized out>) at = hw/virtio/virtio-pci.c:268
#6  0x0000560d55d14fef in virtio_bus_set_host_notifier = (bus=3D0x560d599a9f68, n=3Dn@entry=3D0, = assign=3Dassign@entry=3Dfalse) at hw/virtio/virtio-bus.c:289
#7  = 0x0000560d55ad2886 in virtio_blk_data_plane_stop (vdev=3D<optimized = out>) at = /data00/susieqemu/hw/block/dataplane/virtio-blk.c:295
#8  = 0x0000560d55d1459e in virtio_bus_stop_ioeventfd (bus=3D0x560d599a9f68) at = hw/virtio/virtio-bus.c:246
#9  0x0000560d55b02c4f in virtio_vmstate_change (opaque=3D0x560d599a9fe0, running=3D0, = state=3DRUN_STATE_PAUSED) at = /data00/susieqemu/hw/virtio/virtio.c:2236
#10 0x0000560d55ba2517 in = vm_state_notify (running=3Drunning@entry=3D0, state=3Dstate@entry=3DRUN_STATE_PAUSED) at vl.c:1649
#11 = 0x0000560d55a95c3a in do_vm_stop (state=3Dstate@entry=3DRUN_STATE_PAUSED, = send_stop=3Dsend_stop@entry=3Dtrue) at /data00/susieqemu/cpus.c:1013
#12 = 0x0000560d55a96132 in vm_stop (state=3Dstate@entry=3DRUN_STATE_PAUSED) at = /data00/susieqemu/cpus.c:2036
#13 0x0000560d55bb652b in qmp_stop (errp=3Derrp@entry=3D0x7ffd3691e9b0) at qmp.c:106
#14 = 0x0000560d55bae9aa in qmp_marshal_stop (args=3D<optimized out>, ret=3D<optimized = out>, = errp=3D0x7ffd3691e9f8) at = qapi/qapi-commands-misc.c:784
#15 0x0000560d55e65f89 in do_qmp_dispatch (errp=3D0x7ffd3691e9f0, = request=3D0x7ffd3691e9f0, cmds=3D<optimized out>) at = qapi/qmp-dispatch.c:111
#16 qmp_dispatch (cmds=3D<optimized out>, = request=3Drequest@entry=3D0x560d58467e60) at qapi/qmp-dispatch.c:164
#17 0x0000560d55a9e95d in = monitor_qmp_dispatch_one (req_obj=3Dreq_obj@entry=3D0x560d587b5a20) at = /data00/susieqemu/monitor.c:4084
#18 0x0000560d55a9ec99 in = monitor_qmp_bh_dispatcher (data=3D<optimized out>) at = /data00/susieqemu/monitor.c:4142
#19 0x0000560d55e70e2e in aio_bh_call (bh=3D0x560d58472a20) at = util/async.c:91
#20 aio_bh_poll (ctx=3Dctx@entry=3D0x560d584569e0) at = util/async.c:119
#21 0x0000560d55e7470f in aio_dispatch (ctx=3D0x560d584569e0) at = util/aio-posix.c:456
#22 0x0000560d55e70ced in aio_ctx_dispatch (source=3D<optimized = out>, = callback=3D<optimized out>, user_data=3D<optimized out>) at = util/async.c:262
#23 0x00007ff1a0d087f7 in g_main_context_dispatch () from = /lib/x86_64-linux-gnu/libglib-2.0.so.0
#24 0x0000560d55e737d8 in glib_pollfds_poll = () at = util/main-loop.c:217
#25 os_host_main_loop_wait (timeout=3D<optimized out>) at = util/main-loop.c:265
#26 main_loop_wait (nonblocking=3D<optimized out>) at = util/main-loop.c:524
#27 0x0000560d55a41ff7 in main_loop () at vl.c:1980
#28 main (argc=3D<optimized = out>, = argv=3D<optimized out>, envp=3D<optimized out>) at vl.c:4845
This is because in the virtio_blk_dataplane_stop() function, the context of the = iothread will be set to the maincontext in blk_set_aio_context(). In the middle of this = context change, the iothread = will acquire the original iothread context which is held by the main = thread. After the mainthread change the context of the iothread and then = release the original iothread context, the iothread will get the old iothread context which = is not supposed to happen because the context has been changed to the = main thread context. So when the iothread finish = virtio_blk_rw_complete() and = try to release the main thread context, qemu will crash.

This patch can = solve this problem for me, but I am not sure whether this is right, please give me a = hand.

diff --git a/block/.block-backend.c.swp = b/block/.block-backend.c.swp
deleted file mode 100644
index = 0bf897dd..00000000
Binary files a/block/.block-backend.c.swp and /dev/null = differ
diff --git a/hw/block/virtio-blk.c = b/hw/block/virtio-blk.c
index b9c1b3ba..d8ac782d 100644
--- = a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -91,7 +91,12 @@ static void virtio_blk_rw_complete(void *opaque, int ret)
     VirtIOBlock = *s =3D next->dev;
     VirtIODevice *vdev =3D VIRTIO_DEVICE(s);
     = QEMU_LOG(LOG_INFO, "virtio_blk_rw_complete = before context_acquire: %p\n", (blk_get_aio_context(s->conf.conf.blk)));
-    aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
+    static struct timespec ts_sleep;
+    ts_sleep.tv_sec =3D= 0;
+    ts_sleep.tv_nsec = =3D 50 * 1000 * 1000ull;
+    while (aio_context_tryacquire(blk_get_aio_context(s->conf.conf.blk))) {
+        nanosleep(&ts_sleep, NULL);
+    }
     = while (next) {
     =     VirtIOBlockReq *req =3D next;
         next =3D = req->mr_next;
diff --git = a/include/block/aio.h b/include/block/aio.h
index a1d6b9e2..59be498f = 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -174,6 +174,8 @@ void = aio_context_ref(AioContext = *ctx);
  */
 void = aio_context_unref(AioContext *ctx);

 

+int aio_context_tryacquire(AioContext *ctx);
+
 /* Take ownership of the = AioContext.  If the AioContext will be shared between
  * = threads, and a thread = does not want to be interrupted, it will have to
  * take ownership around calls to = aio_poll().  = Otherwise, = aio_poll()
diff --git = a/util/async.c b/util/async.c
index 27fb4901..009f6c5f 100644
--- = a/util/async.c
+++ b/util/async.c
@@ -498,6 +498,10 @@ void aio_context_unref(AioContext *ctx)
     = g_source_unref(&ctx->source);
 }

 

+int aio_context_acquire(AioContext *ctx) {
+    = return qemu_rec_mutex_trylock(&ctx->lock);
+}
+
 void aio_context_acquire(AioContext *ctx)
 {
     qemu_rec_mutex_lock(&ctx->lock);


= --Apple-Mail=_110C1946-0FD0-4609-A383-653CC8A7192A--