From nobody Mon Feb 9 09:22:48 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1491207967220574.7828645132366; Mon, 3 Apr 2017 01:26:07 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8135480F97; Mon, 3 Apr 2017 08:26:05 +0000 (UTC) Received: from colo-mx.corp.redhat.com (unknown [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 51EAE7E877; Mon, 3 Apr 2017 08:26:05 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 52DF41FF; Mon, 3 Apr 2017 08:25:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v338PRqf001019 for ; Mon, 3 Apr 2017 04:25:31 -0400 Received: by smtp.corp.redhat.com (Postfix) id 6634E17970; Mon, 3 Apr 2017 08:25:27 +0000 (UTC) Received: from mx1.redhat.com (ext-mx09.extmail.prod.ext.phx2.redhat.com [10.5.110.38]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5DCA8A0A11 for ; Mon, 3 Apr 2017 08:25:25 +0000 (UTC) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 57AFB4E4D9 for ; Mon, 3 Apr 2017 08:24:56 +0000 (UTC) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v338J0MF043620 for ; Mon, 3 Apr 2017 04:24:47 -0400 Received: from e06smtp10.uk.ibm.com (e06smtp10.uk.ibm.com [195.75.94.106]) by mx0b-001b2d01.pphosted.com with ESMTP id 29k3yc3q10-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 03 Apr 2017 04:24:47 -0400 Received: from localhost by e06smtp10.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 3 Apr 2017 09:24:45 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp10.uk.ibm.com (192.168.101.140) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 3 Apr 2017 09:24:42 +0100 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v338OgMI36503692; Mon, 3 Apr 2017 08:24:42 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 56233A4069; Mon, 3 Apr 2017 09:24:04 +0100 (BST) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3A5C6A4053; Mon, 3 Apr 2017 09:24:04 +0100 (BST) Received: from marc-ibm.boeblingen.de.ibm.com (unknown [9.152.224.184]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 3 Apr 2017 09:24:04 +0100 (BST) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8135480F97 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 8135480F97 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 57AFB4E4D9 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=none smtp.mailfrom=mhartmay@linux.vnet.ibm.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 57AFB4E4D9 From: Marc Hartmayer To: Libvirt Mailing List Date: Mon, 3 Apr 2017 10:24:35 +0200 In-Reply-To: <20170403082439.10180-1-mhartmay@linux.vnet.ibm.com> References: <20170403082439.10180-1-mhartmay@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17040308-0040-0000-0000-0000035B1035 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17040308-0041-0000-0000-000024C89F98 Message-Id: <20170403082439.10180-2-mhartmay@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-03_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704030076 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 03 Apr 2017 08:25:01 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 03 Apr 2017 08:25:01 +0000 (UTC) for IP:'148.163.158.5' DOMAIN:'mx0b-001b2d01.pphosted.com' HELO:'mx0a-001b2d01.pphosted.com' FROM:'mhartmay@linux.vnet.ibm.com' RCPT:'' X-RedHat-Spam-Score: 1.3 * (BAYES_50, RCVD_IN_SORBS_SPAM) 148.163.158.5 mx0b-001b2d01.pphosted.com 148.163.158.5 mx0b-001b2d01.pphosted.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.38 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Marc Hartmayer Subject: [libvirt] [PATCH 1/5] qemu: Fix two use-after-free situations X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 03 Apr 2017 08:26:06 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" There were multiple race conditions that could lead to segmentation faults. The first precondition for this is qemuProcessLaunch must fail sometime shortly after starting the new QEMU process. The second precondition for the segmentation faults is that the new QEMU process dies - or to be more precise the QEMU monitor has to be closed irregularly. If both happens during qemuProcessStart (starting a domain) there are race windows between the thread with the event loop (T1) and the thread that is starting the domain (T2). First segmentation fault scenario: If qemuProcessLaunch fails during qemuProcessStart the code branches to the 'stop' path where 'qemuMonitorSetDomainLog(priv->mon, NULL, NULL, NULL)' will set the log function of the monitor to NULL (done in T2). In the meantime the event loop of T1 will wake up with an EOF event for the QEMU monitor because the QEMU process has died. The crash occurs if T1 has checked 'mon->logFunc !=3D NULL' in qemuMonitorIO just before the logFunc was set to NULL by T2. If this situation occurs T1 will try to call mon->logFunc which leads to the segmentation fault. Solution: Require the monitor lock for setting the log function. Backtrace: 0 0x0000000000000000 in ?? () 1 0x000003ffe9e45316 in qemuMonitorIO (watch=3D, fd=3D, events=3D, opaque=3D0x3ffe08aa860) at ../../src/qemu/qemu_monitor.c:727 2 0x000003fffda2e1a4 in virEventPollDispatchHandles (nfds=3D, fds=3D0x2aa000fd980) at ../../src/util/vireventpoll.c:508 3 0x000003fffda2e398 in virEventPollRunOnce () at ../../src/util/vireventpoll.c:657 4 0x000003fffda2ca10 in virEventRunDefaultImpl () at ../../src/util/virevent.c:314 5 0x000003fffdba9366 in virNetDaemonRun (dmn=3D0x2aa000cc550) at ../../src/rpc/virnetdaemon.c:818 6 0x000002aa00024668 in main (argc=3D, argv=3D) at ../../daemon/libvirtd.c:1541 Second segmentation fault scenario: If qemuProcessLaunch fails it will unref the log context and with invoking qemuMonitorSetDomainLog(priv->mon, NULL, NULL, NULL) qemuDomainLogContextFree() will be invoked. qemuDomainLogContextFree() invokes virNetClientClose() to close the client and cleans everything up (including unref of _virLogManager.client) when virNetClientClose() returns. When T1 is now trying to report 'qemu unexpectedly closed the monitor' libvirtd will crash because the client has already been freed. Solution: As the critical section in qemuMonitorIO is protected with the monitor lock we can use the same solution as proposed for the first segmentation fault. Backtrace: 0 virClassIsDerivedFrom (klass=3D0x3100979797979797, parent=3D0x2aa000d92f0) at ../../src/util/virobject.c:169 1 0x000003fffda659e6 in virObjectIsClass (anyobj=3D, klass=3D) at ../../src/util/virobject.c:365 2 0x000003fffda65a24 in virObjectLock (anyobj=3D0x3ffe08c1db0) at ../../src/util/virobject.c:317 3 0x000003fffdba4688 in virNetClientIOEventLoop (client=3Dclient@entry=3D0x3ffe08c1db0, thiscall=3Dthiscall@entry=3D0x2aa000fbfa0) at ../../src/rpc/virnetclient.c:1668 4 0x000003fffdba4b4c in virNetClientIO (client=3Dclient@entry=3D0x3ffe08c1db0, thiscall=3D0x2aa000fbfa0) at ../../src/rpc/virnetclient.c:1944 5 0x000003fffdba4d42 in virNetClientSendInternal (client=3Dclient@entry=3D0x3ffe08c1db0, msg=3Dmsg@entry=3D0x2aa000cc710, expectReply=3DexpectReply@entry=3Dtrue, nonBlock=3DnonBlock@entry=3Dfalse) at ../../src/rpc/virnetclient.c:2116 6 0x000003fffdba6268 in virNetClientSendWithReply (client=3D0x3ffe08c1db0, msg=3D0x2aa000cc710) at ../../src/rpc/virnetclient.c:2144 7 0x000003fffdba6e8e in virNetClientProgramCall (prog=3D0x3ffe08c1120, client=3D, serial=3D, proc=3D, noutfds=3D, outfds=3D0x0, ninfds=3D0x0, infds=3D0x0, args_filter=3D0x3fffdb64440 , args=3D0x3ffffffe010, ret_filter=3D0x3fffdb644c0 , ret=3D0x3ffffffe008) at ../../src/rpc/virnetclientprogram.c:329 8 0x000003fffdb64042 in virLogManagerDomainReadLogFile (mgr=3D, path=3D, inode=3D, offset=3D, maxlen=3D, flags=3D0) at ../../src/logging/log_manager.c:272 9 0x000003ffe9e0315c in qemuDomainLogContextRead (ctxt=3D0x3ffe08c2980, msg=3D0x3ffffffe1c0) at ../../src/qemu/qemu_domain.c:4422 10 0x000003ffe9e280a8 in qemuProcessReadLog (logCtxt=3D, msg=3Dmsg@entry=3D0x3ffffffe288) at ../../src/qemu/qemu_process.c:1800 11 0x000003ffe9e28206 in qemuProcessReportLogError (logCtxt=3D, msgprefix=3D0x3ffe9ec276a "qemu unexpectedly closed the monitor") at ../../src/qemu/qemu_process.c:1836 12 0x000003ffe9e28306 in qemuProcessMonitorReportLogError (mon=3Dmon@entry=3D0x3ffe085cf10, msg=3D, opaque=3D) at ../../src/qemu/qemu_process.c:1856 13 0x000003ffe9e452b6 in qemuMonitorIO (watch=3D, fd=3D, events=3D, opaque=3D0x3ffe085cf10) at ../../src/qemu/qemu_monitor.c:726 14 0x000003fffda2e1a4 in virEventPollDispatchHandles (nfds=3D, fds=3D0x2aa000fd980) at ../../src/util/vireventpoll.c:508 15 0x000003fffda2e398 in virEventPollRunOnce () at ../../src/util/vireventpoll.c:657 16 0x000003fffda2ca10 in virEventRunDefaultImpl () at ../../src/util/virevent.c:314 17 0x000003fffdba9366 in virNetDaemonRun (dmn=3D0x2aa000cc550) at ../../src/rpc/virnetdaemon.c:818 18 0x000002aa00024668 in main (argc=3D, argv=3D) at ../../daemon/libvirtd.c:1541 Other code parts where the same problem was possible to occur are fixed as well (qemuMigrationFinish, qemuProcessStart, and qemuDomainSaveImageStartVM). Signed-off-by: Marc Hartmayer Reported-by: Sascha Silbe --- src/qemu/qemu_monitor.c | 44 ++++++++++++++++++++++++++++++++++---------- src/qemu/qemu_monitor.h | 4 ++++ 2 files changed, 38 insertions(+), 10 deletions(-) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index a4fa6ec..b41aaed 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -963,7 +963,7 @@ qemuMonitorClose(qemuMonitorPtr mon) PROBE(QEMU_MONITOR_CLOSE, "mon=3D%p refs=3D%d", mon, mon->parent.parent.u.s.refs); =20 - qemuMonitorSetDomainLog(mon, NULL, NULL, NULL); + qemuMonitorSetDomainLogLocked(mon, NULL, NULL, NULL); =20 if (mon->fd >=3D 0) { qemuMonitorUnregister(mon); @@ -4035,20 +4035,21 @@ qemuMonitorGetDeviceAliases(qemuMonitorPtr mon, =20 =20 /** - * qemuMonitorSetDomainLog: - * Set the file descriptor of the open VM log file to report potential - * early startup errors of qemu. - * - * @mon: Monitor object to set the log file reading on + * qemuMonitorSetDomainLogLocked: + * @mon: Locked monitor object to set the log file reading on * @func: the callback to report errors * @opaque: data to pass to @func * @destroy: optional callback to free @opaque + * + * Set the file descriptor of the open VM log file to report potential + * early startup errors of qemu. This function requires @mon to be + * locked already! */ void -qemuMonitorSetDomainLog(qemuMonitorPtr mon, - qemuMonitorReportDomainLogError func, - void *opaque, - virFreeCallback destroy) +qemuMonitorSetDomainLogLocked(qemuMonitorPtr mon, + qemuMonitorReportDomainLogError func, + void *opaque, + virFreeCallback destroy) { if (mon->logDestroy && mon->logOpaque) mon->logDestroy(mon->logOpaque); @@ -4060,6 +4061,29 @@ qemuMonitorSetDomainLog(qemuMonitorPtr mon, =20 =20 /** + * qemuMonitorSetDomainLog: + * @mon: Unlocked monitor object to set the log file reading on + * @func: the callback to report errors + * @opaque: data to pass to @func + * @destroy: optional callback to free @opaque + * + * Set the file descriptor of the open VM log file to report potential + * early startup errors of qemu. This functions requires @mon to be + * unlocked. + */ +void +qemuMonitorSetDomainLog(qemuMonitorPtr mon, + qemuMonitorReportDomainLogError func, + void *opaque, + virFreeCallback destroy) +{ + virObjectLock(mon); + qemuMonitorSetDomainLogLocked(mon, func, opaque, destroy); + virObjectUnlock(mon); +} + + +/** * qemuMonitorJSONGetGuestCPU: * @mon: Pointer to the monitor * @arch: arch of the guest diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 91ab905..2e42d16 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -1067,6 +1067,10 @@ int qemuMonitorGetDeviceAliases(qemuMonitorPtr mon, typedef void (*qemuMonitorReportDomainLogError)(qemuMonitorPtr mon, const char *msg, void *opaque); +void qemuMonitorSetDomainLogLocked(qemuMonitorPtr mon, + qemuMonitorReportDomainLogError func, + void *opaque, + virFreeCallback destroy); void qemuMonitorSetDomainLog(qemuMonitorPtr mon, qemuMonitorReportDomainLogError func, void *opaque, --=20 2.5.5 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list