From nobody Mon Nov 10 05:16:34 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1489757755102248.0423388501257; Fri, 17 Mar 2017 06:35:55 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A587965728; Fri, 17 Mar 2017 13:35:54 +0000 (UTC) Received: from colo-mx.corp.redhat.com (unknown [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 73B031796B; Fri, 17 Mar 2017 13:35:54 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 267244BB75; Fri, 17 Mar 2017 13:35:54 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v2HDZr1j013965 for ; Fri, 17 Mar 2017 09:35:53 -0400 Received: by smtp.corp.redhat.com (Postfix) id 068B117F5F; Fri, 17 Mar 2017 13:35:53 +0000 (UTC) Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F1BFA1796B for ; Fri, 17 Mar 2017 13:35:51 +0000 (UTC) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 153F8486DC for ; Fri, 17 Mar 2017 13:35:50 +0000 (UTC) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 9662E206BC for ; Fri, 17 Mar 2017 09:35:48 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute5.internal (MEProxy); Fri, 17 Mar 2017 09:35:48 -0400 Received: from localhost.localdomain (tor01.zencurity.dk [185.129.62.62]) by mail.messagingengine.com (Postfix) with ESMTPA id DFB027E0A4 for ; Fri, 17 Mar 2017 09:35:47 -0400 (EDT) Received: by localhost.localdomain (Postfix, from userid 1000) id 8DA5A1C82B; Fri, 17 Mar 2017 14:35:43 +0100 (CET) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A587965728 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=invisiblethingslab.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A587965728 Authentication-Results: mx1.redhat.com; dkim=fail reason="signature verification failed" (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="CJYQfR9G" DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 153F8486DC Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=invisiblethingslab.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=none smtp.mailfrom=woju@invisiblethingslab.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 153F8486DC DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=rvWB2pt3wIF8JJ0axu rf6opVqfw=; b=CJYQfR9GArIIRwi9bZYreOEjWeisXUCZsasrjdRNY4BkSU1L88 VZVuAWYAN+8o88lt4ikHbdQmz3tTBta4kuJ4cXoM1rFKaB5orIVIIHkUq5v4fn6B 5SNJsSGBwQGSZg5m1qkzQUXW3vQg+WZ1uZHUcfi5pEvRMX7rypiNPH4TMU+g/PWO AEnDCMnUxpn97FW9CvbHtUHQ5Xh3En17EtLJ96FylKPCKTjXNZ6lAW141V9CdRfz zhiGc+/pAzJMHxDcQIXfkTQpZvrPEcCaUhhcC0WJ31yOOv6+D0AXh/QhSeNzjluV KNEczokAlaF0ju3SCSAkBzUh5j+M9Fk/XsqA== X-ME-Sender: X-Sasl-enc: kqV42iFjQ15tdkaXvkGxFnR4om4Ih+YheTydkz4DXOzB 1489757747 Date: Fri, 17 Mar 2017 14:35:43 +0100 From: Wojtek Porczyk To: libvirt-list@redhat.com Message-ID: <146cb29018f3eb5a571cfac147030f8285ac74f4.1489756318.git.woju@invisiblethingslab.com> Mail-Followup-To: libvirt-list@redhat.com References: MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 17 Mar 2017 13:35:50 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 17 Mar 2017 13:35:50 +0000 (UTC) for IP:'66.111.4.25' DOMAIN:'out1-smtp.messagingengine.com' HELO:'out1-smtp.messagingengine.com' FROM:'woju@invisiblethingslab.com' RCPT:'' X-RedHat-Spam-Score: -0.32 (BAYES_50, DCC_REPUT_00_12, DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL) 66.111.4.25 out1-smtp.messagingengine.com 66.111.4.25 out1-smtp.messagingengine.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 1/2] Allow for ff callbacks to be called by custom event implementations X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============8678427694849166044==" Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 17 Mar 2017 13:35:55 +0000 (UTC) X-Zoho-Virus-Status: 1 X-ZohoMail: RSF_0 Z_629925259 SPT_0 --===============8678427694849166044== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="AhhlLboLdkugWU4S" Content-Disposition: inline --AhhlLboLdkugWU4S Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable The documentation says: > If the opaque user data requires free'ing when the handle is > unregistered, then a 2nd callback can be supplied for this purpose. > This callback needs to be invoked from a clean stack. If 'ff' > callbacks are invoked directly from the virEventRemoveHandleFunc they > will likely deadlock in libvirt. And they did deadlock. In removeTimeout too. Now we supply a custom function to pick it from the opaque blob and fire. Signed-off-by: Wojtek Porczyk --- libvirt-override.c | 36 ++++++++++++------------------------ libvirt-override.py | 39 +++++++++++++++++++++++++++++++++++++++ sanitytest.py | 5 +++-- 3 files changed, 54 insertions(+), 26 deletions(-) diff --git a/libvirt-override.c b/libvirt-override.c index 9e40f00..37f7ee2 100644 --- a/libvirt-override.c +++ b/libvirt-override.c @@ -5223,6 +5223,9 @@ libvirt_virEventAddHandleFunc(int fd, =20 VIR_PY_TUPLE_SET_GOTO(pyobj_args, 3, cb_args, cleanup); =20 + /* If changing contents of the opaque object, please also change + * virEventExecuteFFCallback() in libvirt-override.py + */ VIR_PY_TUPLE_SET_GOTO(cb_args, 0, libvirt_virEventHandleCallbackWrap(c= b), cleanup); VIR_PY_TUPLE_SET_GOTO(cb_args, 1, libvirt_virVoidPtrWrap(opaque), clea= nup); VIR_PY_TUPLE_SET_GOTO(cb_args, 2, libvirt_virFreeCallbackWrap(ff), cle= anup); @@ -5292,20 +5295,11 @@ libvirt_virEventRemoveHandleFunc(int watch) VIR_PY_TUPLE_SET_GOTO(pyobj_args, 0, libvirt_intWrap(watch), cleanup); =20 result =3D PyEval_CallObject(removeHandleObj, pyobj_args); - if (!result) { + if (result) { + retval =3D 0; + } else { PyErr_Print(); PyErr_Clear(); - } else if (!PyTuple_Check(result) || PyTuple_Size(result) !=3D 3) { - DEBUG("%s: %s must return opaque obj registered with %s" - "to avoid leaking libvirt memory\n", - __FUNCTION__, NAME(removeHandle), NAME(addHandle)); - } else { - opaque =3D PyTuple_GetItem(result, 1); - ff =3D PyTuple_GetItem(result, 2); - cff =3D PyvirFreeCallback_Get(ff); - if (cff) - (*cff)(PyvirVoidPtr_Get(opaque)); - retval =3D 0; } =20 cleanup: @@ -5350,6 +5344,9 @@ libvirt_virEventAddTimeoutFunc(int timeout, =20 VIR_PY_TUPLE_SET_GOTO(pyobj_args, 2, cb_args, cleanup); =20 + /* If changing contents of the opaque object, please also change + * virEventExecuteFFCallback() in libvirt-override.py + */ VIR_PY_TUPLE_SET_GOTO(cb_args, 0, libvirt_virEventTimeoutCallbackWrap(= cb), cleanup); VIR_PY_TUPLE_SET_GOTO(cb_args, 1, libvirt_virVoidPtrWrap(opaque), clea= nup); VIR_PY_TUPLE_SET_GOTO(cb_args, 2, libvirt_virFreeCallbackWrap(ff), cle= anup); @@ -5416,20 +5413,11 @@ libvirt_virEventRemoveTimeoutFunc(int timer) VIR_PY_TUPLE_SET_GOTO(pyobj_args, 0, libvirt_intWrap(timer), cleanup); =20 result =3D PyEval_CallObject(removeTimeoutObj, pyobj_args); - if (!result) { + if (result) { + retval =3D 0; + } else { PyErr_Print(); PyErr_Clear(); - } else if (!PyTuple_Check(result) || PyTuple_Size(result) !=3D 3) { - DEBUG("%s: %s must return opaque obj registered with %s" - "to avoid leaking libvirt memory\n", - __FUNCTION__, NAME(removeTimeout), NAME(addTimeout)); - } else { - opaque =3D PyTuple_GetItem(result, 1); - ff =3D PyTuple_GetItem(result, 2); - cff =3D PyvirFreeCallback_Get(ff); - if (cff) - (*cff)(PyvirVoidPtr_Get(opaque)); - retval =3D 0; } =20 cleanup: diff --git a/libvirt-override.py b/libvirt-override.py index 63f8ecb..3d09d63 100644 --- a/libvirt-override.py +++ b/libvirt-override.py @@ -16,6 +16,7 @@ except ImportError: if str(cyg_e).count("No module named"): raise lib_e =20 +import ctypes import types =20 # The root of all libvirt errors. @@ -211,3 +212,41 @@ def virEventAddTimeout(timeout, cb, opaque): ret =3D libvirtmod.virEventAddTimeout(timeout, cbData) if ret =3D=3D -1: raise libvirtError ('virEventAddTimeout() failed') return ret + + +# +# a caller for the ff callbacks for custom event loop implementations +# + +ctypes.pythonapi.PyCapsule_GetPointer.restype =3D ctypes.c_void_p +ctypes.pythonapi.PyCapsule_GetPointer.argtypes =3D ( + ctypes.py_object, ctypes.c_char_p) + +_virFreeCallback =3D ctypes.CFUNCTYPE(None, ctypes.c_void_p) + +def virEventExecuteFFCallback(opaque): + """ + Execute callback which frees the opaque buffer + + @opaque: the opaque object passed to addHandle or addTimeout + + WARNING: This function should not be called from any call by libvirt's + core. It will most probably cause deadlock in C-level libvirt code. + Instead it should be scheduled and called from implementation's stack. + + See https://libvirt.org/html/libvirt-libvirt-event.html#virEventAddHan= dleFunc + for more information. + + This function is not dependent on any event loop implementation. + """ + # The opaque object is really a 3-tuple, which contains a the real opa= que + # pointer and the ff callback, both of which are inside PyCapsules. If= not + # specified, the ff may be None. See libvirt-override.c. + + dummy, caps_opaque, caps_ff =3D opaque + ff =3D _virFreeCallback(ctypes.pythonapi.PyCapsule_GetPointer( + caps_ff, "virFreeCallback".encode("ascii"))) + if ff: + real_opaque =3D ctypes.pythonapi.PyCapsule_GetPointer( + caps_opaque, "void*".encode("ascii")) + ff(real_opaque) diff --git a/sanitytest.py b/sanitytest.py index a140ba2..6548831 100644 --- a/sanitytest.py +++ b/sanitytest.py @@ -345,11 +345,12 @@ for name in sorted(finalklassmap): =20 # Phase 6: Validate that every python API has a corresponding C API for klass in gotfunctions: - if klass =3D=3D "libvirtError": + if klass in ("libvirtError", "virEventAsyncIOImpl"): continue for func in sorted(gotfunctions[klass]): # These are pure python methods with no C APi - if func in ["connect", "getConnect", "domain", "getDomain"]: + if func in ["connect", "getConnect", "domain", "getDomain", + "virEventRegisterAsyncIOImpl"]: continue =20 key =3D "%s.%s" % (klass, func) --=20 2.5.5 --AhhlLboLdkugWU4S Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYy+YuAAoJEL9r2TIQOiNReFQQAJgWgRTQGg2CCOhuYrQw65D2 DSSwBIzaHD2DWj+BEUDx3j7cXxrceClivymnK+lo9pTu13uy3FpyKj5HBhH8jjR1 3i7tiHSTJt8RZImSV3vAeiYBCWd3tTlTQqFFKQ5PuCl1pBtUTFrTaPa8WtgOQrbz dCdk+UhvOx1BJqnXZMbwx7tUVwPmFyitFwCUW2M2BA/+4ODl0uXZu8RCB5Tnd665 OnSTvk3V9zXhS/nFRUWlxF+Pjcwjuk5DRz/u20zXsqVtF2O7/i8PetIgRLX3PdlO qaffALK7F4EeWop3uI3S3Zu4V6a31qgbCWdLfIxJJ2C+1WPeUR7TnYwQvTuyUbkS xsALeo+5iFX668+PkNoIkntk/1X5GWbVb+eT6jobJupzpoH4ccgW/7ne/VQowfUO 72p1LurHqc6F9qsdOuPwkhwuWN+6pvgzf8hDxj3hN/Z7Fj1sdOGp0UqdVOVCZrEw MZO6cZin9kTrUWiHMXddc4SfWYPbrWgdZe9mg+lRRbDubX6tA5fWYiDijejVM31K xvNUF0xucEQ/9MDpUAtvvJyQHsUz+jtqpbsr2qIACkdqYp0z+J7LBi39ttIIrUZX jqKZQmSR8h1EdAeJIz1HsDPq7hguJtkTszBflluHTIvLNSWhaVroU2jx8V9EtOEy EcgMKNPnwa2wif20Ia8X =uyE1 -----END PGP SIGNATURE----- --AhhlLboLdkugWU4S-- --===============8678427694849166044== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list --===============8678427694849166044==--