From nobody Sat Apr 27 20:00:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1510001602197481.6414448498274; Mon, 6 Nov 2017 12:53:22 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 361805F7AD; Mon, 6 Nov 2017 20:53:17 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1671C5D6A3; Mon, 6 Nov 2017 20:53:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6EE7B1800BDC; Mon, 6 Nov 2017 20:53:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id vA6KrEVF002436 for ; Mon, 6 Nov 2017 15:53:14 -0500 Received: by smtp.corp.redhat.com (Postfix) id B9FB2190C3; Mon, 6 Nov 2017 20:53:14 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-90.phx2.redhat.com [10.3.116.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6E8D1190C2; Mon, 6 Nov 2017 20:53:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 361805F7AD Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=libvir-list-bounces@redhat.com From: John Ferlan To: libvir-list@redhat.com Date: Mon, 6 Nov 2017 15:53:08 -0500 Message-Id: <20171106205309.22525-2-jferlan@redhat.com> In-Reply-To: <20171106205309.22525-1-jferlan@redhat.com> References: <20171106205309.22525-1-jferlan@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: cbosdonnat@suse.com Subject: [libvirt] [PATCH 1/2] storage: Resolve storage driver crash X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 06 Nov 2017 20:53:17 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Resolve a storage driver crash as a result of a long running storageVolCreateXML when the virStorageVolPoolRefreshThread is run as a result of a storageVolUpload complete and ran the virStoragePoolObjClearVols without checking if the creation code was currently processing a buildVol after incrementing the driver->asyncjob count. The refreshThread needs to wait until all creation threads are completed so as to not alter the volume list while the volume is being built. Crash from valgrind is as follows (with a bit of editing): =3D=3D21309=3D=3D Invalid read of size 8 =3D=3D21309=3D=3D at 0x153E47AF: storageBackendUpdateVolTargetInfo =3D=3D21309=3D=3D by 0x153E4C30: virStorageBackendUpdateVolInfo =3D=3D21309=3D=3D by 0x153E52DE: virStorageBackendVolRefreshLocal =3D=3D21309=3D=3D by 0x153DE29E: storageVolCreateXML =3D=3D21309=3D=3D by 0x562035B: virStorageVolCreateXML =3D=3D21309=3D=3D by 0x147366: remoteDispatchStorageVolCreateXML ... =3D=3D21309=3D=3D Address 0x2590a720 is 64 bytes inside a block of size 33= 6 free'd =3D=3D21309=3D=3D at 0x4C2F2BB: free =3D=3D21309=3D=3D by 0x54CB9FA: virFree =3D=3D21309=3D=3D by 0x55BC800: virStorageVolDefFree =3D=3D21309=3D=3D by 0x55BF1D8: virStoragePoolObjClearVols =3D=3D21309=3D=3D by 0x153D967E: virStorageVolPoolRefreshThread ... =3D=3D21309=3D=3D Block was alloc'd at =3D=3D21309=3D=3D at 0x4C300A5: calloc =3D=3D21309=3D=3D by 0x54CB483: virAlloc =3D=3D21309=3D=3D by 0x55BDC1F: virStorageVolDefParseXML =3D=3D21309=3D=3D by 0x55BDC1F: virStorageVolDefParseNode =3D=3D21309=3D=3D by 0x55BE5A4: virStorageVolDefParse =3D=3D21309=3D=3D by 0x153DDFF1: storageVolCreateXML =3D=3D21309=3D=3D by 0x562035B: virStorageVolCreateXML =3D=3D21309=3D=3D by 0x147366: remoteDispatchStorageVolCreateXML ... Signed-off-by: John Ferlan --- src/storage/storage_driver.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index b0edf9f885..5e920fc14c 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -2257,6 +2257,7 @@ virStorageVolPoolRefreshThread(void *opaque) virStorageBackendPtr backend; virObjectEventPtr event =3D NULL; =20 + retry: storageDriverLock(); if (cbdata->vol_path) { if (virStorageBackendPloopRestoreDesc(cbdata->vol_path) < 0) @@ -2270,6 +2271,14 @@ virStorageVolPoolRefreshThread(void *opaque) if (!(backend =3D virStorageBackendForType(def->type))) goto cleanup; =20 + /* Some thread is creating a new volume in the pool, we need to retry = */ + if (virStoragePoolObjGetAsyncjobs(obj) > 0) { + virStoragePoolObjUnlock(obj); + storageDriverUnlock(); + usleep(100 * 1000); + goto retry; + } + virStoragePoolObjClearVols(obj); if (backend->refreshPool(NULL, obj) < 0) VIR_DEBUG("Failed to refresh storage pool"); --=20 2.13.6 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sat Apr 27 20:00:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1510001608562828.1949400625638; Mon, 6 Nov 2017 12:53:28 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3FE2DC058ECE; Mon, 6 Nov 2017 20:53:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0600F5C8BA; Mon, 6 Nov 2017 20:53:26 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C61CE4A467; Mon, 6 Nov 2017 20:53:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id vA6KrFuD002443 for ; Mon, 6 Nov 2017 15:53:15 -0500 Received: by smtp.corp.redhat.com (Postfix) id 3370D190C3; Mon, 6 Nov 2017 20:53:15 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-90.phx2.redhat.com [10.3.116.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC71C190C2; Mon, 6 Nov 2017 20:53:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3FE2DC058ECE Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=libvir-list-bounces@redhat.com From: John Ferlan To: libvir-list@redhat.com Date: Mon, 6 Nov 2017 15:53:09 -0500 Message-Id: <20171106205309.22525-3-jferlan@redhat.com> In-Reply-To: <20171106205309.22525-1-jferlan@redhat.com> References: <20171106205309.22525-1-jferlan@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: cbosdonnat@suse.com Subject: [libvirt] [PATCH 2/2] scsi: Check for long running create in FCRefreshThread X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 06 Nov 2017 20:53:27 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Similar to a recent patch in virStorageVolPoolRefreshThread to ensure that there were no pool AsyncJobs (e.g. nothing being created at the time in a long running buildVol job), modify virStoragePoolFCRefreshThread to check for async jobs before calling virStoragePoolObjClearVols and refreshing the volumes defined in the pool. Signed-off-by: John Ferlan --- src/storage/storage_backend_scsi.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/storage/storage_backend_scsi.c b/src/storage/storage_backe= nd_scsi.c index 02fd4b643c..63a9154102 100644 --- a/src/storage/storage_backend_scsi.c +++ b/src/storage/storage_backend_scsi.c @@ -159,6 +159,7 @@ virStoragePoolFCRefreshThread(void *opaque) pool->def->allocation =3D pool->def->capacity =3D pool->def->avail= able =3D 0; =20 if (virStoragePoolObjIsActive(pool) && + virStoragePoolObjGetAsyncjobs(pool) =3D=3D 0 && virSCSIHostGetNumber(fchost_name, &host) =3D=3D 0 && virStorageBackendSCSITriggerRescan(host) =3D=3D 0) { virStoragePoolObjClearVols(pool); --=20 2.13.6 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list