From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993018; cv=none; d=zohomail.com; s=zohoarc; b=Pe4zfhriHjz2RLltP6DBqCNjIvFy8edqhD7Xg1Emwv+0qbn7O4+yV8CtkrCySdW5tU8/0mfZ2LpUccQLx7bxyi7en7ghZexwXp/smVIDfS5Eu4FlNqbd30B80ICXzfuFHJaVONpmtnNsJPl9/Tc68IBLKFsJ+7xOaSGwtwt0I+I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993018; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bwiDOfVjE/FAb1b/x87NcnJ24Pw7FqnMOItBaWEB77k=; b=K3yHerTEWlLmbnsBKoiLqLNLw/Q3PEO6XCDJB8Q1YXQ/uM2Mk4CwJ2T2NGa01REyrZYvHkbyi4rfKcXg8BJtOEwZgb1hFDhrMkIPHN3BuZOXQHpEnE0lR0C5AJL4tUX5lJ/g4V63VzHEb2Nbwze/fJqw68mGDeCV5Iz7qO+T89M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993018758821.040879197125; Wed, 24 Jun 2020 03:03:38 -0700 (PDT) Received: from localhost ([::1]:39866 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2Fl-0002t8-Fs for importer@patchew.org; Wed, 24 Jun 2020 06:03:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43364) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2Ee-00015v-Kh for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:28 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:31060 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2Ec-0003V4-Ka for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-411-j12izr9NP1O68XIksyYgog-1; Wed, 24 Jun 2020 06:02:21 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AFF3D1B18BC3; Wed, 24 Jun 2020 10:02:20 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 919C47FE87; Wed, 24 Jun 2020 10:02:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bwiDOfVjE/FAb1b/x87NcnJ24Pw7FqnMOItBaWEB77k=; b=FruaT+8uNHy1RDO+FN69wwME6KnKYqIepx6K0fRZLlJzqtMNjzquEv9EZmhZlcELji4C6x WYRiJnMFPEso/zshBOv5h6Ryi2+wJXKz5K5RerVcIB0QvCHkRQjIGTCEY+hOkFrrm52XyS RFiNXSH5XeACCQSHnoUce2t3OlOqP+Q= X-MC-Unique: j12izr9NP1O68XIksyYgog-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 01/12] minikconf: explicitly set encoding to UTF-8 Date: Wed, 24 Jun 2020 11:01:59 +0100 Message-Id: <20200624100210.59975-2-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 01:59:40 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, Richard Henderson , Max Reitz , Stefan Hajnoczi , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" QEMU currently only has ASCII Kconfig files but Linux actually uses UTF-8. Explicitly specify the encoding and that we're doing text file I/O. It's unclear whether or not QEMU will ever need Unicode in its Kconfig files. If we start using the help text then it will become an issue sooner or later. Make this change now for consistency with Linux Kconfig. Reported-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Stefan Hajnoczi Reviewed-by: Richard Henderson Message-id: 20200521153616.307100-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- scripts/minikconf.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/minikconf.py b/scripts/minikconf.py index 90b99517c1..bcd91015d3 100755 --- a/scripts/minikconf.py +++ b/scripts/minikconf.py @@ -402,7 +402,7 @@ class KconfigParser: if incl_abs_fname in self.data.previously_included: return try: - fp =3D open(incl_abs_fname, 'r') + fp =3D open(incl_abs_fname, 'rt', encoding=3D'utf-8') except IOError as e: raise KconfigParserError(self, '%s: %s' % (e.strerror, include)) @@ -696,7 +696,7 @@ if __name__ =3D=3D '__main__': parser.do_assignment(name, value =3D=3D 'y') external_vars.add(name[7:]) else: - fp =3D open(arg, 'r') + fp =3D open(arg, 'rt', encoding=3D'utf-8') parser.parse_file(fp) fp.close() =20 @@ -705,7 +705,7 @@ if __name__ =3D=3D '__main__': if key not in external_vars and config[key]: print ('CONFIG_%s=3Dy' % key) =20 - deps =3D open(argv[2], 'w') + deps =3D open(argv[2], 'wt', encoding=3D'utf-8') for fname in data.previously_included: print ('%s: %s' % (argv[1], fname), file=3Ddeps) deps.close() --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993148; cv=none; d=zohomail.com; s=zohoarc; b=DIOdq3fNrp43M9EpLMV4MbI2ecEPBJgtGx3R6R9nM1fzhkIZwkTvZlZvC+MurjNsPt593WoD3yjTh7CsUzlzox3utm8LU2Z6rJVvEpbZeLJfV9strWRnofyYnWWNoxbd6ZjWLDyTfnwpeLUXMboNub/2BoWbzgaCdoCjizoXldU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993148; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VX3xaEhQbYneUxyvAAyvSsdi9n8zcvzyBvibby1ATqM=; b=Qv62XglTMCNrap/0HTOghoEJM0cwdvXEADzW1+PY2coZJSyVsECfFNMDAD2wMtPVA82h01JMeCw57j1kduyGWrTiGWMYVaQ+B3kG0JtHO3fJnCyK76uW4IpHJ/rWa36NuBSGliSEI0YGVAgggqeHnK5VZ8buXQH/VRzWkWRr6Bw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993148635749.4617298303219; Wed, 24 Jun 2020 03:05:48 -0700 (PDT) Received: from localhost ([::1]:48882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2Hr-0006gX-An for importer@patchew.org; Wed, 24 Jun 2020 06:05:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2Ej-0001Cp-Th for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:33 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:53574 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2Eg-0003Vr-Mx for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:33 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-455-j7dofYqFOKSz7qNfvMLy2g-1; Wed, 24 Jun 2020 06:02:27 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D5CC10059A3; Wed, 24 Jun 2020 10:02:26 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5151D84FA; Wed, 24 Jun 2020 10:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992949; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VX3xaEhQbYneUxyvAAyvSsdi9n8zcvzyBvibby1ATqM=; b=VObzIVpDFmuuuTA9t+qVTfdSYLt/db98AJrYlCbouBn1be8APSLlijiSChplmi3eWBpwOi TCAnxO5kYYg2EXwqq/FNeJuaKad1vt05wO7KwBO4Dtu4/4B7ZQ5Ush1DXtZsM1+LpC8Yat j1rjUYj786YtrrCwCHSTr47v3PX6uQQ= X-MC-Unique: j7dofYqFOKSz7qNfvMLy2g-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 02/12] coroutine: support SafeStack in ucontext backend Date: Wed, 24 Jun 2020 11:02:00 +0100 Message-Id: <20200624100210.59975-3-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 01:59:40 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , Daniele Buono Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Daniele Buono LLVM's SafeStack instrumentation does not yet support programs that make use of the APIs in ucontext.h With the current implementation of coroutine-ucontext, the resulting binary is incorrect, with different coroutines sharing the same unsafe stack and producing undefined behavior at runtime. This fix allocates an additional unsafe stack area for each coroutine, and sets the new unsafe stack pointer before calling swapcontext() in qemu_coroutine_new. This is the only place where the pointer needs to be manually updated, since sigsetjmp/siglongjmp are already instrumented by LLVM to properly support SafeStack. The additional stack is then freed in qemu_coroutine_delete. Signed-off-by: Daniele Buono Message-id: 20200529205122.714-2-dbuono@linux.vnet.ibm.com Signed-off-by: Stefan Hajnoczi --- include/qemu/coroutine_int.h | 5 +++++ util/coroutine-ucontext.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/include/qemu/coroutine_int.h b/include/qemu/coroutine_int.h index bd6b0468e1..1da148552f 100644 --- a/include/qemu/coroutine_int.h +++ b/include/qemu/coroutine_int.h @@ -28,6 +28,11 @@ #include "qemu/queue.h" #include "qemu/coroutine.h" =20 +#ifdef CONFIG_SAFESTACK +/* Pointer to the unsafe stack, defined by the compiler */ +extern __thread void *__safestack_unsafe_stack_ptr; +#endif + #define COROUTINE_STACK_SIZE (1 << 20) =20 typedef enum { diff --git a/util/coroutine-ucontext.c b/util/coroutine-ucontext.c index 613f4c118e..f0b66320e1 100644 --- a/util/coroutine-ucontext.c +++ b/util/coroutine-ucontext.c @@ -45,6 +45,11 @@ typedef struct { Coroutine base; void *stack; size_t stack_size; +#ifdef CONFIG_SAFESTACK + /* Need an unsafe stack for each coroutine */ + void *unsafe_stack; + size_t unsafe_stack_size; +#endif sigjmp_buf env; =20 void *tsan_co_fiber; @@ -179,6 +184,10 @@ Coroutine *qemu_coroutine_new(void) co =3D g_malloc0(sizeof(*co)); co->stack_size =3D COROUTINE_STACK_SIZE; co->stack =3D qemu_alloc_stack(&co->stack_size); +#ifdef CONFIG_SAFESTACK + co->unsafe_stack_size =3D COROUTINE_STACK_SIZE; + co->unsafe_stack =3D qemu_alloc_stack(&co->unsafe_stack_size); +#endif co->base.entry_arg =3D &old_env; /* stash away our jmp_buf */ =20 uc.uc_link =3D &old_uc; @@ -203,6 +212,22 @@ Coroutine *qemu_coroutine_new(void) COROUTINE_YIELD, &fake_stack_save, co->stack, co->stack_size, co->tsan_co_fiber); + +#ifdef CONFIG_SAFESTACK + /* + * Before we swap the context, set the new unsafe stack + * The unsafe stack grows just like the normal stack, so start from + * the last usable location of the memory area. + * NOTE: we don't have to re-set the usp afterwards because we are + * coming back to this context through a siglongjmp. + * The compiler already wrapped the corresponding sigsetjmp call w= ith + * code that saves the usp on the (safe) stack before the call, and + * restores it right after (which is where we return with siglongj= mp). + */ + void *usp =3D co->unsafe_stack + co->unsafe_stack_size; + __safestack_unsafe_stack_ptr =3D usp; +#endif + swapcontext(&old_uc, &uc); } =20 @@ -235,6 +260,9 @@ void qemu_coroutine_delete(Coroutine *co_) #endif =20 qemu_free_stack(co->stack, co->stack_size); +#ifdef CONFIG_SAFESTACK + qemu_free_stack(co->unsafe_stack, co->unsafe_stack_size); +#endif g_free(co); } =20 --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993039; cv=none; d=zohomail.com; s=zohoarc; b=bTAeOg0f7190OOkmD0JOzm0D4GoIPxcBpW2JC7Gdud9ckCxzyU34SRFaJw2zts2SgW0RI1xw56jraJu0K+i+wYZ53LkPZB5VmUYa95mHN9ivWiWXZm3678XnV5XHaiEpT0EjOMVqUafo5NILR402jprtQGVU0QAxYsI71V3STaI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993039; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=E39TEhvqpslW6LMdaJyMYG+MFqK+bgNrvvGjnCuls6c=; b=Vk5ml6ZQBGEYx1/oUJ+E8j3H4PcJn8kacuigMSUOj2HpU3tbAw7FZD8RivQbnDpj3ZeAz1JUUF0QE8QF+KOEl6nALMODNOOvs1jwlDAuaYG/+nCCpdMnqHUyLtXL1A2z+CIi61GydwXZMMK9XCwvFq4eF5aFeAtMrIBAsAedeTU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993039128457.1467883860696; Wed, 24 Jun 2020 03:03:59 -0700 (PDT) Received: from localhost ([::1]:40702 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2G5-0003E3-R6 for importer@patchew.org; Wed, 24 Jun 2020 06:03:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43436) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2Eo-0001NE-1a for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:38 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:41429 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2Em-0003XU-CJ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:37 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-282-2rit4UI1PwK5BFegOvQAyg-1; Wed, 24 Jun 2020 06:02:33 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BE3DB1B18BC6; Wed, 24 Jun 2020 10:02:31 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 877EF5C557; Wed, 24 Jun 2020 10:02:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E39TEhvqpslW6LMdaJyMYG+MFqK+bgNrvvGjnCuls6c=; b=TefbBEk4EZghulMx8mXcDwfZXtFOpksYYPBDGLJn56huBJSvfhd0jCSJqxgxwveFtKCh6m yVKt5PCGTJ0bIDbBpx6qXbjyYPHdiTptcOu4ebAnpWYjseq8pT2Z+iib10+Ik2tR+gPlyW P8R0rzfZHE3fghdVhBUaiEh43y7N7Hc= X-MC-Unique: 2rit4UI1PwK5BFegOvQAyg-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 03/12] coroutine: add check for SafeStack in sigaltstack Date: Wed, 24 Jun 2020 11:02:01 +0100 Message-Id: <20200624100210.59975-4-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 00:34:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , Daniele Buono Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Daniele Buono Current implementation of LLVM's SafeStack is not compatible with code that uses an alternate stack created with sigaltstack(). Since coroutine-sigaltstack relies on sigaltstack(), it is not compatible with SafeStack. The resulting binary is incorrect, with different coroutines sharing the same unsafe stack and producing undefined behavior at runtime. In the future LLVM may provide a SafeStack implementation compatible with sigaltstack(). In the meantime, if SafeStack is desired, the coroutine implementation from coroutine-ucontext should be used. As a safety check, add a control in coroutine-sigaltstack to throw a preprocessor #error if SafeStack is enabled and we are trying to use coroutine-sigaltstack to implement coroutines. Signed-off-by: Daniele Buono Message-id: 20200529205122.714-3-dbuono@linux.vnet.ibm.com Signed-off-by: Stefan Hajnoczi --- util/coroutine-sigaltstack.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/util/coroutine-sigaltstack.c b/util/coroutine-sigaltstack.c index f6fc49a0e5..aade82afb8 100644 --- a/util/coroutine-sigaltstack.c +++ b/util/coroutine-sigaltstack.c @@ -30,6 +30,10 @@ #include "qemu-common.h" #include "qemu/coroutine_int.h" =20 +#ifdef CONFIG_SAFESTACK +#error "SafeStack is not compatible with code run in alternate signal stac= ks" +#endif + typedef struct { Coroutine base; void *stack; --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993143; cv=none; d=zohomail.com; s=zohoarc; b=DhMNv4Y92MAqbn0hx32wMJt39eeEgEsugRSwXhy8Rm9Iw/psH4sv4k+5RydRBCUJP+TDRBl/SQQhSalgIvmvUJvZqyrbvw8Ii2amZmYSARqLoVOeNK4jOX9PzqV8txJ1g65Q1pZ/yxU5642yDpnxpm7nPfJxLT7O0kqK+mCkOtg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993143; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vInTGCffotXESgrOE+sQ/HjsitRb+VUYibAP46TUiF4=; b=XmCLZX+j+qZ8D/keXSzNHCskA8jBfWBexaFB2tbuOH0e02WW3+Y1af+KLYx+tq7XVSL6emcirC5Lpn83srQxyZRqa1zstbXn1bJd5U9JksHJVW/03oUdSFvUWgtqM3lnHVVMB7GPGd3PDuzrOYSV7IlZFC1A15gmS+cs9gmLmvI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159299314370046.00383620725279; Wed, 24 Jun 2020 03:05:43 -0700 (PDT) Received: from localhost ([::1]:48448 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2Hm-0006VS-Fj for importer@patchew.org; Wed, 24 Jun 2020 06:05:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43450) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2Es-0001YZ-VQ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:42 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:45177 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2Er-0003a4-1W for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:42 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-450-_TkRBPyZNpOpvnJchgsTNQ-1; Wed, 24 Jun 2020 06:02:38 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DB3D7107ACCD; Wed, 24 Jun 2020 10:02:36 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 91C385D9D3; Wed, 24 Jun 2020 10:02:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992960; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vInTGCffotXESgrOE+sQ/HjsitRb+VUYibAP46TUiF4=; b=IFrZbl1awfJ1qmAM4azsnxS/0+nFspT2BBezxKhuwZ2yM1X0fgeJwve8cf5uMzNxv4f4MF 7LmXLgiFEsy2fV8iX75uwRnojbLtpZi2J/YmTDPR8rwkef3KSaRwi+DGsUdhv1ZnHjXpMr S3Qej1/BwslLFHR/5GRvgw1H/iU6N7w= X-MC-Unique: _TkRBPyZNpOpvnJchgsTNQ-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 04/12] configure: add flags to support SafeStack Date: Wed, 24 Jun 2020 11:02:02 +0100 Message-Id: <20200624100210.59975-5-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=stefanha@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 03:27:53 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , Daniele Buono Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Daniele Buono This patch adds a flag to enable/disable the SafeStack instrumentation provided by LLVM. On enable, make sure that the compiler supports the flags, and that we are using the proper coroutine implementation (coroutine-ucontext). On disable, explicitly disable the option if it was enabled by default. While SafeStack is supported only on Linux, NetBSD, FreeBSD and macOS, we are not checking for the O.S. since this is already done by LLVM. Signed-off-by: Daniele Buono Message-id: 20200529205122.714-4-dbuono@linux.vnet.ibm.com Signed-off-by: Stefan Hajnoczi --- configure | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/configure b/configure index ba88fd1824..ae8737d5a2 100755 --- a/configure +++ b/configure @@ -307,6 +307,7 @@ audio_win_int=3D"" libs_qga=3D"" debug_info=3D"yes" stack_protector=3D"" +safe_stack=3D"" use_containers=3D"yes" gdb_bin=3D$(command -v "gdb-multiarch" || command -v "gdb") =20 @@ -1287,6 +1288,10 @@ for opt do ;; --disable-stack-protector) stack_protector=3D"no" ;; + --enable-safe-stack) safe_stack=3D"yes" + ;; + --disable-safe-stack) safe_stack=3D"no" + ;; --disable-curses) curses=3D"no" ;; --enable-curses) curses=3D"yes" @@ -1829,6 +1834,8 @@ disabled with --disable-FEATURE, default is enabled i= f available: debug-tcg TCG debugging (default is disabled) debug-info debugging information sparse sparse checker + safe-stack SafeStack Stack Smash Protection. Depends on + clang/llvm >=3D 3.7 and requires coroutine backend ucont= ext. =20 gnutls GNUTLS cryptography support nettle nettle cryptography support @@ -5573,6 +5580,67 @@ if test "$debug_stack_usage" =3D "yes"; then fi fi =20 +################################################## +# SafeStack + + +if test "$safe_stack" =3D "yes"; then +cat > $TMPC << EOF +int main(int argc, char *argv[]) +{ +#if ! __has_feature(safe_stack) +#error SafeStack Disabled +#endif + return 0; +} +EOF + flag=3D"-fsanitize=3Dsafe-stack" + # Check that safe-stack is supported and enabled. + if compile_prog "-Werror $flag" "$flag"; then + # Flag needed both at compilation and at linking + QEMU_CFLAGS=3D"$QEMU_CFLAGS $flag" + QEMU_LDFLAGS=3D"$QEMU_LDFLAGS $flag" + else + error_exit "SafeStack not supported by your compiler" + fi + if test "$coroutine" !=3D "ucontext"; then + error_exit "SafeStack is only supported by the coroutine backend ucont= ext" + fi +else +cat > $TMPC << EOF +int main(int argc, char *argv[]) +{ +#if defined(__has_feature) +#if __has_feature(safe_stack) +#error SafeStack Enabled +#endif +#endif + return 0; +} +EOF +if test "$safe_stack" =3D "no"; then + # Make sure that safe-stack is disabled + if ! compile_prog "-Werror" ""; then + # SafeStack was already enabled, try to explicitly remove the feature + flag=3D"-fno-sanitize=3Dsafe-stack" + if ! compile_prog "-Werror $flag" "$flag"; then + error_exit "Configure cannot disable SafeStack" + fi + QEMU_CFLAGS=3D"$QEMU_CFLAGS $flag" + QEMU_LDFLAGS=3D"$QEMU_LDFLAGS $flag" + fi +else # "$safe_stack" =3D "" + # Set safe_stack to yes or no based on pre-existing flags + if compile_prog "-Werror" ""; then + safe_stack=3D"no" + else + safe_stack=3D"yes" + if test "$coroutine" !=3D "ucontext"; then + error_exit "SafeStack is only supported by the coroutine backend uco= ntext" + fi + fi +fi +fi =20 ########################################## # check if we have open_by_handle_at @@ -6765,6 +6833,7 @@ echo "sparse enabled $sparse" echo "strip binaries $strip_opt" echo "profiler $profiler" echo "static build $static" +echo "safe stack $safe_stack" if test "$darwin" =3D "yes" ; then echo "Cocoa support $cocoa" fi @@ -8370,6 +8439,10 @@ if test "$ccache_cpp2" =3D "yes"; then echo "export CCACHE_CPP2=3Dy" >> $config_host_mak fi =20 +if test "$safe_stack" =3D "yes"; then + echo "CONFIG_SAFESTACK=3Dy" >> $config_host_mak +fi + # If we're using a separate build tree, set it up now. # DIRS are directories which we simply mkdir in the build tree; # LINKS are things to symlink back into the source tree --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993245; cv=none; d=zohomail.com; s=zohoarc; b=nx0pHyVe3FgLybKQpPn/uirecv3ZyFqNKczSgLPlMCbeewht0sEpalHkeOy9sHCmM6Nq4FhiRTsHkFomF+aj3/S1UEf/2ds8aOer/PJMXdSVJVh0LXDmlg3HQQnX7vcIIwCnBvKOV4vEmuQ5WdGtV05VRNMkjF1DU2JCeBC+GWY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993245; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dVpMD9A8TEZKeHH5TOKBr7wEiFBE4Ujs5DlSnNOvwCg=; b=eJs42L+7v0qUeLKyDr6BCW+YzVKDW32rPE7z6iMf9cRwblXBFatErccA3jXxNTeFdzO4az1G8ig6Fv9vCo7BvHyQ5G0cb+45E6nENhCGcewkKv9F4PzgeA4Ms7bkwmFTsML73PSvbJJzXvjMfTRu9XYYRaDnM6PRf85h696nYjU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993245377915.8729805050399; Wed, 24 Jun 2020 03:07:25 -0700 (PDT) Received: from localhost ([::1]:57520 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2JP-0002qd-V4 for importer@patchew.org; Wed, 24 Jun 2020 06:07:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43480) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2Eu-0001dE-NT for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:44 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:60258 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2Es-0003b1-Qc for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:44 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-458-f98Tg0ikMYWH5dLfYM-nIw-1; Wed, 24 Jun 2020 06:02:39 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B3C3B800D5C; Wed, 24 Jun 2020 10:02:38 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4AE3F12CB4; Wed, 24 Jun 2020 10:02:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992962; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dVpMD9A8TEZKeHH5TOKBr7wEiFBE4Ujs5DlSnNOvwCg=; b=WfpfalfEh3NMEdtHcfygsHK5YvPKFCecF/WkPCYz0eAFg0v8DTBQezgQ8RQolS3iR7/5My HnVFhB5dj0M96cKwnbBCZ3qK0ZhVWAR8swJad/NvZXt+0ysoU+2TQN4v4zIA1zZVO+Sc76 blrOpfNOtPBLkUSy86SaihE4hLJqyaE= X-MC-Unique: f98Tg0ikMYWH5dLfYM-nIw-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 05/12] check-block: enable iotests with SafeStack Date: Wed, 24 Jun 2020 11:02:03 +0100 Message-Id: <20200624100210.59975-6-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=stefanha@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 03:27:53 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , Daniele Buono Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Daniele Buono SafeStack is a stack protection technique implemented in llvm. It is enabled with a -fsanitize flag. iotests are currently disabled when any -fsanitize option is used, because such options tend to produce additional warnings and false positives. While common -fsanitize options are used to verify the code and not added in production, SafeStack's main use is in production environments to protect against stack smashing. Since SafeStack does not print any warning or false positive, enable iotests when SafeStack is the only -fsanitize option used. This is likely going to be a production binary and we want to make sure it works correctly. Signed-off-by: Daniele Buono Message-id: 20200529205122.714-5-dbuono@linux.vnet.ibm.com Signed-off-by: Stefan Hajnoczi --- tests/check-block.sh | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/tests/check-block.sh b/tests/check-block.sh index ad320c21ba..8e29c868e5 100755 --- a/tests/check-block.sh +++ b/tests/check-block.sh @@ -21,7 +21,17 @@ if grep -q "CONFIG_GPROF=3Dy" config-host.mak 2>/dev/nul= l ; then exit 0 fi =20 -if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then +# Disable tests with any sanitizer except for SafeStack +CFLAGS=3D$( grep "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ) +SANITIZE_FLAGS=3D"" +#Remove all occurrencies of -fsanitize=3Dsafe-stack +for i in ${CFLAGS}; do + if [ "${i}" !=3D "-fsanitize=3Dsafe-stack" ]; then + SANITIZE_FLAGS=3D"${SANITIZE_FLAGS} ${i}" + fi +done +if echo ${SANITIZE_FLAGS} | grep -q "\-fsanitize" 2>/dev/null; then + # Have a sanitize flag that is not allowed, stop echo "Sanitizers are enabled =3D=3D> Not running the qemu-iotests." exit 0 fi --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993100; cv=none; d=zohomail.com; s=zohoarc; b=Mr6s9lXx47NwWbMyFdOl3W8gysx3P49AuAYzb2nHWBeOsL8JIhopEQWzBXB6qW2Vsa36hKzYVMo3zuY4r6R+BNR64kHCUfUGdOSkrX0SSKv47bOTA008l0tRhXZut5QaLnDFtvBTqhhgpxtGbotIONVJneXyCxDOIhW3luiY6f4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993100; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fcfJpMktAhCTkb8ZUD6UgIjHN+llwqaMaeBYBwKZ4OQ=; b=fpkO6Kn/5tvZKD+7tRvhYvWnjxHXiTSoMcS9RyEYPTq6ZrOQmCbP9teW3oDYAfyN3isVkbTdAsIydC7DiwRiNk0MVOx/He2tHdV5YjK2ygeapFHiMyBSPff7vm0zKWraXv7FNLxXGTD7F/6YsSqTAM4vZliAKhw3QktB6DQeqpw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993100084518.7203293671315; Wed, 24 Jun 2020 03:05:00 -0700 (PDT) Received: from localhost ([::1]:45200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2H4-00059O-Rj for importer@patchew.org; Wed, 24 Jun 2020 06:04:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2F2-0001xU-RL for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:53 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:29401 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2F1-0003gK-2h for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:02:52 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-111-yBa_nef-PkiejBJWsfrykQ-1; Wed, 24 Jun 2020 06:02:48 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 54EDA10059B7; Wed, 24 Jun 2020 10:02:47 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 886FB1010404; Wed, 24 Jun 2020 10:02:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fcfJpMktAhCTkb8ZUD6UgIjHN+llwqaMaeBYBwKZ4OQ=; b=iRfXy+5DtR8bZoi2lTrd3c/yUzrfqVeSWtAkbUxOfSi3q4dbBiqq2MUzt55Lfh3xmulizo H+BuZBM33oUm/vr4x44md5ERtNzA/P1wyw88L4wQn4PA4GX8qOzyYvQTvx6SgC7m6Tbciv Cwym3y4uwA49vBf2hUYkfhvl6Xv/Umg= X-MC-Unique: yBa_nef-PkiejBJWsfrykQ-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 06/12] block/nvme: poll queues without q->lock Date: Wed, 24 Jun 2020 11:02:04 +0100 Message-Id: <20200624100210.59975-7-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=stefanha@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 02:33:25 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Sergio Lopez , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" A lot of CPU time is spent simply locking/unlocking q->lock during polling. Check for completion outside the lock to make q->lock disappear from the profile. Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Message-id: 20200617132201.1832152-2-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/block/nvme.c b/block/nvme.c index eb2f54dd9d..e4375ec245 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -512,6 +512,18 @@ static bool nvme_poll_queues(BDRVNVMeState *s) =20 for (i =3D 0; i < s->nr_queues; i++) { NVMeQueuePair *q =3D s->queues[i]; + const size_t cqe_offset =3D q->cq.head * NVME_CQ_ENTRY_BYTES; + NvmeCqe *cqe =3D (NvmeCqe *)&q->cq.queue[cqe_offset]; + + /* + * Do an early check for completions. q->lock isn't needed because + * nvme_process_completion() only runs in the event loop thread and + * cannot race with itself. + */ + if ((le16_to_cpu(cqe->status) & 0x1) =3D=3D q->cq_phase) { + continue; + } + qemu_mutex_lock(&q->lock); while (nvme_process_completion(s, q)) { /* Keep polling */ --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993164; cv=none; d=zohomail.com; s=zohoarc; b=gXxABHEKT0uIVzTn7lZ7/5aJvOuPseLn0xjV0YaqYQFdV2GpLtudNfRdQULQ/Aq4bqQ3mgyv0XnHOig9ZMFUkbjxHlJ4q/bbeK6aGUIWmWchFTlqia/N+2yHJy6xXmMAwwtvdz/gzyOcmemEP7blv8xbmHbASQr7axND1z6v8CA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993164; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=px6UzXrF2X1M4UOHq2k4dKpAGlLnAFyJPkYcTsXrT7M=; b=lJ798MBCRSUGgk6CpZtL2P5rZ/pa/poig/sz0aOF07HdxnxuV2Cp+omU0YfwNCa4xSlwia6GC5SHsaLkZDOuaqzHlpKP3HYqKJNO8kPqlWPPL8rx60o7qGT0VgF6OBXHyrtIplxuAy2KQthIs4iqBOie7LqZiCFVaF77EizoTw4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993164913608.8075860871462; Wed, 24 Jun 2020 03:06:04 -0700 (PDT) Received: from localhost ([::1]:50636 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2I7-0007Wm-J6 for importer@patchew.org; Wed, 24 Jun 2020 06:06:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43606) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2FB-0002Fu-8w for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:01 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:53166 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2F9-0003jy-MH for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:00 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-86-ADP_mQo5O26SzIT4yZH5Ng-1; Wed, 24 Jun 2020 06:02:56 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 82A8B1B18BCA; Wed, 24 Jun 2020 10:02:55 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id E987D79315; Wed, 24 Jun 2020 10:02:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=px6UzXrF2X1M4UOHq2k4dKpAGlLnAFyJPkYcTsXrT7M=; b=bgoHZCwEsYESwA8RXz/cm6vznIIQNl5fJhVrDQQnSfW6fXniKnPkhpFt/JDdcI/oOJt+k0 zqqVeDUH9gaLceUfOFR+Ugs6QMyUzhWmAdxp9N/Q9vmG6lcua8XnKj8sz8i2cAsBHFLIrj a1SMGwHwN6qMk6UJyhtzysb8Vh42ZJk= X-MC-Unique: ADP_mQo5O26SzIT4yZH5Ng-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 07/12] block/nvme: drop tautologous assertion Date: Wed, 24 Jun 2020 11:02:05 +0100 Message-Id: <20200624100210.59975-8-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 00:34:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Sergio Lopez , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" nvme_process_completion() explicitly checks cid so the assertion that follows is always true: if (cid =3D=3D 0 || cid > NVME_QUEUE_SIZE) { ... continue; } assert(cid <=3D NVME_QUEUE_SIZE); Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-id: 20200617132201.1832152-3-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/nvme.c b/block/nvme.c index e4375ec245..d567ece3f4 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -336,7 +336,6 @@ static bool nvme_process_completion(BDRVNVMeState *s, N= VMeQueuePair *q) cid); continue; } - assert(cid <=3D NVME_QUEUE_SIZE); trace_nvme_complete_command(s, q->index, cid); preq =3D &q->reqs[cid - 1]; req =3D *preq; --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993240; cv=none; d=zohomail.com; s=zohoarc; b=BdfZHhexuFFZDdqc8z9ZhJLIga9D09PkHLFbPg7T5vat/1qZpTf14N3IJj0f3IEjugLA5JWVydtUm9pNBszdNcFvLhkZaWZFQgD8ArMaGfJ2G1dlgxtT/Gx9pCOEp/TRlLUDNbkn/LNLB1Iu1KjS1XSIXdJsFERVZ9qAKc/wKM4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993240; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ygqOlufNJDpj/1ysa1bHRtC92SpCW8Px0NcFXYI/JZk=; b=UgT8TL1dH7MDLi3eBrptb8PQX4fL24K/8RBX/e7U5XPFTjyEQ9J820cVS2P+LAYZ4pHDTl+sSripS364tVSJbF6ZXIXwAaf1PUvGH15W2xhFmR2Mnrso3klvBBEupFbz9JTIWxt9GsQ6hmSM+DvvQJCKZiR563KwpZQPwOm0a3g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993240181981.035056102988; Wed, 24 Jun 2020 03:07:20 -0700 (PDT) Received: from localhost ([::1]:57048 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2JK-0002fD-Uv for importer@patchew.org; Wed, 24 Jun 2020 06:07:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43630) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2FF-0002P7-IJ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:05 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:60474 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2FD-0003lH-Qk for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-236-42DMtY2XMSuKUgkFReMa1A-1; Wed, 24 Jun 2020 06:02:58 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8E521EC1B9; Wed, 24 Jun 2020 10:02:57 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 254B271694; Wed, 24 Jun 2020 10:02:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ygqOlufNJDpj/1ysa1bHRtC92SpCW8Px0NcFXYI/JZk=; b=NWA+ZuMQ4LEMDbCvUa+YAW/+nsdBGuhPdqW2m5XtnLsp3XuwYntl5sTh1pgw9qD+cdVZb6 xkAB5Rmj1kL8FW1BwJueiGPpGqDRhAZZ8BxnZLLYDWpmcmhF+pPFW8Ld3eX08g60h4iK7R vBqGHvPzbskLnkmJYti93wOG+hQAt44= X-MC-Unique: 42DMtY2XMSuKUgkFReMa1A-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 08/12] block/nvme: don't access CQE after moving cq.head Date: Wed, 24 Jun 2020 11:02:06 +0100 Message-Id: <20200624100210.59975-9-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=stefanha@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 03:27:53 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Sergio Lopez , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Do not access a CQE after incrementing q->cq.head and releasing q->lock. It is unlikely that this causes problems in practice but it's a latent bug. The reason why it should be safe at the moment is that completion processing is not re-entrant and the CQ doorbell isn't written until the end of nvme_process_completion(). Make this change now because QEMU expects completion processing to be re-entrant and later patches will do that. Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-id: 20200617132201.1832152-4-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/block/nvme.c b/block/nvme.c index d567ece3f4..344893811a 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -321,11 +321,14 @@ static bool nvme_process_completion(BDRVNVMeState *s,= NVMeQueuePair *q) q->busy =3D true; assert(q->inflight >=3D 0); while (q->inflight) { + int ret; int16_t cid; + c =3D (NvmeCqe *)&q->cq.queue[q->cq.head * NVME_CQ_ENTRY_BYTES]; if ((le16_to_cpu(c->status) & 0x1) =3D=3D q->cq_phase) { break; } + ret =3D nvme_translate_error(c); q->cq.head =3D (q->cq.head + 1) % NVME_QUEUE_SIZE; if (!q->cq.head) { q->cq_phase =3D !q->cq_phase; @@ -344,7 +347,7 @@ static bool nvme_process_completion(BDRVNVMeState *s, N= VMeQueuePair *q) preq->busy =3D false; preq->cb =3D preq->opaque =3D NULL; qemu_mutex_unlock(&q->lock); - req.cb(req.opaque, nvme_translate_error(c)); + req.cb(req.opaque, ret); qemu_mutex_lock(&q->lock); q->inflight--; progress =3D true; --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993350; cv=none; d=zohomail.com; s=zohoarc; b=We+riK2OQ8Nzku9rK2H8j0lnAcU11gcU42+6H/pE74K++mnwgFrFMi2fShj6+7au6xl2UZjG2zOpeBfkLxp3v+0MYuF8uKAZlM9ekghIQljmZi4lJX32liONpbU2S7+FLmEpWS0k5zdzbhM9kkhdMJQP5tf2369d9CpBdOyydKw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993350; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=U6lkZzAaY9OHhFTihf0/hM1V2/Sn3l54fwc5yOhx7GE=; b=InEtB9ikrx8KKbtcdFSh7sLuH6I8809JfqxdfCXtHU4BXyfDMgt8T+UkZXv37cTCnqNlSuOZBQ6dP3Ym4vZ3tz+fDhR7wg1sWQzFI37txwzl6DTobJGIA6EhF0dkZ6dBo5Hx+N7eJi8t+joHnwAbv9IBjiVLO4R1+48+MW+wzwo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993350098619.1964764863884; Wed, 24 Jun 2020 03:09:10 -0700 (PDT) Received: from localhost ([::1]:37696 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2L6-0006R9-Qk for importer@patchew.org; Wed, 24 Jun 2020 06:09:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43656) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2FI-0002VB-5u for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:08 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:38986 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2FF-0003mV-RN for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-290-uR9FKaBMPEeJs6U-58tM2Q-1; Wed, 24 Jun 2020 06:03:00 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CD82610059AC; Wed, 24 Jun 2020 10:02:59 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id D950079317; Wed, 24 Jun 2020 10:02:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U6lkZzAaY9OHhFTihf0/hM1V2/Sn3l54fwc5yOhx7GE=; b=jGvvCyxXdyoA11WAsDS0uJDtg71hklyfedYiBf6Ni26BKQJORFoqh6JLRGYxZy3gJAI2Dp rcx3DlG+LUhBqKBFUhUaVO1sud59xo8/G3mmh3WYgu78ped+96luqLQV03U+CMK08UYKaW MYK/Ew9jsrrC8OHr5VFL/Iu0tD1cxcc= X-MC-Unique: uR9FKaBMPEeJs6U-58tM2Q-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 09/12] block/nvme: switch to a NVMeRequest freelist Date: Wed, 24 Jun 2020 11:02:07 +0100 Message-Id: <20200624100210.59975-10-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=stefanha@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 03:27:53 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Sergio Lopez , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" There are three issues with the current NVMeRequest->busy field: 1. The busy field is accidentally accessed outside q->lock when request submission fails. 2. Waiters on free_req_queue are not woken when a request is returned early due to submission failure. 2. Finding a free request involves scanning all requests. This makes request submission O(n^2). Switch to an O(1) freelist that is always accessed under the lock. Also differentiate between NVME_QUEUE_SIZE, the actual SQ/CQ size, and NVME_NUM_REQS, the number of usable requests. This makes the code simpler than using NVME_QUEUE_SIZE everywhere and having to keep in mind that one slot is reserved. Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Message-id: 20200617132201.1832152-5-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 81 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 54 insertions(+), 27 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 344893811a..8e60882af3 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -33,6 +33,12 @@ #define NVME_QUEUE_SIZE 128 #define NVME_BAR_SIZE 8192 =20 +/* + * We have to leave one slot empty as that is the full queue case where + * head =3D=3D tail + 1. + */ +#define NVME_NUM_REQS (NVME_QUEUE_SIZE - 1) + typedef struct { int32_t head, tail; uint8_t *queue; @@ -47,7 +53,7 @@ typedef struct { int cid; void *prp_list_page; uint64_t prp_list_iova; - bool busy; + int free_req_next; /* q->reqs[] index of next free req */ } NVMeRequest; =20 typedef struct { @@ -61,7 +67,8 @@ typedef struct { /* Fields protected by @lock */ NVMeQueue sq, cq; int cq_phase; - NVMeRequest reqs[NVME_QUEUE_SIZE]; + int free_req_head; + NVMeRequest reqs[NVME_NUM_REQS]; bool busy; int need_kick; int inflight; @@ -200,19 +207,23 @@ static NVMeQueuePair *nvme_create_queue_pair(BlockDri= verState *bs, qemu_mutex_init(&q->lock); q->index =3D idx; qemu_co_queue_init(&q->free_req_queue); - q->prp_list_pages =3D qemu_blockalign0(bs, s->page_size * NVME_QUEUE_S= IZE); + q->prp_list_pages =3D qemu_blockalign0(bs, s->page_size * NVME_NUM_REQ= S); r =3D qemu_vfio_dma_map(s->vfio, q->prp_list_pages, - s->page_size * NVME_QUEUE_SIZE, + s->page_size * NVME_NUM_REQS, false, &prp_list_iova); if (r) { goto fail; } - for (i =3D 0; i < NVME_QUEUE_SIZE; i++) { + q->free_req_head =3D -1; + for (i =3D 0; i < NVME_NUM_REQS; i++) { NVMeRequest *req =3D &q->reqs[i]; req->cid =3D i + 1; + req->free_req_next =3D q->free_req_head; + q->free_req_head =3D i; req->prp_list_page =3D q->prp_list_pages + i * s->page_size; req->prp_list_iova =3D prp_list_iova + i * s->page_size; } + nvme_init_queue(bs, &q->sq, size, NVME_SQ_ENTRY_BYTES, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -254,13 +265,11 @@ static void nvme_kick(BDRVNVMeState *s, NVMeQueuePair= *q) */ static NVMeRequest *nvme_get_free_req(NVMeQueuePair *q) { - int i; - NVMeRequest *req =3D NULL; + NVMeRequest *req; =20 qemu_mutex_lock(&q->lock); - while (q->inflight + q->need_kick > NVME_QUEUE_SIZE - 2) { - /* We have to leave one slot empty as that is the full queue case = (head - * =3D=3D tail + 1). */ + + while (q->free_req_head =3D=3D -1) { if (qemu_in_coroutine()) { trace_nvme_free_req_queue_wait(q); qemu_co_queue_wait(&q->free_req_queue, &q->lock); @@ -269,20 +278,41 @@ static NVMeRequest *nvme_get_free_req(NVMeQueuePair *= q) return NULL; } } - for (i =3D 0; i < NVME_QUEUE_SIZE; i++) { - if (!q->reqs[i].busy) { - q->reqs[i].busy =3D true; - req =3D &q->reqs[i]; - break; - } - } - /* We have checked inflight and need_kick while holding q->lock, so one - * free req must be available. */ - assert(req); + + req =3D &q->reqs[q->free_req_head]; + q->free_req_head =3D req->free_req_next; + req->free_req_next =3D -1; + qemu_mutex_unlock(&q->lock); return req; } =20 +/* With q->lock */ +static void nvme_put_free_req_locked(NVMeQueuePair *q, NVMeRequest *req) +{ + req->free_req_next =3D q->free_req_head; + q->free_req_head =3D req - q->reqs; +} + +/* With q->lock */ +static void nvme_wake_free_req_locked(BDRVNVMeState *s, NVMeQueuePair *q) +{ + if (!qemu_co_queue_empty(&q->free_req_queue)) { + replay_bh_schedule_oneshot_event(s->aio_context, + nvme_free_req_queue_cb, q); + } +} + +/* Insert a request in the freelist and wake waiters */ +static void nvme_put_free_req_and_wake(BDRVNVMeState *s, NVMeQueuePair *q, + NVMeRequest *req) +{ + qemu_mutex_lock(&q->lock); + nvme_put_free_req_locked(q, req); + nvme_wake_free_req_locked(s, q); + qemu_mutex_unlock(&q->lock); +} + static inline int nvme_translate_error(const NvmeCqe *c) { uint16_t status =3D (le16_to_cpu(c->status) >> 1) & 0xFF; @@ -344,7 +374,7 @@ static bool nvme_process_completion(BDRVNVMeState *s, N= VMeQueuePair *q) req =3D *preq; assert(req.cid =3D=3D cid); assert(req.cb); - preq->busy =3D false; + nvme_put_free_req_locked(q, preq); preq->cb =3D preq->opaque =3D NULL; qemu_mutex_unlock(&q->lock); req.cb(req.opaque, ret); @@ -356,10 +386,7 @@ static bool nvme_process_completion(BDRVNVMeState *s, = NVMeQueuePair *q) /* Notify the device so it can post more completions. */ smp_mb_release(); *q->cq.doorbell =3D cpu_to_le32(q->cq.head); - if (!qemu_co_queue_empty(&q->free_req_queue)) { - replay_bh_schedule_oneshot_event(s->aio_context, - nvme_free_req_queue_cb, q); - } + nvme_wake_free_req_locked(s, q); } q->busy =3D false; return progress; @@ -1001,7 +1028,7 @@ static coroutine_fn int nvme_co_prw_aligned(BlockDriv= erState *bs, r =3D nvme_cmd_map_qiov(bs, &cmd, req, qiov); qemu_co_mutex_unlock(&s->dma_map_lock); if (r) { - req->busy =3D false; + nvme_put_free_req_and_wake(s, ioq, req); return r; } nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); @@ -1218,7 +1245,7 @@ static int coroutine_fn nvme_co_pdiscard(BlockDriverS= tate *bs, qemu_co_mutex_unlock(&s->dma_map_lock); =20 if (ret) { - req->busy =3D false; + nvme_put_free_req_and_wake(s, ioq, req); goto out; } =20 --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993205; cv=none; d=zohomail.com; s=zohoarc; b=Yhh6REFnUKbmyAClb1YUjxo+D6yHfP/NKTNuuF2mRFZkkTXnLB36VvFxN3T6nsaJsNO/O4f6OBDEicJe2h6NiN2MQLso49Uv1Km7wZZYqEi/rUTYJSu2+kN7vQM3zQMxi/jIvmIPVjD2Ghx5TntsXZRKBeOBIPDYTTv76tfyA2w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993205; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aK8ekwRHGaQX1JqA3rLumYthk8cTBv5YEUllToKlji4=; b=FDMbZtIgS+JkwbPQyiu2ri6lumaEIT1/FjF8MkkoMuASvyDgn2ZHwJoR1Xqw85TF8eQGUitl3B9vuC6wT+97M4FZEZiI+UjWHjRhzFltFnD80gqUd7DSzqo+Gkco+eGki8JkwWVptgV8l72LjSD7zAD/1lbv+wbK+jPdKO/pyoc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993205463228.13132608833348; Wed, 24 Jun 2020 03:06:45 -0700 (PDT) Received: from localhost ([::1]:54546 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2Im-0001Sm-3i for importer@patchew.org; Wed, 24 Jun 2020 06:06:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43766) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2FO-0002la-OE for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:14 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:42776 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2FL-0003po-Jb for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:14 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-243-EOvOeq4UPm-_HPAuF0qMqg-1; Wed, 24 Jun 2020 06:03:08 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 98445107ACCA; Wed, 24 Jun 2020 10:03:07 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6DBCF61169; Wed, 24 Jun 2020 10:03:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592992990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aK8ekwRHGaQX1JqA3rLumYthk8cTBv5YEUllToKlji4=; b=Wwkh0xRxKuCdIk2VHk/sZnnB7uJmwVOkuw4OF3Bj/iFqplL7igEJOCiLBoFVMs7Ut5Wsi8 L94KgQEvS6XIP9kZRafCkoll3qEeFSaDZWGo4vGBrNhb1BYQ6WYI1t984keofrdBAK2FBl 4wKchGp3lsRItlClyHDEuzMwT/ODHyE= X-MC-Unique: EOvOeq4UPm-_HPAuF0qMqg-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 10/12] block/nvme: clarify that free_req_queue is protected by q->lock Date: Wed, 24 Jun 2020 11:02:08 +0100 Message-Id: <20200624100210.59975-11-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=stefanha@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 02:33:25 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Sergio Lopez , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Existing users access free_req_queue under q->lock. Document this. Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-id: 20200617132201.1832152-6-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/nvme.c b/block/nvme.c index 8e60882af3..426c77e5ab 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -57,7 +57,6 @@ typedef struct { } NVMeRequest; =20 typedef struct { - CoQueue free_req_queue; QemuMutex lock; =20 /* Fields protected by BQL */ @@ -65,6 +64,7 @@ typedef struct { uint8_t *prp_list_pages; =20 /* Fields protected by @lock */ + CoQueue free_req_queue; NVMeQueue sq, cq; int cq_phase; int free_req_head; --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993320; cv=none; d=zohomail.com; s=zohoarc; b=bFAwzO5u7GifYr9hWoS5BykNivPLANbb3CWsZ+PZKIZGYElBt0L/tUJdbl4IsbQT3bn6dnplrRI6mmaFqyGlwNx0MdTbQR2wegpzy+WLs91LujnDloGia/dgAAw2r4u0kMYKZvj9i6QN2j5scu1rQAVYHLsCasXb+giV9C8ym40= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993320; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5G9dX+RCbbwcZvgqUY40WakqF4ueHWtoXpJYfF15QTM=; b=DU7PRTQ16d5EX5M3WNxISTiTvIpnHXTg0rKNXoBB3YNObP+adV+F3ukI8uSFBLo86kzRZvsex/JwE+une++zzocq4xJHoGxTRbgmYnQkoxVE8J7LCSk9I0+wuYwWbLDPAutqlh3Bp5Qavncy10Bx5i+68RWJ0Pnx3ip3Hs4+1fk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993320347596.9791809723298; Wed, 24 Jun 2020 03:08:40 -0700 (PDT) Received: from localhost ([::1]:34798 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2Kc-0005GL-Un for importer@patchew.org; Wed, 24 Jun 2020 06:08:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43816) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2FX-00035n-Ie for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:23 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:35656 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2FV-0003uu-CZ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:23 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-309--yhA1mF4MxGTZKNr4RonYg-1; Wed, 24 Jun 2020 06:03:17 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C7ECC108BD09; Wed, 24 Jun 2020 10:03:15 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 38EA2100238C; Wed, 24 Jun 2020 10:03:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592993000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5G9dX+RCbbwcZvgqUY40WakqF4ueHWtoXpJYfF15QTM=; b=HkMz1Oiabop8icJ6aJNNT1gB+gLNyvPI/ZlIFEkIibmDJbrUWb/oE8YoZheSrl48+9Nn3/ 9uB6HJ8XE76Rx5C8IsTPWUNURCHJnWnG3Xuig/LwbxkhmxeE2LokGWU/AisTyzxA8dwGw5 6AJ8wxjgMXsH1gKUX1AgH4/lLvFp9Ic= X-MC-Unique: -yhA1mF4MxGTZKNr4RonYg-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 11/12] block/nvme: keep BDRVNVMeState pointer in NVMeQueuePair Date: Wed, 24 Jun 2020 11:02:09 +0100 Message-Id: <20200624100210.59975-12-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=stefanha@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 02:33:25 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Sergio Lopez , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Passing around both BDRVNVMeState and NVMeQueuePair is unwieldy. Reduce the number of function arguments by keeping the BDRVNVMeState pointer in NVMeQueuePair. This will come in handly when a BH is introduced in a later patch and only one argument can be passed to it. Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-id: 20200617132201.1832152-7-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 70 ++++++++++++++++++++++++++++------------------------ 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 426c77e5ab..8dc68d3daa 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -39,6 +39,8 @@ */ #define NVME_NUM_REQS (NVME_QUEUE_SIZE - 1) =20 +typedef struct BDRVNVMeState BDRVNVMeState; + typedef struct { int32_t head, tail; uint8_t *queue; @@ -59,8 +61,11 @@ typedef struct { typedef struct { QemuMutex lock; =20 + /* Read from I/O code path, initialized under BQL */ + BDRVNVMeState *s; + int index; + /* Fields protected by BQL */ - int index; uint8_t *prp_list_pages; =20 /* Fields protected by @lock */ @@ -96,7 +101,7 @@ typedef volatile struct { =20 QEMU_BUILD_BUG_ON(offsetof(NVMeRegs, doorbells) !=3D 0x1000); =20 -typedef struct { +struct BDRVNVMeState { AioContext *aio_context; QEMUVFIOState *vfio; NVMeRegs *regs; @@ -130,7 +135,7 @@ typedef struct { =20 /* PCI address (required for nvme_refresh_filename()) */ char *device; -} BDRVNVMeState; +}; =20 #define NVME_BLOCK_OPT_DEVICE "device" #define NVME_BLOCK_OPT_NAMESPACE "namespace" @@ -174,7 +179,7 @@ static void nvme_init_queue(BlockDriverState *bs, NVMeQ= ueue *q, } } =20 -static void nvme_free_queue_pair(BlockDriverState *bs, NVMeQueuePair *q) +static void nvme_free_queue_pair(NVMeQueuePair *q) { qemu_vfree(q->prp_list_pages); qemu_vfree(q->sq.queue); @@ -205,6 +210,7 @@ static NVMeQueuePair *nvme_create_queue_pair(BlockDrive= rState *bs, uint64_t prp_list_iova; =20 qemu_mutex_init(&q->lock); + q->s =3D s; q->index =3D idx; qemu_co_queue_init(&q->free_req_queue); q->prp_list_pages =3D qemu_blockalign0(bs, s->page_size * NVME_NUM_REQ= S); @@ -240,13 +246,15 @@ static NVMeQueuePair *nvme_create_queue_pair(BlockDri= verState *bs, =20 return q; fail: - nvme_free_queue_pair(bs, q); + nvme_free_queue_pair(q); return NULL; } =20 /* With q->lock */ -static void nvme_kick(BDRVNVMeState *s, NVMeQueuePair *q) +static void nvme_kick(NVMeQueuePair *q) { + BDRVNVMeState *s =3D q->s; + if (s->plugged || !q->need_kick) { return; } @@ -295,21 +303,20 @@ static void nvme_put_free_req_locked(NVMeQueuePair *q= , NVMeRequest *req) } =20 /* With q->lock */ -static void nvme_wake_free_req_locked(BDRVNVMeState *s, NVMeQueuePair *q) +static void nvme_wake_free_req_locked(NVMeQueuePair *q) { if (!qemu_co_queue_empty(&q->free_req_queue)) { - replay_bh_schedule_oneshot_event(s->aio_context, + replay_bh_schedule_oneshot_event(q->s->aio_context, nvme_free_req_queue_cb, q); } } =20 /* Insert a request in the freelist and wake waiters */ -static void nvme_put_free_req_and_wake(BDRVNVMeState *s, NVMeQueuePair *q, - NVMeRequest *req) +static void nvme_put_free_req_and_wake(NVMeQueuePair *q, NVMeRequest *req) { qemu_mutex_lock(&q->lock); nvme_put_free_req_locked(q, req); - nvme_wake_free_req_locked(s, q); + nvme_wake_free_req_locked(q); qemu_mutex_unlock(&q->lock); } =20 @@ -336,8 +343,9 @@ static inline int nvme_translate_error(const NvmeCqe *c) } =20 /* With q->lock */ -static bool nvme_process_completion(BDRVNVMeState *s, NVMeQueuePair *q) +static bool nvme_process_completion(NVMeQueuePair *q) { + BDRVNVMeState *s =3D q->s; bool progress =3D false; NVMeRequest *preq; NVMeRequest req; @@ -386,7 +394,7 @@ static bool nvme_process_completion(BDRVNVMeState *s, N= VMeQueuePair *q) /* Notify the device so it can post more completions. */ smp_mb_release(); *q->cq.doorbell =3D cpu_to_le32(q->cq.head); - nvme_wake_free_req_locked(s, q); + nvme_wake_free_req_locked(q); } q->busy =3D false; return progress; @@ -403,8 +411,7 @@ static void nvme_trace_command(const NvmeCmd *cmd) } } =20 -static void nvme_submit_command(BDRVNVMeState *s, NVMeQueuePair *q, - NVMeRequest *req, +static void nvme_submit_command(NVMeQueuePair *q, NVMeRequest *req, NvmeCmd *cmd, BlockCompletionFunc cb, void *opaque) { @@ -413,15 +420,15 @@ static void nvme_submit_command(BDRVNVMeState *s, NVM= eQueuePair *q, req->opaque =3D opaque; cmd->cid =3D cpu_to_le32(req->cid); =20 - trace_nvme_submit_command(s, q->index, req->cid); + trace_nvme_submit_command(q->s, q->index, req->cid); nvme_trace_command(cmd); qemu_mutex_lock(&q->lock); memcpy((uint8_t *)q->sq.queue + q->sq.tail * NVME_SQ_ENTRY_BYTES, cmd, sizeof(*cmd)); q->sq.tail =3D (q->sq.tail + 1) % NVME_QUEUE_SIZE; q->need_kick++; - nvme_kick(s, q); - nvme_process_completion(s, q); + nvme_kick(q); + nvme_process_completion(q); qemu_mutex_unlock(&q->lock); } =20 @@ -436,13 +443,12 @@ static int nvme_cmd_sync(BlockDriverState *bs, NVMeQu= euePair *q, NvmeCmd *cmd) { NVMeRequest *req; - BDRVNVMeState *s =3D bs->opaque; int ret =3D -EINPROGRESS; req =3D nvme_get_free_req(q); if (!req) { return -EBUSY; } - nvme_submit_command(s, q, req, cmd, nvme_cmd_sync_cb, &ret); + nvme_submit_command(q, req, cmd, nvme_cmd_sync_cb, &ret); =20 BDRV_POLL_WHILE(bs, ret =3D=3D -EINPROGRESS); return ret; @@ -554,7 +560,7 @@ static bool nvme_poll_queues(BDRVNVMeState *s) } =20 qemu_mutex_lock(&q->lock); - while (nvme_process_completion(s, q)) { + while (nvme_process_completion(q)) { /* Keep polling */ progress =3D true; } @@ -592,7 +598,7 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Err= or **errp) }; if (nvme_cmd_sync(bs, s->queues[0], &cmd)) { error_setg(errp, "Failed to create io queue [%d]", n); - nvme_free_queue_pair(bs, q); + nvme_free_queue_pair(q); return false; } cmd =3D (NvmeCmd) { @@ -603,7 +609,7 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Err= or **errp) }; if (nvme_cmd_sync(bs, s->queues[0], &cmd)) { error_setg(errp, "Failed to create io queue [%d]", n); - nvme_free_queue_pair(bs, q); + nvme_free_queue_pair(q); return false; } s->queues =3D g_renew(NVMeQueuePair *, s->queues, n + 1); @@ -798,7 +804,7 @@ static void nvme_close(BlockDriverState *bs) BDRVNVMeState *s =3D bs->opaque; =20 for (i =3D 0; i < s->nr_queues; ++i) { - nvme_free_queue_pair(bs, s->queues[i]); + nvme_free_queue_pair(s->queues[i]); } g_free(s->queues); aio_set_event_notifier(bdrv_get_aio_context(bs), &s->irq_notifier, @@ -1028,10 +1034,10 @@ static coroutine_fn int nvme_co_prw_aligned(BlockDr= iverState *bs, r =3D nvme_cmd_map_qiov(bs, &cmd, req, qiov); qemu_co_mutex_unlock(&s->dma_map_lock); if (r) { - nvme_put_free_req_and_wake(s, ioq, req); + nvme_put_free_req_and_wake(ioq, req); return r; } - nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + nvme_submit_command(ioq, req, &cmd, nvme_rw_cb, &data); =20 data.co =3D qemu_coroutine_self(); while (data.ret =3D=3D -EINPROGRESS) { @@ -1131,7 +1137,7 @@ static coroutine_fn int nvme_co_flush(BlockDriverStat= e *bs) assert(s->nr_queues > 1); req =3D nvme_get_free_req(ioq); assert(req); - nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + nvme_submit_command(ioq, req, &cmd, nvme_rw_cb, &data); =20 data.co =3D qemu_coroutine_self(); if (data.ret =3D=3D -EINPROGRESS) { @@ -1184,7 +1190,7 @@ static coroutine_fn int nvme_co_pwrite_zeroes(BlockDr= iverState *bs, req =3D nvme_get_free_req(ioq); assert(req); =20 - nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + nvme_submit_command(ioq, req, &cmd, nvme_rw_cb, &data); =20 data.co =3D qemu_coroutine_self(); while (data.ret =3D=3D -EINPROGRESS) { @@ -1245,13 +1251,13 @@ static int coroutine_fn nvme_co_pdiscard(BlockDrive= rState *bs, qemu_co_mutex_unlock(&s->dma_map_lock); =20 if (ret) { - nvme_put_free_req_and_wake(s, ioq, req); + nvme_put_free_req_and_wake(ioq, req); goto out; } =20 trace_nvme_dsm(s, offset, bytes); =20 - nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + nvme_submit_command(ioq, req, &cmd, nvme_rw_cb, &data); =20 data.co =3D qemu_coroutine_self(); while (data.ret =3D=3D -EINPROGRESS) { @@ -1333,8 +1339,8 @@ static void nvme_aio_unplug(BlockDriverState *bs) for (i =3D 1; i < s->nr_queues; i++) { NVMeQueuePair *q =3D s->queues[i]; qemu_mutex_lock(&q->lock); - nvme_kick(s, q); - nvme_process_completion(s, q); + nvme_kick(q); + nvme_process_completion(q); qemu_mutex_unlock(&q->lock); } } --=20 2.26.2 From nobody Sun May 5 05:17:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592993270; cv=none; d=zohomail.com; s=zohoarc; b=GAHNmXOeU/FufZSOlJKl+ENTsojVDjfH0NNF0CoZvlUfis5j8hEA1Ayn4MM/EbjpHkQnFXX3/FbChOoraJOLiD22+TBYpf7i/Pjddny0X9Avs2wlRe5jqQLL+1EzJC6zn6cmamf+Ls2WeaawjHzjToE1IxL44jaq5ivvPdG45Pc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592993270; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AtKSPkJoUmb+d6RZ+aYTTbLWSYSBVmzUbmT1yW2cRww=; b=QCWZwSR4cjmQHJ4D8rSXYaHLD9KMGu8vhoF7HJe/iM/lv1mkCofHqMbWoGf+yWcC/t+ofGqjuEMiFsva++efbzb3e+3heayLpYiNQGIqf6cwd4a1qNYz4bOoLVoN/BpDj79WnKYivKUP62nbXiNwcPAkaem6stDz2/IK3TChRuo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592993270103279.07751826013657; Wed, 24 Jun 2020 03:07:50 -0700 (PDT) Received: from localhost ([::1]:59176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo2Jo-0003Uw-Lv for importer@patchew.org; Wed, 24 Jun 2020 06:07:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo2Fi-0003W7-SA for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:34 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:33590 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jo2Fd-000428-D0 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 06:03:34 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-115-gtZkyYb_Ppeyule2QlRYNw-1; Wed, 24 Jun 2020 06:03:24 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 61B071B18BC1; Wed, 24 Jun 2020 10:03:23 +0000 (UTC) Received: from localhost (ovpn-114-150.ams2.redhat.com [10.36.114.150]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3570A71660; Wed, 24 Jun 2020 10:03:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592993008; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AtKSPkJoUmb+d6RZ+aYTTbLWSYSBVmzUbmT1yW2cRww=; b=dCC6cJZinYd29JjIIHajMMkpAN+fiGOh5d0GdASKbbNKIeXNNxbPBaLwTGJ6HqnjQq6qPY bnaTBo54Yrx1uRbSGmQjYQlyBmg1WLqIXp5LkaQJNZ+YylYK3PcDXfLLPfCP/FhwG96RUm QE2mdmBEGhDBxYzwt4rfhBLDaF6KCB4= X-MC-Unique: gtZkyYb_Ppeyule2QlRYNw-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 12/12] block/nvme: support nested aio_poll() Date: Wed, 24 Jun 2020 11:02:10 +0100 Message-Id: <20200624100210.59975-13-stefanha@redhat.com> In-Reply-To: <20200624100210.59975-1-stefanha@redhat.com> References: <20200624100210.59975-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/24 01:59:40 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Sergio Lopez , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" QEMU block drivers are supposed to support aio_poll() from I/O completion callback functions. This means completion processing must be re-entrant. The standard approach is to schedule a BH during completion processing and cancel it at the end of processing. If aio_poll() is invoked by a callback function then the BH will run. The BH continues the suspended completion processing. All of this means that request A's cb() can synchronously wait for request B to complete. Previously the nvme block driver would hang because it didn't process completions from nested aio_poll(). Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Message-id: 20200617132201.1832152-8-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 67 ++++++++++++++++++++++++++++++++++++++++------ block/trace-events | 2 +- 2 files changed, 60 insertions(+), 9 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 8dc68d3daa..374e268915 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -74,9 +74,11 @@ typedef struct { int cq_phase; int free_req_head; NVMeRequest reqs[NVME_NUM_REQS]; - bool busy; int need_kick; int inflight; + + /* Thread-safe, no lock necessary */ + QEMUBH *completion_bh; } NVMeQueuePair; =20 /* Memory mapped registers */ @@ -140,6 +142,8 @@ struct BDRVNVMeState { #define NVME_BLOCK_OPT_DEVICE "device" #define NVME_BLOCK_OPT_NAMESPACE "namespace" =20 +static void nvme_process_completion_bh(void *opaque); + static QemuOptsList runtime_opts =3D { .name =3D "nvme", .head =3D QTAILQ_HEAD_INITIALIZER(runtime_opts.head), @@ -181,6 +185,9 @@ static void nvme_init_queue(BlockDriverState *bs, NVMeQ= ueue *q, =20 static void nvme_free_queue_pair(NVMeQueuePair *q) { + if (q->completion_bh) { + qemu_bh_delete(q->completion_bh); + } qemu_vfree(q->prp_list_pages); qemu_vfree(q->sq.queue); qemu_vfree(q->cq.queue); @@ -214,6 +221,8 @@ static NVMeQueuePair *nvme_create_queue_pair(BlockDrive= rState *bs, q->index =3D idx; qemu_co_queue_init(&q->free_req_queue); q->prp_list_pages =3D qemu_blockalign0(bs, s->page_size * NVME_NUM_REQ= S); + q->completion_bh =3D aio_bh_new(bdrv_get_aio_context(bs), + nvme_process_completion_bh, q); r =3D qemu_vfio_dma_map(s->vfio, q->prp_list_pages, s->page_size * NVME_NUM_REQS, false, &prp_list_iova); @@ -352,11 +361,21 @@ static bool nvme_process_completion(NVMeQueuePair *q) NvmeCqe *c; =20 trace_nvme_process_completion(s, q->index, q->inflight); - if (q->busy || s->plugged) { - trace_nvme_process_completion_queue_busy(s, q->index); + if (s->plugged) { + trace_nvme_process_completion_queue_plugged(s, q->index); return false; } - q->busy =3D true; + + /* + * Support re-entrancy when a request cb() function invokes aio_poll(). + * Pending completions must be visible to aio_poll() so that a cb() + * function can wait for the completion of another request. + * + * The aio_poll() loop will execute our BH and we'll resume completion + * processing there. + */ + qemu_bh_schedule(q->completion_bh); + assert(q->inflight >=3D 0); while (q->inflight) { int ret; @@ -384,10 +403,10 @@ static bool nvme_process_completion(NVMeQueuePair *q) assert(req.cb); nvme_put_free_req_locked(q, preq); preq->cb =3D preq->opaque =3D NULL; - qemu_mutex_unlock(&q->lock); - req.cb(req.opaque, ret); - qemu_mutex_lock(&q->lock); q->inflight--; + qemu_mutex_unlock(&q->lock); + req.cb(req.opaque, ret); + qemu_mutex_lock(&q->lock); progress =3D true; } if (progress) { @@ -396,10 +415,28 @@ static bool nvme_process_completion(NVMeQueuePair *q) *q->cq.doorbell =3D cpu_to_le32(q->cq.head); nvme_wake_free_req_locked(q); } - q->busy =3D false; + + qemu_bh_cancel(q->completion_bh); + return progress; } =20 +static void nvme_process_completion_bh(void *opaque) +{ + NVMeQueuePair *q =3D opaque; + + /* + * We're being invoked because a nvme_process_completion() cb() functi= on + * called aio_poll(). The callback may be waiting for further completi= ons + * so notify the device that it has space to fill in more completions = now. + */ + smp_mb_release(); + *q->cq.doorbell =3D cpu_to_le32(q->cq.head); + nvme_wake_free_req_locked(q); + + nvme_process_completion(q); +} + static void nvme_trace_command(const NvmeCmd *cmd) { int i; @@ -1309,6 +1346,13 @@ static void nvme_detach_aio_context(BlockDriverState= *bs) { BDRVNVMeState *s =3D bs->opaque; =20 + for (int i =3D 0; i < s->nr_queues; i++) { + NVMeQueuePair *q =3D s->queues[i]; + + qemu_bh_delete(q->completion_bh); + q->completion_bh =3D NULL; + } + aio_set_event_notifier(bdrv_get_aio_context(bs), &s->irq_notifier, false, NULL, NULL); } @@ -1321,6 +1365,13 @@ static void nvme_attach_aio_context(BlockDriverState= *bs, s->aio_context =3D new_context; aio_set_event_notifier(new_context, &s->irq_notifier, false, nvme_handle_event, nvme_poll_cb); + + for (int i =3D 0; i < s->nr_queues; i++) { + NVMeQueuePair *q =3D s->queues[i]; + + q->completion_bh =3D + aio_bh_new(new_context, nvme_process_completion_bh, q); + } } =20 static void nvme_aio_plug(BlockDriverState *bs) diff --git a/block/trace-events b/block/trace-events index 29dff8881c..dbe76a7613 100644 --- a/block/trace-events +++ b/block/trace-events @@ -158,7 +158,7 @@ nvme_kick(void *s, int queue) "s %p queue %d" nvme_dma_flush_queue_wait(void *s) "s %p" nvme_error(int cmd_specific, int sq_head, int sqid, int cid, int status) "= cmd_specific %d sq_head %d sqid %d cid %d status 0x%x" nvme_process_completion(void *s, int index, int inflight) "s %p queue %d i= nflight %d" -nvme_process_completion_queue_busy(void *s, int index) "s %p queue %d" +nvme_process_completion_queue_plugged(void *s, int index) "s %p queue %d" nvme_complete_command(void *s, int index, int cid) "s %p queue %d cid %d" nvme_submit_command(void *s, int index, int cid) "s %p queue %d cid %d" nvme_submit_command_raw(int c0, int c1, int c2, int c3, int c4, int c5, in= t c6, int c7) "%02x %02x %02x %02x %02x %02x %02x %02x" --=20 2.26.2