From nobody Fri Nov 7 00:59:15 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544678104111193.65964574236966; Wed, 12 Dec 2018 21:15:04 -0800 (PST) Received: from localhost ([::1]:50470 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXJKw-0006pr-OX for importer@patchew.org; Thu, 13 Dec 2018 00:15:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54968) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXJC7-0007Eq-Gq for qemu-devel@nongnu.org; Thu, 13 Dec 2018 00:05:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXJC4-0000w7-Nw for qemu-devel@nongnu.org; Thu, 13 Dec 2018 00:05:54 -0500 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:35369) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXJC3-0000gi-TW for qemu-devel@nongnu.org; Thu, 13 Dec 2018 00:05:52 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id EEDE521B42; Thu, 13 Dec 2018 00:05:37 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Thu, 13 Dec 2018 00:05:37 -0500 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id A29231045F; Thu, 13 Dec 2018 00:05:37 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=mesmtp; bh=RDCNBFN9P/ozafW5L/G18rl+hq0b/x3yzE/vcP4o3r8=; b=jrJjYyir902R kirqRvGYuB/dUKnOIcVMQrJ27lPlhwh9Z1UWolnt3u5dWt/kbTcPlN4zrIovevQL 3aBc89pMagcbtFBiA62PbsLf6Nogirp36kixMKZZLVTxQX7d5GUaeiIBbeI7AB4v EolkaVxAAr0uzZfVH0ie2MJy34K+JSU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=RDCNBFN9P/ozafW5L/G18rl+hq0b/x3yzE/vcP4o3 r8=; b=plL2Ji3vI1X7qOeIilpvvu009eI+avIUHHlEJly0TzGThvg8ZCjybcPHx q7096w3zbaMq3bKcbxC+pJnaJGnklE+7QKBQmPR/dSLNNDxIy8W2nx+xHuWJss+2 lyFDxUYvIdMnGsOTdqK5PWOHIKrs3QQRWTqDjJVlgbMsn0zkb/K4euAiVtILwjdv Lirc1VI+uIvU0/zOwAJCpZ0jIWfGIiBRBzPFV+COMXoeWEducXkP5nUAUiFslPU7 AxKVQGTOJEDDCzbodWXFqWiWwBjBieuF48FhyNPSxB1GpjrjMiMJRA9PsO26tL9E dC1s5azZ6fJ3yQHQYN1/cTFyawxVw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudehuddgkedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfquhhtnecuuegrihhlohhuthemucef tddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvffufffkof gjfhggtgfgsehtkeertdertdejnecuhfhrohhmpedfgfhmihhlihhoucfirdcuvehothgr fdcuoegtohhtrgessghrrggrphdrohhrgheqnecukfhppeduvdekrdehledrvddtrddvud einecurfgrrhgrmhepmhgrihhlfhhrohhmpegtohhtrgessghrrggrphdrohhrghenucev lhhushhtvghrufhiiigvpedt X-ME-Proxy: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Thu, 13 Dec 2018 00:03:44 -0500 Message-Id: <20181213050453.9677-5-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181213050453.9677-1-cota@braap.org> References: <20181213050453.9677-1-cota@braap.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH v5 04/73] cpu: make qemu_work_cond per-cpu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" This eliminates the need to use the BQL to queue CPU work. While at it, give the per-cpu field a generic name ("cond") since it will soon be used for more than just queueing CPU work. Reviewed-by: Richard Henderson Reviewed-by: Alex Benn=C3=A9e Signed-off-by: Emilio G. Cota --- include/qom/cpu.h | 6 ++-- cpus-common.c | 72 ++++++++++++++++++++++++++++++++++++++--------- cpus.c | 2 +- qom/cpu.c | 1 + 4 files changed, 63 insertions(+), 18 deletions(-) diff --git a/include/qom/cpu.h b/include/qom/cpu.h index bb5a36a324..aff42c41dd 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -316,6 +316,7 @@ struct qemu_work_item; * @mem_io_vaddr: Target virtual address at which the memory was accessed. * @kvm_fd: vCPU file descriptor for KVM. * @lock: Lock to prevent multiple access to per-CPU fields. + * @cond: Condition variable for per-CPU events. * @work_list: List of pending asynchronous work. * @trace_dstate_delayed: Delayed changes to trace_dstate (includes all ch= anges * to @trace_dstate). @@ -358,6 +359,7 @@ struct CPUState { =20 QemuMutex lock; /* fields below protected by @lock */ + QemuCond cond; QSIMPLEQ_HEAD(, qemu_work_item) work_list; =20 CPUAddressSpace *cpu_ases; @@ -763,12 +765,10 @@ bool cpu_is_stopped(CPUState *cpu); * @cpu: The vCPU to run on. * @func: The function to be executed. * @data: Data to pass to the function. - * @mutex: Mutex to release while waiting for @func to run. * * Used internally in the implementation of run_on_cpu. */ -void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data da= ta, - QemuMutex *mutex); +void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data da= ta); =20 /** * run_on_cpu: diff --git a/cpus-common.c b/cpus-common.c index 2913294cb7..71469c85ce 100644 --- a/cpus-common.c +++ b/cpus-common.c @@ -26,7 +26,6 @@ static QemuMutex qemu_cpu_list_lock; static QemuCond exclusive_cond; static QemuCond exclusive_resume; -static QemuCond qemu_work_cond; =20 /* >=3D 1 if a thread is inside start_exclusive/end_exclusive. Written * under qemu_cpu_list_lock, read with atomic operations. @@ -42,7 +41,6 @@ void qemu_init_cpu_list(void) qemu_mutex_init(&qemu_cpu_list_lock); qemu_cond_init(&exclusive_cond); qemu_cond_init(&exclusive_resume); - qemu_cond_init(&qemu_work_cond); } =20 void cpu_list_lock(void) @@ -113,23 +111,37 @@ struct qemu_work_item { bool free, exclusive, done; }; =20 -static void queue_work_on_cpu(CPUState *cpu, struct qemu_work_item *wi) +/* Called with the CPU's lock held */ +static void queue_work_on_cpu_locked(CPUState *cpu, struct qemu_work_item = *wi) { - qemu_mutex_lock(&cpu->lock); QSIMPLEQ_INSERT_TAIL(&cpu->work_list, wi, node); wi->done =3D false; - qemu_mutex_unlock(&cpu->lock); =20 qemu_cpu_kick(cpu); } =20 -void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data da= ta, - QemuMutex *mutex) +static void queue_work_on_cpu(CPUState *cpu, struct qemu_work_item *wi) +{ + cpu_mutex_lock(cpu); + queue_work_on_cpu_locked(cpu, wi); + cpu_mutex_unlock(cpu); +} + +void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data da= ta) { struct qemu_work_item wi; + bool has_bql =3D qemu_mutex_iothread_locked(); + + g_assert(no_cpu_mutex_locked()); =20 if (qemu_cpu_is_self(cpu)) { - func(cpu, data); + if (has_bql) { + func(cpu, data); + } else { + qemu_mutex_lock_iothread(); + func(cpu, data); + qemu_mutex_unlock_iothread(); + } return; } =20 @@ -139,13 +151,34 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func fun= c, run_on_cpu_data data, wi.free =3D false; wi.exclusive =3D false; =20 - queue_work_on_cpu(cpu, &wi); + cpu_mutex_lock(cpu); + queue_work_on_cpu_locked(cpu, &wi); + + /* + * We are going to sleep on the CPU lock, so release the BQL. + * + * During the transition to per-CPU locks, we release the BQL _after_ + * having kicked the destination CPU (from queue_work_on_cpu_locked ab= ove). + * This makes sure that the enqueued work will be seen by the CPU + * after being woken up from the kick, since the CPU sleeps on the BQL. + * Once we complete the transition to per-CPU locks, we will release + * the BQL earlier in this function. + */ + if (has_bql) { + qemu_mutex_unlock_iothread(); + } + while (!atomic_mb_read(&wi.done)) { CPUState *self_cpu =3D current_cpu; =20 - qemu_cond_wait(&qemu_work_cond, mutex); + qemu_cond_wait(&cpu->cond, &cpu->lock); current_cpu =3D self_cpu; } + cpu_mutex_unlock(cpu); + + if (has_bql) { + qemu_mutex_lock_iothread(); + } } =20 void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data= data) @@ -307,6 +340,7 @@ void async_safe_run_on_cpu(CPUState *cpu, run_on_cpu_fu= nc func, void process_queued_cpu_work(CPUState *cpu) { struct qemu_work_item *wi; + bool has_bql =3D qemu_mutex_iothread_locked(); =20 qemu_mutex_lock(&cpu->lock); if (QSIMPLEQ_EMPTY(&cpu->work_list)) { @@ -324,13 +358,23 @@ void process_queued_cpu_work(CPUState *cpu) * BQL, so it goes to sleep; start_exclusive() is sleeping too= , so * neither CPU can proceed. */ - qemu_mutex_unlock_iothread(); + if (has_bql) { + qemu_mutex_unlock_iothread(); + } start_exclusive(); wi->func(cpu, wi->data); end_exclusive(); - qemu_mutex_lock_iothread(); + if (has_bql) { + qemu_mutex_lock_iothread(); + } } else { - wi->func(cpu, wi->data); + if (has_bql) { + wi->func(cpu, wi->data); + } else { + qemu_mutex_lock_iothread(); + wi->func(cpu, wi->data); + qemu_mutex_unlock_iothread(); + } } qemu_mutex_lock(&cpu->lock); if (wi->free) { @@ -340,5 +384,5 @@ void process_queued_cpu_work(CPUState *cpu) } } qemu_mutex_unlock(&cpu->lock); - qemu_cond_broadcast(&qemu_work_cond); + qemu_cond_broadcast(&cpu->cond); } diff --git a/cpus.c b/cpus.c index 980d62cd58..7ae859f35c 100644 --- a/cpus.c +++ b/cpus.c @@ -1236,7 +1236,7 @@ void qemu_init_cpu_loop(void) =20 void run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data) { - do_run_on_cpu(cpu, func, data, &qemu_global_mutex); + do_run_on_cpu(cpu, func, data); } =20 static void qemu_kvm_destroy_vcpu(CPUState *cpu) diff --git a/qom/cpu.c b/qom/cpu.c index 411b9b2eec..aa15ea4af5 100644 --- a/qom/cpu.c +++ b/qom/cpu.c @@ -372,6 +372,7 @@ static void cpu_common_initfn(Object *obj) cpu->nr_threads =3D 1; =20 qemu_mutex_init(&cpu->lock); + qemu_cond_init(&cpu->cond); QSIMPLEQ_INIT(&cpu->work_list); QTAILQ_INIT(&cpu->breakpoints); QTAILQ_INIT(&cpu->watchpoints); --=20 2.17.1