From nobody Wed Oct 23 00:25:49 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1728986946; cv=none; d=zohomail.com; s=zohoarc; b=XLTbQoh0vNTnlwISfFqJqc5jaiU5P5m/B14TrTvP8zYdN2DhG98oM5XzF51kbxTMQqJ8ru41oqO6H9EmkID9xvUN/lLxb2AANBTROUo7pWNOH+iJwdNGZkHkOdPk5ceg2lwaHV4aZllH2gZ82xJzyXcG6shwphBkWkv2A1NVTIw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1728986946; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=+rAMZa0mwMX2J0hw4kTVyh6LzQdk2qhhFgWuT8d10R0=; b=jgCY56hcMUzd0aqBrM745d/C2cMqd4PS8pd/SBESqCBhGjtIO6aTlGCI4byxeC8+UTG55iUTPbRyLw4IqbYaoPwaFhqsMmr0nmkMhvL9MJ9sCjscp/4QboYMfthLF+LdnfYmGcl0jRjZTKPq5wHJWEiOmNZasGR39tqGnvHNv/4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1728986946310331.00717909257673; Tue, 15 Oct 2024 03:09:06 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t0eU8-00062q-S3; Tue, 15 Oct 2024 06:09:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t0eTz-0005j0-8Q; Tue, 15 Oct 2024 06:08:51 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t0eTw-0002PO-Fw; Tue, 15 Oct 2024 06:08:50 -0400 Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4XSV7N1nZWz6DB4H; Tue, 15 Oct 2024 18:04:16 +0800 (CST) Received: from frapeml500007.china.huawei.com (unknown [7.182.85.172]) by mail.maildlp.com (Postfix) with ESMTPS id 91FF5140519; Tue, 15 Oct 2024 18:08:44 +0800 (CST) Received: from 00293818-MRGF.huawei.com (10.48.146.149) by frapeml500007.china.huawei.com (7.182.85.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Tue, 15 Oct 2024 12:08:24 +0200 To: , , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH RFC V5 23/30] tcg: Update tcg_register_thread() leg to handle region alloc for hotplugged vCPU Date: Tue, 15 Oct 2024 11:00:05 +0100 Message-ID: <20241015100012.254223-24-salil.mehta@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241015100012.254223-1-salil.mehta@huawei.com> References: <20241015100012.254223-1-salil.mehta@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.48.146.149] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To frapeml500007.china.huawei.com (7.182.85.172) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=salil.mehta@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Salil Mehta From: Salil Mehta via Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1728986947618116600 The TCG code cache consists of multiple regions shared among vCPUs in multi-threaded TCG mode. For cold-plugged vCPUs, these regions are sized and allocated during initialization in the `tcg_register_thread()` function whe= n the vCPUs are realized. Later, these regions must be reallocated for hot-plugged vCPUs as well. If region allocation fails for hot-plugged vCPUs=E2=80=94due to the code ca= che being under stress=E2=80=94the TCG code cache must be flushed to create space for= the newly hot-plugged vCPU. The only safe way to perform `tb_flush()` is to execute it synchronously within the `cpu_exec()` loop. Reported-by: Miguel Luis Signed-off-by: Miguel Luis Signed-off-by: Salil Mehta --- accel/tcg/tcg-accel-ops-mttcg.c | 2 +- accel/tcg/tcg-accel-ops-rr.c | 2 +- include/tcg/startup.h | 6 ++++++ include/tcg/tcg.h | 1 + tcg/region.c | 14 ++++++++++++++ tcg/tcg.c | 13 ++++++++++++- 6 files changed, 35 insertions(+), 3 deletions(-) diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttc= g.c index 49814ec4af..ab2f79d2c7 100644 --- a/accel/tcg/tcg-accel-ops-mttcg.c +++ b/accel/tcg/tcg-accel-ops-mttcg.c @@ -74,7 +74,7 @@ static void *mttcg_cpu_thread_fn(void *arg) force_rcu.notifier.notify =3D mttcg_force_rcu; force_rcu.cpu =3D cpu; rcu_add_force_rcu_notifier(&force_rcu.notifier); - tcg_register_thread(); + tcg_register_thread(cpu); =20 bql_lock(); qemu_thread_get_self(cpu->thread); diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c index 8ebadf8e9e..953231837c 100644 --- a/accel/tcg/tcg-accel-ops-rr.c +++ b/accel/tcg/tcg-accel-ops-rr.c @@ -186,7 +186,7 @@ static void *rr_cpu_thread_fn(void *arg) rcu_register_thread(); force_rcu.notify =3D rr_force_rcu; rcu_add_force_rcu_notifier(&force_rcu); - tcg_register_thread(); + tcg_register_thread(cpu); =20 bql_lock(); qemu_thread_get_self(cpu->thread); diff --git a/include/tcg/startup.h b/include/tcg/startup.h index f71305765c..a565071516 100644 --- a/include/tcg/startup.h +++ b/include/tcg/startup.h @@ -25,6 +25,8 @@ #ifndef TCG_STARTUP_H #define TCG_STARTUP_H =20 +#include "hw/core/cpu.h" + /** * tcg_init: Initialize the TCG runtime * @tb_size: translation buffer size @@ -43,7 +45,11 @@ void tcg_init(size_t tb_size, int splitwx, unsigned max_= cpus); * accelerator's init_machine() method) must register with this * function before initiating translation. */ +#ifdef CONFIG_USER_ONLY void tcg_register_thread(void); +#else +void tcg_register_thread(CPUState *cpu); +#endif =20 /** * tcg_prologue_init(): Generate the code for the TCG prologue diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index 21d5884741..e3328cc600 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -518,6 +518,7 @@ struct TCGContext { =20 /* Track which vCPU triggers events */ CPUState *cpu; /* *_trans */ + bool tbflush_pend; /* TB flush pending due to vCPU hotplug */ =20 /* These structures are private to tcg-target.c.inc. */ #ifdef TCG_TARGET_NEED_LDST_LABELS diff --git a/tcg/region.c b/tcg/region.c index 478ec051c4..9007bfd71e 100644 --- a/tcg/region.c +++ b/tcg/region.c @@ -393,6 +393,20 @@ bool tcg_region_alloc(TCGContext *s) static void tcg_region_initial_alloc__locked(TCGContext *s) { bool err =3D tcg_region_alloc__locked(s); + + /* + * Hotplugged vCPUs may initially fail to find even a single available + * region. This could be due to the TB cache being under stress from t= he + * existing vCPUs. To mitigate this, the TB cache should be flushed. + * Therefore, the region allocation failure should be ignored, and a f= lag + * set to mark `tb_flush()` as pending. The flush will be performed la= ter, + * synchronously in the context of `cpu_exec_loop()`/`tb_gen_code()`. + */ + if (err && s->cpu && DEVICE(s->cpu)->hotplugged) { + s->tbflush_pend =3D true; + return; + } + g_assert(!err); } =20 diff --git a/tcg/tcg.c b/tcg/tcg.c index 34e3056380..5e9c6b2b4b 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -783,12 +783,13 @@ void tcg_register_thread(void) tcg_ctx =3D &tcg_init_ctx; } #else -void tcg_register_thread(void) +void tcg_register_thread(CPUState *cpu) { TCGContext *s =3D g_malloc(sizeof(*s)); unsigned int i, n; =20 *s =3D tcg_init_ctx; + s->cpu =3D cpu; =20 /* Relink mem_base. */ for (i =3D 0, n =3D tcg_init_ctx.nb_globals; i < n; ++i) { @@ -1388,6 +1389,16 @@ TranslationBlock *tcg_tb_alloc(TCGContext *s) TranslationBlock *tb; void *next; =20 + /* + * The hotplugged vCPU's TCG context might not have any regions alloca= ted. + * If this condition is detected, we should flush the TB cache to ensu= re + * that regions can be allocated for the newly hotplugged vCPU's TCGCo= ntext. + */ + if (s->tbflush_pend) { + s->tbflush_pend =3D false; + return NULL; + } + retry: tb =3D (void *)ROUND_UP((uintptr_t)s->code_gen_ptr, align); next =3D (void *)ROUND_UP((uintptr_t)(tb + 1), align); --=20 2.34.1