From nobody Sun May 12 23:43:53 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1683592161; cv=none; d=zohomail.com; s=zohoarc; b=Lpwhp4bRKZQ1xnB8ZvtZso21fV/N95f+Fae7tmsnKemZttGCHvRVsgJ1fs3gJmVXA6Iwr3G7qsjzkTKxCF2/4ITpA3WVlkLBeN1tmCbN1PVdsV3r/U7962/zilgSdm0d0Rfa59r0VofwelF9nQYDjEDmzi/sZmZiRqAbxLCqOvI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683592161; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/1w77KaIoEw3rO4tby6tQKfdSg9DR3DbTkGBALJKypU=; b=JBe4NGnIb1HzREAjLZ2XceP4oJN2tyGJD3NMpiDzVaPWmrRp7Jh0RQR0t4sLJ6vnbrqFi+6/hleClDomq6AMIrJ6go163aUwPGdJBVUsHNLNt9uQeoI5xH8mJjB6rg7Uhuc/tX/mqAZLME9OPQz8AKnmROal4fi97uQidkG+nok= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683592161560592.4676229825459; Mon, 8 May 2023 17:29:21 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwBDO-0004Mn-Oa; Mon, 08 May 2023 20:28:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwBDH-0004LM-PD for qemu-devel@nongnu.org; Mon, 08 May 2023 20:28:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwBD7-000449-HO for qemu-devel@nongnu.org; Mon, 08 May 2023 20:28:18 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-340-B6LzUqOMMNyoWIrm3ZbKaA-1; Mon, 08 May 2023 20:27:59 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 713E3878521; Tue, 9 May 2023 00:27:58 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-118.bne.redhat.com [10.64.54.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A9BB14171BC; Tue, 9 May 2023 00:27:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683592088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/1w77KaIoEw3rO4tby6tQKfdSg9DR3DbTkGBALJKypU=; b=BYiJ18J4nK72NdcHjvqqNG14VGpoSQZwl5WPcyeelz1Ba5wFxfWpzy9Fk47tO6Ifg0eIKK pREhviCn8qPwjo5/RR9g4L7qlKk/DQIaprXLnZ5A8fTsjzupcXcGnpbN0SNHndfGgOBw/X 0p/VFNTjdtA8xnwp3o9Un41pTNm/CMg= X-MC-Unique: B6LzUqOMMNyoWIrm3ZbKaA-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, qemu-riscv@nongnu.org, pbonzini@redhat.com, imammedo@redhat.com, peter.maydell@linaro.org, rad@semihalf.com, quic_llindhol@quicinc.com, eduardo@habkost.net, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, thuth@redhat.com, lvivier@redhat.com, ajones@ventanamicro.com, berrange@redhat.com, dbarboza@ventanamicro.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v5 1/3] numa: Validate cluster and NUMA node boundary if required Date: Tue, 9 May 2023 10:27:37 +1000 Message-Id: <20230509002739.18388-2-gshan@redhat.com> In-Reply-To: <20230509002739.18388-1-gshan@redhat.com> References: <20230509002739.18388-1-gshan@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1683592163529100007 For some architectures like ARM64, multiple CPUs in one cluster can be associated with different NUMA nodes, which is irregular configuration because we shouldn't have this in baremetal environment. The irregular configuration causes Linux guest to misbehave, as the following warning messages indicate. -smp 6,maxcpus=3D6,sockets=3D2,clusters=3D1,cores=3D3,threads=3D1 \ -numa node,nodeid=3D0,cpus=3D0-1,memdev=3Dram0 \ -numa node,nodeid=3D1,cpus=3D2-3,memdev=3Dram1 \ -numa node,nodeid=3D2,cpus=3D4-5,memdev=3Dram2 \ ------------[ cut here ]------------ WARNING: CPU: 0 PID: 1 at kernel/sched/topology.c:2271 build_sched_domain= s+0x284/0x910 Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.14.0-268.el9.aarch64 #1 pstate: 00400005 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=3D--) pc : build_sched_domains+0x284/0x910 lr : build_sched_domains+0x184/0x910 sp : ffff80000804bd50 x29: ffff80000804bd50 x28: 0000000000000002 x27: 0000000000000000 x26: ffff800009cf9a80 x25: 0000000000000000 x24: ffff800009cbf840 x23: ffff000080325000 x22: ffff0000005df800 x21: ffff80000a4ce508 x20: 0000000000000000 x19: ffff000080324440 x18: 0000000000000014 x17: 00000000388925c0 x16: 000000005386a066 x15: 000000009c10cc2e x14: 00000000000001c0 x13: 0000000000000001 x12: ffff00007fffb1a0 x11: ffff00007fffb180 x10: ffff80000a4ce508 x9 : 0000000000000041 x8 : ffff80000a4ce500 x7 : ffff80000a4cf920 x6 : 0000000000000001 x5 : 0000000000000001 x4 : 0000000000000007 x3 : 0000000000000002 x2 : 0000000000001000 x1 : ffff80000a4cf928 x0 : 0000000000000001 Call trace: build_sched_domains+0x284/0x910 sched_init_domains+0xac/0xe0 sched_init_smp+0x48/0xc8 kernel_init_freeable+0x140/0x1ac kernel_init+0x28/0x140 ret_from_fork+0x10/0x20 Improve the situation to warn when multiple CPUs in one cluster have been associated with different NUMA nodes. However, one NUMA node is allowed to be associated with different clusters. Signed-off-by: Gavin Shan Acked-by: Philippe Mathieu-Daud=C3=A9 Acked-by: Igor Mammedov --- hw/core/machine.c | 42 ++++++++++++++++++++++++++++++++++++++++++ include/hw/boards.h | 1 + 2 files changed, 43 insertions(+) diff --git a/hw/core/machine.c b/hw/core/machine.c index 47a34841a5..b718d89441 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -1261,6 +1261,45 @@ static void machine_numa_finish_cpu_init(MachineStat= e *machine) g_string_free(s, true); } =20 +static void validate_cpu_cluster_to_numa_boundary(MachineState *ms) +{ + MachineClass *mc =3D MACHINE_GET_CLASS(ms); + NumaState *state =3D ms->numa_state; + const CPUArchIdList *possible_cpus =3D mc->possible_cpu_arch_ids(ms); + const CPUArchId *cpus =3D possible_cpus->cpus; + int i, j; + + if (state->num_nodes <=3D 1 || possible_cpus->len <=3D 1) { + return; + } + + /* + * The Linux scheduling domain can't be parsed when the multiple CPUs + * in one cluster have been associated with different NUMA nodes. Howe= ver, + * it's fine to associate one NUMA node with CPUs in different cluster= s. + */ + for (i =3D 0; i < possible_cpus->len; i++) { + for (j =3D i + 1; j < possible_cpus->len; j++) { + if (cpus[i].props.has_socket_id && + cpus[i].props.has_cluster_id && + cpus[i].props.has_node_id && + cpus[j].props.has_socket_id && + cpus[j].props.has_cluster_id && + cpus[j].props.has_node_id && + cpus[i].props.socket_id =3D=3D cpus[j].props.socket_id && + cpus[i].props.cluster_id =3D=3D cpus[j].props.cluster_id && + cpus[i].props.node_id !=3D cpus[j].props.node_id) { + warn_report("CPU-%d and CPU-%d in socket-%ld-cluster-%ld " + "have been associated with node-%ld and node-= %ld " + "respectively. It can cause OSes like Linux t= o " + "misbehave", i, j, cpus[i].props.socket_id, + cpus[i].props.cluster_id, cpus[i].props.node_= id, + cpus[j].props.node_id); + } + } + } +} + MemoryRegion *machine_consume_memdev(MachineState *machine, HostMemoryBackend *backend) { @@ -1346,6 +1385,9 @@ void machine_run_board_init(MachineState *machine, co= nst char *mem_path, Error * numa_complete_configuration(machine); if (machine->numa_state->num_nodes) { machine_numa_finish_cpu_init(machine); + if (machine_class->cpu_cluster_has_numa_boundary) { + validate_cpu_cluster_to_numa_boundary(machine); + } } } =20 diff --git a/include/hw/boards.h b/include/hw/boards.h index f4117fdb9a..f609cc9aed 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -273,6 +273,7 @@ struct MachineClass { bool nvdimm_supported; bool numa_mem_supported; bool auto_enable_numa; + bool cpu_cluster_has_numa_boundary; SMPCompatProps smp_props; const char *default_ram_id; =20 --=20 2.23.0 From nobody Sun May 12 23:43:53 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1683592183; cv=none; d=zohomail.com; s=zohoarc; b=Q2ycL183XfCiBmXln0jCZCipnc6DO4orLEmdw9B7zeOK4cc4pvP6ci/MuPSiRYplfq2SlJBmsTTYYTZ6vo7Q2iG/PUvEUl2Vma9/EQRjus8j6kTakT3H0mzKUhxW+9QRv6ULr/5cTtyno0kW9eovjxazjmI2z7oUz8Z96PHuesw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683592183; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Om+yHZCioNW3THbOGuEmFPAAwBXcsErx+2ZivRpwpRk=; b=PzfSYdLkKNi65oZ5yf6LIZNDCHL/GcFzxTlFfatONfFja5uP1Wmrtq3Px/NSB1lEZjQ9v1J8jpM4C7R3FriNZni9vHx1E8qlplzPMF/rmfLOe1r4lGJBTnPOQbOeKpXGwX+BJTpQEFumvKIK8esIOS85fG1ILIfAvgIr75LRi/Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683592183786833.1282668127637; Mon, 8 May 2023 17:29:43 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwBDO-0004Lx-06; Mon, 08 May 2023 20:28:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwBDF-0004KB-6w for qemu-devel@nongnu.org; Mon, 08 May 2023 20:28:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwBDB-00047J-5s for qemu-devel@nongnu.org; Mon, 08 May 2023 20:28:15 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-665-vZ8NPbONPzGwbZQFb0O4og-1; Mon, 08 May 2023 20:28:07 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6E55485A588; Tue, 9 May 2023 00:28:06 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-118.bne.redhat.com [10.64.54.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E71A14171BC; Tue, 9 May 2023 00:27:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683592091; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Om+yHZCioNW3THbOGuEmFPAAwBXcsErx+2ZivRpwpRk=; b=DtW6vqoIVKpI7pCwKC4eeDdAT1REkugN7sHw6MeELwVsYhKSwG0qYZULOeTKf9lSQ50Uz2 tasv3sPtaeBdRssUVfR+70yklABX7Gp09Oet6ZIdcITh/WXzNlqhUKpLSi9GXZkvFsn8BT Cz+2arm5Y29taBjlAhk2BER2HuK9AA8= X-MC-Unique: vZ8NPbONPzGwbZQFb0O4og-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, qemu-riscv@nongnu.org, pbonzini@redhat.com, imammedo@redhat.com, peter.maydell@linaro.org, rad@semihalf.com, quic_llindhol@quicinc.com, eduardo@habkost.net, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, thuth@redhat.com, lvivier@redhat.com, ajones@ventanamicro.com, berrange@redhat.com, dbarboza@ventanamicro.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v5 2/3] hw/arm: Validate cluster and NUMA node boundary Date: Tue, 9 May 2023 10:27:38 +1000 Message-Id: <20230509002739.18388-3-gshan@redhat.com> In-Reply-To: <20230509002739.18388-1-gshan@redhat.com> References: <20230509002739.18388-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1683592185659100006 Content-Type: text/plain; charset="utf-8" There are two ARM machines where NUMA is aware: 'virt' and 'sbsa-ref'. Both of them are required to follow cluster-NUMA-node boundary. To enable the validation to warn about the irregular configuration where multiple CPUs in one cluster have been associated with different NUMA nodes. Signed-off-by: Gavin Shan Acked-by: Igor Mammedov --- hw/arm/sbsa-ref.c | 2 ++ hw/arm/virt.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index 0b93558dde..efb380e7c8 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -864,6 +864,8 @@ static void sbsa_ref_class_init(ObjectClass *oc, void *= data) mc->possible_cpu_arch_ids =3D sbsa_ref_possible_cpu_arch_ids; mc->cpu_index_to_instance_props =3D sbsa_ref_cpu_index_to_props; mc->get_default_cpu_node_id =3D sbsa_ref_get_default_cpu_node_id; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; } =20 static const TypeInfo sbsa_ref_info =3D { diff --git a/hw/arm/virt.c b/hw/arm/virt.c index b99ae18501..5c88b78aab 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -3032,6 +3032,8 @@ static void virt_machine_class_init(ObjectClass *oc, = void *data) mc->smp_props.clusters_supported =3D true; mc->auto_enable_numa_with_memhp =3D true; mc->auto_enable_numa_with_memdev =3D true; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; mc->default_ram_id =3D "mach-virt.ram"; =20 object_class_property_add(oc, "acpi", "OnOffAuto", --=20 2.23.0 From nobody Sun May 12 23:43:53 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1683592184; cv=none; d=zohomail.com; s=zohoarc; b=Qz9qV+iDZ0joCBtpxgvdsxdfXOiM7HmhWValYBR+8lYIhiHAJAVNrcfHcFqrjmiVPG62gIPWpYpi8+iCnu1KJXqBCHmbR3OrnMAWv1anUP2D1pA22N+oxQV99o6TrjUhGRceyCwsU20k2/po4hQHDbF3w1x331iK1L+CyXLRu9o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683592184; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=h0e9kG0FR3IP27E3gxDdD+6BjsHxySQBdLDMEqFNo6w=; b=mbRaq5bQ3vk0w19UHhvMFAAIVpxPXmJgTAltSJHtzR4VmiiJf8TVngrRUPQW0+Quzp8DXt8jHGE6E+z/JXsDzfzus1c9lApjshJiuIxP0MYa7yehWnadMJ7wRck/FALsuen/lYs4zl/17XvajRotJfm2y+FdOGmqp3soyCUI0Nw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683592184225297.0586646023794; Mon, 8 May 2023 17:29:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwBDQ-0004Nw-9m; Mon, 08 May 2023 20:28:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwBDJ-0004LW-B2 for qemu-devel@nongnu.org; Mon, 08 May 2023 20:28:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwBDH-0004Cd-RP for qemu-devel@nongnu.org; Mon, 08 May 2023 20:28:21 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-522-tgb_eNJAOb2ZMgLQXmuYiQ-1; Mon, 08 May 2023 20:28:16 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3F75B100F654; Tue, 9 May 2023 00:28:15 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-118.bne.redhat.com [10.64.54.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B51714171BC; Tue, 9 May 2023 00:28:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683592099; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h0e9kG0FR3IP27E3gxDdD+6BjsHxySQBdLDMEqFNo6w=; b=VzfkiFYkvRjgQbtIarZu3VuVPl+pyxElEFfio0U+2Gpf+DzP7/45S7glAtQjE8bGjCwXIr qdJLZEmzz2dgdXb4gbo6mGXDezEyx3onn0BJUIXr8ZAB/SbbhO6xesLhFuX9Hnaq4QeDlM AD/XAeEsbNRGeKoaKOwhJn9xqCmscIw= X-MC-Unique: tgb_eNJAOb2ZMgLQXmuYiQ-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, qemu-riscv@nongnu.org, pbonzini@redhat.com, imammedo@redhat.com, peter.maydell@linaro.org, rad@semihalf.com, quic_llindhol@quicinc.com, eduardo@habkost.net, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, thuth@redhat.com, lvivier@redhat.com, ajones@ventanamicro.com, berrange@redhat.com, dbarboza@ventanamicro.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v5 3/3] hw/riscv: Validate cluster and NUMA node boundary Date: Tue, 9 May 2023 10:27:39 +1000 Message-Id: <20230509002739.18388-4-gshan@redhat.com> In-Reply-To: <20230509002739.18388-1-gshan@redhat.com> References: <20230509002739.18388-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1683592185637100005 Content-Type: text/plain; charset="utf-8" There are two RISCV machines where NUMA is aware: 'virt' and 'spike'. Both of them are required to follow cluster-NUMA-node boundary. To enable the validation to warn about the irregular configuration where multiple CPUs in one cluster has been associated with multiple NUMA nodes. Signed-off-by: Gavin Shan Reviewed-by: Daniel Henrique Barboza Acked-by: Igor Mammedov Acked-by: Alistair Francis --- hw/riscv/spike.c | 2 ++ hw/riscv/virt.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c index 2c5546560a..81f7e53aed 100644 --- a/hw/riscv/spike.c +++ b/hw/riscv/spike.c @@ -354,6 +354,8 @@ static void spike_machine_class_init(ObjectClass *oc, v= oid *data) mc->cpu_index_to_instance_props =3D riscv_numa_cpu_index_to_props; mc->get_default_cpu_node_id =3D riscv_numa_get_default_cpu_node_id; mc->numa_mem_supported =3D true; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; mc->default_ram_id =3D "riscv.spike.ram"; object_class_property_add_str(oc, "signature", NULL, spike_set_signatu= re); object_class_property_set_description(oc, "signature", diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 4e3efbee16..84a2bca460 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -1678,6 +1678,8 @@ static void virt_machine_class_init(ObjectClass *oc, = void *data) mc->cpu_index_to_instance_props =3D riscv_numa_cpu_index_to_props; mc->get_default_cpu_node_id =3D riscv_numa_get_default_cpu_node_id; mc->numa_mem_supported =3D true; + /* platform instead of architectural choice */ + mc->cpu_cluster_has_numa_boundary =3D true; mc->default_ram_id =3D "riscv_virt_board.ram"; assert(!mc->get_hotplug_handler); mc->get_hotplug_handler =3D virt_machine_get_hotplug_handler; --=20 2.23.0