From nobody Wed May 1 20:38:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496161710417600.171752667428; Tue, 30 May 2017 09:28:30 -0700 (PDT) Received: from localhost ([::1]:54967 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFk0T-0001KZ-1x for importer@patchew.org; Tue, 30 May 2017 12:28:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60917) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjwN-0005rg-NK for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dFjwM-00055H-Ee for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:15 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43368) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dFjwJ-00052Z-Ao; Tue, 30 May 2017 12:24:11 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 484F785540; Tue, 30 May 2017 16:24:10 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.34.112.60]) by smtp.corp.redhat.com (Postfix) with ESMTP id C605E171A0; Tue, 30 May 2017 16:24:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 484F785540 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=imammedo@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 484F785540 From: Igor Mammedov To: qemu-devel@nongnu.org Date: Tue, 30 May 2017 18:23:56 +0200 Message-Id: <1496161442-96665-2-git-send-email-imammedo@redhat.com> In-Reply-To: <1496161442-96665-1-git-send-email-imammedo@redhat.com> References: <1496161442-96665-1-git-send-email-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 30 May 2017 16:24:10 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 1/7] numa: consolidate cpu_preplug fixups/checks for pc/arm/spapr X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Eduardo Habkost , David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Igor Mammedov Reviewed-by: David Gibson --- v3: retain opriginal error checking logic in case of "node-id" set && !slot->props.has_node_id it will be cleaned up in a follow up patch Eduardo Habkost v2: user error_abort in numa_cpu_pre_plug() Eduardo Habkost --- include/sysemu/numa.h | 1 + hw/arm/virt.c | 16 ++-------------- hw/i386/pc.c | 17 +---------------- hw/ppc/spapr.c | 17 +---------------- numa.c | 23 +++++++++++++++++++++++ 5 files changed, 28 insertions(+), 46 deletions(-) diff --git a/include/sysemu/numa.h b/include/sysemu/numa.h index 7ffde5b..610eece 100644 --- a/include/sysemu/numa.h +++ b/include/sysemu/numa.h @@ -35,4 +35,5 @@ void numa_legacy_auto_assign_ram(MachineClass *mc, NodeIn= fo *nodes, int nb_nodes, ram_addr_t size); void numa_default_auto_assign_ram(MachineClass *mc, NodeInfo *nodes, int nb_nodes, ram_addr_t size); +void numa_cpu_pre_plug(const CPUArchId *slot, DeviceState *dev, Error **er= rp); #endif diff --git a/hw/arm/virt.c b/hw/arm/virt.c index c7c8159..ce676df 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1351,7 +1351,6 @@ static void machvirt_init(MachineState *machine) for (n =3D 0; n < possible_cpus->len; n++) { Object *cpuobj; CPUState *cs; - int node_id; =20 if (n >=3D smp_cpus) { break; @@ -1364,19 +1363,8 @@ static void machvirt_init(MachineState *machine) cs =3D CPU(cpuobj); cs->cpu_index =3D n; =20 - node_id =3D possible_cpus->cpus[cs->cpu_index].props.node_id; - if (!possible_cpus->cpus[cs->cpu_index].props.has_node_id) { - /* by default CPUState::numa_node was 0 if it's not set via CLI - * keep it this way for now but in future we probably should - * refuse to start up with incomplete numa mapping */ - node_id =3D 0; - } - if (cs->numa_node =3D=3D CPU_UNSET_NUMA_NODE_ID) { - cs->numa_node =3D node_id; - } else { - /* CPU isn't device_add compatible yet, this shouldn't happen = */ - error_setg(&error_abort, "user set node-id not implemented"); - } + numa_cpu_pre_plug(&possible_cpus->cpus[cs->cpu_index], DEVICE(cpuo= bj), + &error_fatal); =20 if (!vms->secure) { object_property_set_bool(cpuobj, false, "has_el3", NULL); diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 816bfa8..cf09949 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -1893,7 +1893,6 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_d= ev, DeviceState *dev, Error **errp) { int idx; - int node_id; CPUState *cs; CPUArchId *cpu_slot; X86CPUTopoInfo topo; @@ -1984,21 +1983,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_= dev, cs =3D CPU(cpu); cs->cpu_index =3D idx; =20 - node_id =3D cpu_slot->props.node_id; - if (!cpu_slot->props.has_node_id) { - /* by default CPUState::numa_node was 0 if it's not set via CLI - * keep it this way for now but in future we probably should - * refuse to start up with incomplete numa mapping */ - node_id =3D 0; - } - if (cs->numa_node =3D=3D CPU_UNSET_NUMA_NODE_ID) { - cs->numa_node =3D node_id; - } else if (cs->numa_node !=3D node_id) { - error_setg(errp, "node-id %d must match numa node specified" - "with -numa option for cpu-index %d", - cs->numa_node, cs->cpu_index); - return; - } + numa_cpu_pre_plug(cpu_slot, dev, errp); } =20 static void pc_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 0980d73..c7fee8b 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2831,11 +2831,9 @@ static void spapr_core_pre_plug(HotplugHandler *hotp= lug_dev, DeviceState *dev, MachineClass *mc =3D MACHINE_GET_CLASS(hotplug_dev); Error *local_err =3D NULL; CPUCore *cc =3D CPU_CORE(dev); - sPAPRCPUCore *sc =3D SPAPR_CPU_CORE(dev); char *base_core_type =3D spapr_get_cpu_core_type(machine->cpu_model); const char *type =3D object_get_typename(OBJECT(dev)); CPUArchId *core_slot; - int node_id; int index; =20 if (dev->hotplugged && !mc->has_hotpluggable_cpus) { @@ -2870,20 +2868,7 @@ static void spapr_core_pre_plug(HotplugHandler *hotp= lug_dev, DeviceState *dev, goto out; } =20 - node_id =3D core_slot->props.node_id; - if (!core_slot->props.has_node_id) { - /* by default CPUState::numa_node was 0 if it's not set via CLI - * keep it this way for now but in future we probably should - * refuse to start up with incomplete numa mapping */ - node_id =3D 0; - } - if (sc->node_id =3D=3D CPU_UNSET_NUMA_NODE_ID) { - sc->node_id =3D node_id; - } else if (sc->node_id !=3D node_id) { - error_setg(&local_err, "node-id %d must match numa node specified" - "with -numa option for cpu-index %d", sc->node_id, cc->core_id= ); - goto out; - } + numa_cpu_pre_plug(core_slot, dev, &local_err); =20 out: g_free(base_core_type); diff --git a/numa.c b/numa.c index ca73145..19fd1e0 100644 --- a/numa.c +++ b/numa.c @@ -534,6 +534,29 @@ void parse_numa_opts(MachineState *ms) } } =20 +void numa_cpu_pre_plug(const CPUArchId *slot, DeviceState *dev, Error **er= rp) +{ + int mapped_node_id; /* set by -numa option */ + int node_id =3D object_property_get_int(OBJECT(dev), "node-id", &error= _abort); + + /* by default CPUState::numa_node was 0 if it wasn't set explicitly + * TODO: make it error when incomplete numa mapping support is removed + */ + mapped_node_id =3D slot->props.node_id; + if (!slot->props.has_node_id) { + mapped_node_id =3D 0; + } + + if (node_id =3D=3D CPU_UNSET_NUMA_NODE_ID) { + /* due to bug in libvirt, it doesn't pass node-id from props on + * device_add as expected, so we have to fix it up here */ + object_property_set_int(OBJECT(dev), mapped_node_id, "node-id", er= rp); + } else if (node_id !=3D mapped_node_id) { + error_setg(errp, "node-id=3D%d must match numa node specified " + "with -numa option", node_id); + } +} + static void allocate_system_memory_nonnuma(MemoryRegion *mr, Object *owner, const char *name, uint64_t ram_size) --=20 2.7.4 From nobody Wed May 1 20:38:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496161605119303.0815038252048; Tue, 30 May 2017 09:26:45 -0700 (PDT) Received: from localhost ([::1]:54958 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjyl-0007tE-CX for importer@patchew.org; Tue, 30 May 2017 12:26:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60938) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjwO-0005t0-RX for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dFjwN-00055t-NV for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40610) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dFjwL-000537-5l; Tue, 30 May 2017 12:24:13 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D4F361E4C; Tue, 30 May 2017 16:24:12 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.34.112.60]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D9AE171A0; Tue, 30 May 2017 16:24:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1D4F361E4C Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=imammedo@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1D4F361E4C From: Igor Mammedov To: qemu-devel@nongnu.org Date: Tue, 30 May 2017 18:23:57 +0200 Message-Id: <1496161442-96665-3-git-send-email-imammedo@redhat.com> In-Reply-To: <1496161442-96665-1-git-send-email-imammedo@redhat.com> References: <1496161442-96665-1-git-send-email-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 30 May 2017 16:24:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 2/7] numa: move default mapping init to machine X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Eduardo Habkost , David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" there is no need use cpu_index_to_instance_props() for setting default cpu -> node mapping. Generic machine code can do it without cpu_index by just enabling already preset defaults in possible_cpus. PS: as bonus it makes one less user of cpu_index_to_instance_props() Signed-off-by: Igor Mammedov --- v2: - make default_mapping boolean, Eduardo Habkost - redo default mapping detection loop as a combo of for/if, Eduardo Habkost - return back lost TODO comment, Eduardo Habkost --- hw/core/machine.c | 33 +++++++++++++++++++++++---------- numa.c | 26 -------------------------- 2 files changed, 23 insertions(+), 36 deletions(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index fd6a436..aaf3cff 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -700,26 +700,39 @@ static char *cpu_slot_to_string(const CPUArchId *cpu) return g_string_free(s, false); } =20 -static void machine_numa_validate(MachineState *machine) +static void machine_numa_finish_init(MachineState *machine) { int i; + bool default_mapping; GString *s =3D g_string_new(NULL); MachineClass *mc =3D MACHINE_GET_CLASS(machine); const CPUArchIdList *possible_cpus =3D mc->possible_cpu_arch_ids(machi= ne); =20 assert(nb_numa_nodes); for (i =3D 0; i < possible_cpus->len; i++) { + if (possible_cpus->cpus[i].props.has_node_id) { + break; + } + } + default_mapping =3D (i =3D=3D possible_cpus->len); + + for (i =3D 0; i < possible_cpus->len; i++) { const CPUArchId *cpu_slot =3D &possible_cpus->cpus[i]; =20 - /* at this point numa mappings are initilized by CLI options - * or with default mappings so it's sufficient to list - * all not yet mapped CPUs here */ - /* TODO: make it hard error in future */ if (!cpu_slot->props.has_node_id) { - char *cpu_str =3D cpu_slot_to_string(cpu_slot); - g_string_append_printf(s, "%sCPU %d [%s]", s->len ? ", " : "",= i, - cpu_str); - g_free(cpu_str); + if (default_mapping) { + /* fetch default mapping from board and enable it */ + CpuInstanceProperties props =3D cpu_slot->props; + props.has_node_id =3D true; + machine_set_cpu_numa_node(machine, &props, &error_fatal); + } else { + /* record slots with not set mapping, + * TODO: make it hard error in future */ + char *cpu_str =3D cpu_slot_to_string(cpu_slot); + g_string_append_printf(s, "%sCPU %d [%s]", + s->len ? ", " : "", i, cpu_str); + g_free(cpu_str); + } } } if (s->len) { @@ -737,7 +750,7 @@ void machine_run_board_init(MachineState *machine) MachineClass *machine_class =3D MACHINE_GET_CLASS(machine); =20 if (nb_numa_nodes) { - machine_numa_validate(machine); + machine_numa_finish_init(machine); } machine_class->init(machine); } diff --git a/numa.c b/numa.c index 19fd1e0..45ad70a 100644 --- a/numa.c +++ b/numa.c @@ -427,7 +427,6 @@ void numa_default_auto_assign_ram(MachineClass *mc, Nod= eInfo *nodes, void parse_numa_opts(MachineState *ms) { int i; - const CPUArchIdList *possible_cpus; MachineClass *mc =3D MACHINE_GET_CLASS(ms); =20 if (qemu_opts_foreach(qemu_find_opts("numa"), parse_numa, ms, NULL)) { @@ -485,31 +484,6 @@ void parse_numa_opts(MachineState *ms) =20 numa_set_mem_ranges(); =20 - /* assign CPUs to nodes using board provided default mapping */ - if (!mc->cpu_index_to_instance_props || !mc->possible_cpu_arch_ids= ) { - error_report("default CPUs to NUMA node mapping isn't supporte= d"); - exit(1); - } - - possible_cpus =3D mc->possible_cpu_arch_ids(ms); - for (i =3D 0; i < possible_cpus->len; i++) { - if (possible_cpus->cpus[i].props.has_node_id) { - break; - } - } - - /* no CPUs are assigned to NUMA nodes */ - if (i =3D=3D possible_cpus->len) { - for (i =3D 0; i < max_cpus; i++) { - CpuInstanceProperties props; - /* fetch default mapping from board and enable it */ - props =3D mc->cpu_index_to_instance_props(ms, i); - props.has_node_id =3D true; - - machine_set_cpu_numa_node(ms, &props, &error_fatal); - } - } - /* QEMU needs at least all unique node pair distances to build * the whole NUMA distance table. QEMU treats the distance table * as symmetric by default, i.e. distance A->B =3D=3D distance B->= A. --=20 2.7.4 From nobody Wed May 1 20:38:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496161807324251.46463201911524; Tue, 30 May 2017 09:30:07 -0700 (PDT) Received: from localhost ([::1]:54972 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFk21-0002RQ-OJ for importer@patchew.org; Tue, 30 May 2017 12:30:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60997) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjwR-0005vt-0i for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dFjwP-00057A-P8 for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44328) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dFjwM-00055B-Sz; Tue, 30 May 2017 12:24:15 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DA982C049D5B; Tue, 30 May 2017 16:24:13 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.34.112.60]) by smtp.corp.redhat.com (Postfix) with ESMTP id 64310171A0; Tue, 30 May 2017 16:24:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com DA982C049D5B Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=imammedo@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com DA982C049D5B From: Igor Mammedov To: qemu-devel@nongnu.org Date: Tue, 30 May 2017 18:23:58 +0200 Message-Id: <1496161442-96665-4-git-send-email-imammedo@redhat.com> In-Reply-To: <1496161442-96665-1-git-send-email-imammedo@redhat.com> References: <1496161442-96665-1-git-send-email-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 30 May 2017 16:24:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 3/7] numa: make sure that all cpus have has_node_id set if numa is enabled X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Eduardo Habkost , David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It fixes/add missing _PXM object for non mapped CPU (x86) and missing fdt node (virt-arm). It ensures that possible_cpus contains complete mapping if numa is enabled by the time machine_init() is executed. As result non completely mapped CPUs: 1) appear in ACPI/fdt blobs 2) QMP query-hotpluggable-cpus command shows bound nodes for such CPUs 3) allows to drop checks for has_node_id in numa only code, reducing number of invariants incomplete mapping could produce 4) moves fixup/implicit node init from runtime numa_cpu_pre_plug() (when CPU object is created) to machine_numa_finish_init() which helps to fix [1, 2] and make possible_cpus complete source of numa mapping available even before CPUs are created. Signed-off-by: Igor Mammedov --- hw/arm/virt-acpi-build.c | 4 +--- hw/core/machine.c | 16 ++++++++++------ hw/i386/acpi-build.c | 3 +-- hw/i386/pc.c | 4 +--- numa.c | 16 +++++----------- 5 files changed, 18 insertions(+), 25 deletions(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index e585206..977a794 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -496,12 +496,10 @@ build_srat(GArray *table_data, BIOSLinker *linker, Vi= rtMachineState *vms) srat->reserved1 =3D cpu_to_le32(1); =20 for (i =3D 0; i < cpu_list->len; ++i) { - int node_id =3D cpu_list->cpus[i].props.has_node_id ? - cpu_list->cpus[i].props.node_id : 0; core =3D acpi_data_push(table_data, sizeof(*core)); core->type =3D ACPI_SRAT_PROCESSOR_GICC; core->length =3D sizeof(*core); - core->proximity =3D cpu_to_le32(node_id); + core->proximity =3D cpu_to_le32(cpu_list->cpus[i].props.node_id); core->acpi_processor_uid =3D cpu_to_le32(i); core->flags =3D cpu_to_le32(1); } diff --git a/hw/core/machine.c b/hw/core/machine.c index aaf3cff..964b75d 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -720,19 +720,23 @@ static void machine_numa_finish_init(MachineState *ma= chine) const CPUArchId *cpu_slot =3D &possible_cpus->cpus[i]; =20 if (!cpu_slot->props.has_node_id) { - if (default_mapping) { - /* fetch default mapping from board and enable it */ - CpuInstanceProperties props =3D cpu_slot->props; - props.has_node_id =3D true; - machine_set_cpu_numa_node(machine, &props, &error_fatal); - } else { + /* fetch default mapping from board and enable it */ + CpuInstanceProperties props =3D cpu_slot->props; + + if (!default_mapping) { /* record slots with not set mapping, * TODO: make it hard error in future */ char *cpu_str =3D cpu_slot_to_string(cpu_slot); g_string_append_printf(s, "%sCPU %d [%s]", s->len ? ", " : "", i, cpu_str); g_free(cpu_str); + + /* non mapped cpus used to fallback to node 0 */ + props.node_id =3D 0; } + + props.has_node_id =3D true; + machine_set_cpu_numa_node(machine, &props, &error_fatal); } } if (s->len) { diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index afcadac..873880d 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -2335,8 +2335,7 @@ build_srat(GArray *table_data, BIOSLinker *linker, Ma= chineState *machine) srat->reserved1 =3D cpu_to_le32(1); =20 for (i =3D 0; i < apic_ids->len; i++) { - int node_id =3D apic_ids->cpus[i].props.has_node_id ? - apic_ids->cpus[i].props.node_id : 0; + int node_id =3D apic_ids->cpus[i].props.node_id; uint32_t apic_id =3D apic_ids->cpus[i].arch_id; =20 if (apic_id < 255) { diff --git a/hw/i386/pc.c b/hw/i386/pc.c index cf09949..84f0a6f 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -788,9 +788,7 @@ static FWCfgState *bochs_bios_init(AddressSpace *as, PC= MachineState *pcms) for (i =3D 0; i < cpus->len; i++) { unsigned int apic_id =3D cpus->cpus[i].arch_id; assert(apic_id < pcms->apic_id_limit); - if (cpus->cpus[i].props.has_node_id) { - numa_fw_cfg[apic_id + 1] =3D cpu_to_le64(cpus->cpus[i].props.n= ode_id); - } + numa_fw_cfg[apic_id + 1] =3D cpu_to_le64(cpus->cpus[i].props.node_= id); } for (i =3D 0; i < nb_numa_nodes; i++) { numa_fw_cfg[pcms->apic_id_limit + 1 + i] =3D diff --git a/numa.c b/numa.c index 45ad70a..abed45c 100644 --- a/numa.c +++ b/numa.c @@ -510,22 +510,16 @@ void parse_numa_opts(MachineState *ms) =20 void numa_cpu_pre_plug(const CPUArchId *slot, DeviceState *dev, Error **er= rp) { - int mapped_node_id; /* set by -numa option */ int node_id =3D object_property_get_int(OBJECT(dev), "node-id", &error= _abort); =20 - /* by default CPUState::numa_node was 0 if it wasn't set explicitly - * TODO: make it error when incomplete numa mapping support is removed - */ - mapped_node_id =3D slot->props.node_id; - if (!slot->props.has_node_id) { - mapped_node_id =3D 0; - } - if (node_id =3D=3D CPU_UNSET_NUMA_NODE_ID) { /* due to bug in libvirt, it doesn't pass node-id from props on * device_add as expected, so we have to fix it up here */ - object_property_set_int(OBJECT(dev), mapped_node_id, "node-id", er= rp); - } else if (node_id !=3D mapped_node_id) { + if (slot->props.has_node_id) { + object_property_set_int(OBJECT(dev), slot->props.node_id, + "node-id", errp); + } + } else if (node_id !=3D slot->props.node_id) { error_setg(errp, "node-id=3D%d must match numa node specified " "with -numa option", node_id); } --=20 2.7.4 From nobody Wed May 1 20:38:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496161939742571.1613688570029; Tue, 30 May 2017 09:32:19 -0700 (PDT) Received: from localhost ([::1]:54994 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFk49-0003sp-CO for importer@patchew.org; Tue, 30 May 2017 12:32:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32803) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjwU-0005z4-74 for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dFjwT-00058k-As for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:22 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38640) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dFjwO-000566-Q9; Tue, 30 May 2017 12:24:16 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B784C37E65; Tue, 30 May 2017 16:24:15 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.34.112.60]) by smtp.corp.redhat.com (Postfix) with ESMTP id 315A0171A0; Tue, 30 May 2017 16:24:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B784C37E65 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=imammedo@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B784C37E65 From: Igor Mammedov To: qemu-devel@nongnu.org Date: Tue, 30 May 2017 18:23:59 +0200 Message-Id: <1496161442-96665-5-git-send-email-imammedo@redhat.com> In-Reply-To: <1496161442-96665-1-git-send-email-imammedo@redhat.com> References: <1496161442-96665-1-git-send-email-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 30 May 2017 16:24:15 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 4/7] numa: make hmp 'info numa' fetch numa nodes from qmp_query_cpus() result X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Eduardo Habkost , David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" HMP command 'info numa' is the last external user that access CPUState::numa_node field directly. In order to move it to CPU classes that actually use it, eliminate direct access and use an alternative approach by using result of qmp_query_cpus(), which provides topology properties CPU threads are associated with (including node-id). Signed-off-by: Igor Mammedov --- monitor.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/monitor.c b/monitor.c index baa73c9..9d91631 100644 --- a/monitor.c +++ b/monitor.c @@ -1696,23 +1696,26 @@ static void hmp_info_mtree(Monitor *mon, const QDic= t *qdict) static void hmp_info_numa(Monitor *mon, const QDict *qdict) { int i; - CPUState *cpu; uint64_t *node_mem; + CpuInfoList *cpu_list, *cpu; =20 + cpu_list =3D qmp_query_cpus(&error_abort); node_mem =3D g_new0(uint64_t, nb_numa_nodes); query_numa_node_mem(node_mem); monitor_printf(mon, "%d nodes\n", nb_numa_nodes); for (i =3D 0; i < nb_numa_nodes; i++) { monitor_printf(mon, "node %d cpus:", i); - CPU_FOREACH(cpu) { - if (cpu->numa_node =3D=3D i) { - monitor_printf(mon, " %d", cpu->cpu_index); + for (cpu =3D cpu_list; cpu; cpu =3D cpu->next) { + if (cpu->value->has_props && cpu->value->props->has_node_id && + cpu->value->props->node_id =3D=3D i) { + monitor_printf(mon, " %" PRIi64, cpu->value->CPU); } } monitor_printf(mon, "\n"); monitor_printf(mon, "node %d size: %" PRId64 " MB\n", i, node_mem[i] >> 20); } + qapi_free_CpuInfoList(cpu_list); g_free(node_mem); } =20 --=20 2.7.4 From nobody Wed May 1 20:38:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496161642886974.4941149131793; Tue, 30 May 2017 09:27:22 -0700 (PDT) Received: from localhost ([::1]:54961 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjzM-0008OO-EM for importer@patchew.org; Tue, 30 May 2017 12:27:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32837) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjwV-000619-Pl for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dFjwU-00059Q-Kb for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43538) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dFjwQ-00057J-Ip; Tue, 30 May 2017 12:24:18 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 81A4D80B22; Tue, 30 May 2017 16:24:17 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.34.112.60]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0787D171A0; Tue, 30 May 2017 16:24:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 81A4D80B22 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=imammedo@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 81A4D80B22 From: Igor Mammedov To: qemu-devel@nongnu.org Date: Tue, 30 May 2017 18:24:00 +0200 Message-Id: <1496161442-96665-6-git-send-email-imammedo@redhat.com> In-Reply-To: <1496161442-96665-1-git-send-email-imammedo@redhat.com> References: <1496161442-96665-1-git-send-email-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 30 May 2017 16:24:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 5/7] numa: move numa_node from CPUState into target specific classes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Eduardo Habkost , David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Move vcpu's assocciated numa_node field out of generic CPUState into inherited classes that actually care about cpu<->numa mapping, i.e: ARMCPU, PowerPCCPU, X86CPU. Signed-off-by: Igor Mammedov --- include/qom/cpu.h | 2 -- target/arm/cpu.h | 2 ++ target/i386/cpu.h | 1 + target/ppc/cpu.h | 1 + hw/ppc/spapr.c | 2 +- hw/ppc/spapr_cpu_core.c | 4 +++- target/arm/cpu.c | 2 +- target/i386/cpu.c | 2 +- 8 files changed, 10 insertions(+), 6 deletions(-) diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 55214ce..89ddb68 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -265,7 +265,6 @@ struct qemu_work_item; * @cpu_index: CPU index (informative). * @nr_cores: Number of cores within this CPU package. * @nr_threads: Number of threads within this CPU. - * @numa_node: NUMA node this CPU is belonging to. * @host_tid: Host thread ID. * @running: #true if CPU is currently running (lockless). * @has_waiter: #true if a CPU is currently waiting for the cpu_exec_end; @@ -314,7 +313,6 @@ struct CPUState { =20 int nr_cores; int nr_threads; - int numa_node; =20 struct QemuThread *thread; #ifdef _WIN32 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 048faed..5ffc9d8 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -703,6 +703,8 @@ struct ARMCPU { =20 ARMELChangeHook *el_change_hook; void *el_change_hook_opaque; + + int32_t node_id; /* NUMA node this CPU is belonging to */ }; =20 static inline ARMCPU *arm_env_get_cpu(CPUARMState *env) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index c4602ca..dec4067 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1275,6 +1275,7 @@ struct X86CPU { =20 struct kvm_msrs *kvm_msr_buf; =20 + int32_t node_id; /* NUMA node this CPU is belonging to */ int32_t socket_id; int32_t core_id; int32_t thread_id; diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h index 401e10e..31c052d 100644 --- a/target/ppc/cpu.h +++ b/target/ppc/cpu.h @@ -1205,6 +1205,7 @@ struct PowerPCCPU { uint32_t compat_pvr; PPCVirtualHypervisor *vhyp; Object *intc; + int32_t node_id; /* NUMA node this CPU is belonging to */ =20 /* Fields related to migration compatibility hacks */ bool pre_2_8_migration; diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index c7fee8b..34bb03d 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -187,7 +187,7 @@ static int spapr_fixup_cpu_numa_dt(void *fdt, int offse= t, CPUState *cs) cpu_to_be32(0x0), cpu_to_be32(0x0), cpu_to_be32(0x0), - cpu_to_be32(cs->numa_node), + cpu_to_be32(cpu->node_id), cpu_to_be32(index)}; =20 /* Advertise NUMA via ibm,associativity */ diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c index a17ea07..60baf02 100644 --- a/hw/ppc/spapr_cpu_core.c +++ b/hw/ppc/spapr_cpu_core.c @@ -183,15 +183,17 @@ static void spapr_cpu_core_realize(DeviceState *dev, = Error **errp) for (i =3D 0; i < cc->nr_threads; i++) { char id[32]; CPUState *cs; + PowerPCCPU *cpu; =20 obj =3D sc->threads + i * size; =20 object_initialize(obj, size, typename); cs =3D CPU(obj); + cpu =3D POWERPC_CPU(cs); cs->cpu_index =3D cc->core_id + i; =20 /* Set NUMA node for the threads belonged to core */ - cs->numa_node =3D sc->node_id; + cpu->node_id =3D sc->node_id; =20 snprintf(id, sizeof(id), "thread[%d]", i); object_property_add_child(OBJECT(sc), id, obj, &local_err); diff --git a/target/arm/cpu.c b/target/arm/cpu.c index c185eb1..09ef3a6 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1573,7 +1573,7 @@ static Property arm_cpu_properties[] =3D { DEFINE_PROP_UINT32("midr", ARMCPU, midr, 0), DEFINE_PROP_UINT64("mp-affinity", ARMCPU, mp_affinity, ARM64_AFFINITY_INVALID), - DEFINE_PROP_INT32("node-id", CPUState, numa_node, CPU_UNSET_NUMA_NODE_= ID), + DEFINE_PROP_INT32("node-id", ARMCPU, node_id, CPU_UNSET_NUMA_NODE_ID), DEFINE_PROP_END_OF_LIST() }; =20 diff --git a/target/i386/cpu.c b/target/i386/cpu.c index a41d595..ffb5267 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -3986,7 +3986,7 @@ static Property x86_cpu_properties[] =3D { DEFINE_PROP_INT32("core-id", X86CPU, core_id, -1), DEFINE_PROP_INT32("socket-id", X86CPU, socket_id, -1), #endif - DEFINE_PROP_INT32("node-id", CPUState, numa_node, CPU_UNSET_NUMA_NODE_= ID), + DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID), DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false), { .name =3D "hv-spinlocks", .info =3D &qdev_prop_spinlocks }, DEFINE_PROP_BOOL("hv-relaxed", X86CPU, hyperv_relaxed_timing, false), --=20 2.7.4 From nobody Wed May 1 20:38:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496161761103776.4552387031679; Tue, 30 May 2017 09:29:21 -0700 (PDT) Received: from localhost ([::1]:54970 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFk1H-0001vW-8X for importer@patchew.org; Tue, 30 May 2017 12:29:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32862) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjwX-00063c-Ph for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dFjwW-0005AL-S2 for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:25 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44530) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dFjwU-00058x-IN; Tue, 30 May 2017 12:24:22 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 89CD0C049E32; Tue, 30 May 2017 16:24:21 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.34.112.60]) by smtp.corp.redhat.com (Postfix) with ESMTP id C41B4171A0; Tue, 30 May 2017 16:24:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 89CD0C049E32 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=imammedo@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 89CD0C049E32 From: Igor Mammedov To: qemu-devel@nongnu.org Date: Tue, 30 May 2017 18:24:01 +0200 Message-Id: <1496161442-96665-7-git-send-email-imammedo@redhat.com> In-Reply-To: <1496161442-96665-1-git-send-email-imammedo@redhat.com> References: <1496161442-96665-1-git-send-email-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 30 May 2017 16:24:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 6/7] spapr: cleanup spapr_fixup_cpu_numa_dt() usage X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Eduardo Habkost , David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" even though spapr_fixup_cpu_numa_dt() has no effect on FDT if numa is disabled, don't call it uselessly. It makes it obvious at call sites that function is need only when numa is enabled. Signed-off-by: Igor Mammedov Reviewed-by: Greg Kurz --- hw/ppc/spapr.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 34bb03d..96a2a74 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -178,10 +178,8 @@ static int spapr_fixup_cpu_smt_dt(void *fdt, int offse= t, PowerPCCPU *cpu, return ret; } =20 -static int spapr_fixup_cpu_numa_dt(void *fdt, int offset, CPUState *cs) +static int spapr_fixup_cpu_numa_dt(void *fdt, int offset, PowerPCCPU *cpu) { - int ret =3D 0; - PowerPCCPU *cpu =3D POWERPC_CPU(cs); int index =3D ppc_get_vcpu_dt_id(cpu); uint32_t associativity[] =3D {cpu_to_be32(0x5), cpu_to_be32(0x0), @@ -191,12 +189,8 @@ static int spapr_fixup_cpu_numa_dt(void *fdt, int offs= et, CPUState *cs) cpu_to_be32(index)}; =20 /* Advertise NUMA via ibm,associativity */ - if (nb_numa_nodes > 1) { - ret =3D fdt_setprop(fdt, offset, "ibm,associativity", associativit= y, + return fdt_setprop(fdt, offset, "ibm,associativity", associativity, sizeof(associativity)); - } - - return ret; } =20 /* Populate the "ibm,pa-features" property */ @@ -321,9 +315,11 @@ static int spapr_fixup_cpu_dt(void *fdt, sPAPRMachineS= tate *spapr) return ret; } =20 - ret =3D spapr_fixup_cpu_numa_dt(fdt, offset, cs); - if (ret < 0) { - return ret; + if (nb_numa_nodes > 1) { + ret =3D spapr_fixup_cpu_numa_dt(fdt, offset, cpu); + if (ret < 0) { + return ret; + } } =20 ret =3D spapr_fixup_cpu_smt_dt(fdt, offset, cpu, compat_smt); @@ -538,7 +534,9 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *f= dt, int offset, _FDT((fdt_setprop(fdt, offset, "ibm,pft-size", pft_size_prop, sizeof(pft_size_prop)))); =20 - _FDT(spapr_fixup_cpu_numa_dt(fdt, offset, cs)); + if (nb_numa_nodes > 1) { + _FDT(spapr_fixup_cpu_numa_dt(fdt, offset, cpu)); + } =20 _FDT(spapr_fixup_cpu_smt_dt(fdt, offset, cpu, compat_smt)); =20 --=20 2.7.4 From nobody Wed May 1 20:38:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496161642774536.303561633024; Tue, 30 May 2017 09:27:22 -0700 (PDT) Received: from localhost ([::1]:54960 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjzM-0008O1-9P for importer@patchew.org; Tue, 30 May 2017 12:27:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32910) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dFjwf-0006Ez-Al for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dFjwd-0005Cu-Qp for qemu-devel@nongnu.org; Tue, 30 May 2017 12:24:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:28890) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dFjwY-0005Ah-HM; Tue, 30 May 2017 12:24:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 75D2280F75; Tue, 30 May 2017 16:24:25 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.34.112.60]) by smtp.corp.redhat.com (Postfix) with ESMTP id CFF48171A0; Tue, 30 May 2017 16:24:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 75D2280F75 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=imammedo@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 75D2280F75 From: Igor Mammedov To: qemu-devel@nongnu.org Date: Tue, 30 May 2017 18:24:02 +0200 Message-Id: <1496161442-96665-8-git-send-email-imammedo@redhat.com> In-Reply-To: <1496161442-96665-1-git-send-email-imammedo@redhat.com> References: <1496161442-96665-1-git-send-email-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 30 May 2017 16:24:25 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 7/7] numa: cpu: calculate/set default node-ids after all -numa CLI options are parsed X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Eduardo Habkost , David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Calculating default node-ids for CPUs in possible_cpu_arch_ids() is rather fragile since defaults calculation uses nb_numa_nodes but callback might be potentially called early before all -numa CLI options are parsed, which would lead to cpus assigned only upto nb_numa_nodes at the time possible_cpu_arch_ids() is called. Issue was introduced by (7c88e65 numa: mirror cpu to node mapping in MachineState::possible_cpus) and for example CLI: -smp 4 -numa node,cpus=3D0 -numa node would set props.node-id in possible_cpus array for every non explicitly mapped CPU to the first node. Issue is not visible to guest nor to mgmt interface due to 1) implictly mapped cpus are forced to the first node in case of partial mapping 2) in case of default mapping possible_cpu_arch_ids() is called after all -numa options are parsed (resulting in correct mapping). However it's fragile to rely on late execution of possible_cpu_arch_ids(), therefore add machine specific callback that returns node-id for CPU and use it to calculate/ set defaults at machine_numa_finish_init() time when all -numa options are parsed. Reported-by: Eduardo Habkost Signed-off-by: Igor Mammedov --- include/hw/boards.h | 3 +++ include/sysemu/numa.h | 9 +++++++++ hw/arm/virt.c | 16 ++++++++-------- hw/core/machine.c | 1 + hw/i386/pc.c | 21 ++++++++++++--------- hw/ppc/spapr.c | 16 +++++++--------- 6 files changed, 40 insertions(+), 26 deletions(-) diff --git a/include/hw/boards.h b/include/hw/boards.h index 76ce021..063f329 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -94,6 +94,8 @@ typedef struct { * Returns an array of @CPUArchId architecture-dependent CPU IDs * which includes CPU IDs for present and possible to hotplug CPUs. * Caller is responsible for freeing returned list. + * @get_default_cpu_node_id: + * returns default board specific node_id value for CPU * @has_hotpluggable_cpus: * If true, board supports CPUs creation with -device/device_add. * @minimum_page_bits: @@ -151,6 +153,7 @@ struct MachineClass { CpuInstanceProperties (*cpu_index_to_instance_props)(MachineState *mac= hine, unsigned cpu_inde= x); const CPUArchIdList *(*possible_cpu_arch_ids)(MachineState *machine); + int64_t (*get_default_cpu_node_id)(const MachineState *ms, int idx); }; =20 /** diff --git a/include/sysemu/numa.h b/include/sysemu/numa.h index 610eece..ea123ef 100644 --- a/include/sysemu/numa.h +++ b/include/sysemu/numa.h @@ -36,4 +36,13 @@ void numa_legacy_auto_assign_ram(MachineClass *mc, NodeI= nfo *nodes, void numa_default_auto_assign_ram(MachineClass *mc, NodeInfo *nodes, int nb_nodes, ram_addr_t size); void numa_cpu_pre_plug(const CPUArchId *slot, DeviceState *dev, Error **er= rp); + +static inline void assert_nb_numa_nodes_change(void) +{ + static int saved_nb_numa_nodes; + assert(nb_numa_nodes); + assert(saved_nb_numa_nodes =3D=3D 0 || saved_nb_numa_nodes =3D=3D nb_n= uma_nodes); + saved_nb_numa_nodes =3D nb_numa_nodes; +} + #endif diff --git a/hw/arm/virt.c b/hw/arm/virt.c index ce676df..74f1453 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1553,6 +1553,13 @@ virt_cpu_index_to_props(MachineState *ms, unsigned c= pu_index) return possible_cpus->cpus[cpu_index].props; } =20 +static int64_t virt_get_default_cpu_node_id(const MachineState *ms, int id= x) +{ + assert(nb_numa_nodes); + assert_nb_numa_nodes_change(); + return idx % nb_numa_nodes; +} + static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms) { int n; @@ -1571,14 +1578,6 @@ static const CPUArchIdList *virt_possible_cpu_arch_i= ds(MachineState *ms) virt_cpu_mp_affinity(vms, n); ms->possible_cpus->cpus[n].props.has_thread_id =3D true; ms->possible_cpus->cpus[n].props.thread_id =3D n; - - /* default distribution of CPUs over NUMA nodes */ - if (nb_numa_nodes) { - /* preset values but do not enable them i.e. 'has_node_id =3D = false', - * numa init code will enable them later if manual mapping was= n't - * present on CLI */ - ms->possible_cpus->cpus[n].props.node_id =3D n % nb_numa_nodes; - } } return ms->possible_cpus; } @@ -1601,6 +1600,7 @@ static void virt_machine_class_init(ObjectClass *oc, = void *data) mc->minimum_page_bits =3D 12; mc->possible_cpu_arch_ids =3D virt_possible_cpu_arch_ids; mc->cpu_index_to_instance_props =3D virt_cpu_index_to_props; + mc->get_default_cpu_node_id =3D virt_get_default_cpu_node_id; } =20 static const TypeInfo virt_machine_info =3D { diff --git a/hw/core/machine.c b/hw/core/machine.c index 964b75d..01028c8 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -723,6 +723,7 @@ static void machine_numa_finish_init(MachineState *mach= ine) /* fetch default mapping from board and enable it */ CpuInstanceProperties props =3D cpu_slot->props; =20 + props.node_id =3D mc->get_default_cpu_node_id(machine, i); if (!default_mapping) { /* record slots with not set mapping, * TODO: make it hard error in future */ diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 84f0a6f..51d5a1b 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -2253,6 +2253,17 @@ pc_cpu_index_to_props(MachineState *ms, unsigned cpu= _index) return possible_cpus->cpus[cpu_index].props; } =20 +static int64_t pc_get_default_cpu_node_id(const MachineState *ms, int idx) +{ + X86CPUTopoInfo topo; + + assert_nb_numa_nodes_change(); + assert(ms->possible_cpus && (idx < ms->possible_cpus->len)); + x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id, + smp_cores, smp_threads, &topo); + return topo.pkg_id % nb_numa_nodes; +} + static const CPUArchIdList *pc_possible_cpu_arch_ids(MachineState *ms) { int i; @@ -2282,15 +2293,6 @@ static const CPUArchIdList *pc_possible_cpu_arch_ids= (MachineState *ms) ms->possible_cpus->cpus[i].props.core_id =3D topo.core_id; ms->possible_cpus->cpus[i].props.has_thread_id =3D true; ms->possible_cpus->cpus[i].props.thread_id =3D topo.smt_id; - - /* default distribution of CPUs over NUMA nodes */ - if (nb_numa_nodes) { - /* preset values but do not enable them i.e. 'has_node_id =3D = false', - * numa init code will enable them later if manual mapping was= n't - * present on CLI */ - ms->possible_cpus->cpus[i].props.node_id =3D - topo.pkg_id % nb_numa_nodes; - } } return ms->possible_cpus; } @@ -2335,6 +2337,7 @@ static void pc_machine_class_init(ObjectClass *oc, vo= id *data) pcmc->linuxboot_dma_enabled =3D true; mc->get_hotplug_handler =3D pc_get_hotpug_handler; mc->cpu_index_to_instance_props =3D pc_cpu_index_to_props; + mc->get_default_cpu_node_id =3D pc_get_default_cpu_node_id; mc->possible_cpu_arch_ids =3D pc_possible_cpu_arch_ids; mc->has_hotpluggable_cpus =3D true; mc->default_boot_order =3D "cad"; diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 96a2a74..06d0fb3 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -3002,6 +3002,12 @@ spapr_cpu_index_to_props(MachineState *machine, unsi= gned cpu_index) return core_slot->props; } =20 +static int64_t spapr_get_default_cpu_node_id(const MachineState *ms, int i= dx) +{ + assert_nb_numa_nodes_change(); + return idx / smp_cores % nb_numa_nodes; +} + static const CPUArchIdList *spapr_possible_cpu_arch_ids(MachineState *mach= ine) { int i; @@ -3026,15 +3032,6 @@ static const CPUArchIdList *spapr_possible_cpu_arch_= ids(MachineState *machine) machine->possible_cpus->cpus[i].arch_id =3D core_id; machine->possible_cpus->cpus[i].props.has_core_id =3D true; machine->possible_cpus->cpus[i].props.core_id =3D core_id; - - /* default distribution of CPUs over NUMA nodes */ - if (nb_numa_nodes) { - /* preset values but do not enable them i.e. 'has_node_id =3D = false', - * numa init code will enable them later if manual mapping was= n't - * present on CLI */ - machine->possible_cpus->cpus[i].props.node_id =3D - core_id / smp_threads / smp_cores % nb_numa_nodes; - } } return machine->possible_cpus; } @@ -3160,6 +3157,7 @@ static void spapr_machine_class_init(ObjectClass *oc,= void *data) hc->plug =3D spapr_machine_device_plug; hc->unplug =3D spapr_machine_device_unplug; mc->cpu_index_to_instance_props =3D spapr_cpu_index_to_props; + mc->get_default_cpu_node_id =3D spapr_get_default_cpu_node_id; mc->possible_cpu_arch_ids =3D spapr_possible_cpu_arch_ids; hc->unplug_request =3D spapr_machine_device_unplug_request; =20 --=20 2.7.4