From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689285448786.7879075906312; Mon, 5 Jun 2017 12:01:25 -0700 (PDT) Received: from localhost ([::1]:34726 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxFi-0002sy-Dv for importer@patchew.org; Mon, 05 Jun 2017 15:01:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60988) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxE0-0001ht-PO for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxDy-0007k8-SG for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36600) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxDy-0007jL-KK; Mon, 05 Jun 2017 14:59:34 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 75CECC0467C7; Mon, 5 Jun 2017 18:59:33 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id E3ED286E0D; Mon, 5 Jun 2017 18:59:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 75CECC0467C7 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 75CECC0467C7 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:18 -0300 Message-Id: <20170605185927.12111-2-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 05 Jun 2017 18:59:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 01/10] pc: Use "min-[x]level" on compat_props X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Since the automatic cpuid-level code was introduced in commit c39c0edf9bb3b968ba95484465a50c7b19f4aa3a ("target-i386: Automatically set level/xlevel/xlevel2 when needed"), the CPU model tables just define the default CPUID level code (set using "min-level"). Setting "[x]level" forces CPUID level to a specific value and disable the automatic-level logic. But the PC compat code was not updated and the existing "[x]level" compat properties broke compatibility for people using features that triggered the auto-level code. To keep previous behavior, we should set "min-[x]level" instead of "[x]level" on compat_props. This was not a problem for most cases, because old machine-types don't have full-cpuid-auto-level enabled. The only common use case it broke was the CPUID[7] auto-level code, that was already enabled since the first CPUID[7] feature was introduced (in QEMU 1.4.0). This causes the regression reported at: https://bugzilla.redhat.com/show_bug.cgi?id=3D1454641 Change the PC compat code to use "min-[x]level" instead of "[x]level" on compat_props, and add new test cases to ensure we don't break this again. Reported-by: "Guo, Zhiyi" Fixes: c39c0edf9bb ("target-i386: Automatically set level/xlevel/xlevel2 wh= en needed") Cc: qemu-stable@nongnu.org Acked-by: Michael S. Tsirkin Signed-off-by: Eduardo Habkost --- include/hw/i386/pc.h | 42 +++++++++++++++++++++------------------= --- tests/test-x86-cpuid-compat.c | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+), 21 deletions(-) diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index e447f5d8f4..d071c9c0e9 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -566,75 +566,75 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64= _t *); .value =3D "off",\ },{\ .driver =3D "qemu64" "-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(4),\ },{\ .driver =3D "kvm64" "-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(5),\ },{\ .driver =3D "pentium3" "-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(2),\ },{\ .driver =3D "n270" "-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(5),\ },{\ .driver =3D "Conroe" "-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(4),\ },{\ .driver =3D "Penryn" "-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(4),\ },{\ .driver =3D "Nehalem" "-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(4),\ },{\ .driver =3D "n270" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Penryn" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Conroe" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Nehalem" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Westmere" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "SandyBridge" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "IvyBridge" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Haswell" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Haswell-noTSX" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Broadwell" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D "Broadwell-noTSX" "-" TYPE_X86_CPU,\ - .property =3D "xlevel",\ + .property =3D "min-xlevel",\ .value =3D stringify(0x8000000a),\ },{\ .driver =3D TYPE_X86_CPU,\ @@ -860,7 +860,7 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t= *); .value =3D stringify(2),\ },{\ .driver =3D "Conroe-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(2),\ },{\ .driver =3D "Penryn-" TYPE_X86_CPU,\ @@ -868,7 +868,7 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t= *); .value =3D stringify(2),\ },{\ .driver =3D "Penryn-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(2),\ },{\ .driver =3D "Nehalem-" TYPE_X86_CPU,\ @@ -876,7 +876,7 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t= *); .value =3D stringify(2),\ },{\ .driver =3D "Nehalem-" TYPE_X86_CPU,\ - .property =3D "level",\ + .property =3D "min-level",\ .value =3D stringify(2),\ },{\ .driver =3D "virtio-net-pci",\ diff --git a/tests/test-x86-cpuid-compat.c b/tests/test-x86-cpuid-compat.c index 6c71e46391..4166ce54b7 100644 --- a/tests/test-x86-cpuid-compat.c +++ b/tests/test-x86-cpuid-compat.c @@ -313,6 +313,44 @@ int main(int argc, char **argv) add_cpuid_test("x86/cpuid/auto-xlevel2/pc-2.7", "-machine pc-i440fx-2.7 -cpu 486,+xstore", "xlevel2", 0); + /* + * QEMU 1.4.0 had auto-level enabled for CPUID[7], already, + * and the compat code that sets default level shouldn't + * disable the auto-level=3D7 code: + */ + add_cpuid_test("x86/cpuid/auto-level7/pc-i440fx-1.4/off", + "-machine pc-i440fx-1.4 -cpu Nehalem", + "level", 2); + add_cpuid_test("x86/cpuid/auto-level7/pc-i440fx-1.5/on", + "-machine pc-i440fx-1.4 -cpu Nehalem,+smap", + "level", 7); + add_cpuid_test("x86/cpuid/auto-level7/pc-i440fx-2.3/off", + "-machine pc-i440fx-2.3 -cpu Penryn", + "level", 4); + add_cpuid_test("x86/cpuid/auto-level7/pc-i440fx-2.3/on", + "-machine pc-i440fx-2.3 -cpu Penryn,+erms", + "level", 7); + add_cpuid_test("x86/cpuid/auto-level7/pc-i440fx-2.9/off", + "-machine pc-i440fx-2.9 -cpu Conroe", + "level", 10); + add_cpuid_test("x86/cpuid/auto-level7/pc-i440fx-2.9/on", + "-machine pc-i440fx-2.9 -cpu Conroe,+erms", + "level", 10); + + /* + * xlevel doesn't have any feature that triggers auto-level + * code on old machine-types. Just check that the compat code + * is working correctly: + */ + add_cpuid_test("x86/cpuid/xlevel-compat/pc-i440fx-2.3", + "-machine pc-i440fx-2.3 -cpu SandyBridge", + "xlevel", 0x8000000a); + add_cpuid_test("x86/cpuid/xlevel-compat/pc-i440fx-2.4/npt-off", + "-machine pc-i440fx-2.4 -cpu SandyBridge,", + "xlevel", 0x80000008); + add_cpuid_test("x86/cpuid/xlevel-compat/pc-i440fx-2.4/npt-on", + "-machine pc-i440fx-2.4 -cpu SandyBridge,+npt", + "xlevel", 0x80000008); =20 /* Test feature parsing */ add_feature_test("x86/cpuid/features/plus", --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689286292167.3645663967892; Mon, 5 Jun 2017 12:01:26 -0700 (PDT) Received: from localhost ([::1]:34727 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxFk-0002vV-P8 for importer@patchew.org; Mon, 05 Jun 2017 15:01:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32777) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxE3-0001je-Qs for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxE2-0007oh-Do for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33688) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxE2-0007nr-4o for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:38 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D52680461; Mon, 5 Jun 2017 18:59:37 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id D44247B131; Mon, 5 Jun 2017 18:59:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1D52680461 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1D52680461 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:19 -0300 Message-Id: <20170605185927.12111-3-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 05 Jun 2017 18:59:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 02/10] numa: consolidate cpu_preplug fixups/checks for pc/arm/spapr X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Mammedov , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Igor Mammedov Signed-off-by: Igor Mammedov Reviewed-by: David Gibson Message-Id: <1496161442-96665-2-git-send-email-imammedo@redhat.com> [ehabkost: Fix indentation] Signed-off-by: Eduardo Habkost --- include/sysemu/numa.h | 1 + hw/arm/virt.c | 16 ++-------------- hw/i386/pc.c | 17 +---------------- hw/ppc/spapr.c | 17 +---------------- numa.c | 23 +++++++++++++++++++++++ 5 files changed, 28 insertions(+), 46 deletions(-) diff --git a/include/sysemu/numa.h b/include/sysemu/numa.h index 7ffde5b119..610eece211 100644 --- a/include/sysemu/numa.h +++ b/include/sysemu/numa.h @@ -35,4 +35,5 @@ void numa_legacy_auto_assign_ram(MachineClass *mc, NodeIn= fo *nodes, int nb_nodes, ram_addr_t size); void numa_default_auto_assign_ram(MachineClass *mc, NodeInfo *nodes, int nb_nodes, ram_addr_t size); +void numa_cpu_pre_plug(const CPUArchId *slot, DeviceState *dev, Error **er= rp); #endif diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 4db2d4207c..010f7244bf 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1372,7 +1372,6 @@ static void machvirt_init(MachineState *machine) for (n =3D 0; n < possible_cpus->len; n++) { Object *cpuobj; CPUState *cs; - int node_id; =20 if (n >=3D smp_cpus) { break; @@ -1385,19 +1384,8 @@ static void machvirt_init(MachineState *machine) cs =3D CPU(cpuobj); cs->cpu_index =3D n; =20 - node_id =3D possible_cpus->cpus[cs->cpu_index].props.node_id; - if (!possible_cpus->cpus[cs->cpu_index].props.has_node_id) { - /* by default CPUState::numa_node was 0 if it's not set via CLI - * keep it this way for now but in future we probably should - * refuse to start up with incomplete numa mapping */ - node_id =3D 0; - } - if (cs->numa_node =3D=3D CPU_UNSET_NUMA_NODE_ID) { - cs->numa_node =3D node_id; - } else { - /* CPU isn't device_add compatible yet, this shouldn't happen = */ - error_setg(&error_abort, "user set node-id not implemented"); - } + numa_cpu_pre_plug(&possible_cpus->cpus[cs->cpu_index], DEVICE(cpuo= bj), + &error_fatal); =20 if (!vms->secure) { object_property_set_bool(cpuobj, false, "has_el3", NULL); diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 107a34125b..5b07be61cf 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -1893,7 +1893,6 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_d= ev, DeviceState *dev, Error **errp) { int idx; - int node_id; CPUState *cs; CPUArchId *cpu_slot; X86CPUTopoInfo topo; @@ -1984,21 +1983,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_= dev, cs =3D CPU(cpu); cs->cpu_index =3D idx; =20 - node_id =3D cpu_slot->props.node_id; - if (!cpu_slot->props.has_node_id) { - /* by default CPUState::numa_node was 0 if it's not set via CLI - * keep it this way for now but in future we probably should - * refuse to start up with incomplete numa mapping */ - node_id =3D 0; - } - if (cs->numa_node =3D=3D CPU_UNSET_NUMA_NODE_ID) { - cs->numa_node =3D node_id; - } else if (cs->numa_node !=3D node_id) { - error_setg(errp, "node-id %d must match numa node specified" - "with -numa option for cpu-index %d", - cs->numa_node, cs->cpu_index); - return; - } + numa_cpu_pre_plug(cpu_slot, dev, errp); } =20 static void pc_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index ab3aab1279..94563cd8f5 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2922,11 +2922,9 @@ static void spapr_core_pre_plug(HotplugHandler *hotp= lug_dev, DeviceState *dev, MachineClass *mc =3D MACHINE_GET_CLASS(hotplug_dev); Error *local_err =3D NULL; CPUCore *cc =3D CPU_CORE(dev); - sPAPRCPUCore *sc =3D SPAPR_CPU_CORE(dev); char *base_core_type =3D spapr_get_cpu_core_type(machine->cpu_model); const char *type =3D object_get_typename(OBJECT(dev)); CPUArchId *core_slot; - int node_id; int index; =20 if (dev->hotplugged && !mc->has_hotpluggable_cpus) { @@ -2967,20 +2965,7 @@ static void spapr_core_pre_plug(HotplugHandler *hotp= lug_dev, DeviceState *dev, goto out; } =20 - node_id =3D core_slot->props.node_id; - if (!core_slot->props.has_node_id) { - /* by default CPUState::numa_node was 0 if it's not set via CLI - * keep it this way for now but in future we probably should - * refuse to start up with incomplete numa mapping */ - node_id =3D 0; - } - if (sc->node_id =3D=3D CPU_UNSET_NUMA_NODE_ID) { - sc->node_id =3D node_id; - } else if (sc->node_id !=3D node_id) { - error_setg(&local_err, "node-id %d must match numa node specified" - "with -numa option for cpu-index %d", sc->node_id, cc->core_id= ); - goto out; - } + numa_cpu_pre_plug(core_slot, dev, &local_err); =20 out: g_free(base_core_type); diff --git a/numa.c b/numa.c index be50c62aa9..1c35a79b57 100644 --- a/numa.c +++ b/numa.c @@ -533,6 +533,29 @@ void parse_numa_opts(MachineState *ms) } } =20 +void numa_cpu_pre_plug(const CPUArchId *slot, DeviceState *dev, Error **er= rp) +{ + int mapped_node_id; /* set by -numa option */ + int node_id =3D object_property_get_int(OBJECT(dev), "node-id", &error= _abort); + + /* by default CPUState::numa_node was 0 if it wasn't set explicitly + * TODO: make it error when incomplete numa mapping support is removed + */ + mapped_node_id =3D slot->props.node_id; + if (!slot->props.has_node_id) { + mapped_node_id =3D 0; + } + + if (node_id =3D=3D CPU_UNSET_NUMA_NODE_ID) { + /* due to bug in libvirt, it doesn't pass node-id from props on + * device_add as expected, so we have to fix it up here */ + object_property_set_int(OBJECT(dev), mapped_node_id, "node-id", er= rp); + } else if (node_id !=3D mapped_node_id) { + error_setg(errp, "node-id=3D%d must match numa node specified " + "with -numa option", node_id); + } +} + static void allocate_system_memory_nonnuma(MemoryRegion *mr, Object *owner, const char *name, uint64_t ram_size) --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689304464658.110200494689; Mon, 5 Jun 2017 12:01:44 -0700 (PDT) Received: from localhost ([::1]:34729 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxFx-000369-Q6 for importer@patchew.org; Mon, 05 Jun 2017 15:01:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32802) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxE6-0001mC-P2 for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxE5-0007sM-Bz for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57468) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxE5-0007rx-41 for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:41 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 11F5FC9B3A; Mon, 5 Jun 2017 18:59:40 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7425986E00; Mon, 5 Jun 2017 18:59:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 11F5FC9B3A Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 11F5FC9B3A From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:20 -0300 Message-Id: <20170605185927.12111-4-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 05 Jun 2017 18:59:40 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 03/10] numa: move default mapping init to machine X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Mammedov , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Igor Mammedov there is no need use cpu_index_to_instance_props() for setting default cpu -> node mapping. Generic machine code can do it without cpu_index by just enabling already preset defaults in possible_cpus. PS: as bonus it makes one less user of cpu_index_to_instance_props() Signed-off-by: Igor Mammedov Message-Id: <1496161442-96665-3-git-send-email-imammedo@redhat.com> Signed-off-by: Eduardo Habkost --- hw/core/machine.c | 33 +++++++++++++++++++++++---------- numa.c | 26 -------------------------- 2 files changed, 23 insertions(+), 36 deletions(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index 3adebf14c4..c1434e6a69 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -701,26 +701,39 @@ static char *cpu_slot_to_string(const CPUArchId *cpu) return g_string_free(s, false); } =20 -static void machine_numa_validate(MachineState *machine) +static void machine_numa_finish_init(MachineState *machine) { int i; + bool default_mapping; GString *s =3D g_string_new(NULL); MachineClass *mc =3D MACHINE_GET_CLASS(machine); const CPUArchIdList *possible_cpus =3D mc->possible_cpu_arch_ids(machi= ne); =20 assert(nb_numa_nodes); for (i =3D 0; i < possible_cpus->len; i++) { + if (possible_cpus->cpus[i].props.has_node_id) { + break; + } + } + default_mapping =3D (i =3D=3D possible_cpus->len); + + for (i =3D 0; i < possible_cpus->len; i++) { const CPUArchId *cpu_slot =3D &possible_cpus->cpus[i]; =20 - /* at this point numa mappings are initilized by CLI options - * or with default mappings so it's sufficient to list - * all not yet mapped CPUs here */ - /* TODO: make it hard error in future */ if (!cpu_slot->props.has_node_id) { - char *cpu_str =3D cpu_slot_to_string(cpu_slot); - g_string_append_printf(s, "%sCPU %d [%s]", s->len ? ", " : "",= i, - cpu_str); - g_free(cpu_str); + if (default_mapping) { + /* fetch default mapping from board and enable it */ + CpuInstanceProperties props =3D cpu_slot->props; + props.has_node_id =3D true; + machine_set_cpu_numa_node(machine, &props, &error_fatal); + } else { + /* record slots with not set mapping, + * TODO: make it hard error in future */ + char *cpu_str =3D cpu_slot_to_string(cpu_slot); + g_string_append_printf(s, "%sCPU %d [%s]", + s->len ? ", " : "", i, cpu_str); + g_free(cpu_str); + } } } if (s->len && !qtest_enabled()) { @@ -738,7 +751,7 @@ void machine_run_board_init(MachineState *machine) MachineClass *machine_class =3D MACHINE_GET_CLASS(machine); =20 if (nb_numa_nodes) { - machine_numa_validate(machine); + machine_numa_finish_init(machine); } machine_class->init(machine); } diff --git a/numa.c b/numa.c index 1c35a79b57..4ec3eaf687 100644 --- a/numa.c +++ b/numa.c @@ -426,7 +426,6 @@ void numa_default_auto_assign_ram(MachineClass *mc, Nod= eInfo *nodes, void parse_numa_opts(MachineState *ms) { int i; - const CPUArchIdList *possible_cpus; MachineClass *mc =3D MACHINE_GET_CLASS(ms); =20 if (qemu_opts_foreach(qemu_find_opts("numa"), parse_numa, ms, NULL)) { @@ -484,31 +483,6 @@ void parse_numa_opts(MachineState *ms) =20 numa_set_mem_ranges(); =20 - /* assign CPUs to nodes using board provided default mapping */ - if (!mc->cpu_index_to_instance_props || !mc->possible_cpu_arch_ids= ) { - error_report("default CPUs to NUMA node mapping isn't supporte= d"); - exit(1); - } - - possible_cpus =3D mc->possible_cpu_arch_ids(ms); - for (i =3D 0; i < possible_cpus->len; i++) { - if (possible_cpus->cpus[i].props.has_node_id) { - break; - } - } - - /* no CPUs are assigned to NUMA nodes */ - if (i =3D=3D possible_cpus->len) { - for (i =3D 0; i < max_cpus; i++) { - CpuInstanceProperties props; - /* fetch default mapping from board and enable it */ - props =3D mc->cpu_index_to_instance_props(ms, i); - props.has_node_id =3D true; - - machine_set_cpu_numa_node(ms, &props, &error_fatal); - } - } - /* QEMU needs at least all unique node pair distances to build * the whole NUMA distance table. QEMU treats the distance table * as symmetric by default, i.e. distance A->B =3D=3D distance B->= A. --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689433703140.10136818923036; Mon, 5 Jun 2017 12:03:53 -0700 (PDT) Received: from localhost ([::1]:34734 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxI4-0005L4-AI for importer@patchew.org; Mon, 05 Jun 2017 15:03:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32831) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxEC-0001rI-Jt for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxEA-0007zW-FB for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35526) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxEA-0007yE-6a for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:46 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 147D3381285; Mon, 5 Jun 2017 18:59:45 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6E7F860461; Mon, 5 Jun 2017 18:59:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 147D3381285 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 147D3381285 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:21 -0300 Message-Id: <20170605185927.12111-5-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 05 Jun 2017 18:59:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 04/10] numa: make sure that all cpus have has_node_id set if numa is enabled X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Mammedov , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Igor Mammedov It fixes/add missing _PXM object for non mapped CPU (x86) and missing fdt node (virt-arm). It ensures that possible_cpus contains complete mapping if numa is enabled by the time machine_init() is executed. As result non completely mapped CPUs: 1) appear in ACPI/fdt blobs 2) QMP query-hotpluggable-cpus command shows bound nodes for such CPUs 3) allows to drop checks for has_node_id in numa only code, reducing number of invariants incomplete mapping could produce 4) moves fixup/implicit node init from runtime numa_cpu_pre_plug() (when CPU object is created) to machine_numa_finish_init() which helps to fix [1, 2] and make possible_cpus complete source of numa mapping available even before CPUs are created. Signed-off-by: Igor Mammedov Message-Id: <1496161442-96665-4-git-send-email-imammedo@redhat.com> Signed-off-by: Eduardo Habkost --- hw/arm/virt-acpi-build.c | 4 +--- hw/core/machine.c | 16 ++++++++++------ hw/i386/acpi-build.c | 3 +-- hw/i386/pc.c | 4 +--- numa.c | 16 +++++----------- 5 files changed, 18 insertions(+), 25 deletions(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 2079828c22..3d78ff68e6 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -496,12 +496,10 @@ build_srat(GArray *table_data, BIOSLinker *linker, Vi= rtMachineState *vms) srat->reserved1 =3D cpu_to_le32(1); =20 for (i =3D 0; i < cpu_list->len; ++i) { - int node_id =3D cpu_list->cpus[i].props.has_node_id ? - cpu_list->cpus[i].props.node_id : 0; core =3D acpi_data_push(table_data, sizeof(*core)); core->type =3D ACPI_SRAT_PROCESSOR_GICC; core->length =3D sizeof(*core); - core->proximity =3D cpu_to_le32(node_id); + core->proximity =3D cpu_to_le32(cpu_list->cpus[i].props.node_id); core->acpi_processor_uid =3D cpu_to_le32(i); core->flags =3D cpu_to_le32(1); } diff --git a/hw/core/machine.c b/hw/core/machine.c index c1434e6a69..2e7e9778cd 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -721,19 +721,23 @@ static void machine_numa_finish_init(MachineState *ma= chine) const CPUArchId *cpu_slot =3D &possible_cpus->cpus[i]; =20 if (!cpu_slot->props.has_node_id) { - if (default_mapping) { - /* fetch default mapping from board and enable it */ - CpuInstanceProperties props =3D cpu_slot->props; - props.has_node_id =3D true; - machine_set_cpu_numa_node(machine, &props, &error_fatal); - } else { + /* fetch default mapping from board and enable it */ + CpuInstanceProperties props =3D cpu_slot->props; + + if (!default_mapping) { /* record slots with not set mapping, * TODO: make it hard error in future */ char *cpu_str =3D cpu_slot_to_string(cpu_slot); g_string_append_printf(s, "%sCPU %d [%s]", s->len ? ", " : "", i, cpu_str); g_free(cpu_str); + + /* non mapped cpus used to fallback to node 0 */ + props.node_id =3D 0; } + + props.has_node_id =3D true; + machine_set_cpu_numa_node(machine, &props, &error_fatal); } } if (s->len && !qtest_enabled()) { diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index 82bd44f38e..ce74c84460 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -2335,8 +2335,7 @@ build_srat(GArray *table_data, BIOSLinker *linker, Ma= chineState *machine) srat->reserved1 =3D cpu_to_le32(1); =20 for (i =3D 0; i < apic_ids->len; i++) { - int node_id =3D apic_ids->cpus[i].props.has_node_id ? - apic_ids->cpus[i].props.node_id : 0; + int node_id =3D apic_ids->cpus[i].props.node_id; uint32_t apic_id =3D apic_ids->cpus[i].arch_id; =20 if (apic_id < 255) { diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 5b07be61cf..5b8c6fbbea 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -788,9 +788,7 @@ static FWCfgState *bochs_bios_init(AddressSpace *as, PC= MachineState *pcms) for (i =3D 0; i < cpus->len; i++) { unsigned int apic_id =3D cpus->cpus[i].arch_id; assert(apic_id < pcms->apic_id_limit); - if (cpus->cpus[i].props.has_node_id) { - numa_fw_cfg[apic_id + 1] =3D cpu_to_le64(cpus->cpus[i].props.n= ode_id); - } + numa_fw_cfg[apic_id + 1] =3D cpu_to_le64(cpus->cpus[i].props.node_= id); } for (i =3D 0; i < nb_numa_nodes; i++) { numa_fw_cfg[pcms->apic_id_limit + 1 + i] =3D diff --git a/numa.c b/numa.c index 4ec3eaf687..65701cb6c8 100644 --- a/numa.c +++ b/numa.c @@ -509,22 +509,16 @@ void parse_numa_opts(MachineState *ms) =20 void numa_cpu_pre_plug(const CPUArchId *slot, DeviceState *dev, Error **er= rp) { - int mapped_node_id; /* set by -numa option */ int node_id =3D object_property_get_int(OBJECT(dev), "node-id", &error= _abort); =20 - /* by default CPUState::numa_node was 0 if it wasn't set explicitly - * TODO: make it error when incomplete numa mapping support is removed - */ - mapped_node_id =3D slot->props.node_id; - if (!slot->props.has_node_id) { - mapped_node_id =3D 0; - } - if (node_id =3D=3D CPU_UNSET_NUMA_NODE_ID) { /* due to bug in libvirt, it doesn't pass node-id from props on * device_add as expected, so we have to fix it up here */ - object_property_set_int(OBJECT(dev), mapped_node_id, "node-id", er= rp); - } else if (node_id !=3D mapped_node_id) { + if (slot->props.has_node_id) { + object_property_set_int(OBJECT(dev), slot->props.node_id, + "node-id", errp); + } + } else if (node_id !=3D slot->props.node_id) { error_setg(errp, "node-id=3D%d must match numa node specified " "with -numa option", node_id); } --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689409941644.367488580828; Mon, 5 Jun 2017 12:03:29 -0700 (PDT) Received: from localhost ([::1]:34733 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxHi-00056j-OH for importer@patchew.org; Mon, 05 Jun 2017 15:03:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32833) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxEC-0001rg-Ui for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxEC-000828-5T for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36804) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxEB-00081J-VD for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:48 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E4217C049D5C; Mon, 5 Jun 2017 18:59:46 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6709C7B133; Mon, 5 Jun 2017 18:59:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E4217C049D5C Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com E4217C049D5C From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:22 -0300 Message-Id: <20170605185927.12111-6-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 05 Jun 2017 18:59:47 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 05/10] numa: make hmp 'info numa' fetch numa nodes from qmp_query_cpus() result X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Mammedov , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Igor Mammedov HMP command 'info numa' is the last external user that access CPUState::numa_node field directly. In order to move it to CPU classes that actually use it, eliminate direct access and use an alternative approach by using result of qmp_query_cpus(), which provides topology properties CPU threads are associated with (including node-id). Signed-off-by: Igor Mammedov Message-Id: <1496161442-96665-5-git-send-email-imammedo@redhat.com> Signed-off-by: Eduardo Habkost --- monitor.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/monitor.c b/monitor.c index 75e7cd26d0..1e63ace2d4 100644 --- a/monitor.c +++ b/monitor.c @@ -1696,23 +1696,26 @@ static void hmp_info_mtree(Monitor *mon, const QDic= t *qdict) static void hmp_info_numa(Monitor *mon, const QDict *qdict) { int i; - CPUState *cpu; uint64_t *node_mem; + CpuInfoList *cpu_list, *cpu; =20 + cpu_list =3D qmp_query_cpus(&error_abort); node_mem =3D g_new0(uint64_t, nb_numa_nodes); query_numa_node_mem(node_mem); monitor_printf(mon, "%d nodes\n", nb_numa_nodes); for (i =3D 0; i < nb_numa_nodes; i++) { monitor_printf(mon, "node %d cpus:", i); - CPU_FOREACH(cpu) { - if (cpu->numa_node =3D=3D i) { - monitor_printf(mon, " %d", cpu->cpu_index); + for (cpu =3D cpu_list; cpu; cpu =3D cpu->next) { + if (cpu->value->has_props && cpu->value->props->has_node_id && + cpu->value->props->node_id =3D=3D i) { + monitor_printf(mon, " %" PRIi64, cpu->value->CPU); } } monitor_printf(mon, "\n"); monitor_printf(mon, "node %d size: %" PRId64 " MB\n", i, node_mem[i] >> 20); } + qapi_free_CpuInfoList(cpu_list); g_free(node_mem); } =20 --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689512048747.2656837856292; Mon, 5 Jun 2017 12:05:12 -0700 (PDT) Received: from localhost ([::1]:34738 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxJM-0006B1-KN for importer@patchew.org; Mon, 05 Jun 2017 15:05:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32854) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxEF-0001ty-7w for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxEE-00083U-66 for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57578) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxED-000838-TU for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:50 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D0F03C9B36; Mon, 5 Jun 2017 18:59:48 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5CAF05C540; Mon, 5 Jun 2017 18:59:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D0F03C9B36 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D0F03C9B36 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:23 -0300 Message-Id: <20170605185927.12111-7-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 05 Jun 2017 18:59:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 06/10] numa: move numa_node from CPUState into target specific classes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Mammedov , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Igor Mammedov Move vcpu's associated numa_node field out of generic CPUState into inherited classes that actually care about cpu<->numa mapping, i.e: ARMCPU, PowerPCCPU, X86CPU. Signed-off-by: Igor Mammedov Message-Id: <1496161442-96665-6-git-send-email-imammedo@redhat.com> [ehabkost: s/CPU is belonging to/CPU belongs to/ on comments] Signed-off-by: Eduardo Habkost --- include/qom/cpu.h | 2 -- target/arm/cpu.h | 2 ++ target/i386/cpu.h | 1 + target/ppc/cpu.h | 1 + hw/ppc/spapr.c | 2 +- hw/ppc/spapr_cpu_core.c | 4 +++- target/arm/cpu.c | 2 +- target/i386/cpu.c | 2 +- 8 files changed, 10 insertions(+), 6 deletions(-) diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 55214ce131..89ddb686fb 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -265,7 +265,6 @@ struct qemu_work_item; * @cpu_index: CPU index (informative). * @nr_cores: Number of cores within this CPU package. * @nr_threads: Number of threads within this CPU. - * @numa_node: NUMA node this CPU is belonging to. * @host_tid: Host thread ID. * @running: #true if CPU is currently running (lockless). * @has_waiter: #true if a CPU is currently waiting for the cpu_exec_end; @@ -314,7 +313,6 @@ struct CPUState { =20 int nr_cores; int nr_threads; - int numa_node; =20 struct QemuThread *thread; #ifdef _WIN32 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 13da5036bc..16a1e59615 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -704,6 +704,8 @@ struct ARMCPU { =20 ARMELChangeHook *el_change_hook; void *el_change_hook_opaque; + + int32_t node_id; /* NUMA node this CPU belongs to */ }; =20 static inline ARMCPU *arm_env_get_cpu(CPUARMState *env) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index c4602ca80d..cfe825f0a4 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1275,6 +1275,7 @@ struct X86CPU { =20 struct kvm_msrs *kvm_msr_buf; =20 + int32_t node_id; /* NUMA node this CPU belongs to */ int32_t socket_id; int32_t core_id; int32_t thread_id; diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h index 401e10e7da..d10808d9f4 100644 --- a/target/ppc/cpu.h +++ b/target/ppc/cpu.h @@ -1205,6 +1205,7 @@ struct PowerPCCPU { uint32_t compat_pvr; PPCVirtualHypervisor *vhyp; Object *intc; + int32_t node_id; /* NUMA node this CPU belongs to */ =20 /* Fields related to migration compatibility hacks */ bool pre_2_8_migration; diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 94563cd8f5..96471a5d89 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -191,7 +191,7 @@ static int spapr_fixup_cpu_numa_dt(void *fdt, int offse= t, CPUState *cs) cpu_to_be32(0x0), cpu_to_be32(0x0), cpu_to_be32(0x0), - cpu_to_be32(cs->numa_node), + cpu_to_be32(cpu->node_id), cpu_to_be32(index)}; =20 /* Advertise NUMA via ibm,associativity */ diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c index ff7058ecc0..029a14120e 100644 --- a/hw/ppc/spapr_cpu_core.c +++ b/hw/ppc/spapr_cpu_core.c @@ -184,15 +184,17 @@ static void spapr_cpu_core_realize(DeviceState *dev, = Error **errp) for (i =3D 0; i < cc->nr_threads; i++) { char id[32]; CPUState *cs; + PowerPCCPU *cpu; =20 obj =3D sc->threads + i * size; =20 object_initialize(obj, size, typename); cs =3D CPU(obj); + cpu =3D POWERPC_CPU(cs); cs->cpu_index =3D cc->core_id + i; =20 /* Set NUMA node for the threads belonged to core */ - cs->numa_node =3D sc->node_id; + cpu->node_id =3D sc->node_id; =20 snprintf(id, sizeof(id), "thread[%d]", i); object_property_add_child(OBJECT(sc), id, obj, &local_err); diff --git a/target/arm/cpu.c b/target/arm/cpu.c index e748097860..82d739f832 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1587,7 +1587,7 @@ static Property arm_cpu_properties[] =3D { DEFINE_PROP_UINT32("midr", ARMCPU, midr, 0), DEFINE_PROP_UINT64("mp-affinity", ARMCPU, mp_affinity, ARM64_AFFINITY_INVALID), - DEFINE_PROP_INT32("node-id", CPUState, numa_node, CPU_UNSET_NUMA_NODE_= ID), + DEFINE_PROP_INT32("node-id", ARMCPU, node_id, CPU_UNSET_NUMA_NODE_ID), DEFINE_PROP_END_OF_LIST() }; =20 diff --git a/target/i386/cpu.c b/target/i386/cpu.c index a41d595c23..ffb5267162 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -3986,7 +3986,7 @@ static Property x86_cpu_properties[] =3D { DEFINE_PROP_INT32("core-id", X86CPU, core_id, -1), DEFINE_PROP_INT32("socket-id", X86CPU, socket_id, -1), #endif - DEFINE_PROP_INT32("node-id", CPUState, numa_node, CPU_UNSET_NUMA_NODE_= ID), + DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID), DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false), { .name =3D "hv-spinlocks", .info =3D &qdev_prop_spinlocks }, DEFINE_PROP_BOOL("hv-relaxed", X86CPU, hyperv_relaxed_timing, false), --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689612765381.90212644528265; Mon, 5 Jun 2017 12:06:52 -0700 (PDT) Received: from localhost ([::1]:34747 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxKu-0007JN-RT for importer@patchew.org; Mon, 05 Jun 2017 15:06:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32867) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxEG-0001vX-SQ for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxEF-00084y-VR for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33894) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxEF-00084C-NH for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:51 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A8A3780B22; Mon, 5 Jun 2017 18:59:50 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 35D665C540; Mon, 5 Jun 2017 18:59:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A8A3780B22 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A8A3780B22 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:24 -0300 Message-Id: <20170605185927.12111-8-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 05 Jun 2017 18:59:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 07/10] spapr: cleanup spapr_fixup_cpu_numa_dt() usage X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Mammedov , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Igor Mammedov even though spapr_fixup_cpu_numa_dt() has no effect on FDT if numa is disabled, don't call it uselessly. It makes it obvious at call sites that function is needed only when numa is enabled. Signed-off-by: Igor Mammedov Message-Id: <1496161442-96665-7-git-send-email-imammedo@redhat.com> Reviewed-by: Greg Kurz Signed-off-by: Eduardo Habkost --- hw/ppc/spapr.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 96471a5d89..70eb60efed 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -182,10 +182,8 @@ static int spapr_fixup_cpu_smt_dt(void *fdt, int offse= t, PowerPCCPU *cpu, return ret; } =20 -static int spapr_fixup_cpu_numa_dt(void *fdt, int offset, CPUState *cs) +static int spapr_fixup_cpu_numa_dt(void *fdt, int offset, PowerPCCPU *cpu) { - int ret =3D 0; - PowerPCCPU *cpu =3D POWERPC_CPU(cs); int index =3D ppc_get_vcpu_dt_id(cpu); uint32_t associativity[] =3D {cpu_to_be32(0x5), cpu_to_be32(0x0), @@ -195,12 +193,8 @@ static int spapr_fixup_cpu_numa_dt(void *fdt, int offs= et, CPUState *cs) cpu_to_be32(index)}; =20 /* Advertise NUMA via ibm,associativity */ - if (nb_numa_nodes > 1) { - ret =3D fdt_setprop(fdt, offset, "ibm,associativity", associativit= y, + return fdt_setprop(fdt, offset, "ibm,associativity", associativity, sizeof(associativity)); - } - - return ret; } =20 /* Populate the "ibm,pa-features" property */ @@ -325,9 +319,11 @@ static int spapr_fixup_cpu_dt(void *fdt, sPAPRMachineS= tate *spapr) return ret; } =20 - ret =3D spapr_fixup_cpu_numa_dt(fdt, offset, cs); - if (ret < 0) { - return ret; + if (nb_numa_nodes > 1) { + ret =3D spapr_fixup_cpu_numa_dt(fdt, offset, cpu); + if (ret < 0) { + return ret; + } } =20 ret =3D spapr_fixup_cpu_smt_dt(fdt, offset, cpu, compat_smt); @@ -542,7 +538,9 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *f= dt, int offset, _FDT((fdt_setprop(fdt, offset, "ibm,pft-size", pft_size_prop, sizeof(pft_size_prop)))); =20 - _FDT(spapr_fixup_cpu_numa_dt(fdt, offset, cs)); + if (nb_numa_nodes > 1) { + _FDT(spapr_fixup_cpu_numa_dt(fdt, offset, cpu)); + } =20 _FDT(spapr_fixup_cpu_smt_dt(fdt, offset, cpu, compat_smt)); =20 --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689674947175.10810663846655; Mon, 5 Jun 2017 12:07:54 -0700 (PDT) Received: from localhost ([::1]:34750 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxLy-0008Bi-D9 for importer@patchew.org; Mon, 05 Jun 2017 15:07:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32898) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxEM-00020i-Ig for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxEJ-00086s-Cy for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36896) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxEJ-00086V-7b for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:55 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 31B3DC04B929; Mon, 5 Jun 2017 18:59:54 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E1A486E0D; Mon, 5 Jun 2017 18:59:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 31B3DC04B929 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 31B3DC04B929 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:25 -0300 Message-Id: <20170605185927.12111-9-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 05 Jun 2017 18:59:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 08/10] qemu.py: Don't set _popen=None on error/shutdown X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Keep the Popen object around to we can query its exit code later. To keep the existing 'self._popen is None' checks working, add a is_running() method, that will check if the process is still running. Signed-off-by: Eduardo Habkost Message-Id: <20170526181200.17227-2-ehabkost@redhat.com> Reviewed-by: Markus Armbruster Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 6d1b6230b7..16934f1e02 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -85,8 +85,11 @@ class QEMUMachine(object): return raise =20 + def is_running(self): + return self._popen and (self._popen.returncode is None) + def get_pid(self): - if not self._popen: + if not self.is_running(): return None return self._popen.pid =20 @@ -128,16 +131,16 @@ class QEMUMachine(object): stderr=3Dsubprocess.STDOUT, she= ll=3DFalse) self._post_launch() except: - if self._popen: + if self.is_running(): self._popen.kill() + self._popen.wait() self._load_io_log() self._post_shutdown() - self._popen =3D None raise =20 def shutdown(self): '''Terminate the VM and clean up''' - if not self._popen is None: + if self.is_running(): try: self._qmp.cmd('quit') self._qmp.close() @@ -149,7 +152,6 @@ class QEMUMachine(object): sys.stderr.write('qemu received signal %i: %s\n' % (-exitc= ode, ' '.join(self._args))) self._load_io_log() self._post_shutdown() - self._popen =3D None =20 underscore_to_dash =3D string.maketrans('_', '-') def qmp(self, cmd, conv_keys=3DTrue, **args): --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689570072188.29610379609835; Mon, 5 Jun 2017 12:06:10 -0700 (PDT) Received: from localhost ([::1]:34746 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxKI-0006rQ-L6 for importer@patchew.org; Mon, 05 Jun 2017 15:06:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32897) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxEM-00020h-IU for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxEL-00087c-7c for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:32976) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxEL-00087B-2C for qemu-devel@nongnu.org; Mon, 05 Jun 2017 14:59:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 076EB142A70; Mon, 5 Jun 2017 18:59:56 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8FD2D86E0F; Mon, 5 Jun 2017 18:59:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 076EB142A70 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 076EB142A70 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:26 -0300 Message-Id: <20170605185927.12111-10-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 05 Jun 2017 18:59:56 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 09/10] qemu.py: Add QEMUMachine.exitcode() method X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Allow the exit code of QEMU to be queried by scripts. Signed-off-by: Eduardo Habkost Message-Id: <20170526181200.17227-3-ehabkost@redhat.com> Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/qemu.py b/scripts/qemu.py index 16934f1e02..880e3e8219 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -88,6 +88,11 @@ class QEMUMachine(object): def is_running(self): return self._popen and (self._popen.returncode is None) =20 + def exitcode(self): + if self._popen is None: + return None + return self._popen.returncode + def get_pid(self): if not self.is_running(): return None --=20 2.11.0.259.g40922b1 From nobody Wed Nov 5 15:55:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1496689757446550.7867793052029; Mon, 5 Jun 2017 12:09:17 -0700 (PDT) Received: from localhost ([::1]:34758 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxNK-0000pk-RF for importer@patchew.org; Mon, 05 Jun 2017 15:09:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32931) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dHxEX-0002Bu-Kq for qemu-devel@nongnu.org; Mon, 05 Jun 2017 15:00:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dHxEP-0008A6-Ft for qemu-devel@nongnu.org; Mon, 05 Jun 2017 15:00:09 -0400 Received: from mx1.redhat.com ([209.132.183.28]:10092) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dHxEP-00089Q-5F for qemu-devel@nongnu.org; Mon, 05 Jun 2017 15:00:01 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0A40E19CBD1; Mon, 5 Jun 2017 19:00:00 +0000 (UTC) Received: from localhost (ovpn-116-6.gru2.redhat.com [10.97.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58C2A78C0B; Mon, 5 Jun 2017 18:59:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0A40E19CBD1 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 0A40E19CBD1 From: Eduardo Habkost To: Peter Maydell , Stefan Hajnoczi Date: Mon, 5 Jun 2017 15:59:27 -0300 Message-Id: <20170605185927.12111-11-ehabkost@redhat.com> In-Reply-To: <20170605185927.12111-1-ehabkost@redhat.com> References: <20170605185927.12111-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 05 Jun 2017 19:00:00 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 10/10] scripts: Test script to look for -device crashes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Test code to check if we can crash QEMU using -device. It will test all accel/machine/device combinations by default, which may take a few hours (it's more than 90k test cases). There's a "-r" option that makes it test a random sample of combinations. The scripts contains a whitelist for: 1) known error messages that make QEMU exit cleanly; 2) known QEMU crashes. This is the behavior when the script finds a failure: * Known clean (exitcode=3D1) errors generate DEBUG messages (hidden by default) * Unknown clean (exitcode=3D1) errors will generate INFO messages (visible by default) * Known crashes generate error messages, but are not fatal (unless --strict mode is used) * Unknown crashes generate fatal error messages Having an updated whitelist of known clean errors is useful to make the script less verbose and run faster when in --quick mode, but the whitelist doesn't need to be always up to date. Signed-off-by: Eduardo Habkost Message-Id: <20170526181200.17227-4-ehabkost@redhat.com> Signed-off-by: Eduardo Habkost --- scripts/device-crash-test | 624 ++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 624 insertions(+) create mode 100755 scripts/device-crash-test diff --git a/scripts/device-crash-test b/scripts/device-crash-test new file mode 100755 index 0000000000..5f90e9bb54 --- /dev/null +++ b/scripts/device-crash-test @@ -0,0 +1,624 @@ +#!/usr/bin/env python2.7 +# +# Copyright (c) 2017 Red Hat Inc +# +# Author: +# Eduardo Habkost +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +""" +Run QEMU with all combinations of -machine and -device types, +check for crashes and unexpected errors. +""" + +import sys +import os +import glob +import logging +import traceback +import re +import random +import argparse +from itertools import chain + +sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'scripts')) +from qemu import QEMUMachine + +logger =3D logging.getLogger('device-crash-test') +dbg =3D logger.debug + + +# Purposes of the following whitelist: +# * Avoiding verbose log messages when we find known non-fatal +# (exitcode=3D1) errors +# * Avoiding fatal errors when we find known crashes +# * Skipping machines/devices that are known not to work out of +# the box, when running in --quick mode +# +# Keeping the whitelist updated is desirable, but not required, +# because unexpected cases where QEMU exits with exitcode=3D1 will +# just trigger a INFO message. + +# Valid whitelist entry keys: +# * accel: regexp, full match only +# * machine: regexp, full match only +# * device: regexp, full match only +# * log: regexp, partial match allowed +# * exitcode: if not present, defaults to 1. If None, matches any exitcode +# * warn: if True, matching failures will be logged as warnings +# * expected: if True, QEMU is expected to always fail every time +# when testing the corresponding test case +# * loglevel: log level of log output when there's a match. +ERROR_WHITELIST =3D [ + # Machines that won't work out of the box: + # MACHINE | ERROR MESSAGE + {'machine':'niagara', 'expected':True}, # Unable to load a firmw= are for -M niagara + {'machine':'boston', 'expected':True}, # Please provide either = a -kernel or -bios argument + {'machine':'leon3_generic', 'expected':True}, # Can't read bios image = (null) + + # devices that don't work out of the box because they require extra op= tions to "-device DEV": + # DEVICE | ERROR MESSAGE + {'device':'.*-(i386|x86_64)-cpu', 'expected':True}, # CPU socket-id= is not set + {'device':'ARM,bitband-memory', 'expected':True}, # source-memory= property not set + {'device':'arm.cortex-a9-global-timer', 'expected':True}, # a9_gtimer_= realize: num-cpu must be between 1 and 4 + {'device':'arm_mptimer', 'expected':True}, # num-cpu must = be between 1 and 4 + {'device':'armv7m', 'expected':True}, # memory proper= ty was not set + {'device':'aspeed.scu', 'expected':True}, # Unknown silic= on revision: 0x0 + {'device':'aspeed.sdmc', 'expected':True}, # Unknown silic= on revision: 0x0 + {'device':'bcm2835-dma', 'expected':True}, # bcm2835_dma_r= ealize: required dma-mr link not found: Property '.dma-mr' not found + {'device':'bcm2835-fb', 'expected':True}, # bcm2835_fb_re= alize: required vcram-base property not set + {'device':'bcm2835-mbox', 'expected':True}, # bcm2835_mbox_= realize: required mbox-mr link not found: Property '.mbox-mr' not found + {'device':'bcm2835-peripherals', 'expected':True}, # bcm2835_perip= herals_realize: required ram link not found: Property '.ram' not found + {'device':'bcm2835-property', 'expected':True}, # bcm2835_prope= rty_realize: required fb link not found: Property '.fb' not found + {'device':'bcm2835_gpio', 'expected':True}, # bcm2835_gpio_= realize: required sdhci link not found: Property '.sdbus-sdhci' not found + {'device':'bcm2836', 'expected':True}, # bcm2836_reali= ze: required ram link not found: Property '.ram' not found + {'device':'cfi.pflash01', 'expected':True}, # attribute "se= ctor-length" not specified or zero. + {'device':'cfi.pflash02', 'expected':True}, # attribute "se= ctor-length" not specified or zero. + {'device':'icp', 'expected':True}, # icp_realize: = required link 'xics' not found: Property '.xics' not found + {'device':'ics', 'expected':True}, # ics_base_real= ize: required link 'xics' not found: Property '.xics' not found + # "-device ide-cd" does work on more recent QEMU versions, so it doesn= 't have expected=3DTrue + {'device':'ide-cd'}, # No drive specif= ied + {'device':'ide-drive', 'expected':True}, # No drive spec= ified + {'device':'ide-hd', 'expected':True}, # No drive spec= ified + {'device':'ipmi-bmc-extern', 'expected':True}, # IPMI external= bmc requires chardev attribute + {'device':'isa-debugcon', 'expected':True}, # Can't create = serial device, empty char device + {'device':'isa-ipmi-bt', 'expected':True}, # IPMI device r= equires a bmc attribute to be set + {'device':'isa-ipmi-kcs', 'expected':True}, # IPMI device r= equires a bmc attribute to be set + {'device':'isa-parallel', 'expected':True}, # Can't create = serial device, empty char device + {'device':'isa-serial', 'expected':True}, # Can't create = serial device, empty char device + {'device':'ivshmem', 'expected':True}, # You must spec= ify either 'shm' or 'chardev' + {'device':'ivshmem-doorbell', 'expected':True}, # You must spec= ify a 'chardev' + {'device':'ivshmem-plain', 'expected':True}, # You must spec= ify a 'memdev' + {'device':'kvm-pci-assign', 'expected':True}, # no host devic= e specified + {'device':'loader', 'expected':True}, # please includ= e valid arguments + {'device':'nand', 'expected':True}, # Unsupported N= AND block size 0x1 + {'device':'nvdimm', 'expected':True}, # 'memdev' prop= erty is not set + {'device':'nvme', 'expected':True}, # Device initia= lization failed + {'device':'pc-dimm', 'expected':True}, # 'memdev' prop= erty is not set + {'device':'pci-bridge', 'expected':True}, # Bridge chassi= s not specified. Each bridge is required to be assigned a unique chassis id= > 0. + {'device':'pci-bridge-seat', 'expected':True}, # Bridge chassi= s not specified. Each bridge is required to be assigned a unique chassis id= > 0. + {'device':'pci-serial', 'expected':True}, # Can't create = serial device, empty char device + {'device':'pci-serial-2x', 'expected':True}, # Can't create = serial device, empty char device + {'device':'pci-serial-4x', 'expected':True}, # Can't create = serial device, empty char device + {'device':'pxa2xx-dma', 'expected':True}, # channels valu= e invalid + {'device':'pxb', 'expected':True}, # Bridge chassi= s not specified. Each bridge is required to be assigned a unique chassis id= > 0. + {'device':'scsi-block', 'expected':True}, # drive propert= y not set + {'device':'scsi-disk', 'expected':True}, # drive propert= y not set + {'device':'scsi-generic', 'expected':True}, # drive propert= y not set + {'device':'scsi-hd', 'expected':True}, # drive propert= y not set + {'device':'spapr-pci-host-bridge', 'expected':True}, # BUID not spec= ified for PHB + {'device':'spapr-pci-vfio-host-bridge', 'expected':True}, # BUID not s= pecified for PHB + {'device':'spapr-rng', 'expected':True}, # spapr-rng nee= ds an RNG backend! + {'device':'spapr-vty', 'expected':True}, # chardev prope= rty not set + {'device':'tpm-tis', 'expected':True}, # tpm_tis: back= end driver with id (null) could not be found + {'device':'unimplemented-device', 'expected':True}, # property 'siz= e' not specified or zero + {'device':'usb-braille', 'expected':True}, # Property char= dev is required + {'device':'usb-mtp', 'expected':True}, # x-root proper= ty must be configured + {'device':'usb-redir', 'expected':True}, # Parameter 'ch= ardev' is missing + {'device':'usb-serial', 'expected':True}, # Property char= dev is required + {'device':'usb-storage', 'expected':True}, # drive propert= y not set + {'device':'vfio-amd-xgbe', 'expected':True}, # -device vfio-= amd-xgbe: vfio error: wrong host device name + {'device':'vfio-calxeda-xgmac', 'expected':True}, # -device vfio-= calxeda-xgmac: vfio error: wrong host device name + {'device':'vfio-pci', 'expected':True}, # No provided h= ost device + {'device':'vfio-pci-igd-lpc-bridge', 'expected':True}, # VFIO dummy IS= A/LPC bridge must have address 1f.0 + {'device':'vhost-scsi.*', 'expected':True}, # vhost-scsi: m= issing wwpn + {'device':'vhost-vsock-device', 'expected':True}, # guest-cid pro= perty must be greater than 2 + {'device':'vhost-vsock-pci', 'expected':True}, # guest-cid pro= perty must be greater than 2 + {'device':'virtio-9p-ccw', 'expected':True}, # 9pfs device c= ouldn't find fsdev with the id =3D NULL + {'device':'virtio-9p-device', 'expected':True}, # 9pfs device c= ouldn't find fsdev with the id =3D NULL + {'device':'virtio-9p-pci', 'expected':True}, # 9pfs device c= ouldn't find fsdev with the id =3D NULL + {'device':'virtio-blk-ccw', 'expected':True}, # drive propert= y not set + {'device':'virtio-blk-device', 'expected':True}, # drive propert= y not set + {'device':'virtio-blk-device', 'expected':True}, # drive propert= y not set + {'device':'virtio-blk-pci', 'expected':True}, # drive propert= y not set + {'device':'virtio-crypto-ccw', 'expected':True}, # 'cryptodev' p= arameter expects a valid object + {'device':'virtio-crypto-device', 'expected':True}, # 'cryptodev' p= arameter expects a valid object + {'device':'virtio-crypto-pci', 'expected':True}, # 'cryptodev' p= arameter expects a valid object + {'device':'virtio-input-host-device', 'expected':True}, # evdev proper= ty is required + {'device':'virtio-input-host-pci', 'expected':True}, # evdev propert= y is required + {'device':'xen-pvdevice', 'expected':True}, # Device ID inv= alid, it must always be supplied + {'device':'vhost-vsock-ccw', 'expected':True}, # guest-cid pro= perty must be greater than 2 + {'device':'ALTR.timer', 'expected':True}, # "clock-freque= ncy" property must be provided + {'device':'zpci', 'expected':True}, # target must b= e defined + {'device':'pnv-(occ|icp|lpc)', 'expected':True}, # required link= 'xics' not found: Property '.xics' not found + {'device':'powernv-cpu-.*', 'expected':True}, # pnv_core_real= ize: required link 'xics' not found: Property '.xics' not found + + # ioapic devices are already created by pc and will fail: + {'machine':'q35|pc.*', 'device':'kvm-ioapic', 'expected':True}, # Only= 1 ioapics allowed + {'machine':'q35|pc.*', 'device':'ioapic', 'expected':True}, # Only= 1 ioapics allowed + + # KVM-specific devices shouldn't be tried without accel=3Dkvm: + {'accel':'(?!kvm).*', 'device':'kvmclock', 'expected':True}, + {'accel':'(?!kvm).*', 'device':'kvm-pci-assign', 'expected':True}, + + # xen-specific machines and devices: + {'accel':'(?!xen).*', 'machine':'xen.*', 'expected':True}, + {'accel':'(?!xen).*', 'device':'xen-.*', 'expected':True}, + + # this fails on some machine-types, but not all, so they don't have ex= pected=3DTrue: + {'device':'vmgenid'}, # vmgenid requires DMA write support in fw_cfg, = which this machine type does not provide + + # Silence INFO messages for errors that are common on multiple + # devices/machines: + {'log':r"No '[\w-]+' bus found for device '[\w-]+'"}, + {'log':r"images* must be given with the 'pflash' parameter"}, + {'log':r"(Guest|ROM|Flash|Kernel) image must be specified"}, + {'log':r"[cC]ould not load [\w ]+ (BIOS|bios) '[\w-]+\.bin'"}, + {'log':r"Couldn't find rom image '[\w-]+\.bin'"}, + {'log':r"speed mismatch trying to attach usb device"}, + {'log':r"Can't create a second ISA bus"}, + {'log':r"duplicate fw_cfg file name"}, + # sysbus-related error messages: most machines reject most dynamic sys= bus devices: + {'log':r"Option '-device [\w.,-]+' cannot be handled by this machine"}, + {'log':r"Device [\w.,-]+ is not supported by this machine yet"}, + {'log':r"Device [\w.,-]+ can not be dynamically instantiated"}, + {'log':r"Platform Bus: Can not fit MMIO region of size "}, + # other more specific errors we will ignore: + {'device':'allwinner-a10', 'log':"Unsupported NIC model:"}, + {'device':'.*-spapr-cpu-core', 'log':r"CPU core type should be"}, + {'log':r"MSI(-X)? is not supported by interrupt controller"}, + {'log':r"pxb-pcie? devices cannot reside on a PCIe? bus"}, + {'log':r"Ignoring smp_cpus value"}, + {'log':r"sd_init failed: Drive 'sd0' is already in use because it has = been automatically connected to another device"}, + {'log':r"This CPU requires a smaller page size than the system is usin= g"}, + {'log':r"MSI-X support is mandatory in the S390 architecture"}, + {'log':r"rom check and register reset failed"}, + {'log':r"Unable to initialize GIC, CPUState for CPU#0 not valid"}, + {'log':r"Multiple VT220 operator consoles are not supported"}, + {'log':r"core 0 already populated"}, + {'log':r"could not find stage1 bootloader"}, + + # other exitcode=3D1 failures not listed above will just generate INFO= messages: + {'exitcode':1, 'loglevel':logging.INFO}, + + # KNOWN CRASHES: + # Known crashes will generate error messages, but won't be fatal. + # Those entries must be removed once we fix the crashes. + {'exitcode':-6, 'log':r"Device 'serial0' is in use", 'loglevel':loggin= g.ERROR}, + {'exitcode':-6, 'log':r"spapr_rtas_register: Assertion .*rtas_table\[t= oken\]\.name.* failed", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"qemu_net_client_setup: Assertion `!peer->peer'= failed", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r'RAMBlock "[\w.-]+" already registered', 'logle= vel':logging.ERROR}, + {'exitcode':-6, 'log':r"find_ram_offset: Assertion `size !=3D 0' faile= d.", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"puv3_load_kernel: Assertion `kernel_filename != =3D NULL' failed", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"add_cpreg_to_hashtable: code should not be rea= ched", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"qemu_alloc_display: Assertion `surface->image = !=3D NULL' failed", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"Unexpected error in error_set_from_qdev_prop_e= rror", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"Object .* is not an instance of type spapr-mac= hine", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"Object .* is not an instance of type generic-p= c-machine", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"Object .* is not an instance of type e500-ccsr= ", 'loglevel':logging.ERROR}, + {'exitcode':-6, 'log':r"vmstate_register_with_alias_id: Assertion `!se= ->compat || se->instance_id =3D=3D 0' failed", 'loglevel':logging.ERROR}, + {'exitcode':-11, 'device':'stm32f205-soc', 'loglevel':logging.ERROR, '= expected':True}, + {'exitcode':-11, 'device':'xlnx,zynqmp', 'loglevel':logging.ERROR, 'ex= pected':True}, + {'exitcode':-11, 'device':'mips-cps', 'loglevel':logging.ERROR, 'expec= ted':True}, + {'exitcode':-11, 'device':'gus', 'loglevel':logging.ERROR, 'expected':= True}, + {'exitcode':-11, 'device':'a9mpcore_priv', 'loglevel':logging.ERROR, '= expected':True}, + {'exitcode':-11, 'device':'a15mpcore_priv', 'loglevel':logging.ERROR, = 'expected':True}, + {'exitcode':-11, 'device':'isa-serial', 'loglevel':logging.ERROR, 'exp= ected':True}, + {'exitcode':-11, 'device':'sb16', 'loglevel':logging.ERROR, 'expected'= :True}, + {'exitcode':-11, 'device':'cs4231a', 'loglevel':logging.ERROR, 'expect= ed':True}, + {'exitcode':-11, 'device':'arm-gicv3', 'loglevel':logging.ERROR, 'expe= cted':True}, + {'exitcode':-11, 'machine':'isapc', 'device':'.*-iommu', 'loglevel':lo= gging.ERROR, 'expected':True}, + + # everything else (including SIGABRT and SIGSEGV) will be a fatal erro= r: + {'exitcode':None, 'fatal':True, 'loglevel':logging.FATAL}, +] + + +def whitelistTestCaseMatch(wl, t): + """Check if a test case specification can match a whitelist entry + + This only checks if a whitelist entry is a candidate match + for a given test case, it won't check if the test case + results/output match the entry. See whitelistResultMatch(). + """ + return (('machine' not in wl or + 'machine' not in t or + re.match(wl['machine'] + '$', t['machine'])) and + ('accel' not in wl or + 'accel' not in t or + re.match(wl['accel'] + '$', t['accel'])) and + ('device' not in wl or + 'device' not in t or + re.match(wl['device'] + '$', t['device']))) + + +def whitelistCandidates(t): + """Generate the list of candidates that can match a test case""" + for i, wl in enumerate(ERROR_WHITELIST): + if whitelistTestCaseMatch(wl, t): + yield (i, wl) + + +def findExpectedResult(t): + """Check if there's an expected=3DTrue whitelist entry for a test case + + Returns (i, wl) tuple, where i is the index in + ERROR_WHITELIST and wl is the whitelist entry itself. + """ + for i, wl in whitelistCandidates(t): + if wl.get('expected'): + return (i, wl) + + +def whitelistResultMatch(wl, r): + """Check if test case results/output match a whitelist entry + + It is valid to call this function only if + whitelistTestCaseMatch() is True for the entry (e.g. on + entries returned by whitelistCandidates()) + """ + assert whitelistTestCaseMatch(wl, r['testcase']) + return ((wl.get('exitcode', 1) is None or + r['exitcode'] =3D=3D wl.get('exitcode', 1)) and + ('log' not in wl or + re.search(wl['log'], r['log'], re.MULTILINE))) + + +def checkResultWhitelist(r): + """Look up whitelist entry for a given test case result + + Returns (i, wl) tuple, where i is the index in + ERROR_WHITELIST and wl is the whitelist entry itself. + """ + for i, wl in whitelistCandidates(r['testcase']): + if whitelistResultMatch(wl, r): + return i, wl + + raise Exception("this should never happen") + + +def qemuOptsEscape(s): + """Escape option value QemuOpts""" + return s.replace(",", ",,") + + +def formatTestCase(t): + """Format test case info as "key=3Dvalue key=3Dvalue" for prettier log= ging output""" + return ' '.join('%s=3D%s' % (k, v) for k, v in t.items()) + + +def qomListTypeNames(vm, **kwargs): + """Run qom-list-types QMP command, return type names""" + types =3D vm.command('qom-list-types', **kwargs) + return [t['name'] for t in types] + + +def infoQDM(vm): + """Parse 'info qdm' output""" + args =3D {'command-line': 'info qdm'} + devhelp =3D vm.command('human-monitor-command', **args) + for l in devhelp.split('\n'): + l =3D l.strip() + if l =3D=3D '' or l.endswith(':'): + continue + d =3D {'name': re.search(r'name "([^"]+)"', l).group(1), + 'no-user': (re.search(', no-user', l) is not None)} + yield d + + +class QemuBinaryInfo(object): + def __init__(self, binary, devtype): + if devtype is None: + devtype =3D 'device' + + self.binary =3D binary + self._machine_info =3D {} + + dbg("devtype: %r", devtype) + args =3D ['-S', '-machine', 'none,accel=3Dkvm:tcg'] + dbg("querying info for QEMU binary: %s", binary) + vm =3D QEMUMachine(binary=3Dbinary, args=3Dargs) + vm.launch() + try: + self.alldevs =3D set(qomListTypeNames(vm, implements=3Ddevtype= , abstract=3DFalse)) + # there's no way to query DeviceClass::user_creatable using QM= P, + # so use 'info qdm': + self.no_user_devs =3D set([d['name'] for d in infoQDM(vm, ) if= d['no-user']]) + self.machines =3D list(m['name'] for m in vm.command('query-ma= chines')) + self.user_devs =3D self.alldevs.difference(self.no_user_devs) + self.kvm_available =3D vm.command('query-kvm')['enabled'] + finally: + vm.shutdown() + + def machineInfo(self, machine): + """Query for information on a specific machine-type + + Results are cached internally, in case the same machine- + type is queried multiple times. + """ + if machine in self._machine_info: + return self._machine_info[machine] + + mi =3D {} + args =3D ['-S', '-machine', '%s' % (machine)] + dbg("querying machine info for binary=3D%s machine=3D%s", self.bin= ary, machine) + vm =3D QEMUMachine(binary=3Dself.binary, args=3Dargs) + try: + vm.launch() + mi['runnable'] =3D True + except KeyboardInterrupt: + raise + except: + dbg("exception trying to run binary=3D%s machine=3D%s", self.b= inary, machine, exc_info=3Dsys.exc_info()) + dbg("log: %r", vm.get_log()) + mi['runnable'] =3D False + + vm.shutdown() + self._machine_info[machine] =3D mi + return mi + + +BINARY_INFO =3D {} + + +def getBinaryInfo(args, binary): + if binary not in BINARY_INFO: + BINARY_INFO[binary] =3D QemuBinaryInfo(binary, args.devtype) + return BINARY_INFO[binary] + + +def checkOneCase(args, testcase): + """Check one specific case + + Returns a dictionary containing failure information on error, + or None on success + """ + binary =3D testcase['binary'] + accel =3D testcase['accel'] + machine =3D testcase['machine'] + device =3D testcase['device'] + + dbg("will test: %r", testcase) + + args =3D ['-S', '-machine', '%s,accel=3D%s' % (machine, accel), + '-device', qemuOptsEscape(device)] + cmdline =3D ' '.join([binary] + args) + dbg("will launch QEMU: %s", cmdline) + vm =3D QEMUMachine(binary=3Dbinary, args=3Dargs) + + exc_traceback =3D None + try: + vm.launch() + except KeyboardInterrupt: + raise + except: + exc_traceback =3D traceback.format_exc() + dbg("Exception while running test case") + finally: + vm.shutdown() + ec =3D vm.exitcode() + log =3D vm.get_log() + + if exc_traceback is not None or ec !=3D 0: + return {'exc_traceback':exc_traceback, + 'exitcode':ec, + 'log':log, + 'testcase':testcase, + 'cmdline':cmdline} + + +def binariesToTest(args, testcase): + if args.qemu: + r =3D args.qemu + else: + r =3D glob.glob('./*-softmmu/qemu-system-*') + return r + + +def accelsToTest(args, testcase): + if getBinaryInfo(args, testcase['binary']).kvm_available: + yield 'kvm' + yield 'tcg' + + +def machinesToTest(args, testcase): + return getBinaryInfo(args, testcase['binary']).machines + + +def devicesToTest(args, testcase): + return getBinaryInfo(args, testcase['binary']).user_devs + + +TESTCASE_VARIABLES =3D [ + ('binary', binariesToTest), + ('accel', accelsToTest), + ('machine', machinesToTest), + ('device', devicesToTest), +] + + +def genCases1(args, testcases, var, fn): + """Generate new testcases for one variable + + If an existing item already has a variable set, don't + generate new items and just return it directly. This + allows the "-t" command-line option to be used to choose + a specific test case. + """ + for testcase in testcases: + if var in testcase: + yield testcase.copy() + else: + for i in fn(args, testcase): + t =3D testcase.copy() + t[var] =3D i + yield t + + +def genCases(args, testcase): + """Generate test cases for all variables + """ + cases =3D [testcase.copy()] + for var, fn in TESTCASE_VARIABLES: + dbg("var: %r, fn: %r", var, fn) + cases =3D genCases1(args, cases, var, fn) + return cases + + +def casesToTest(args, testcase): + cases =3D genCases(args, testcase) + if args.random: + cases =3D list(cases) + cases =3D random.sample(cases, min(args.random, len(cases))) + if args.debug: + cases =3D list(cases) + dbg("%d test cases to test", len(cases)) + if args.shuffle: + cases =3D list(cases) + random.shuffle(cases) + return cases + + +def logFailure(f, level): + t =3D f['testcase'] + logger.log(level, "failed: %s", formatTestCase(t)) + logger.log(level, "cmdline: %s", f['cmdline']) + for l in f['log'].strip().split('\n'): + logger.log(level, "log: %s", l) + logger.log(level, "exit code: %r", f['exitcode']) + if f['exc_traceback']: + logger.log(level, "exception:") + for l in f['exc_traceback'].split('\n'): + logger.log(level, " %s", l.rstrip('\n')) + + +def main(): + parser =3D argparse.ArgumentParser(description=3D"QEMU -device crash t= est") + parser.add_argument('-t', metavar=3D'KEY=3DVALUE', nargs=3D'*', + help=3D"Limit test cases to KEY=3DVALUE", + action=3D'append', dest=3D'testcases', default=3D[= ]) + parser.add_argument('-d', '--debug', action=3D'store_true', + help=3D'debug output') + parser.add_argument('-v', '--verbose', action=3D'store_true', default= =3DTrue, + help=3D'verbose output') + parser.add_argument('-q', '--quiet', dest=3D'verbose', action=3D'store= _false', + help=3D'non-verbose output') + parser.add_argument('-r', '--random', type=3Dint, metavar=3D'COUNT', + help=3D'run a random sample of COUNT test cases', + default=3D0) + parser.add_argument('--shuffle', action=3D'store_true', + help=3D'Run test cases in random order') + parser.add_argument('--dry-run', action=3D'store_true', + help=3D"Don't run any tests, just generate list") + parser.add_argument('-D', '--devtype', metavar=3D'TYPE', + help=3D"Test only device types that implement TYPE= ") + parser.add_argument('-Q', '--quick', action=3D'store_true', default=3D= True, + help=3D"Quick mode: skip test cases that are expec= ted to fail") + parser.add_argument('-F', '--full', action=3D'store_false', dest=3D'qu= ick', + help=3D"Full mode: test cases that are expected to= fail") + parser.add_argument('--strict', action=3D'store_true', dest=3D'strict', + help=3D"Treat all warnings as fatal") + parser.add_argument('qemu', nargs=3D'*', metavar=3D'QEMU', + help=3D'QEMU binary to run') + args =3D parser.parse_args() + + if args.debug: + lvl =3D logging.DEBUG + elif args.verbose: + lvl =3D logging.INFO + else: + lvl =3D logging.WARN + logging.basicConfig(stream=3Dsys.stdout, level=3Dlvl, format=3D'%(leve= lname)s: %(message)s') + + fatal_failures =3D [] + wl_stats =3D {} + skipped =3D 0 + total =3D 0 + + tc =3D {} + dbg("testcases: %r", args.testcases) + if args.testcases: + for t in chain(*args.testcases): + for kv in t.split(): + k, v =3D kv.split('=3D', 1) + tc[k] =3D v + + if len(binariesToTest(args, tc)) =3D=3D 0: + print >>sys.stderr, "No QEMU binary found" + parser.print_usage(sys.stderr) + return 1 + + for t in casesToTest(args, tc): + logger.info("running test case: %s", formatTestCase(t)) + total +=3D 1 + + expected_match =3D findExpectedResult(t) + if (args.quick and + (expected_match or + not getBinaryInfo(args, t['binary']).machineInfo(t['machi= ne'])['runnable'])): + dbg("skipped: %s", formatTestCase(t)) + skipped +=3D 1 + continue + + if args.dry_run: + continue + + try: + f =3D checkOneCase(args, t) + except KeyboardInterrupt: + break + + if f: + i, wl =3D checkResultWhitelist(f) + dbg("testcase: %r, whitelist match: %r", t, wl) + wl_stats.setdefault(i, []).append(f) + level =3D wl.get('loglevel', logging.DEBUG) + logFailure(f, level) + if wl.get('fatal') or (args.strict and level >=3D logging.WARN= ): + fatal_failures.append(f) + else: + dbg("success: %s", formatTestCase(t)) + if expected_match: + logger.warn("Didn't fail as expected: %s", formatTestCase(= t)) + + logger.info("Total: %d test cases", total) + if skipped: + logger.info("Skipped %d test cases", skipped) + + if args.debug: + stats =3D sorted([(len(wl_stats.get(i, [])), wl) for i, wl in enum= erate(ERROR_WHITELIST)]) + for count, wl in stats: + dbg("whitelist entry stats: %d: %r", count, wl) + + if fatal_failures: + for f in fatal_failures: + t =3D f['testcase'] + logger.error("Fatal failure: %s", formatTestCase(t)) + logger.error("Fatal failures on some machine/device combinations") + return 1 + +if __name__ =3D=3D '__main__': + sys.exit(main()) --=20 2.11.0.259.g40922b1