From nobody Sat Feb 7 06:35:30 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626205339672307.7977001029367; Tue, 13 Jul 2021 12:42:19 -0700 (PDT) Received: from localhost ([::1]:49564 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3OIM-0003CP-J6 for importer@patchew.org; Tue, 13 Jul 2021 15:42:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54778) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3OH8-0000oO-Ic; Tue, 13 Jul 2021 15:41:03 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]:46715) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m3OH4-0006Oq-AU; Tue, 13 Jul 2021 15:41:01 -0400 Received: by mail-pl1-x62e.google.com with SMTP id c15so11582980pls.13; Tue, 13 Jul 2021 12:40:57 -0700 (PDT) Received: from rekt.ibmuc.com ([191.193.148.134]) by smtp.gmail.com with ESMTPSA id o72sm20077506pfg.44.2021.07.13.12.40.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jul 2021 12:40:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AqJ/CANn+mLhqYcCkmI2qej991jLbuz2Di1qLnOGcV8=; b=jslYnNe23rnzuLK8IeGxfLkHlaFfq/mslSwvTCgLzkAOkB/d9vSK7H5MrYJ7V/7M9d cvpuGgJUjtNFQBjxDDNjj1ZIs/BW0o2j0b3bzLIfvkWswkdD1mGDF7qeVhcyeoH750fq b0DmpFXc7+ESKlKlUW9DU3sWxvbWMqIpxjwV6JfalHlvxDCJl+56j9CL9UTdz3mMCPX9 +jStBUQ1htmAQcfFpVTsZBQULiGAtkM2FLhgDBCnP5ym9ZLJuLcrEoeUrrAPoboFxjhF gDTFmpemcypuIY5x1sngTgiwXEPcLtLdZ08U4qSsCXEotMbqxynSDr/fhe8sWSSNbGDw xj0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AqJ/CANn+mLhqYcCkmI2qej991jLbuz2Di1qLnOGcV8=; b=en9XufI89iVsbMzwGntTmrcz6VDKArtDOw7uFj0yZDZ7sK53icCjSkiY0YMvJ3htBu re2KwIQ103oQ+jOkUcU3iTEK3AE+1doRRopNou8zKFP4n1ToXm0RM18zD8fJ9x1fwkDr CsxR176DBDRy0gFpFuCqgYb1V5eSUj3nE7i9+xIjDz0b/TId0IAYnwfeDQS0bnY8q6eE TdWT7tyelE/tRlTsM5BE9+3QCADYEahpXJrMyW6JRgFhXDXksDlPiS/yfVduj+a2oxEx BjHjtr0wF/TJi0VyMml0W+uMzalwYt3XaGLIx9Lvholsj/SwvGNEuW+4rUJMstkgRVqZ t8VA== X-Gm-Message-State: AOAM5308CPFfnWeKDM/JcnPrWiJzFonh0vEzAtyAR4f2hR2VccXmpi5K ygwcDWRSnc1A6z/YbU5GWKcEDFcXTlw/nM7Ym8E= X-Google-Smtp-Source: ABdhPJy1ygD8BGqH1wcEl3/9jyja7tI88yO9NbvUGLPj33M583r0x5FFi9c1d9DR86yMOZI56sOrVQ== X-Received: by 2002:a17:902:864c:b029:10d:8c9e:5f56 with SMTP id y12-20020a170902864cb029010d8c9e5f56mr4866182plt.8.1626205256185; Tue, 13 Jul 2021 12:40:56 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH v3 1/5] spapr_numa.c: split FORM1 code into helpers Date: Tue, 13 Jul 2021 16:40:41 -0300 Message-Id: <20210713194045.916263-2-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210713194045.916263-1-danielhb413@gmail.com> References: <20210713194045.916263-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=danielhb413@gmail.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626205341775100002 Content-Type: text/plain; charset="utf-8" The upcoming FORM2 NUMA affinity will support asymmetric NUMA topologies and doesn't need be concerned with all the legacy support for older pseries FORM1 guests. We're also not going to calculate associativity domains based on numa distance (via spapr_numa_define_associativity_domains) since the distances will be written directly into new DT properties. Let's split FORM1 code into its own functions to allow for easier insertion of FORM2 logic later on. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_numa.c | 61 +++++++++++++++++++++++++++++---------------- 1 file changed, 39 insertions(+), 22 deletions(-) diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 779f18b994..04a86f9b5b 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -155,6 +155,32 @@ static void spapr_numa_define_associativity_domains(Sp= aprMachineState *spapr) =20 } =20 +/* + * Set NUMA machine state data based on FORM1 affinity semantics. + */ +static void spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr, + MachineState *machine) +{ + bool using_legacy_numa =3D spapr_machine_using_legacy_numa(spapr); + + /* + * Legacy NUMA guests (pseries-5.1 and older, or guests with only + * 1 NUMA node) will not benefit from anything we're going to do + * after this point. + */ + if (using_legacy_numa) { + return; + } + + if (!spapr_numa_is_symmetrical(machine)) { + error_report("Asymmetrical NUMA topologies aren't supported " + "in the pSeries machine"); + exit(EXIT_FAILURE); + } + + spapr_numa_define_associativity_domains(spapr); +} + void spapr_numa_associativity_init(SpaprMachineState *spapr, MachineState *machine) { @@ -210,22 +236,7 @@ void spapr_numa_associativity_init(SpaprMachineState *= spapr, spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] =3D cpu_to_be3= 2(i); } =20 - /* - * Legacy NUMA guests (pseries-5.1 and older, or guests with only - * 1 NUMA node) will not benefit from anything we're going to do - * after this point. - */ - if (using_legacy_numa) { - return; - } - - if (!spapr_numa_is_symmetrical(machine)) { - error_report("Asymmetrical NUMA topologies aren't supported " - "in the pSeries machine"); - exit(EXIT_FAILURE); - } - - spapr_numa_define_associativity_domains(spapr); + spapr_numa_FORM1_affinity_init(spapr, machine); } =20 void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, @@ -302,12 +313,8 @@ int spapr_numa_write_assoc_lookup_arrays(SpaprMachineS= tate *spapr, void *fdt, return ret; } =20 -/* - * Helper that writes ibm,associativity-reference-points and - * max-associativity-domains in the RTAS pointed by @rtas - * in the DT @fdt. - */ -void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s) +static void spapr_numa_FORM1_write_rtas_dt(SpaprMachineState *spapr, + void *fdt, int rtas) { MachineState *ms =3D MACHINE(spapr); SpaprMachineClass *smc =3D SPAPR_MACHINE_GET_CLASS(spapr); @@ -365,6 +372,16 @@ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr= , void *fdt, int rtas) maxdomains, sizeof(maxdomains))); } =20 +/* + * Helper that writes ibm,associativity-reference-points and + * max-associativity-domains in the RTAS pointed by @rtas + * in the DT @fdt. + */ +void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s) +{ + spapr_numa_FORM1_write_rtas_dt(spapr, fdt, rtas); +} + static target_ulong h_home_node_associativity(PowerPCCPU *cpu, SpaprMachineState *spapr, target_ulong opcode, --=20 2.31.1 From nobody Sat Feb 7 06:35:30 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 162620534757714.142542737332633; Tue, 13 Jul 2021 12:42:27 -0700 (PDT) Received: from localhost ([::1]:50210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3OIU-0003co-E7 for importer@patchew.org; Tue, 13 Jul 2021 15:42:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54790) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3OHA-0000od-2c; Tue, 13 Jul 2021 15:41:05 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]:46716) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m3OH7-0006Pi-Iq; Tue, 13 Jul 2021 15:41:03 -0400 Received: by mail-pl1-x62f.google.com with SMTP id c15so11583033pls.13; Tue, 13 Jul 2021 12:40:59 -0700 (PDT) Received: from rekt.ibmuc.com ([191.193.148.134]) by smtp.gmail.com with ESMTPSA id o72sm20077506pfg.44.2021.07.13.12.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jul 2021 12:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uk3ggyAXUlG7fmdizF0mrMSzFALcRmmgb/G9sj6wbXI=; b=bIXo3X3zN1QpgzLcQHWkFJvs24kIA5d5jrcFxSXJgTGW5O7QzmGTheu4Aeuu15Gn6M lnnRITiy4NPtS1IIe+PefjAPyd5KFQj55EqO6vZJwqTdAgG7wPx17HHOA1DVSGe30Ek0 NkaR2bkZveFNbSYetH+PIjbAGg03KDg4tojQmSOI4o2Rkqiwgo2nTmVO1EgrPpUgec1U 5A0i7oU6eiKuhy/TeoEXkaUVW3qBgQF8+aMXYEuudhIdjEpuTzjBw35ccqNUKrdV6Vev ThVTYyRhoM4aFwaOlgl+1gDpxohQZCvVzt399wPRZQmiCVfD4moqFPundLMIwTLPxzb/ eyqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uk3ggyAXUlG7fmdizF0mrMSzFALcRmmgb/G9sj6wbXI=; b=diC8oGkIVoATRFWpbYboEOqhGnHeXsccydN6R3wZoTZvvJoqjwczg4MTiNBCZCL/cv Fxe6uqCzFxNc67ohUocJZNXER/B4ASWaHl5robPzsab51BRlnpe/PSdhG3JV0babvSaG J3TJRJ6+JdduJk3LKs8l/3gvBeKvdzobJUq7KdqcHbr26tCwJ5IHPZeKIQZbdFfes7D4 QJ/drWDZ4WwoZyLwWm6mH3FuAJg5GGKZccnl2u5fKG7ss/q0uPTItjiwRD0jNhCcg5M5 EY4ez7fuHoyduMCph55X14BQBrib2yFS3oS+j27c/dxB9DES86LWsBTMbON/nB8OQdyq X6eg== X-Gm-Message-State: AOAM533rRPfAOhRhXVywg41AoN7ziRDCycVN5XYVrZ+x3/+desJqYHfj PNudTw8z1gPpv9SKog++Akl9d3Q6hzD6qTwxzBs= X-Google-Smtp-Source: ABdhPJw8u95PUz+nyEtSBZLFW6WZRPlQU2UVjvtO3WQKC4qQQUYxTZ6cVRxmZbPzZG1GS+s97Sdk2g== X-Received: by 2002:a17:90a:6391:: with SMTP id f17mr893413pjj.27.1626205258527; Tue, 13 Jul 2021 12:40:58 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH v3 2/5] spapr: move NUMA data init to post-CAS Date: Tue, 13 Jul 2021 16:40:42 -0300 Message-Id: <20210713194045.916263-3-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210713194045.916263-1-danielhb413@gmail.com> References: <20210713194045.916263-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=danielhb413@gmail.com; helo=mail-pl1-x62f.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626205349848100001 Content-Type: text/plain; charset="utf-8" The pSeries machine will support a new NUMA affinity form, FORM2. This new FORM will be negotiated via ibm,architecture-vec5 during CAS. All artifacts and assumptions that are currently on use for FORM1 affinity will be deprecated in a guest that chooses to use FORM2. This means that we're going to wait until CAS to determine whether we're going to use FORM1 and FORM2. This patch does that by moving all NUMA data init functions to post-CAS time. spapr_numa_associativity_init() is moved from spapr_machine_init() to do_client_architecture_support(). Straightforward change since the initialization of spapr->numa_assoc_array is transparent to the guest. spapr_numa_write_rtas_dt() is more complex. The function is called from spapr_dt_rtas(), which in turned is called by spapr_build_fdt(). spapr_build_fdt() is called in two places: spapr_machine_reset() and do_client_architecture_support(). This means that we're writing RTAS nodes with NUMA artifacts without going through CAS first, and then writing it again post CAS. This is not an issue because, at this moment, we always write the same FORM1 NUMA affinity properties in the DT. With the upcoming FORM2 support, we're now reliant on guest choice to decide what to write. The proposed solution is to change spapr_numa_write_rtas_dt() to not write the DT until we're post-CAS. This is a benign guest visible change (a well behaved guest wouldn't bother to read NUMA properties before CAs), but to be on the safe side, let's wrap it with a machine class flag to skip this logic unless we're running with the latest machine type. This also means that FORM2 support will not be available for older machine types. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 6 +++--- hw/ppc/spapr_hcall.c | 4 ++++ hw/ppc/spapr_numa.c | 11 +++++++++++ include/hw/ppc/spapr.h | 1 + 4 files changed, 19 insertions(+), 3 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 81699d4f8b..725d0f3c6e 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2808,9 +2808,6 @@ static void spapr_machine_init(MachineState *machine) =20 spapr->gpu_numa_id =3D spapr_numa_initial_nvgpu_numa_id(machine); =20 - /* Init numa_assoc_array */ - spapr_numa_associativity_init(spapr, machine); - if ((!kvm_enabled() || kvmppc_has_cap_mmu_radix()) && ppc_type_check_compat(machine->cpu_type, CPU_POWERPC_LOGICAL_3_00,= 0, spapr->max_compat_pvr)) { @@ -4700,8 +4697,11 @@ DEFINE_SPAPR_MACHINE(6_1, "6.1", true); */ static void spapr_machine_6_0_class_options(MachineClass *mc) { + SpaprMachineClass *smc =3D SPAPR_MACHINE_CLASS(mc); + spapr_machine_6_1_class_options(mc); compat_props_add(mc->compat_props, hw_compat_6_0, hw_compat_6_0_len); + smc->pre_6_1_numa_affinity =3D true; } =20 DEFINE_SPAPR_MACHINE(6_0, "6.0", false); diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index 0e9a5b2e40..1cc88716c1 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -11,6 +11,7 @@ #include "helper_regs.h" #include "hw/ppc/spapr.h" #include "hw/ppc/spapr_cpu_core.h" +#include "hw/ppc/spapr_numa.h" #include "mmu-hash64.h" #include "cpu-models.h" #include "trace.h" @@ -1197,6 +1198,9 @@ target_ulong do_client_architecture_support(PowerPCCP= U *cpu, spapr->cas_pre_isa3_guest =3D !spapr_ovec_test(ov1_guest, OV1_PPC_3_00= ); spapr_ovec_cleanup(ov1_guest); =20 + /* Init numa_assoc_array */ + spapr_numa_associativity_init(spapr, MACHINE(spapr)); + /* * Ensure the guest asks for an interrupt mode we support; * otherwise terminate the boot. diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 04a86f9b5b..b0bd056546 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -379,6 +379,17 @@ static void spapr_numa_FORM1_write_rtas_dt(SpaprMachin= eState *spapr, */ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s) { + SpaprMachineClass *smc =3D SPAPR_MACHINE_GET_CLASS(spapr); + + /* + * pre-6.1 machine types were writing RTAS information before + * CAS. Check if that's case before changing their existing + * behavior. + */ + if (spapr_ovec_empty(spapr->ov5_cas) && !smc->pre_6_1_numa_affinity) { + return; + } + spapr_numa_FORM1_write_rtas_dt(spapr, fdt, rtas); } =20 diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index 637652ad16..068a29535a 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -145,6 +145,7 @@ struct SpaprMachineClass { hwaddr rma_limit; /* clamp the RMA to this size */ bool pre_5_1_assoc_refpoints; bool pre_5_2_numa_associativity; + bool pre_6_1_numa_affinity; =20 bool (*phb_placement)(SpaprMachineState *spapr, uint32_t index, uint64_t *buid, hwaddr *pio, --=20 2.31.1 From nobody Sat Feb 7 06:35:30 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626205691600341.5177116717898; Tue, 13 Jul 2021 12:48:11 -0700 (PDT) Received: from localhost ([::1]:36026 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3OO2-0004fr-7x for importer@patchew.org; Tue, 13 Jul 2021 15:48:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54802) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3OHC-0000pF-6c; Tue, 13 Jul 2021 15:41:06 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]:33339) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m3OH9-0006Qy-CL; Tue, 13 Jul 2021 15:41:05 -0400 Received: by mail-pf1-x42c.google.com with SMTP id m83so12891692pfd.0; Tue, 13 Jul 2021 12:41:02 -0700 (PDT) Received: from rekt.ibmuc.com ([191.193.148.134]) by smtp.gmail.com with ESMTPSA id o72sm20077506pfg.44.2021.07.13.12.40.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jul 2021 12:41:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1ZVGTC59DaSZLjnHdhrPnUSRYmJd9wlS7nS7IrWb0go=; b=jCmZQdqvhXJsJtq+gx87+eYl5jtylcHawEF3EGE9cvt5rj7JbmSvZIacPS7TbJbneO aFXM7SaXUgHWNGsWgwlEqna76e1Y+AqAdPJte/seBSHw1M65WALP7l4yvWUsW57VgQhb 2ignd9myY38nwtA7JWdPjmLqPZxT3aPRpjdPYdBlCUXd4a/2Kk+uDXKm+3mzcW6H9VD2 JYipdZaCWAHbbffWFD2HEtSq5ohO/i/XmEKgQoe8XGxQ5NETuvRpo0q8Nn+kr1MjqGeD 21YLzFjxiAVfg4Fogz7s8mG79acDpPiQ5kEMrHPClP5xXoYfK2qJRyiSKFMJxEBkLdJE dMsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1ZVGTC59DaSZLjnHdhrPnUSRYmJd9wlS7nS7IrWb0go=; b=Kt2Os7sxKM+NRXaYXBD0m+ZYEfXivs3TQd/reGqJmRW8Ow/v0FzQx1p9LBxJrNnvaj FHxlvN/1MK/TZocfGipP6g7rz/pA67Oa45zXXX4Cg603QIcllgazQ8OqfhXxjHEhnPw9 dcizqtfstbSZDtnFZg18dLhJX+dp8raLjTJFfK4DvvD+6/bn4DpENFFUW8O4fng8P/Yt QGKE9qXJEGkbzHmQcyK/mcXCcGtUn5X0lrq3s3aSp2f2xv3c36J+DbS0n5yiCOSuJalZ 6aNFRQweAQqOHDrHGIjZDi8E2gyUc9vcbjn48HToALfomgkLF7WxGz1Z5hONrovFItCS K+5Q== X-Gm-Message-State: AOAM530F1NyGyAcKREh+WRfhyuUNPTRLKRfRnB8KBhwRmxe1YgntJmmN C+EXoGwzYGDHqnvY4hAraOtFua08gXBlfv+lqes= X-Google-Smtp-Source: ABdhPJwsCh+mijzB7daxfLqBzBLRthsuSSbtxIuABl+Yrb6cXGweZcwhZ6C1TQIdWSiNIgu+dqCr3g== X-Received: by 2002:aa7:820d:0:b029:2f1:d22d:f21d with SMTP id k13-20020aa7820d0000b02902f1d22df21dmr6045476pfi.7.1626205260906; Tue, 13 Jul 2021 12:41:00 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH v3 3/5] spapr_numa.c: base FORM2 NUMA affinity support Date: Tue, 13 Jul 2021 16:40:43 -0300 Message-Id: <20210713194045.916263-4-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210713194045.916263-1-danielhb413@gmail.com> References: <20210713194045.916263-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=danielhb413@gmail.com; helo=mail-pf1-x42c.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626205692220100001 Content-Type: text/plain; charset="utf-8" The main feature of FORM2 affinity support is the separation of NUMA distances from ibm,associativity information. This allows for a more flexible and straightforward NUMA distance assignment without relying on complex associations between several levels of NUMA via ibm,associativity matches. Another feature is its extensibility. This base support contains the facilities for NUMA distance assignment, but in the future more facilities will be added for latency, performance, bandwidth and so on. This patch implements the base FORM2 affinity support as follows: - the use of FORM2 associativity is indicated by using bit 2 of byte 5 of ibm,architecture-vec-5. A FORM2 aware guest can choose to use FORM1 or FORM2 affinity. Setting both forms will default to FORM2. We rely on CAS to decide what to write in the DT, so all writing is postponed until client architecture support is done. This also mean that we're not advertising FORM2 for pseries-6.0 and older machine versions to prevent guest visible changes; - ibm,associativity-reference-points has a new semantic. Instead of being used to calculate distances via NUMA levels, it's now used to indicate the primary domain index in the ibm,associativity domain of each resource. In our case it's set to {0x4}, matching the position where we already place logical_domain_id; - two new RTAS DT artifacts are introduced: ibm,numa-lookup-index-table and ibm,numa-distance-table. The index table is used to list all the NUMA logical domains of the platform, in ascending order, and allows for spartial NUMA configurations (although QEMU ATM doesn't support that). ibm,numa-distance-table is an array that contains all the distances from the first NUMA node to all other nodes, then the second NUMA node distances to all other nodes and so on; - spapr_post_load changes: since we're adding a new NUMA affinity that isn't compatible with the existing one, migration must be handled accordingly because we can't be certain of whether the guest went through CAS in the source. The solution chosen is to initiate the NUMA associativity data in spapr_post_load() unconditionally. The worst case would be to write the DT twice if the guest is in pre-CAS stage. Otherwise, we're making sure that a FORM1 guest will have the spapr->numa_assoc_array initialized with the proper information based on user distance, something that we're not doing with FORM2. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 21 ++++++ hw/ppc/spapr_numa.c | 127 +++++++++++++++++++++++++++++++++++- include/hw/ppc/spapr_ovec.h | 1 + 3 files changed, 148 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 725d0f3c6e..1b7f706996 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1786,6 +1786,22 @@ static int spapr_post_load(void *opaque, int version= _id) return err; } =20 + /* + * NUMA data init is made in CAS time. There is no reliable + * way of telling whether the guest already went through CAS + * in the source due to how spapr_ov5_cas_needed works: a + * FORM1 guest can be migrated with ov5_cas empty regardless + * of going through CAS first. + * + * One solution is to always call numa_associativity_init. The + * downside is that a guest migrated before CAS will run + * numa_associativity_init again when going through it, but + * at least we're making sure spapr->numa_assoc_array will be + * initialized and hotplug operations won't fail in both before + * and after CAS migration cases. + */ + spapr_numa_associativity_init(spapr, MACHINE(spapr)); + return err; } =20 @@ -2752,6 +2768,11 @@ static void spapr_machine_init(MachineState *machine) =20 spapr_ovec_set(spapr->ov5, OV5_FORM1_AFFINITY); =20 + /* Do not advertise FORM2 support for pseries-6.0 and older */ + if (!smc->pre_6_1_numa_affinity) { + spapr_ovec_set(spapr->ov5, OV5_FORM2_AFFINITY); + } + /* advertise support for dedicated HP event source to guests */ if (spapr->use_hotplug_event_source) { spapr_ovec_set(spapr->ov5, OV5_HP_EVT); diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index b0bd056546..1c894bf877 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -202,6 +202,16 @@ void spapr_numa_associativity_init(SpaprMachineState *= spapr, spapr->numa_assoc_array[i][0] =3D cpu_to_be32(MAX_DISTANCE_REF_POI= NTS); spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] =3D cpu_to_be3= 2(i); =20 + /* + * For FORM2 affinity the initialization above is enough. No + * need to fill non-zero NUMA nodes with node_id because + * there is no associativity domain match to calculate + * NUMA distances in FORM2. + */ + if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { + continue; + } + /* * Fill all associativity domains of non-zero NUMA nodes with * node_id. This is required because the default value (0) is @@ -236,7 +246,16 @@ void spapr_numa_associativity_init(SpaprMachineState *= spapr, spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] =3D cpu_to_be3= 2(i); } =20 - spapr_numa_FORM1_affinity_init(spapr, machine); + /* + * We test for !FORM2 instead of testing for FORM1 because, + * as per spapr_ov5_cas_needed, setting FORM1 is not enough + * to get ov5_cas migrated, but setting FORM2 is. Since we're + * dealing with either FORM1 or FORM2, test for the option + * that is guaranteed to be set after a migration. + */ + if (!spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { + spapr_numa_FORM1_affinity_init(spapr, machine); + } } =20 void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, @@ -313,6 +332,107 @@ int spapr_numa_write_assoc_lookup_arrays(SpaprMachine= State *spapr, void *fdt, return ret; } =20 +static void spapr_numa_FORM2_write_rtas_tables(SpaprMachineState *spapr, + void *fdt, int rtas) +{ + MachineState *ms =3D MACHINE(spapr); + NodeInfo *numa_info =3D ms->numa_state->nodes; + int nb_numa_nodes =3D ms->numa_state->num_nodes; + int distance_table_entries =3D nb_numa_nodes * nb_numa_nodes; + g_autofree uint32_t *lookup_index_table =3D NULL; + g_autofree uint32_t *distance_table =3D NULL; + int src, dst, i, distance_table_size; + uint8_t *node_distances; + + /* + * ibm,numa-lookup-index-table: array with length and a + * list of NUMA ids present in the guest. + */ + lookup_index_table =3D g_new0(uint32_t, nb_numa_nodes + 1); + lookup_index_table[0] =3D cpu_to_be32(nb_numa_nodes); + + for (i =3D 0; i < nb_numa_nodes; i++) { + lookup_index_table[i + 1] =3D cpu_to_be32(i); + } + + _FDT(fdt_setprop(fdt, rtas, "ibm,numa-lookup-index-table", + lookup_index_table, + (nb_numa_nodes + 1) * sizeof(uint32_t))); + + /* + * ibm,numa-distance-table: contains all node distances. First + * element is the size of the table as uint32, followed up + * by all the uint8 distances from the first NUMA node, then all + * distances from the second NUMA node and so on. + * + * ibm,numa-lookup-index-table is used by guest to navigate this + * array because NUMA ids can be sparse (node 0 is the first, + * node 8 is the second ...). + */ + distance_table =3D g_new0(uint32_t, distance_table_entries + 1); + distance_table[0] =3D cpu_to_be32(distance_table_entries); + + node_distances =3D (uint8_t *)&distance_table[1]; + i =3D 0; + + for (src =3D 0; src < nb_numa_nodes; src++) { + for (dst =3D 0; dst < nb_numa_nodes; dst++) { + node_distances[i++] =3D numa_info[src].distance[dst]; + } + } + + distance_table_size =3D distance_table_entries * sizeof(uint8_t) + + sizeof(uint32_t); + _FDT(fdt_setprop(fdt, rtas, "ibm,numa-distance-table", + distance_table, distance_table_size)); +} + +/* + * This helper could be compressed in a single function with + * FORM1 logic since we're setting the same DT values, with the + * difference being a call to spapr_numa_FORM2_write_rtas_tables() + * in the end. The separation was made to avoid clogging FORM1 code + * which already has to deal with compat modes from previous + * QEMU machine types. + */ +static void spapr_numa_FORM2_write_rtas_dt(SpaprMachineState *spapr, + void *fdt, int rtas) +{ + MachineState *ms =3D MACHINE(spapr); + uint32_t number_nvgpus_nodes =3D spapr->gpu_numa_id - + spapr_numa_initial_nvgpu_numa_id(ms); + + /* + * In FORM2, ibm,associativity-reference-points will point to + * the element in the ibm,associativity array that contains the + * primary domain index. This value (in our case, the numa-id) is + * then used as an index to retrieve all other attributes of the + * node (distance, bandwidth, latency) via ibm,numa-lookup-index-table + * and other ibm,numa-*-table properties. + */ + uint32_t refpoints[] =3D { + cpu_to_be32(0x4), + }; + + uint32_t nr_refpoints =3D ARRAY_SIZE(refpoints); + uint32_t maxdomain =3D ms->numa_state->num_nodes + number_nvgpus_nodes; + uint32_t maxdomains[] =3D { + cpu_to_be32(4), + cpu_to_be32(maxdomain), + cpu_to_be32(maxdomain), + cpu_to_be32(maxdomain), + cpu_to_be32(maxdomain) + }; + + _FDT(fdt_setprop(fdt, rtas, "ibm,associativity-reference-points", + refpoints, nr_refpoints * sizeof(refpoints[0]))); + + _FDT(fdt_setprop(fdt, rtas, "ibm,max-associativity-domains", + maxdomains, sizeof(maxdomains))); + + spapr_numa_FORM2_write_rtas_tables(spapr, fdt, rtas); +} + static void spapr_numa_FORM1_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rtas) { @@ -390,6 +510,11 @@ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr= , void *fdt, int rtas) return; } =20 + if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { + spapr_numa_FORM2_write_rtas_dt(spapr, fdt, rtas); + return; + } + spapr_numa_FORM1_write_rtas_dt(spapr, fdt, rtas); } =20 diff --git a/include/hw/ppc/spapr_ovec.h b/include/hw/ppc/spapr_ovec.h index 48b716a060..c3e8b98e7e 100644 --- a/include/hw/ppc/spapr_ovec.h +++ b/include/hw/ppc/spapr_ovec.h @@ -49,6 +49,7 @@ typedef struct SpaprOptionVector SpaprOptionVector; /* option vector 5 */ #define OV5_DRCONF_MEMORY OV_BIT(2, 2) #define OV5_FORM1_AFFINITY OV_BIT(5, 0) +#define OV5_FORM2_AFFINITY OV_BIT(5, 2) #define OV5_HP_EVT OV_BIT(6, 5) #define OV5_HPT_RESIZE OV_BIT(6, 7) #define OV5_DRMEM_V2 OV_BIT(22, 0) --=20 2.31.1 From nobody Sat Feb 7 06:35:30 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626205439158911.9677236550249; Tue, 13 Jul 2021 12:43:59 -0700 (PDT) Received: from localhost ([::1]:56596 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3OJy-0007ro-3k for importer@patchew.org; Tue, 13 Jul 2021 15:43:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54816) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3OHD-0000t1-La; Tue, 13 Jul 2021 15:41:07 -0400 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]:43761) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m3OHB-0006Sy-54; Tue, 13 Jul 2021 15:41:07 -0400 Received: by mail-pg1-x536.google.com with SMTP id y4so20001667pgl.10; Tue, 13 Jul 2021 12:41:04 -0700 (PDT) Received: from rekt.ibmuc.com ([191.193.148.134]) by smtp.gmail.com with ESMTPSA id o72sm20077506pfg.44.2021.07.13.12.41.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jul 2021 12:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2k+CaPOOV81A+TC8lkent/ZTVe10msUzKY0Lu+Jglxg=; b=WNGLDOd04y8XA5UsXAtWMNQSSXwgkudwMP7x8D/tOkWmyl07+0xRHYT9XuYQjyCwLW HU2arC8y/9/pk5/hLdnBqHjCsbch2qiOu0cXpXnKiWphFTd5W9DvVQYu3yzyVvCUCUs7 rcfQAlwcqhBeD17iP3kkFpG5R0ge0zPYfbaYYxDE0YYsJA1VRcmGmFlcLxEHvg+qaIDv rhG6LXrtQglp8/yClCs3AY1kvwmCzw3RxW7X7o23obFo6Pa1BNyvvJcqfCkuOqjjzQgz oTMVk92p79HhgMavdwpRNoJj6gmMOx44PxPFctjcIWU7aPFXsqU11ViwGuPdBlYCNj+u hepw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2k+CaPOOV81A+TC8lkent/ZTVe10msUzKY0Lu+Jglxg=; b=I+NXYu0vjc9REv2zGwvHNKf/WnGvz6UjJcAtSVWD01BVyLvY1WQtnynWmZPFBozJcU LykqeLuZmFlqQ/fIBhwQBHTeTv23z5wCa51YAVqr1aCdXjAOG0VzRb8EAE5SCtpXrfh1 Y/9P62ZPglmzCEbER/fdWSBbqWuusna4H137qWUDGwUq5s7eOH9Po1KBUG2zp9H1WDsg o62bumHEyDyf8bjQhUcPrn92x3DqL/h3PjqPJDkPskvQ7K2dDsLuFsmvHVkusXO5QgTF XfhWH8NSxEwFz5EPIDFw3ybigELOqsLLOQwC2VUjqEw9HFg0pd5KGHC2VQ56GqmbIze9 zkww== X-Gm-Message-State: AOAM530gWQ4jObJqgb98FnMAI0u9f1RiLM+nOz/dM7eEaot0i+exmNRe 5fh35w8HQfEGDDOSSPCeQ4uuni9OnGj4vBVfpLQ= X-Google-Smtp-Source: ABdhPJzfNS9yMDv71L+FVdg2K7IQosTfORCag5QvePrhbJGCWKCN08CNBBrWKkt6aMKbvi6TznkhCQ== X-Received: by 2002:aa7:8148:0:b029:31b:10b4:f391 with SMTP id d8-20020aa781480000b029031b10b4f391mr5911171pfn.69.1626205263426; Tue, 13 Jul 2021 12:41:03 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH v3 4/5] spapr: simplify spapr_numa_associativity_init params Date: Tue, 13 Jul 2021 16:40:44 -0300 Message-Id: <20210713194045.916263-5-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210713194045.916263-1-danielhb413@gmail.com> References: <20210713194045.916263-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::536; envelope-from=danielhb413@gmail.com; helo=mail-pg1-x536.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626205441220100001 Content-Type: text/plain; charset="utf-8" When spapr_numa_associativity_init() was introduced it was being called from spapr_machine_init(), where we have pointers to a SpaprMachineState and a MachineState. Having both being passed as params spared us from calling a macro to get the MachineState. Previous patches moved the function away from spapr_machine_init(), and the new locations doesn't have a pointer to MachineState ready. This resulted in MACHINE(spapr) macro calls as the second parameter in both callers. Simplify the function by folding the 'MACHINE(spapr)' macro into the function body, which can now receive only a pointer to SpaprMachineState. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 2 +- hw/ppc/spapr_hcall.c | 2 +- hw/ppc/spapr_numa.c | 4 ++-- include/hw/ppc/spapr_numa.h | 3 +-- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 1b7f706996..76f1637959 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1800,7 +1800,7 @@ static int spapr_post_load(void *opaque, int version_= id) * initialized and hotplug operations won't fail in both before * and after CAS migration cases. */ - spapr_numa_associativity_init(spapr, MACHINE(spapr)); + spapr_numa_associativity_init(spapr); =20 return err; } diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index 1cc88716c1..a721d3edce 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -1199,7 +1199,7 @@ target_ulong do_client_architecture_support(PowerPCCP= U *cpu, spapr_ovec_cleanup(ov1_guest); =20 /* Init numa_assoc_array */ - spapr_numa_associativity_init(spapr, MACHINE(spapr)); + spapr_numa_associativity_init(spapr); =20 /* * Ensure the guest asks for an interrupt mode we support; diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 1c894bf877..bba5a984e4 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -181,9 +181,9 @@ static void spapr_numa_FORM1_affinity_init(SpaprMachine= State *spapr, spapr_numa_define_associativity_domains(spapr); } =20 -void spapr_numa_associativity_init(SpaprMachineState *spapr, - MachineState *machine) +void spapr_numa_associativity_init(SpaprMachineState *spapr) { + MachineState *machine =3D MACHINE(spapr); SpaprMachineClass *smc =3D SPAPR_MACHINE_GET_CLASS(spapr); int nb_numa_nodes =3D machine->numa_state->num_nodes; int i, j, max_nodes_with_gpus; diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index 6f9f02d3de..adaec8e163 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -22,8 +22,7 @@ * function. spapr_machine_init() is the only caller for it, and * it has both pointers resolved already. */ -void spapr_numa_associativity_init(SpaprMachineState *spapr, - MachineState *machine); +void spapr_numa_associativity_init(SpaprMachineState *spapr); void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s); void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, int offset, int nodeid); --=20 2.31.1 From nobody Sat Feb 7 06:35:30 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626205509468319.5846744667557; Tue, 13 Jul 2021 12:45:09 -0700 (PDT) Received: from localhost ([::1]:59216 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3OL5-0001CJ-G7 for importer@patchew.org; Tue, 13 Jul 2021 15:45:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3OHG-0000zt-7P; Tue, 13 Jul 2021 15:41:10 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]:46744) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m3OHE-0006Ux-Gd; Tue, 13 Jul 2021 15:41:09 -0400 Received: by mail-pf1-x431.google.com with SMTP id x16so20526549pfa.13; Tue, 13 Jul 2021 12:41:07 -0700 (PDT) Received: from rekt.ibmuc.com ([191.193.148.134]) by smtp.gmail.com with ESMTPSA id o72sm20077506pfg.44.2021.07.13.12.41.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jul 2021 12:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JQbLYJekVFXHz1j8A8zyhgeWXPfW3Rmufm8WbLN+aw4=; b=CTNZZkHg59AMBFZ7jpPlupOVjD3KtQ5YFnLvG/SKH6+ohAXndoaZMdUE9NZvM2Xlis v5mPm2rbAzP+EUWXnknG7QhzfkAue/48WcEpkKGB1g6/7qw3UUV2F6IPvZFQ05TL2EP6 8O51Ec9f4HiQ1anDn0PpSSCC6jDeSWxUpTdPrPszri/MezJDgU62A8UHXdYio3rtlIlt 7bqrhRW122LOKRu+oIBcr/OB7rK6GkNpElqUKnrf336Ny++daumkXa5Sar+mVaGyfg/p kYEUOwHGg5pITeFmHA8vDGoZTc+aAqf027nSZChnCh8yHtALZqMY1Q1dezmGjZs0aXwR Xhmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JQbLYJekVFXHz1j8A8zyhgeWXPfW3Rmufm8WbLN+aw4=; b=doh3GQusKy+C0DiUbrb5Z05ePbM+Ij9h6/vU6i3wRtKY4b9MbmkERKaZS9nR+Z7MZ3 4jOwUKlIZaPJwDnbZ8mMxtd9twN+8k8iWrI0OG7xphtd8LHuKKsQjSkmRbQjIhDVyfhz LmQ/7AMwctml852tM7isH27NxsvOs/+5k/NJSMZXF3FGR08QT3rgCq5Ia1lhGKWqvo5x EXRNDhm7gyunlucHfyFKdC2aq9NEXzPkpcbiiUoNSAvvOXRcA4U/WtHi4dpFAs8rshNa 0Lc7kajj/vsATmYVozhP2Dmt7ksfjNIGVEuCizFJwjrVPm2iQmd2LPVZuwqM6ZHjAq7H WRnw== X-Gm-Message-State: AOAM532GX/fYvnr/s8xGDlxFgzBjeXzjMnhNx3YmoS3SC+Hy1eCh3OXy fMyvaA33QtsYXxB0Ccz4PotsxSKyod+nHJxIazo= X-Google-Smtp-Source: ABdhPJye85PcMPJOl2ccsjbKMWida0mnORZgjeXpsXca4ai+z8oBpkR6ffAlvbODN9Xz5MfdhRHIdA== X-Received: by 2002:a63:110c:: with SMTP id g12mr5667058pgl.139.1626205266140; Tue, 13 Jul 2021 12:41:06 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH v3 5/5] spapr: move memory/cpu less check to spapr_numa_FORM1_affinity_init() Date: Tue, 13 Jul 2021 16:40:45 -0300 Message-Id: <20210713194045.916263-6-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210713194045.916263-1-danielhb413@gmail.com> References: <20210713194045.916263-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=danielhb413@gmail.com; helo=mail-pf1-x431.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626205511391100001 Content-Type: text/plain; charset="utf-8" FORM2 NUMA affinity is prepared to deal with memory/cpu less NUMA nodes. This is used by the DAX KMEM driver to locate a PAPR SCM device that has a different latency than the original NUMA node from the regular memory. Move this verification to FORM1 affinity code. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 33 --------------------------------- hw/ppc/spapr_numa.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+), 33 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 76f1637959..538c5bcd94 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2794,39 +2794,6 @@ static void spapr_machine_init(MachineState *machine) /* init CPUs */ spapr_init_cpus(spapr); =20 - /* - * check we don't have a memory-less/cpu-less NUMA node - * Firmware relies on the existing memory/cpu topology to provide the - * NUMA topology to the kernel. - * And the linux kernel needs to know the NUMA topology at start - * to be able to hotplug CPUs later. - */ - if (machine->numa_state->num_nodes) { - for (i =3D 0; i < machine->numa_state->num_nodes; ++i) { - /* check for memory-less node */ - if (machine->numa_state->nodes[i].node_mem =3D=3D 0) { - CPUState *cs; - int found =3D 0; - /* check for cpu-less node */ - CPU_FOREACH(cs) { - PowerPCCPU *cpu =3D POWERPC_CPU(cs); - if (cpu->node_id =3D=3D i) { - found =3D 1; - break; - } - } - /* memory-less and cpu-less node */ - if (!found) { - error_report( - "Memory-less/cpu-less nodes are not supported (node= %d)", - i); - exit(1); - } - } - } - - } - spapr->gpu_numa_id =3D spapr_numa_initial_nvgpu_numa_id(machine); =20 if ((!kvm_enabled() || kvmppc_has_cap_mmu_radix()) && diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index bba5a984e4..98740c732d 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -162,6 +162,39 @@ static void spapr_numa_FORM1_affinity_init(SpaprMachin= eState *spapr, MachineState *machine) { bool using_legacy_numa =3D spapr_machine_using_legacy_numa(spapr); + int i; + + /* + * check we don't have a memory-less/cpu-less NUMA node + * Firmware relies on the existing memory/cpu topology to provide the + * NUMA topology to the kernel. + * And the linux kernel needs to know the NUMA topology at start + * to be able to hotplug CPUs later. + */ + if (machine->numa_state->num_nodes) { + for (i =3D 0; i < machine->numa_state->num_nodes; ++i) { + /* check for memory-less node */ + if (machine->numa_state->nodes[i].node_mem =3D=3D 0) { + CPUState *cs; + int found =3D 0; + /* check for cpu-less node */ + CPU_FOREACH(cs) { + PowerPCCPU *cpu =3D POWERPC_CPU(cs); + if (cpu->node_id =3D=3D i) { + found =3D 1; + break; + } + } + /* memory-less and cpu-less node */ + if (!found) { + error_report( + "Memory-less/cpu-less nodes are not supported (node= %d)", + i); + exit(1); + } + } + } + } =20 /* * Legacy NUMA guests (pseries-5.1 and older, or guests with only --=20 2.31.1