From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623720887942429.79910263105944; Mon, 14 Jun 2021 18:34:47 -0700 (PDT) Received: from localhost ([::1]:46734 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsxyY-0001s5-Nk for importer@patchew.org; Mon, 14 Jun 2021 21:34:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39866) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxQ-00085C-HN; Mon, 14 Jun 2021 21:33:36 -0400 Received: from mail-qk1-x732.google.com ([2607:f8b0:4864:20::732]:41590) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxO-0002RP-Sy; Mon, 14 Jun 2021 21:33:36 -0400 Received: by mail-qk1-x732.google.com with SMTP id 5so2437911qkc.8; Mon, 14 Jun 2021 18:33:34 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t7etlJeKfXCsFCXTplqCexddhzu9ub2O7NLBBdXFErY=; b=lhGbJK3sPpSwHvZnDWvmCoS0DouIB9CF5yQA6aUn/l2UGz9c4gCb5ePyMmtQu+o0TG 55NJvqlcn6j7f+/Nm8R5sxsCxJq7ZWDp9JcPotnGF4q1t9q5+pQuKSSQziCsLC8LWdXK cZ4lZme6MCt7N2oJ7foMvJTgGj/aXHmZRxAyCz4UYf2LmQvujOrkxY/P1w5yhyLyE14d oEAC5FIeDIm9xH6GL31byip5lwMVBJn6YvqKnOIiNTEGAZk7xklNQBKBeFQE9JOCQLlA pLga9JoUlOYIzrSZ3AtBwWz1P2RjDTAbSXyPsMe67VaIChfNVf/sdEuspGqlGfq3dv+z lPxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t7etlJeKfXCsFCXTplqCexddhzu9ub2O7NLBBdXFErY=; b=G6eLjVZWS7iaypb986OlgpCuk80XCFf/0IpZ6vXnG0o7kj6ojXyBoGo5bQ95L3xR+P jkGx606DLWSUVd3VGJ81G8JvdxbSdDqKHXEGiC3C91jZ6FmoSCH1vJnBz41Bcy1Nm7Qe 0LBVM6ODO62e1M/9qZ6cpp+7lAWJ5QEfrDHYQCV8UxfYJYo2sqCE9sQWEjLmin3MwRt3 T7SM/2tfpKKof+612Nbdb1xLs1A6U8OlPlNCWeGMdhmylhOkiNnnSDLHIjz1LuNdnOvo p1fw7e6cY+8OWgrjMhF4QmjO0zDR9ew4eO1siKn2CI3Pw28BeOgfPZe5GqvCZjemiGy9 WyZg== X-Gm-Message-State: AOAM531pmqS0AuuiDAlUVh1xwWR3My+yum/diWMQT92GTvWxWT0o7tKU fZM7ZuU4hDKdakzOnFd+GZcvrpwY28tCBQ== X-Google-Smtp-Source: ABdhPJytDmkO1S2RgZdItiyBfl6Mn/Feosma1/QUT0jLsHiooqBRIHWoYjBWS9wlKVg8LcmJZr3KlA== X-Received: by 2002:a05:620a:146f:: with SMTP id j15mr2113704qkl.484.1623720813378; Mon, 14 Jun 2021 18:33:33 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 1/8] spapr: move NUMA data init to do_client_architecture_support() Date: Mon, 14 Jun 2021 22:33:02 -0300 Message-Id: <20210615013309.2833323-2-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::732; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x732.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The pSeries machine will support a new NUMA affinity form, FORM2. This new FORM will be negotiated via ibm,architecture-vec5 during CAS. All artifacts and assumptions that are currently on use for FORM1 affinity will be deprecated in a guest that chooses to use FORM2. This means that we're going to wait until CAS to determine whether we're going to use FORM1 and FORM2. This patch starts doing that by moving spapr_numa_associativity_init() from spapr_machine_init() to do_client_architecture_support(). Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 3 --- hw/ppc/spapr_hcall.c | 4 ++++ 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 4dd90b75cc..dd57b30cff 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2790,9 +2790,6 @@ static void spapr_machine_init(MachineState *machine) =20 spapr->gpu_numa_id =3D spapr_numa_initial_nvgpu_numa_id(machine); =20 - /* Init numa_assoc_array */ - spapr_numa_associativity_init(spapr, machine); - if ((!kvm_enabled() || kvmppc_has_cap_mmu_radix()) && ppc_type_check_compat(machine->cpu_type, CPU_POWERPC_LOGICAL_3_00,= 0, spapr->max_compat_pvr)) { diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index f25014afda..6d6ffcc92b 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -11,6 +11,7 @@ #include "helper_regs.h" #include "hw/ppc/spapr.h" #include "hw/ppc/spapr_cpu_core.h" +#include "hw/ppc/spapr_numa.h" #include "mmu-hash64.h" #include "cpu-models.h" #include "trace.h" @@ -1197,6 +1198,9 @@ target_ulong do_client_architecture_support(PowerPCCP= U *cpu, spapr->cas_pre_isa3_guest =3D !spapr_ovec_test(ov1_guest, OV1_PPC_3_00= ); spapr_ovec_cleanup(ov1_guest); =20 + /* Init numa_assoc_array */ + spapr_numa_associativity_init(spapr, MACHINE(spapr)); + /* * Ensure the guest asks for an interrupt mode we support; * otherwise terminate the boot. --=20 2.31.1 From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623721065949524.289191836675; Mon, 14 Jun 2021 18:37:45 -0700 (PDT) Received: from localhost ([::1]:53592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsy1Q-0006Sb-SJ for importer@patchew.org; Mon, 14 Jun 2021 21:37:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39886) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxT-0008Aq-6r; Mon, 14 Jun 2021 21:33:39 -0400 Received: from mail-qv1-xf35.google.com ([2607:f8b0:4864:20::f35]:39595) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxR-0002So-7H; Mon, 14 Jun 2021 21:33:38 -0400 Received: by mail-qv1-xf35.google.com with SMTP id u14so18635757qvq.6; Mon, 14 Jun 2021 18:33:36 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AqJ/CANn+mLhqYcCkmI2qej991jLbuz2Di1qLnOGcV8=; b=u4LeJcGt0f+FZDmmAX0KhHnDur1WL2obCsS/dsqUa8dYBqLQUSOlLpXF5cV+cVvjur V0doKp4xeq8v0HAdCM/frJ+8+mC1LPv0RPyQFY5HggGVaWagGO0flxGa1t9iN7+WYW/P CWm+mrzlenD+r1VmNM4xLc/NVp5cs2U1LLjPon9UgdS1Qe0GKlBWRRrFdaeTc0pptHq2 DZ+d6f4plov46Y1jU+XEMYSef58F0tIEnpxCnWTpbrth+Gyz4BRwKEqbAIymABKIGxSV +TKg72Va81DejSXHSWflslzYnLDHfHqlBfyp5iLZGsEFKORqc3oJBYkYeWSy/bzPr0zx 9APw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AqJ/CANn+mLhqYcCkmI2qej991jLbuz2Di1qLnOGcV8=; b=Z0sTJ/Y53XRuvnJwvLK31T/whA3rLVloRTN3WhJTwwlzxoQd+3b6+584rmdpsA9IX/ 8JA1XrI/kKxOETmjMJ+JbEUodpbByhKntiPu7Tgoq870wEeE/lKG26Re7LYgXkIjTfzR IULxGwQRiMZymPSHPJrui/oYH8j50mUhmKdBV6881qmKZgtHRZkRut84Llr+fv0j/wYZ DyavjUr5b5m85E8OiONEf3pYbE8bJ/ITUbiVjrRAxyOlOxf14YHMcjxZ2iRAmM+npmWH o3WhRPCvKBD7CLS9giBxdb4s9PEnMfPnqtHtXAgysUaX87nZsI5LtC1cbVsAryMkpo4S nNgA== X-Gm-Message-State: AOAM532H4oMV/qpDr88jTuYXpJO/wschcPlYUK7cBVrjmgpNQfK/L/wq qenaLN4PxXQWlqgXFPcNE58iEEuVfbMnzA== X-Google-Smtp-Source: ABdhPJzBKbBABZSBm/mn4ZbcbrK2Kll1HVHpunX959TlJ4u8drl02XuBI4LYJBt1cw9bPtCemOEQeQ== X-Received: by 2002:a0c:c30e:: with SMTP id f14mr2332776qvi.19.1623720815593; Mon, 14 Jun 2021 18:33:35 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 2/8] spapr_numa.c: split FORM1 code into helpers Date: Mon, 14 Jun 2021 22:33:03 -0300 Message-Id: <20210615013309.2833323-3-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::f35; envelope-from=danielhb413@gmail.com; helo=mail-qv1-xf35.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The upcoming FORM2 NUMA affinity will support asymmetric NUMA topologies and doesn't need be concerned with all the legacy support for older pseries FORM1 guests. We're also not going to calculate associativity domains based on numa distance (via spapr_numa_define_associativity_domains) since the distances will be written directly into new DT properties. Let's split FORM1 code into its own functions to allow for easier insertion of FORM2 logic later on. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_numa.c | 61 +++++++++++++++++++++++++++++---------------- 1 file changed, 39 insertions(+), 22 deletions(-) diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 779f18b994..04a86f9b5b 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -155,6 +155,32 @@ static void spapr_numa_define_associativity_domains(Sp= aprMachineState *spapr) =20 } =20 +/* + * Set NUMA machine state data based on FORM1 affinity semantics. + */ +static void spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr, + MachineState *machine) +{ + bool using_legacy_numa =3D spapr_machine_using_legacy_numa(spapr); + + /* + * Legacy NUMA guests (pseries-5.1 and older, or guests with only + * 1 NUMA node) will not benefit from anything we're going to do + * after this point. + */ + if (using_legacy_numa) { + return; + } + + if (!spapr_numa_is_symmetrical(machine)) { + error_report("Asymmetrical NUMA topologies aren't supported " + "in the pSeries machine"); + exit(EXIT_FAILURE); + } + + spapr_numa_define_associativity_domains(spapr); +} + void spapr_numa_associativity_init(SpaprMachineState *spapr, MachineState *machine) { @@ -210,22 +236,7 @@ void spapr_numa_associativity_init(SpaprMachineState *= spapr, spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] =3D cpu_to_be3= 2(i); } =20 - /* - * Legacy NUMA guests (pseries-5.1 and older, or guests with only - * 1 NUMA node) will not benefit from anything we're going to do - * after this point. - */ - if (using_legacy_numa) { - return; - } - - if (!spapr_numa_is_symmetrical(machine)) { - error_report("Asymmetrical NUMA topologies aren't supported " - "in the pSeries machine"); - exit(EXIT_FAILURE); - } - - spapr_numa_define_associativity_domains(spapr); + spapr_numa_FORM1_affinity_init(spapr, machine); } =20 void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, @@ -302,12 +313,8 @@ int spapr_numa_write_assoc_lookup_arrays(SpaprMachineS= tate *spapr, void *fdt, return ret; } =20 -/* - * Helper that writes ibm,associativity-reference-points and - * max-associativity-domains in the RTAS pointed by @rtas - * in the DT @fdt. - */ -void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s) +static void spapr_numa_FORM1_write_rtas_dt(SpaprMachineState *spapr, + void *fdt, int rtas) { MachineState *ms =3D MACHINE(spapr); SpaprMachineClass *smc =3D SPAPR_MACHINE_GET_CLASS(spapr); @@ -365,6 +372,16 @@ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr= , void *fdt, int rtas) maxdomains, sizeof(maxdomains))); } =20 +/* + * Helper that writes ibm,associativity-reference-points and + * max-associativity-domains in the RTAS pointed by @rtas + * in the DT @fdt. + */ +void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s) +{ + spapr_numa_FORM1_write_rtas_dt(spapr, fdt, rtas); +} + static target_ulong h_home_node_associativity(PowerPCCPU *cpu, SpaprMachineState *spapr, target_ulong opcode, --=20 2.31.1 From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16237210329421015.5770391497455; Mon, 14 Jun 2021 18:37:12 -0700 (PDT) Received: from localhost ([::1]:52714 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsy0t-0005s5-WD for importer@patchew.org; Mon, 14 Jun 2021 21:37:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39910) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxW-0008K3-6n; Mon, 14 Jun 2021 21:33:42 -0400 Received: from mail-qk1-x733.google.com ([2607:f8b0:4864:20::733]:39821) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxU-0002VP-GM; Mon, 14 Jun 2021 21:33:41 -0400 Received: by mail-qk1-x733.google.com with SMTP id j184so40436470qkd.6; Mon, 14 Jun 2021 18:33:39 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HoasKmU0HCAdf95ydCAzDe4K/CXVQEtjNSxIVUKPTWo=; b=MCbgDPNSSKMPnjNkWknY4imEkt+QqvUnGTYFjuMdrB43f4yCrNe3t0cymbbgfkd+Tt pMFdJW7jqgWsIdnfWCtDgutWOf+8zrQF8/q65k8e3MKrrIbJc1kCNiPO3155+WijE/zV COgfsiErArTNxnGlnMnAE55U+WOjYk2EHSTNaaXlhsR/ugVnY4Nx/GZytYsxoacMqn4l EMfRqd1y/6VzBHNYGnRYtFdUC41rMSK5hF6Bru7mjX+0Elpcsz+33zd26q8wDg6EDKID GWzRCd1tcT66YvlSYqOV5/HpnEsda0fxKr0OhokABlpk89C7gVjJbZ145xW9nDKpUyB4 DuLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HoasKmU0HCAdf95ydCAzDe4K/CXVQEtjNSxIVUKPTWo=; b=DLdAzKUdm5LYvOf60AY/PMPZBPbxuCAX77IBz8bwCNhXbQMHdMxbjxFPORHmGeik5c oc080kKk9X/U4KP5zpLIc/1/GAAcjzlfqdC0wvh3nkzZ6/hPfu6Auc1vXy2X49BBOvg1 R9eHaUFs1DsEacT1Q1aDhm/LVSnuTdbeGaNw7jWZBYRRuw4VZFkT6dKKidw67RJZyPlN H66JQLUXumQsf3pLT3FFs0nZNcqUdgdh0S1FiHAz+q0gKacH1JA/Dx0+f95M73n6K4Sm /hwJHsxdUG7hq87qFr1erTF5b+84z20REbnQpkuWHeAf/wxpl5KAZ9h7KjFb2wFD0xWY dTHg== X-Gm-Message-State: AOAM532MeGEcb04F9VNsc3Z6QO2k+BmXJWOfTDZTMyRCU5VDtNCXiQZR bix1ZK8rQN50KjPRxVWfm0TxYweBeANTFw== X-Google-Smtp-Source: ABdhPJwzqdRsEjqITJObsfQ/gtdJ0Ow4+UisDlxrAmqdeB5b+mcQ1rIQmet7s0kpQmmjJvFSnPDoaA== X-Received: by 2002:a05:620a:816:: with SMTP id s22mr20240097qks.222.1623720819164; Mon, 14 Jun 2021 18:33:39 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 3/8] spapr_numa.c: wait for CAS before writing rtas DT Date: Mon, 14 Jun 2021 22:33:04 -0300 Message-Id: <20210615013309.2833323-4-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::733; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x733.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" spapr_numa_write_rtas_dt() is called from spapr_dt_rtas(), which in turned is called by spapr_build_fdt(). spapr_build_fdt() is called in two places: spapr_machine_reset() and do_client_architecture_support(). When called in machine_reset() we're writing RTAS nodes with NUMA artifacts without going through CAS first. This is not an issue because we always write the same thing in DT, since we support just FORM1 NUMA affinity. With the upcoming FORM2 support, we're now reliant on guest choice to decide what to write. Instead of taking a guess (e.g. default to FORM1, switch to FORM2 if guest chooses it), postpone the writing of ibm,associativity-reference-points and ibm,max-associativity-domains until we're sure what was negotiated with the guest. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_numa.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 04a86f9b5b..e1a7f80076 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -379,6 +379,10 @@ static void spapr_numa_FORM1_write_rtas_dt(SpaprMachin= eState *spapr, */ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s) { + if (spapr_ovec_empty(spapr->ov5_cas)) { + return; + } + spapr_numa_FORM1_write_rtas_dt(spapr, fdt, rtas); } =20 --=20 2.31.1 From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623720898501679.2764072855389; Mon, 14 Jun 2021 18:34:58 -0700 (PDT) Received: from localhost ([::1]:47650 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsxyj-0002TB-5f for importer@patchew.org; Mon, 14 Jun 2021 21:34:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39922) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxZ-00005f-Pu; Mon, 14 Jun 2021 21:33:45 -0400 Received: from mail-qk1-x732.google.com ([2607:f8b0:4864:20::732]:44882) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxX-0002X9-EX; Mon, 14 Jun 2021 21:33:45 -0400 Received: by mail-qk1-x732.google.com with SMTP id c18so25343799qkc.11; Mon, 14 Jun 2021 18:33:42 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z/U84gKjj6AZ/Dy1SeSTnXgrsQdMC5j4d9ApcGQBaxA=; b=OnonqFBGafCpLyBYJukta2C2PnIZV39MIONC4Nv8e8wO9YgwcBeWALpqAmPuq98tWv IopfX5SN6H1U5GC4ZJtLcb9+qByMrVOLii+nHBtm9nO95AsZNewzCpwheu9/SbhlvAY4 qYKkb0R2P/XMAc9pEITxuIxyDl1oMlRVoqSKdRO2oN9rdv+Q6WG8z8iFylPsds6YUNFN kOtRAQEIdCLP3Gaq8e7tzlw8o7mQg26RgMFvOq/v4E/oDjGqpJP1XS+wveAWF6HfQoR5 cXcJLHdikA6Q1yGsS5mMQeYmAhd6PaG6zPpqfazgbaM7TKFVW8w+egTXdZNDgzQiCIZL zkjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z/U84gKjj6AZ/Dy1SeSTnXgrsQdMC5j4d9ApcGQBaxA=; b=pkbOVPfceG1UESJKSDu8ft5/bfXKeqVLmeZkj6CjlFJcXqif35iVHRTygE9IvSgBNK SfXI9OChXFBEc1Vw9eFpZRZWEHUyKDeBbgfA3DYxqlu3ozbF1gQOgrF5VOjd5hiz7MWy 0AH4dbzCP+/PJEQQ8vrEDE6W6A+RSfllMbGTTWUaGQzIj32Qp6uRUvgYHxz8DanluU5W 0puLRhQ8f8mQYV6HcB2HsM4CZ+KymaamZZ/SwrLkDzJAVykUJOr913mlmx60iKRkkTla wVudOOlQGSiuXcKaveaFBs4+YXN8A+qXGJNUNHUB8fk1f4Sohy73r8l7ugQmbNqHoyWz ZgCA== X-Gm-Message-State: AOAM530UTvyR2gL/Vzm0B5UtW+VqNQWenWKacCCJRSEFmhbFcEv+HpYh 3ZlGkxUleaPEJ0w3V1rS5dy8XP0WZy5+Gg== X-Google-Smtp-Source: ABdhPJxbnrTGwzd2GSnCbPHqhqJ4F2MUMQerZgLNFpwfWnf3acdnzlUoX3EWeeGPjtxdVLRGSB3CBA== X-Received: by 2002:a37:b147:: with SMTP id a68mr19432292qkf.4.1623720821914; Mon, 14 Jun 2021 18:33:41 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 4/8] spapr_numa.c: base FORM2 NUMA affinity support Date: Mon, 14 Jun 2021 22:33:05 -0300 Message-Id: <20210615013309.2833323-5-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::732; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x732.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The main feature of FORM2 affinity support is the separation of NUMA distances from ibm,associativity information. This allows for a more flexible and straightforward NUMA distance assignment without relying on complex associations between several levels of NUMA via ibm,associativity matches. Another feature is its extensibility. This base support contains the facilities for NUMA distance assignment. In the future more facilities will be added for latency, performance, bandwidth and so on. This patch implements the base FORM2 affinity support as follows: - the use of FORM2 associativity is indicated by using bit 2 of byte 5 of ibm,architecture-vec-5. A FORM2 aware guest can choose to use FORM1 or FORM2 affinity. Setting both forms will default to FORM2. We rely on CAS to decide what to write in the DT, so all writing is postponed until client architecture support is done; - ibm,associativity-reference-points has a new semantic. Instead of being used to calculate distances via NUMA levels, the concept of subdomain was introduced. The primary domain is the first element of the array, secondary domain is the second element and so on. At this moment, the only usage of this semantic is with NVDIMMs. This will be explained further in the next patches; - two new RTAS DT artifacts are introduced: ibm,numa-lookup-index-table and ibm,numa-distance-table. The index table is used to list all the NUMA logical domains of the platform, in ascending order, and allows for spartial NUMA configurations (although QEMU ATM doesn't support that). ibm,numa-distance-table is an array that contains all the distances from the first NUMA node to all other nodes, then the second NUMA node distances to all other nodes and so on; - spapr_post_load changes: since we're adding a new NUMA affinity that isn't compatible with the existing one, migration must be handled accordingly because we can't be certain of whether the guest went through CAS in the source. The solution chosen is to initiate the NUMA associativity data in spapr_post_load() unconditionally. The worst case would be to write the DT twice if the guest is in pre-CAS stage. Otherwise, we're making sure that a FORM1 guest will have the spapr->numa_assoc_array initialized with the proper information based on user distance, something that we're not doing with FORM2. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 17 +++++ hw/ppc/spapr_numa.c | 140 +++++++++++++++++++++++++++++++++++- include/hw/ppc/spapr_ovec.h | 1 + 3 files changed, 157 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index dd57b30cff..c6def3690a 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1770,6 +1770,22 @@ static int spapr_post_load(void *opaque, int version= _id) return err; } =20 + /* + * NUMA data init is made in CAS time. There is no reliable + * way of telling whether the guest already went through CAS + * in the source due to how spapr_ov5_cas_needed works: a + * FORM1 guest can be migrated with ov5_cas empty regardless + * of going through CAS first. + * + * One solution is to always call numa_associativity_init. The + * downside is that a guest migrated before CAS will run + * numa_associativity_init again when going through it, but + * at least we're making sure spapr->numa_assoc_array will be + * initialized and hotplug operations won't fail in both before + * and after CAS migration cases. + */ + spapr_numa_associativity_init(spapr, MACHINE(spapr)); + return err; } =20 @@ -2733,6 +2749,7 @@ static void spapr_machine_init(MachineState *machine) } =20 spapr_ovec_set(spapr->ov5, OV5_FORM1_AFFINITY); + spapr_ovec_set(spapr->ov5, OV5_FORM2_AFFINITY); =20 /* advertise support for dedicated HP event source to guests */ if (spapr->use_hotplug_event_source) { diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index e1a7f80076..c8fd66b53a 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -202,6 +202,16 @@ void spapr_numa_associativity_init(SpaprMachineState *= spapr, spapr->numa_assoc_array[i][0] =3D cpu_to_be32(MAX_DISTANCE_REF_POI= NTS); spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] =3D cpu_to_be3= 2(i); =20 + /* + * For FORM2 affinity the initialization above is enough. No + * need to fill non-zero NUMA nodes with node_id because + * there is no associativity domain match to calculate + * NUMA distances in FORM2. + */ + if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { + continue; + } + /* * Fill all associativity domains of non-zero NUMA nodes with * node_id. This is required because the default value (0) is @@ -236,7 +246,16 @@ void spapr_numa_associativity_init(SpaprMachineState *= spapr, spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] =3D cpu_to_be3= 2(i); } =20 - spapr_numa_FORM1_affinity_init(spapr, machine); + /* + * We test for !FORM2 instead of testing for FORM1 because, + * as per spapr_ov5_cas_needed, setting FORM1 is not enough + * to get ov5_cas migrated, but setting FORM2 is. Since we're + * dealing with either FORM1 or FORM2, test for the option + * that is guaranteed to be set after a migration. + */ + if (!spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { + spapr_numa_FORM1_affinity_init(spapr, machine); + } } =20 void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, @@ -313,6 +332,120 @@ int spapr_numa_write_assoc_lookup_arrays(SpaprMachine= State *spapr, void *fdt, return ret; } =20 +static void spapr_numa_FORM2_write_rtas_tables(SpaprMachineState *spapr, + void *fdt, int rtas) +{ + MachineState *ms =3D MACHINE(spapr); + NodeInfo *numa_info =3D ms->numa_state->nodes; + int nb_numa_nodes =3D ms->numa_state->num_nodes; + int distance_table_entries =3D nb_numa_nodes * nb_numa_nodes; + g_autofree uint32_t *lookup_index_table =3D NULL; + g_autofree uint32_t *distance_table =3D NULL; + int src, dst, i, distance_table_size; + uint8_t *node_distances; + + /* + * ibm,numa-lookup-index-table: array with length and a + * list of NUMA ids present in the guest. + */ + lookup_index_table =3D g_new0(uint32_t, nb_numa_nodes + 1); + lookup_index_table[0] =3D cpu_to_be32(nb_numa_nodes); + + for (i =3D 0; i < nb_numa_nodes; i++) { + lookup_index_table[i + 1] =3D cpu_to_be32(i); + } + + _FDT(fdt_setprop(fdt, rtas, "ibm,numa-lookup-index-table", + lookup_index_table, + (nb_numa_nodes + 1) * sizeof(uint32_t))); + + /* + * ibm,numa-distance-table: contains all node distances. First + * element is the size of the table as uint32, followed up + * by all the uint8 distances from the first NUMA node, then all + * distances from the second NUMA node and so on. + * + * ibm,numa-lookup-index-table is used by guest to navigate this + * array because NUMA ids can be sparse (node 0 is the first, + * node 8 is the second ...). + */ + distance_table =3D g_new0(uint32_t, distance_table_entries + 1); + distance_table[0] =3D cpu_to_be32(distance_table_entries); + + node_distances =3D (uint8_t *)&distance_table[1]; + i =3D 0; + + for (src =3D 0; src < nb_numa_nodes; src++) { + for (dst =3D 0; dst < nb_numa_nodes; dst++) { + node_distances[i++] =3D numa_info[src].distance[dst]; + } + } + + distance_table_size =3D distance_table_entries * sizeof(uint8_t) + + sizeof(uint32_t); + _FDT(fdt_setprop(fdt, rtas, "ibm,numa-distance-table", + distance_table, distance_table_size)); +} + +/* + * This helper could be compressed in a single function with + * FORM1 logic since we're setting the same DT values, with the + * difference being a call to spapr_numa_FORM2_write_rtas_tables() + * in the end. The separation was made to avoid clogging FORM1 code + * which already has to deal with compat modes from previous + * QEMU machine types. + */ +static void spapr_numa_FORM2_write_rtas_dt(SpaprMachineState *spapr, + void *fdt, int rtas) +{ + MachineState *ms =3D MACHINE(spapr); + uint32_t number_nvgpus_nodes =3D spapr->gpu_numa_id - + spapr_numa_initial_nvgpu_numa_id(ms); + + /* + * From the kernel docs: "With Form2 the primary domainID and + * secondary domainID are used to identify the NUMA nodes + * the kernel should use when using persistent memory devices." + * + * Persistent memory devices, such as spapr-nvdimm, will use the + * primary domainID to indicate the NUMA node number the guest + * OS should use when using the device as regular memory. The + * secondary domainID indicates the numa node number that should + * be used when the device is used as persistent memory. + * + * FORM2 specification allows for further subdomains such as + * tertiary and quaternary. To avoid having to update + * ibm,associativity-reference-points every time a new subdomain + * level is going to be used by the kernel, let's set all available + * subdomains QEMU is willing to support without increasing + * MAX_DISTANCE_REF_POINTS. + */ + uint32_t refpoints[] =3D { + cpu_to_be32(0x4), + cpu_to_be32(0x3), + cpu_to_be32(0x2), + cpu_to_be32(0x1), + }; + + uint32_t nr_refpoints =3D ARRAY_SIZE(refpoints); + uint32_t maxdomain =3D ms->numa_state->num_nodes + number_nvgpus_nodes; + uint32_t maxdomains[] =3D { + cpu_to_be32(4), + cpu_to_be32(maxdomain), + cpu_to_be32(maxdomain), + cpu_to_be32(maxdomain), + cpu_to_be32(maxdomain) + }; + + _FDT(fdt_setprop(fdt, rtas, "ibm,associativity-reference-points", + refpoints, nr_refpoints * sizeof(refpoints[0]))); + + _FDT(fdt_setprop(fdt, rtas, "ibm,max-associativity-domains", + maxdomains, sizeof(maxdomains))); + + spapr_numa_FORM2_write_rtas_tables(spapr, fdt, rtas); +} + static void spapr_numa_FORM1_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rtas) { @@ -383,6 +516,11 @@ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr= , void *fdt, int rtas) return; } =20 + if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { + spapr_numa_FORM2_write_rtas_dt(spapr, fdt, rtas); + return; + } + spapr_numa_FORM1_write_rtas_dt(spapr, fdt, rtas); } =20 diff --git a/include/hw/ppc/spapr_ovec.h b/include/hw/ppc/spapr_ovec.h index 48b716a060..c3e8b98e7e 100644 --- a/include/hw/ppc/spapr_ovec.h +++ b/include/hw/ppc/spapr_ovec.h @@ -49,6 +49,7 @@ typedef struct SpaprOptionVector SpaprOptionVector; /* option vector 5 */ #define OV5_DRCONF_MEMORY OV_BIT(2, 2) #define OV5_FORM1_AFFINITY OV_BIT(5, 0) +#define OV5_FORM2_AFFINITY OV_BIT(5, 2) #define OV5_HP_EVT OV_BIT(6, 5) #define OV5_HPT_RESIZE OV_BIT(6, 7) #define OV5_DRMEM_V2 OV_BIT(22, 0) --=20 2.31.1 From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623721161528942.176483863379; Mon, 14 Jun 2021 18:39:21 -0700 (PDT) Received: from localhost ([::1]:58414 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsy2y-0001Ec-Gv for importer@patchew.org; Mon, 14 Jun 2021 21:39:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39936) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxc-0000HN-QL; Mon, 14 Jun 2021 21:33:48 -0400 Received: from mail-qk1-x72c.google.com ([2607:f8b0:4864:20::72c]:35723) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxa-0002ZK-VH; Mon, 14 Jun 2021 21:33:48 -0400 Received: by mail-qk1-x72c.google.com with SMTP id g19so7673841qkk.2; Mon, 14 Jun 2021 18:33:46 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zYGtZHRoFpB+hNiVeJI9fya+wJI+iK+132NMdUE4Zt0=; b=Atu5l02QMnsCVTEL7zVMthUoqeen9P/Bqms3Gxgwa/6d4gpsoh4+7kp9o2rL2Y9n5Z R6yct0NCsywxJgefN89qdnChOs0CjTXTF+f+jUckkr2+PPDbO4OcsLb2PE74rgpAq3jH zVcqR9H86yIeTKO7MiogDAijQlOidYo/gU4Zn1RXoH8EJXi825JeH5F62tzfTZoypvl+ eeuqKdZtnSl+Nn5Cffm8yD0MT2Es9PCZscPHFav3XsqC6piZTHlHiwnBpFxG+z2t4O6C xTd2Ln8hV9Q16jXEoPM0deUGUNiJu6FhyETy1kZQltGkWfDryI2T1yH1XeJGjxIBngPD DZog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zYGtZHRoFpB+hNiVeJI9fya+wJI+iK+132NMdUE4Zt0=; b=gFdQiQYNwirgTvpLdfX0Zx0P42u3ihazIZqzulPtZKmRmDqUyc7M4HM2pT50wNw5e6 RkCIDRDj3fbjHba2Hhy0kxIjxjPGvRuBp6S1jSp9nNtQSYOU5X2cZ7pj9/TzPMLpMrez BhRbCL0pAYTqwpb20vpFhxyNVrDMQ46NVKEkt+1KD+BxIZFHNP/nKlIcQ6ajLQh+JJxL s2p+wP4ss6xDLv737O7l8NQNS2LQNzqrQN5JZVPjki2tzJKMUWN0wEurYiQAx9iCsKkw mVaTCMAaS8kNz0VkzRVR0MQzX26+SvqGIzblm0H5uLUqk2rte+jJMcK4gbIjo3aWd9+c ocyA== X-Gm-Message-State: AOAM532UR1sMhwr5zJduZeRi7OQeS5uwy9iLF5z0YaoEG0rFNyIb9lqT Tv5A0d6LG5PrZt8NC7fUaracDmn+WMqmUQ== X-Google-Smtp-Source: ABdhPJyOAP7KL3/GnzJN55nOMDFIq/ZW7TFuGMHeXcfn91C5VGR98VOArDv9qzLlIou/kAJ2lkLedg== X-Received: by 2002:a37:9d93:: with SMTP id g141mr666948qke.350.1623720825502; Mon, 14 Jun 2021 18:33:45 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 5/8] spapr: simplify spapr_numa_associativity_init params Date: Mon, 14 Jun 2021 22:33:06 -0300 Message-Id: <20210615013309.2833323-6-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::72c; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x72c.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" When spapr_numa_associativity_init() was introduced it was being called from spapr_machine_init(), where we have pointers to a SpaprMachineState and a MachineState. Having both being passed as params spared us from calling a macro to get the MachineState. Previous patches moved the function away from spapr_machine_init(), and the new locations doesn't have a pointer to MachineState ready. This resulted in MACHINE(spapr) macro calls as the second parameter in both callers. Simplify the function by folding the 'MACHINE(spapr)' macro into the function body, which can now receive only a pointer to SpaprMachineState. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 2 +- hw/ppc/spapr_hcall.c | 2 +- hw/ppc/spapr_numa.c | 4 ++-- include/hw/ppc/spapr_numa.h | 3 +-- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index c6def3690a..c70b8b2f44 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1784,7 +1784,7 @@ static int spapr_post_load(void *opaque, int version_= id) * initialized and hotplug operations won't fail in both before * and after CAS migration cases. */ - spapr_numa_associativity_init(spapr, MACHINE(spapr)); + spapr_numa_associativity_init(spapr); =20 return err; } diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index 6d6ffcc92b..73aca0aed6 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -1199,7 +1199,7 @@ target_ulong do_client_architecture_support(PowerPCCP= U *cpu, spapr_ovec_cleanup(ov1_guest); =20 /* Init numa_assoc_array */ - spapr_numa_associativity_init(spapr, MACHINE(spapr)); + spapr_numa_associativity_init(spapr); =20 /* * Ensure the guest asks for an interrupt mode we support; diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index c8fd66b53a..b14dd543c8 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -181,9 +181,9 @@ static void spapr_numa_FORM1_affinity_init(SpaprMachine= State *spapr, spapr_numa_define_associativity_domains(spapr); } =20 -void spapr_numa_associativity_init(SpaprMachineState *spapr, - MachineState *machine) +void spapr_numa_associativity_init(SpaprMachineState *spapr) { + MachineState *machine =3D MACHINE(spapr); SpaprMachineClass *smc =3D SPAPR_MACHINE_GET_CLASS(spapr); int nb_numa_nodes =3D machine->numa_state->num_nodes; int i, j, max_nodes_with_gpus; diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index 6f9f02d3de..adaec8e163 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -22,8 +22,7 @@ * function. spapr_machine_init() is the only caller for it, and * it has both pointers resolved already. */ -void spapr_numa_associativity_init(SpaprMachineState *spapr, - MachineState *machine); +void spapr_numa_associativity_init(SpaprMachineState *spapr); void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s); void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, int offset, int nodeid); --=20 2.31.1 From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623721143733671.8050467187442; Mon, 14 Jun 2021 18:39:03 -0700 (PDT) Received: from localhost ([::1]:57572 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsy2g-0000gd-Qk for importer@patchew.org; Mon, 14 Jun 2021 21:39:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39958) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxg-0000UG-2H; Mon, 14 Jun 2021 21:33:52 -0400 Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]:34596) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxe-0002av-46; Mon, 14 Jun 2021 21:33:51 -0400 Received: by mail-qt1-x831.google.com with SMTP id u20so10095921qtx.1; Mon, 14 Jun 2021 18:33:49 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lfy76N0NY/1aMHF2KprIQN24YDxSOiwpQMaSev64vXg=; b=nzDoSlZpuUpIhcs2v+K4+dOL880TV7lyM8+7ux2gmQ7X0fV9fVSblUYicAA7HlBzj6 jRfI4by7DqJk22Haozh/813A4jEvAoIdWIe4TpReNAXOdETPDyrxxzVEcyQn7czrEu9c GumrevTGti8ttTse4Grg8aJDn6YmOaaZXM1MXR9NjZ+qF+3KMVdnfMojo7M0szIkijEE e2hIzn8CQw5JIwbt+9+Y/RYAD0JPOOTvEx6U4JaRoAOPOd2w4T///BnuOtxWYnXJJaer nNInc+ZqjKSjrC4fuHGm04CX2K+jKWMbt/QgRWpmhKA3s+R/nvdTuPAI4Yse/a+Nuzfx QOWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lfy76N0NY/1aMHF2KprIQN24YDxSOiwpQMaSev64vXg=; b=qYQjVcFX0EP/0MHd1FS0er8qVbugSa8p9xhP+67YILqf/2CeE4VGgwekqb+viYfDwa SWeceeAIuyhNOe/l4QCYV8Dwv7SThrzjYLOa+cpYGhx/gjR79V9qjhAF+e20khQ8ElsF MaIo2C1Az4vCqxCvLxBU4j7GvDfdFC3aBWxUxjEZO0aIvHVLT7o46s5ZgB9qsW7Sue7g LGRJnrVCudKEsSLaVj5poah0U0CmcoEJSNJD2UvIuUy4OJ+jLM2xoMoO9EGXcHytmT0X Wgu28u28F8KjeUF3BRsQfSLSRrxSewYn66oWcSmip/Y2LQ1auq1RfNyCCFCkWRW/twoa Vclw== X-Gm-Message-State: AOAM532hwG1bFdtgQgOFpMMf7SjzCXjRQymTzfGYN77KIOYfhsd3UHIT AQtS2+scSl3akInYxvdE9p+nx1y5dzNKTA== X-Google-Smtp-Source: ABdhPJyr5UVADKkb2QcX8Gmg3JfyvAeTEPPe3XYgAaHR0sxlDuD6ln410ENZ04Dqs+XuHEwn7jhzoA== X-Received: by 2002:a05:622a:20a:: with SMTP id b10mr669248qtx.349.1623720828669; Mon, 14 Jun 2021 18:33:48 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 6/8] nvdimm: add PPC64 'device-node' property Date: Mon, 14 Jun 2021 22:33:07 -0300 Message-Id: <20210615013309.2833323-7-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::831; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x831.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Shivaprasad G Bhat , aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , groug@kaod.org, qemu-ppc@nongnu.org, Igor Mammedov , david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The spapr-nvdimm driver is able to operate in two ways. The first one is as a regular memory in which the NUMA node of the parent pc-dimm class is used. The second mode, as persistent memory, allows for a different NUMA node to be used based on the locality of the device. At this moment we don't have a way to express this second NUMA node for the persistent memory mode. This patch introduces a new 'device-node' property that will be used by the PPC64 spapr-nvdimm driver to set a second NUMA node for the nvdimm. CC: Shivaprasad G Bhat CC: Igor Mammedov Signed-off-by: Daniel Henrique Barboza --- hw/mem/nvdimm.c | 28 ++++++++++++++++++++++++++++ include/hw/mem/nvdimm.h | 12 ++++++++++++ 2 files changed, 40 insertions(+) diff --git a/hw/mem/nvdimm.c b/hw/mem/nvdimm.c index 7397b67156..030ccf7575 100644 --- a/hw/mem/nvdimm.c +++ b/hw/mem/nvdimm.c @@ -96,6 +96,29 @@ static void nvdimm_set_uuid(Object *obj, Visitor *v, con= st char *name, g_free(value); } =20 +static void nvdimm_get_device_node(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + NVDIMMDevice *nvdimm =3D NVDIMM(obj); + uint8_t value =3D nvdimm->device_node; + + visit_type_uint8(v, name, &value, errp); +} + +static void nvdimm_set_device_node(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + NVDIMMDevice *nvdimm =3D NVDIMM(obj); + uint8_t value; + + if (!visit_type_uint8(v, name, &value, errp)) { + return; + } + + nvdimm->device_node =3D value; +} =20 static void nvdimm_init(Object *obj) { @@ -105,6 +128,11 @@ static void nvdimm_init(Object *obj) =20 object_property_add(obj, NVDIMM_UUID_PROP, "QemuUUID", nvdimm_get_uuid, nvdimm_set_uuid, NULL, NULL); + + object_property_add(obj, NVDIMM_DEVICE_NODE, "uint8", + nvdimm_get_device_node, + nvdimm_set_device_node, + NULL, NULL); } =20 static void nvdimm_finalize(Object *obj) diff --git a/include/hw/mem/nvdimm.h b/include/hw/mem/nvdimm.h index bcf62f825c..430169322f 100644 --- a/include/hw/mem/nvdimm.h +++ b/include/hw/mem/nvdimm.h @@ -49,6 +49,7 @@ OBJECT_DECLARE_TYPE(NVDIMMDevice, NVDIMMClass, NVDIMM) =20 #define NVDIMM_LABEL_SIZE_PROP "label-size" +#define NVDIMM_DEVICE_NODE "device-node" #define NVDIMM_UUID_PROP "uuid" #define NVDIMM_UNARMED_PROP "unarmed" =20 @@ -89,6 +90,17 @@ struct NVDIMMDevice { * The PPC64 - spapr requires each nvdimm device have a uuid. */ QemuUUID uuid; + + /* + * The spapr-nvdimm (PPC64 NVDIMM) driver supports two modes of + * operation: regular memory and persistent memory. When using the + * device as regular memory, the NUMA nodeid that comes from + * PC_DIMM_NODEPROP is to be used. When used as persistent memory, + * the guest should consider the 'device-node' instead since it + * indicates the locality of the device to an established NUMA + * node, which is more relevant to this type of usage. + */ + uint8_t device_node; }; =20 struct NVDIMMClass { --=20 2.31.1 From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623721287670573.1603500351088; Mon, 14 Jun 2021 18:41:27 -0700 (PDT) Received: from localhost ([::1]:34906 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsy50-0004MN-GI for importer@patchew.org; Mon, 14 Jun 2021 21:41:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39970) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxj-0000hC-KB; Mon, 14 Jun 2021 21:33:55 -0400 Received: from mail-qt1-x836.google.com ([2607:f8b0:4864:20::836]:42924) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxh-0002ct-Rl; Mon, 14 Jun 2021 21:33:55 -0400 Received: by mail-qt1-x836.google.com with SMTP id v6so10076425qta.9; Mon, 14 Jun 2021 18:33:53 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lukq+gNciQn9g7tr1zSOYGZblQeIN4FqWF9uIUuzVjs=; b=tG40xa5wwrU/GNfx8anTCEDyZMTqvAE3OXSGoQ5lBxYW+F6sVE4mB+MM5TaDDf5Etc ia09WniblRNh0N9ud5BQ2TlKvXGCWWkbvlFgPRZTZSln45ntSxu0qExY2o2WkU0JoGzY X+wMubqvo5kVGcHOlvjQMZ8EEddHa+gSfSuJMZkC6GJ0N9pybKJ1cYdKkaDS7JJ3cbT2 y1QNhoUSPzVdjuNJ8xfQNfU9Lbudqy/mlmOwzNQcEsa4MHgtA+T/rROXHGk8dbUcw2P4 4Soy0VZcam0uMDVjRDO7/Z1pHYYUzKBonBLNiMKv2iE1NbEZ8+gmrUm+tbYQFYQix8Yq THPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lukq+gNciQn9g7tr1zSOYGZblQeIN4FqWF9uIUuzVjs=; b=NiuJNQ/Ld/+AGHkyy+09U1d+EDAFKwECR9Go0TPi0E4Irf3e0JA9wBWDOzivUScf78 UNmXssqPK/LgiCGlpC+LboNB7nGAboAXzZRUgN9mIc5ZRK6IdMKWwqz9N/8sHGaGW5A5 6EeT/wvzPHK//HPMC6woOzYP8dEuXK84a61KbQRa9H8c8pAis7cYsqGRD0TqvODK14Uf zfx2wbNOBbOJm/3KZy3sWtMnyFRL9d3pfyMYxCqaoc7Pysg5yFPSo4xGrRk85CsNemq6 5cJtlnYm6Cw7OlT/XTTeqcfzKpl9ODrBAHsHVH1H6/65NoUjC0pq2GjWQtwTuYK3pNlG y3Eg== X-Gm-Message-State: AOAM532fHmDSmvfhBtulADb9c5P9PF9lUPWE41n/w3fTUF194SEqNjek A7nl3wQ5RYVlOHGngxsKI+S7gTq4DBqKhw== X-Google-Smtp-Source: ABdhPJzju+Vji+Qdpml36xBaFu9bi2J9JIfdq+91htzNVZAWv0iSfsHmzRXLA9hTHWgfbEAGYPlxhQ== X-Received: by 2002:ac8:7e81:: with SMTP id w1mr1490756qtj.130.1623720832469; Mon, 14 Jun 2021 18:33:52 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 7/8] spapr_numa, spapar_nvdimm: write secondary NUMA domain for nvdimms Date: Mon, 14 Jun 2021 22:33:08 -0300 Message-Id: <20210615013309.2833323-8-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::836; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x836.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Shivaprasad G Bhat , aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , groug@kaod.org, qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Using the new 'device-node' property, write it in the nvdimm DT to set a secondary domain for the persistent memory operation mode. Note that this is only available in FORM2 affinity. FORM1 affinity NVDIMMs aren't affected by this change. CC: Shivaprasad G Bhat Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_numa.c | 20 ++++++++++++++++++++ hw/ppc/spapr_nvdimm.c | 3 ++- include/hw/ppc/spapr_numa.h | 3 +++ 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index b14dd543c8..0cabb6757f 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -266,6 +266,26 @@ void spapr_numa_write_associativity_dt(SpaprMachineSta= te *spapr, void *fdt, sizeof(spapr->numa_assoc_array[nodeid])))); } =20 +void spapr_numa_write_nvdimm_assoc_dt(SpaprMachineState *spapr, void *fdt, + int offset, int nodeid, + int device_node) +{ + uint32_t *nvdimm_assoc_array =3D spapr->numa_assoc_array[nodeid]; + + /* + * 'device-node' is the secondary domain for NVDIMMs when + * using FORM2. The secondary domain for FORM2 in QEMU + * is 0x3. + */ + if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { + nvdimm_assoc_array[0x3] =3D cpu_to_be32(device_node); + } + + _FDT((fdt_setprop(fdt, offset, "ibm,associativity", + nvdimm_assoc_array, + sizeof(spapr->numa_assoc_array[nodeid])))); +} + static uint32_t *spapr_numa_get_vcpu_assoc(SpaprMachineState *spapr, PowerPCCPU *cpu) { diff --git a/hw/ppc/spapr_nvdimm.c b/hw/ppc/spapr_nvdimm.c index 91de1052f2..49a05736fe 100644 --- a/hw/ppc/spapr_nvdimm.c +++ b/hw/ppc/spapr_nvdimm.c @@ -142,7 +142,8 @@ static int spapr_dt_nvdimm(SpaprMachineState *spapr, vo= id *fdt, _FDT((fdt_setprop_string(fdt, child_offset, "compatible", "ibm,pmemory= "))); _FDT((fdt_setprop_string(fdt, child_offset, "device_type", "ibm,pmemor= y"))); =20 - spapr_numa_write_associativity_dt(spapr, fdt, child_offset, node); + spapr_numa_write_nvdimm_assoc_dt(spapr, fdt, child_offset, node, + nvdimm->device_node); =20 buf =3D qemu_uuid_unparse_strdup(&nvdimm->uuid); _FDT((fdt_setprop_string(fdt, child_offset, "ibm,unit-guid", buf))); diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index adaec8e163..af25741e70 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -26,6 +26,9 @@ void spapr_numa_associativity_init(SpaprMachineState *spa= pr); void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rta= s); void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, int offset, int nodeid); +void spapr_numa_write_nvdimm_assoc_dt(SpaprMachineState *spapr, void *fdt, + int offset, int nodeid, + int device_node); int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, int offset, PowerPCCPU *cpu); int spapr_numa_write_assoc_lookup_arrays(SpaprMachineState *spapr, void *f= dt, --=20 2.31.1 From nobody Wed May 22 02:44:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623721247628969.0298999897168; Mon, 14 Jun 2021 18:40:47 -0700 (PDT) Received: from localhost ([::1]:34136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lsy4M-0003pQ-80 for importer@patchew.org; Mon, 14 Jun 2021 21:40:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39990) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lsxxm-0000r3-7N; Mon, 14 Jun 2021 21:33:58 -0400 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]:40947) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lsxxk-0002em-DF; Mon, 14 Jun 2021 21:33:57 -0400 Received: by mail-qk1-x736.google.com with SMTP id u30so40433409qke.7; Mon, 14 Jun 2021 18:33:55 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7c6:4d9a:784f:a7de:c020:5f1e]) by smtp.gmail.com with ESMTPSA id h2sm11639320qkf.106.2021.06.14.18.33.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 18:33:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wUgGKo4Xbe8PZndEW+FwB83m/Vv6a6cLHQWWfrS+caU=; b=W/buIhwtArAVb5xWO+Lcflhvoe5FqlTwevuQiVYRfRehduvN9jDPHv52hQNYBi3Sk7 5TyzsdPqvBMoTJInals0pOThReAY+d5I76oKzPr4jKqfoF28e23Jo2SL/kOSc5rr0dtF am6rycITNYFLwvNNtiBmwCBrEISCDS/xaFJJKuSnjLbJYQw0d4SuZJoJCSnZw0kNbypU OLOR2bp8JsKt+PwtZj6EFHNzq6SJqH81zHj3KRhswB2gOx+lebtkbUpR3s85h0btjefV QSLi8yGu1ZNyeOXvR9/6cVuaYE2t92ULW0fAxbmo6MLUnnUnyBH/yLk/XDc3RWw2f3G0 Rcww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wUgGKo4Xbe8PZndEW+FwB83m/Vv6a6cLHQWWfrS+caU=; b=IbKLsgXcLTHu2Ro+VWXALxdCNoqyo214P3UXcRHlk2z70KZ6tkxDc1bfFyhH55LNhn Mpr8P2jAsW808uCVKncn6pvwcBV8DBmTwzM4Oa+QNCjuePKti3T9vQ6RxSYH1zbqXfZu WEX1B4VRqWdIwefsE3j6dKvRUBsf1ruHp4PKrP67NPdKdxX4bTZ4WLBjOjwLPd9Z62t4 Ey4t7o9y+ob+0Ylo8wxnwpOjtHSaLie5t9LyMJljtELPRxCbhJztRpXVxsXZ19FAJpvN Hn2ojGn6UtZSD8kbDvV/jNbJq0SbAqGf5jeLF8NNMRqIm1MQ2focfhnEKS4UsuexsCYP jzeg== X-Gm-Message-State: AOAM532e2dzm7cnzjbsYLYre0tcdhUhr2sx+Vvl72LmBEeXBXtNLg7W0 eVXm4Qs8j6P1Qv3LZaxxoxS2/qWjt5rGLA== X-Google-Smtp-Source: ABdhPJyoyFVv0qDK9eSpDJ6fdRYbch5juC2wqaXTKNDw/12BVQnXF32UqogUQGLoCB5SjfvMAgUxJw== X-Received: by 2002:a05:620a:2058:: with SMTP id d24mr13273434qka.472.1623720834999; Mon, 14 Jun 2021 18:33:54 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [RFC PATCH 8/8] spapr: move memory/cpu less check to spapr_numa_FORM1_affinity_init() Date: Mon, 14 Jun 2021 22:33:09 -0300 Message-Id: <20210615013309.2833323-9-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615013309.2833323-1-danielhb413@gmail.com> References: <20210615013309.2833323-1-danielhb413@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::736; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x736.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" FORM2 NUMA affinity is prepared to deal with memory/cpu less NUMA nodes. This is used by the DAX KMEM driver to locate a PAPR SCM device that has a different latency than the original NUMA node from the regular memory. Move this verification to FORM1 affinity code. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 33 --------------------------------- hw/ppc/spapr_numa.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+), 33 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index c70b8b2f44..e42bfd314b 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2772,39 +2772,6 @@ static void spapr_machine_init(MachineState *machine) /* init CPUs */ spapr_init_cpus(spapr); =20 - /* - * check we don't have a memory-less/cpu-less NUMA node - * Firmware relies on the existing memory/cpu topology to provide the - * NUMA topology to the kernel. - * And the linux kernel needs to know the NUMA topology at start - * to be able to hotplug CPUs later. - */ - if (machine->numa_state->num_nodes) { - for (i =3D 0; i < machine->numa_state->num_nodes; ++i) { - /* check for memory-less node */ - if (machine->numa_state->nodes[i].node_mem =3D=3D 0) { - CPUState *cs; - int found =3D 0; - /* check for cpu-less node */ - CPU_FOREACH(cs) { - PowerPCCPU *cpu =3D POWERPC_CPU(cs); - if (cpu->node_id =3D=3D i) { - found =3D 1; - break; - } - } - /* memory-less and cpu-less node */ - if (!found) { - error_report( - "Memory-less/cpu-less nodes are not supported (node= %d)", - i); - exit(1); - } - } - } - - } - spapr->gpu_numa_id =3D spapr_numa_initial_nvgpu_numa_id(machine); =20 if ((!kvm_enabled() || kvmppc_has_cap_mmu_radix()) && diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 0cabb6757f..c7b0745455 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -162,6 +162,39 @@ static void spapr_numa_FORM1_affinity_init(SpaprMachin= eState *spapr, MachineState *machine) { bool using_legacy_numa =3D spapr_machine_using_legacy_numa(spapr); + int i; + + /* + * check we don't have a memory-less/cpu-less NUMA node + * Firmware relies on the existing memory/cpu topology to provide the + * NUMA topology to the kernel. + * And the linux kernel needs to know the NUMA topology at start + * to be able to hotplug CPUs later. + */ + if (machine->numa_state->num_nodes) { + for (i =3D 0; i < machine->numa_state->num_nodes; ++i) { + /* check for memory-less node */ + if (machine->numa_state->nodes[i].node_mem =3D=3D 0) { + CPUState *cs; + int found =3D 0; + /* check for cpu-less node */ + CPU_FOREACH(cs) { + PowerPCCPU *cpu =3D POWERPC_CPU(cs); + if (cpu->node_id =3D=3D i) { + found =3D 1; + break; + } + } + /* memory-less and cpu-less node */ + if (!found) { + error_report( + "Memory-less/cpu-less nodes are not supported (node= %d)", + i); + exit(1); + } + } + } + } =20 /* * Legacy NUMA guests (pseries-5.1 and older, or guests with only --=20 2.31.1