From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE4F11DF978; Mon, 28 Oct 2024 17:59:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138385; cv=none; b=TW5GjhT5orMLZrHEl5mVbL0sebaubHezofyXYpRGrEohISW/Hz3a7pe6DUiAwK+pFSAerBIe3CVxDAR4XXc0WNCYzqlZ4eo5TQY6XM1JgzxPBITtZMNVYbTN11DcGq0kpzEFh9fvHnDKu/aiicS0R1k8AekdyinYmKpyIyhcCZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138385; c=relaxed/simple; bh=Rtrn4VLgu5DfagKoC7TxUEvBsOj+n2T7PA++iG8Vjp8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kX3Q8+SqDnr1xLnnzctXZNJd1WWWuLr5fZ+L+TcAH1wrYLqus5KpTB4iz58uPcGPtWQq+SRcrCSOUi7knRN/0NAzIMV8Fr4kqYE5H5fvHXa05qA1Wi5Ctl5D5TLOQVpDpysn6XY62w9XnqQeU6vKfLyqmHBjm96/MjvBF6q5tOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PTiiCqgy; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PTiiCqgy" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a9a0472306cso630878966b.3; Mon, 28 Oct 2024 10:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138381; x=1730743181; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8f9PcoY43I62EwhuLpsEoAimQzVe7SbZIt1u4SOlej4=; b=PTiiCqgy4zYAGWpqGKB8ZkvHIJfhWwf+ZT5S8XlWob832eYRIx+YuiAZPSx2WgHrWm KNh6OdcHXgBfyik0mv8jOeGZ238gxvILgYZz4vOQFVAm5udyvHANftbNXEkAWKP+fgbg hHc6kO0nZHJjIU7rqVHOO36Nk8zkmy5Wg8yFG/ZxSDCy6lfRWGLzUce0puFmVtp96SKn KuOClq6oZB4EBPyappYRoE27bFzfkxpl6W4ZEHm1lI5Lp8zlCU6DWMp/PhlJkZkTd+hr r+MVK2KruxiLMdLY1OzGD3wuSO/epAaKDad7YtzJdNCW7i9hyIvbhE9uLXOD9tHc7G/t UNnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138381; x=1730743181; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8f9PcoY43I62EwhuLpsEoAimQzVe7SbZIt1u4SOlej4=; b=sSyB8qNvb+9XOsxIHm7aIaSjltN2HIh/bDBhsfya8FAuwjonAehmDayVrxPrWR2D/j u8IDpu1wCAH/CsvuX7UM5rpSFT1oormQJxmoyiL9l0hwGVelDqg5kCqFt4M8fCDli4uu sewN6Zb31XfFk5nxPgEbNcKtkuEWKxQzLfKioopTN/ZGm/YRA0KeTdVe5pIfqM/PxZiK kXcTpyHj9PeW/HG0tkjwWo1pbCCTQuCLYVQ3XdTvinFA6E8SlxcRd33Gnr81vHlE1Hp0 psW6brlGSP+VysreK8kpQeio8LtEWrdtolNWKkMjlKahkL02cev1ZbBEnHpCCkR23sCr OXVQ== X-Forwarded-Encrypted: i=1; AJvYcCVEvX5Q+OnwqzcsiQN0/BGOS+NfJ74OBvE94uJ5E8x+0xtBBalS+rLfl6Gh4oPgXJZGvWxf9Te8R7FE@vger.kernel.org, AJvYcCXT9jMLKAUjttg5nKGwtDUIYF8xVq/bpmYsQlL5OaZ/yZVw3V42c5VxquaRjS69o45ItxYHG5zU9Guokw==@vger.kernel.org, AJvYcCXwFqaEPRgrR/GpfPeI8ViPRAHSiWCjAJHsAEuewiz1WChMWjUZVrkRprs9hGnWiQGln2yV3NrDVcln51fz@vger.kernel.org X-Gm-Message-State: AOJu0Yx+W+GVbgGc9la1ZCyMGR6ATPVEIHKzbs1WxuIriITQ0zntFOaz cQxEgnN2wKY51Jdji1utnGsp6aMXxwl0lqDKax6/cMuCQPr66b8C X-Google-Smtp-Source: AGHT+IG7C7sZZ8GB7+eMPJDp7gB/Ed+PcxCWYwIERQr4mXe5Tv0nG2H0hoCUZ7oYsfB1HkHda01TVg== X-Received: by 2002:a17:907:3f0a:b0:a9a:ca1:5e09 with SMTP id a640c23a62f3a-a9de5f23b50mr736555966b.29.1730138380969; Mon, 28 Oct 2024 10:59:40 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:40 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 01/13] irqchip/mips-gic: Introduce for_each_online_cpu_gic() Date: Mon, 28 Oct 2024 18:59:23 +0100 Message-Id: <20241028175935.51250-2-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paul Burton Parts of code in the MIPS GIC driver operate on the GIC local register block for each online CPU, accessing each via the GIC's other/redirect register block. Abstract the process of iterating over online CPUs & configuring the other/redirect region to access their registers through a new for_each_online_cpu_gic() macro. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- drivers/irqchip/irq-mips-gic.c | 59 +++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 76253e864f23..6c7a7d2f0438 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -66,6 +66,44 @@ static struct gic_all_vpes_chip_data { bool mask; } gic_all_vpes_chip_data[GIC_NUM_LOCAL_INTRS]; =20 +static int __gic_with_next_online_cpu(int prev) +{ + unsigned int cpu; + + /* Discover the next online CPU */ + cpu =3D cpumask_next(prev, cpu_online_mask); + + /* If there isn't one, we're done */ + if (cpu >=3D nr_cpu_ids) + return cpu; + + /* + * Move the access lock to the next CPU's GIC local register block. + * + * Set GIC_VL_OTHER. Since the caller holds gic_lock nothing can + * clobber the written value. + */ + write_gic_vl_other(mips_cm_vp_id(cpu)); + + return cpu; +} + +/** + * for_each_online_cpu_gic() - Iterate over online CPUs, access local regi= sters + * @cpu: An integer variable to hold the current CPU number + * @gic_lock: A pointer to raw spin lock used as a guard + * + * Iterate over online CPUs & configure the other/redirect register region= to + * access each CPUs GIC local register block, which can be accessed from t= he + * loop body using read_gic_vo_*() or write_gic_vo_*() accessor functions = or + * their derivatives. + */ +#define for_each_online_cpu_gic(cpu, gic_lock) \ + guard(raw_spinlock_irqsave)(gic_lock); \ + for ((cpu) =3D __gic_with_next_online_cpu(-1); \ + (cpu) < nr_cpu_ids; \ + (cpu) =3D __gic_with_next_online_cpu(cpu)) + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -350,37 +388,27 @@ static struct irq_chip gic_local_irq_controller =3D { static void gic_mask_local_irq_all_vpes(struct irq_data *d) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; int intr, cpu; =20 intr =3D GIC_HWIRQ_TO_LOCAL(d->hwirq); cd =3D irq_data_get_irq_chip_data(d); cd->mask =3D false; =20 - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_rmask(BIT(intr)); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); } =20 static void gic_unmask_local_irq_all_vpes(struct irq_data *d) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; int intr, cpu; =20 intr =3D GIC_HWIRQ_TO_LOCAL(d->hwirq); cd =3D irq_data_get_irq_chip_data(d); cd->mask =3D true; =20 - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_smask(BIT(intr)); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); } =20 static void gic_all_vpes_irq_cpu_online(void) @@ -469,7 +497,6 @@ static int gic_irq_domain_map(struct irq_domain *d, uns= igned int virq, irq_hw_number_t hwirq) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; unsigned int intr; int err, cpu; u32 map; @@ -533,12 +560,8 @@ static int gic_irq_domain_map(struct irq_domain *d, un= signed int virq, if (!gic_local_irq_is_routable(intr)) return -EPERM; =20 - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_map(mips_gic_vx_map_reg(intr), map); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); =20 return 0; } --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E84921DF98E; Mon, 28 Oct 2024 17:59:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138387; cv=none; b=Q6VtZu8n/PRX+lHbnDZLfeq/vSro9CWkexrD7D4MbZKmRGqpDQ2SXye2soWgnzV0iYK3pMvqykCwHF8f/b4JRoEO2C8aHGdDbpeG7fyRxP77vCb93HFW+9KFhn4PtRjolI0PIZY4e73rnm8PQyOie1Vs+OKDmugXhbJfaVSSGMQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138387; c=relaxed/simple; bh=u6vvKO6ochcmouGVbVlP93MlD8ggXFRsMmANCvVfLCc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Q99O6C9bonWTLs4VILaUdIhvqeuxe3igt1U0P8ZUA6edA/cGHLEwF6SPtHQN35w+LnJGwZWVL5y+izfwINCPnmkxICI+zMJKSgaQyoe6c7mEzovpPZ6ztpv0Z2s3PmFm7ZJpUKp/1hQHgcGKZUeOwrIS1hivYUOWJbERkXrLlaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PRtgT8iq; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PRtgT8iq" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a99ebb390a5so1039887066b.1; Mon, 28 Oct 2024 10:59:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138383; x=1730743183; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZRX466fP4uw99H0/bcfPFf3v++c0P8v6HBEx3du66O8=; b=PRtgT8iqh9XWULEojznF7jDKWeDUylGLNus55NxD2tzTDCqtrNTvL83v4NPiNUBkL8 V0Q989QFPd2YF47GdH/w7aqHnZ8/Z0aehu1iOMOK12rAs8aKBdW857HiWU9+gld5GSjW QoczYjvZwcrtumpB3mULiEv885YcxqFTpJjHgdyvUqf3ngsJdC1qdlp+c6YEVEmh/fXc dsHpunjNVXRIWg9gCbjIYau01+HkI6OG6oe1nqbcz2Okccs1x/3xapSao98qJPZql5tb IajFuyIDsiBm/xt/zIzUi/DVueN7eAJ85JzPNECkW4S8VK9UvtCekV+DWmVUhw1DYXf4 WRWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138383; x=1730743183; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZRX466fP4uw99H0/bcfPFf3v++c0P8v6HBEx3du66O8=; b=vjn+jMBgAgpkqEWAcERMXBPyWxatrxbaFv7hqinWQwHVhKwuYSD1YF5laQhmonhwuL JRhUbfanNSvBs2pG4IOzss8HPJgdMPH1ureEQU0PBmQ6TeUmpJZItc7+TGGBKziwVsTZ JJIQK8OEantSDLRsyvtSSllRuomTmkDsNCGr+RQqdIp6MiHtBe1q8zAklVTy8Riry7CZ GXUgP58Tu6fBDZD3TGsCtYB29+sTL55+bteoHWbH6ovDGFCWdM0zdJIgHmld5zwXo0Ww cPboG+ig2E7MmU0kPhBunKopuctV6+J5agJPEXKK/OIEZhQbwwXFbt7+IPzjlfAf3Kh+ fkBQ== X-Forwarded-Encrypted: i=1; AJvYcCUlK0JqfgLtAOGppqpA6nP19qIzyo8A2kMgXrcS/giEoQ/qPrxvbvI30INFlUjvA/hX7teYBeflgo+5HbSx@vger.kernel.org, AJvYcCWf0Uv9VToDsbiNA1msWYCV2l2QTvqKIxfyjx5UJqpNMUdr144KtU/T8cLLxr/orQSjwAeNAb3fvCCmPg==@vger.kernel.org, AJvYcCXAAq8K6MDzwspZ/ARR5vPa6c4uTxN2Zems4/gBFQaQ2jhciP56A3Zr4mUBC3XSPLaF/5gPIcE9SXs6@vger.kernel.org X-Gm-Message-State: AOJu0Yxd3VQF5fbIY/Za0SkP70KesrN2hUlYe0oXpBYnj63IMt7sPGTX 3tR+WxAokDYffHKQaHq9ITxLDGmDvYHbzanwYINwt2Ut75EM2AcL X-Google-Smtp-Source: AGHT+IEOK7cSDa5E6xpn6lKVUU6dFZoleezY5QuGa3zJzk33nx+xsdhMG009sH1i/o6gzfQwQ3WZOA== X-Received: by 2002:a17:907:3d91:b0:a91:1699:f8eb with SMTP id a640c23a62f3a-a9e22ae20f7mr39162966b.28.1730138383015; Mon, 28 Oct 2024 10:59:43 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:42 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 02/13] irqchip/mips-gic: Support multi-cluster in for_each_online_cpu_gic() Date: Mon, 28 Oct 2024 18:59:24 +0100 Message-Id: <20241028175935.51250-3-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paul Burton Use CM's GCR_CL_REDIRECT register to access registers in remote clusters, so users of gic_with_each_online_cpu() gain support for multi-cluster with no further changes. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- drivers/irqchip/irq-mips-gic.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 6c7a7d2f0438..29bdfdce2123 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -88,6 +88,12 @@ static int __gic_with_next_online_cpu(int prev) return cpu; } =20 +static inline void gic_unlock_cluster(void) +{ + if (mips_cps_multicluster_cpus()) + mips_cm_unlock_other(); +} + /** * for_each_online_cpu_gic() - Iterate over online CPUs, access local regi= sters * @cpu: An integer variable to hold the current CPU number @@ -102,6 +108,7 @@ static int __gic_with_next_online_cpu(int prev) guard(raw_spinlock_irqsave)(gic_lock); \ for ((cpu) =3D __gic_with_next_online_cpu(-1); \ (cpu) < nr_cpu_ids; \ + gic_unlock_cluster(), \ (cpu) =3D __gic_with_next_online_cpu(cpu)) =20 static void gic_clear_pcpu_masks(unsigned int intr) --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0CA11DFD90; Mon, 28 Oct 2024 17:59:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138389; cv=none; b=N+F8eX5EbZlTXBsSYFMabIcNMOsdhSdJYAzwnd3fROD0o0syrM6V66Fotg1KiDEtcaYVqtatKCGhIXVc6Md1yrywm1yVEz0eGwCJzTmZlWOHl/t2jHTtRpYqPrxlQ2EbSlb+wXgR6B/eCRdA2vMeKI/V/WuToMRvPU/uhACBPgI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138389; c=relaxed/simple; bh=OL3VID4hY2eu8UwEYOyDoL2nQ3kOIwJbWponl9rfmWg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=M72c3vMRM2F+8u3dnrkNY5JAoD2qjFNw9IPenDGRD0jZXLYnAb0jPsJFnKWthoy5Bsi25nN5cZmHbW8KMZASrWxwh1hQJRA8AruPam5h8rPi+QR5KBpjqqAZSFy0sV0RS1nt3XzC1qY/5OtUo9boCx9EQF+kUM4sfP6EnuS0Wqk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bMCSP6Ny; arc=none smtp.client-ip=209.85.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bMCSP6Ny" Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-5c903f5bd0eso9038843a12.3; Mon, 28 Oct 2024 10:59:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138385; x=1730743185; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GZMIvPMuWtT6vOInn7H8aGGpvxbD/cbeLjklWeSSNdg=; b=bMCSP6NyIXGOzdcn0Pa2FWRAmMmueUTJylF7YCsz1oaQeHAuEyBtySAHuc6qoJ8/nD Z9cN9CPkSQnJLw31iHXsWpQsjsA/0DQZBwbkC+90diJRfVrbBOaMhalXpnsquH5fqjTo 5LX1dnWDO8ZCyOdyh/KZM4fbxgQmX1uZhECtCIQmHxf+2NT+srA81C4qA959kzXMbThB pA5G5Jx72f8lKa8xmCBqZB+xZvl2Mi7k6qroCfUwBcQPC3SX7fnR4pqaGvvjoOJrSgC4 AF5tjNXq0tjeF1pee1ILGCEcsFepVdVJzpv7MybrNUjB5BocP5C1JeuQ5bDxQbmeFaRO wvAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138385; x=1730743185; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GZMIvPMuWtT6vOInn7H8aGGpvxbD/cbeLjklWeSSNdg=; b=hAOcDGUFWl4Yw23JA2Vd2ExblFJzq58ErVpPjXoR0g4AB5hWwJ5dV1a3ujPGO2Ytqf 5ObjgUejuyTBeNVy9C6w3eDjeNqp4dUhDSTqSmTXvCWltSkV+sldsgUrkKJ1aoAG8Y1Y 4QyTvhTOc4mI6QnRCEWrklNIsQrVnWdMX7sIVNO/Vkyf/cTJQd41GdqFz/vX9tl90uMh n36YwTRM663Hjqgyzc/QCFb8q9Irm7pCv/kdoIZdhhQhDI6G9rvpHmLysjcBuvBPw9Sa we5CvCDj8CAIzC6JL6OuP2wu6tk6c+diZATUngDdmO8PC9HDp6QE5QI5Z/5hANyTsuqJ HHpQ== X-Forwarded-Encrypted: i=1; AJvYcCWIbWTEXvyytxHnSaAJ8MEXwQ0b2CHV7tP4N8zyFXUSl+1hd/dx1TdqzTs/xyv/eZvlxNYsbIDxjErQ4wKX@vger.kernel.org, AJvYcCWOLPhP58Duzd+dxB0NYCpvdQiQURyDnd14HtoQiDgNyi6WyQVjB2yLorRDNUa4iIPbT7BkNCfSyCOy@vger.kernel.org, AJvYcCXFT44m6oG2l5zOAOBNR1IaoKqU6zBB9bXiI5bGOhn80tIuQEQvQQs12YwNqdj8QDyFdusuTv/S8BUm2A==@vger.kernel.org X-Gm-Message-State: AOJu0YzEP6opBrdVkQ4RQXpmJm+8X2iJ6tMtadLDoG2AjrYAXD3pDCwa UOiVskcIKypplYtkPIyvjvPTj+kYQdgSOZMolnOAaRf5H6KUQa/L X-Google-Smtp-Source: AGHT+IHpsQwEdo/tU6Dy2HQo/rMxHtRL5C3XUuHugh6QQB+BIaQOId+pkyGUl2FLu2N6qetqW5+KRQ== X-Received: by 2002:a17:907:9444:b0:a9a:558:3929 with SMTP id a640c23a62f3a-a9de615f94emr856744066b.48.1730138384966; Mon, 28 Oct 2024 10:59:44 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:44 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 03/13] irqchip/mips-gic: Setup defaults in each cluster Date: Mon, 28 Oct 2024 18:59:25 +0100 Message-Id: <20241028175935.51250-4-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Chao-ying Fu In multi-cluster MIPS I6500 systems, there is a GIC per cluster. The default shared interrupt setup configured in gic_of_init() will only apply to the GIC in the cluster containing the boot CPU, leaving the GICs of other clusters unconfigured. Similarly configure other clusters. Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- drivers/irqchip/irq-mips-gic.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 29bdfdce2123..d93a076620c7 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -764,7 +764,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, gicconfig; + unsigned int cpu_vec, i, gicconfig, cl, nclusters; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -845,11 +845,29 @@ static int __init gic_of_init(struct device_node *nod= e, =20 board_bind_eic_interrupt =3D &gic_bind_eic_interrupt; =20 - /* Setup defaults */ - for (i =3D 0; i < gic_shared_intrs; i++) { - change_gic_pol(i, GIC_POL_ACTIVE_HIGH); - change_gic_trig(i, GIC_TRIG_LEVEL); - write_gic_rmask(i); + /* + * Initialise each cluster's GIC shared registers to sane default + * values. + * Otherwise, the IPI set up will be erased if we move code + * to gic_cpu_startup for each cpu. + */ + nclusters =3D mips_cps_numclusters(); + for (cl =3D 0; cl < nclusters; cl++) { + if (cl =3D=3D cpu_cluster(¤t_cpu_data)) { + for (i =3D 0; i < gic_shared_intrs; i++) { + change_gic_pol(i, GIC_POL_ACTIVE_HIGH); + change_gic_trig(i, GIC_TRIG_LEVEL); + write_gic_rmask(i); + } + } else { + mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + for (i =3D 0; i < gic_shared_intrs; i++) { + change_gic_redir_pol(i, GIC_POL_ACTIVE_HIGH); + change_gic_redir_trig(i, GIC_TRIG_LEVEL); + write_gic_redir_rmask(i); + } + mips_cm_unlock_other(); + } } =20 return cpuhp_setup_state(CPUHP_AP_IRQ_MIPS_GIC_STARTING, --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7880B1DFE1C; Mon, 28 Oct 2024 17:59:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138392; cv=none; b=pziRSaTjnEwe/hhyUnEj3rPDyEm+0dKNIJ7p0MEACmEuEdHTwgN05CjYHHEY9sXtvKaL38FBIk/QydAUdrsDpugbe1J7CQx9+hRpXP7uyLphfCV1d00BdYt9nYkajmiVJPNWv+4ZYFCLSD6F0KvfGQZBqrNbOiigP98peNRwh88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138392; c=relaxed/simple; bh=xbT/mgkvA1/pzY6aN9D52QMznUnwyQC4d1EnOUIm7yo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tOvFBkiXBfc2o9Ruhw48/6kOTj3rTX9YaMYZPMYmmjkEhoE2IdEcctyx4P2Gt1WVz75hCzJKDVQGI3Mdx6Y0soE+ja/f7D4dX3ELVuKyA6zemd9skgkX+5lRqNqox7Tfw5WzVK6soR/rfCQdbOLswlMEL0LDQ+Un3CZEjVmaFvM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UOCo8q29; arc=none smtp.client-ip=209.85.208.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UOCo8q29" Received: by mail-lj1-f169.google.com with SMTP id 38308e7fff4ca-2fb4af0b6beso68640561fa.3; Mon, 28 Oct 2024 10:59:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138388; x=1730743188; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jLJuGQqhe3edbNXfQYPGMoDXheUeKQlWXtJxjbW/mxo=; b=UOCo8q296aH1I56c1ksg3Bz/gUuUgoNetN/dg6mcuOh41CNbgaUWfU/Jbp+14xZ3WY x0vWreCVvpDL5DaA7/jKB9cdcg3U2Ve5fMRWC3Y/odIhQvAmI4hAPz6oaMuUwyp+nwDh H9mrCQ5Ju7EWsn/NMJSBElxjPybk+GIEn2Hzra9j2EKmYSyy0fJKRS9uw5xPfZtS/krq 0UxQMS168zl8eu4FRAP4qtx9jt6rgd4OYBTcQmqkD5ePLJIz47HdsjLPRwbONeYBaxoI 4LRvp6ZJj8/JA9VlxE602EL3hWvZX6Pxwgfb5qKEbYC51mG8nVVLBqnH4PqxFE46qa+Q U9mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138388; x=1730743188; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jLJuGQqhe3edbNXfQYPGMoDXheUeKQlWXtJxjbW/mxo=; b=psnOiuUROlqmHdSRQ53I6yVOkGx9KkOfAQjQwX7XHdaClYteiXkZfXxMsMziyC0PIJ VNqZW5X0x73Jukk8KsYNTXT7mL/vGB6R+DE8z7PnSP3vLbgDGhyEat1njuhbAIpPzBVS zh6D6BC4mGSU3a9Xhf2Jl6vOnVg4Mv+FFMGiH8oTqqnHKaT9+SjGWiGhJIJAaDXNlVqj 48OCNhovVwfXTr5/1m2vpu/P9DFwIuvYcxBo0kcAIBH2hB2lllNh2RmZXx6L6UXW0+Rm ju76MxsrXdJJPrkGyTfLCbfuk0BLtVV+kuLeM7H3uYZQ5FH6knlvG6C2DZVVOMtfPnxr /gZQ== X-Forwarded-Encrypted: i=1; AJvYcCW4tj9pBGylfKN6yiqNFuF7Pgui4/55aQUQbD+tkCzyR7ZdLc5Arymd2pBF2V1/C3EWiTlKE1vbOXaT9a9H@vger.kernel.org, AJvYcCWjXavqVcLvTGL93z44Hhn4ILc3IA8DPmRpnahLhvV3eTms+n6P+c9WYzzhJVtThGoa+iJ8dfGZhY3/xA==@vger.kernel.org, AJvYcCX6zOSbHot6/fDZcDlNL6rStSrVcjcQQACfXOpSPSoQQFqhc8g25+H0yfZrOXKRi7Mhpu9fWsWr5vbQ@vger.kernel.org X-Gm-Message-State: AOJu0YwvtP5ohnALwvaf8y+1qanWP9InRoA9Au2Ufn9nijzd7x87hk/E L78PX+OY35KO63s3SPONcCXxOqQkyrRmDO1aAEY0dxMp5WH7v6gP X-Google-Smtp-Source: AGHT+IEd1Bbr0aAc2gXCZPoGTLfySGtBI8xCPYEnrLomKp1K/eJSAoA/SW8ocEHaObPgCMatuDbY5A== X-Received: by 2002:a05:6512:32c9:b0:53b:1e70:6ab4 with SMTP id 2adb3069b0e04-53b348cbac4mr6983522e87.14.1730138387150; Mon, 28 Oct 2024 10:59:47 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:46 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 04/13] irqchip/mips-gic: Multi-cluster support Date: Mon, 28 Oct 2024 18:59:26 +0100 Message-Id: <20241028175935.51250-5-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paul Burton The MIPS I6500 CPU & CM (Coherence Manager) 3.5 introduce the concept of multiple clusters to the system. In these systems, each cluster contains its own GIC, so the GIC isn't truly global any longer. Access to registers in the GICs of remote clusters is possible using a redirect register block much like the redirect register blocks provided by the CM & CPC, and configured through the same GCR_REDIRECT register that mips_cm_lock_other() abstraction builds upon. It is expected that external interrupts are connected identically on all clusters. That is, if there is a device providing an interrupt connected to GIC interrupt pin 0 then it should be connected to pin 0 of every GIC in the system. For the most part, the GIC can be treated as though it is still truly global, so long as interrupts in the cluster are configured properly. This patch introduces support for such multi-cluster systems in the MIPS GIC irqchip driver. A newly introduced gic_irq_lock_cluster() function allows: 1) Configure access to a GIC in a remote cluster via the redirect register block, using mips_cm_lock_other(). Or: 2) Detect that the interrupt in question is affine to the local cluster and plain old GIC register access to the GIC in the local cluster should be used. It is possible to access the local cluster's GIC registers via the redirect block, but keeping the special case for them is both good for performance (because we avoid the locking & indirection overhead of using the redirect block) and necessary to maintain compatibility with systems using CM revisions prior to 3.5 which don't support the redirect block. The gic_irq_lock_cluster() function relies upon an IRQs effective affinity in order to discover which cluster the IRQ is affine to. In order to track this & allow it to be updated at an appropriate point during gic_set_affinity() we select the generic support for effective affinity using CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK. gic_set_affinity() is the one function which gains much complexity. It now deconfigures routing to any VP(E), ie. CPU, on the old cluster when moving affinity to a new cluster. gic_shared_irq_domain_map() moves its update of the IRQs effective affinity to before its use of gic_irq_lock_cluster(), to ensure that operation is on the cluster the IRQ is affine to. The remaining changes are straightforward use of the gic_irq_lock_cluster() function to select between local cluster & remote cluster code-paths when configuring interrupts. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- drivers/irqchip/Kconfig | 1 + drivers/irqchip/irq-mips-gic.c | 161 +++++++++++++++++++++++++++++---- 2 files changed, 143 insertions(+), 19 deletions(-) diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index d82bcab233a1..deaafbfea169 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -338,6 +338,7 @@ config KEYSTONE_IRQ =20 config MIPS_GIC bool + select GENERIC_IRQ_EFFECTIVE_AFF_MASK select GENERIC_IRQ_IPI if SMP select IRQ_DOMAIN_HIERARCHY select MIPS_CM diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index d93a076620c7..f42f69bbd6fb 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -111,6 +111,41 @@ static inline void gic_unlock_cluster(void) gic_unlock_cluster(), \ (cpu) =3D __gic_with_next_online_cpu(cpu)) =20 +/** + * gic_irq_lock_cluster() - Lock redirect block access to IRQ's cluster + * @d: struct irq_data corresponding to the interrupt we're interested in + * + * Locks redirect register block access to the global register block of th= e GIC + * within the remote cluster that the IRQ corresponding to @d is affine to, + * returning true when this redirect block setup & locking has been perfor= med. + * + * If @d is affine to the local cluster then no locking is performed and t= his + * function will return false, indicating to the caller that it should acc= ess + * the local clusters registers without the overhead of indirection throug= h the + * redirect block. + * + * In summary, if this function returns true then the caller should access= GIC + * registers using redirect register block accessors & then call + * mips_cm_unlock_other() when done. If this function returns false then t= he + * caller should trivially access GIC registers in the local cluster. + * + * Returns true if locking performed, else false. + */ +static bool gic_irq_lock_cluster(struct irq_data *d) +{ + unsigned int cpu, cl; + + cpu =3D cpumask_first(irq_data_get_effective_affinity_mask(d)); + BUG_ON(cpu >=3D NR_CPUS); + + cl =3D cpu_cluster(&cpu_data[cpu]); + if (cl =3D=3D cpu_cluster(¤t_cpu_data)) + return false; + + mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + return true; +} + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -157,7 +192,12 @@ static void gic_send_ipi(struct irq_data *d, unsigned = int cpu) { irq_hw_number_t hwirq =3D GIC_HWIRQ_TO_SHARED(irqd_to_hwirq(d)); =20 - write_gic_wedge(GIC_WEDGE_RW | hwirq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(GIC_WEDGE_RW | hwirq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(GIC_WEDGE_RW | hwirq); + } } =20 int gic_get_c0_compare_int(void) @@ -225,7 +265,13 @@ static void gic_mask_irq(struct irq_data *d) { unsigned int intr =3D GIC_HWIRQ_TO_SHARED(d->hwirq); =20 - write_gic_rmask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_rmask(intr); + mips_cm_unlock_other(); + } else { + write_gic_rmask(intr); + } + gic_clear_pcpu_masks(intr); } =20 @@ -234,7 +280,12 @@ static void gic_unmask_irq(struct irq_data *d) unsigned int intr =3D GIC_HWIRQ_TO_SHARED(d->hwirq); unsigned int cpu; =20 - write_gic_smask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_smask(intr); + mips_cm_unlock_other(); + } else { + write_gic_smask(intr); + } =20 gic_clear_pcpu_masks(intr); cpu =3D cpumask_first(irq_data_get_effective_affinity_mask(d)); @@ -245,7 +296,12 @@ static void gic_ack_irq(struct irq_data *d) { unsigned int irq =3D GIC_HWIRQ_TO_SHARED(d->hwirq); =20 - write_gic_wedge(irq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(irq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(irq); + } } =20 static int gic_set_type(struct irq_data *d, unsigned int type) @@ -285,9 +341,16 @@ static int gic_set_type(struct irq_data *d, unsigned i= nt type) break; } =20 - change_gic_pol(irq, pol); - change_gic_trig(irq, trig); - change_gic_dual(irq, dual); + if (gic_irq_lock_cluster(d)) { + change_gic_redir_pol(irq, pol); + change_gic_redir_trig(irq, trig); + change_gic_redir_dual(irq, dual); + mips_cm_unlock_other(); + } else { + change_gic_pol(irq, pol); + change_gic_trig(irq, trig); + change_gic_dual(irq, dual); + } =20 if (trig =3D=3D GIC_TRIG_EDGE) irq_set_chip_handler_name_locked(d, &gic_edge_irq_controller, @@ -305,25 +368,72 @@ static int gic_set_affinity(struct irq_data *d, const= struct cpumask *cpumask, bool force) { unsigned int irq =3D GIC_HWIRQ_TO_SHARED(d->hwirq); + unsigned int cpu, cl, old_cpu, old_cl; unsigned long flags; - unsigned int cpu; =20 + /* + * The GIC specifies that we can only route an interrupt to one VP(E), + * ie. CPU in Linux parlance, at a time. Therefore we always route to + * the first online CPU in the mask. + */ cpu =3D cpumask_first_and(cpumask, cpu_online_mask); if (cpu >=3D NR_CPUS) return -EINVAL; =20 - /* Assumption : cpumask refers to a single CPU */ - raw_spin_lock_irqsave(&gic_lock, flags); + old_cpu =3D cpumask_first(irq_data_get_effective_affinity_mask(d)); + old_cl =3D cpu_cluster(&cpu_data[old_cpu]); + cl =3D cpu_cluster(&cpu_data[cpu]); =20 - /* Re-route this IRQ */ - write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + raw_spin_lock_irqsave(&gic_lock, flags); =20 - /* Update the pcpu_masks */ - gic_clear_pcpu_masks(irq); - if (read_gic_mask(irq)) - set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + /* + * If we're moving affinity between clusters, stop routing the + * interrupt to any VP(E) in the old cluster. + */ + if (cl !=3D old_cl) { + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_vp(irq, 0); + mips_cm_unlock_other(); + } else { + write_gic_map_vp(irq, 0); + } + } =20 + /* + * Update effective affinity - after this gic_irq_lock_cluster() will + * begin operating on the new cluster. + */ irq_data_update_effective_affinity(d, cpumask_of(cpu)); + + /* + * If we're moving affinity between clusters, configure the interrupt + * trigger type in the new cluster. + */ + if (cl !=3D old_cl) + gic_set_type(d, irqd_get_trigger_type(d)); + + /* Route the interrupt to its new VP(E) */ + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_pin(irq, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_redir_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + + mips_cm_unlock_other(); + } else { + write_gic_map_pin(irq, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); =20 return IRQ_SET_MASK_OK; @@ -471,11 +581,21 @@ static int gic_shared_irq_domain_map(struct irq_domai= n *d, unsigned int virq, unsigned long flags; =20 data =3D irq_get_irq_data(virq); + irq_data_update_effective_affinity(data, cpumask_of(cpu)); =20 raw_spin_lock_irqsave(&gic_lock, flags); - write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); - write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); - irq_data_update_effective_affinity(data, cpumask_of(cpu)); + + /* Route the interrupt to its VP(E) */ + if (gic_irq_lock_cluster(data)) { + write_gic_redir_map_pin(intr, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + mips_cm_unlock_other(); + } else { + write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); =20 return 0; @@ -651,6 +771,9 @@ static int gic_ipi_domain_alloc(struct irq_domain *d, u= nsigned int virq, if (ret) goto error; =20 + /* Set affinity to cpu. */ + irq_data_update_effective_affinity(irq_get_irq_data(virq + i), + cpumask_of(cpu)); ret =3D irq_set_irq_type(virq + i, IRQ_TYPE_EDGE_RISING); if (ret) goto error; --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE5B81DF98E; Mon, 28 Oct 2024 17:59:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138393; cv=none; b=qKF4Y4IxqUHn6HN1x3bieNm8ZxJxPEUio0KaAtdeLwPCZ/S77ws8NGFG8oOPxJ50fRHEzY6qEGDbf5jTVly/dIqeM+mhy/g9YVco2y9fFKCxsqU9XTs/28R4B69ak/BFl/Fy2K+NOfprtetesaxwtqARus6j9roJhYwhVqOZbVc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138393; c=relaxed/simple; bh=p2Tp/HcuSBEX14Tfk4llfvH2YxkZR1LNd5lRMCyquIY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qfHgJis/OK9zH3RlKh/8XhInfZtF/PxaiaQNcgU2s8qNJ/Suh4Ntli/htC0vCFaGl/SPqlKgXm6PSvxwxbaZr6cln+pMiyVAf+bKS9mB0mm7JWV7fPLuqitPgla/Ly1RLJHCoMkykuA09Cd9IqslsYoI4tJMgyPddmIa1Pw6YTA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gnm8sjtT; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gnm8sjtT" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a9a0ef5179dso670335566b.1; Mon, 28 Oct 2024 10:59:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138389; x=1730743189; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HeLRARWoyg8Va94Z6ay+CauVKyiZ38OKRS9Vk7LJXu0=; b=gnm8sjtTPqCKEP3DPRLqQ4VSz1ABLCBde87RPy5whqbkDP43I6uWs6yi+t7tCtu/uR LlthCCpkDblcVGsbHLgkZnT5zXWfbhnLgsLM2H6azhm0AgDH98rVp7lsCswfKahAlGk4 naTxXnqOb9rM7mSxG3gZfDxekU97oDbpizcNR+6ZLTejJMD0a2l1ipz3tlYr+c/OlL0a GLAS61jvFYL0omQauEB0UnepQLERrXgT6C2/LwjWmRHQVP5XTgz5X+J8DRbjfkpRXaQb 9USYME4bdYg4w0kvPE5RLkzf9iA5OSBR/WoEBBDzFaAxUkZwWGMChd3m4AqklTYHdm4f ZKkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138389; x=1730743189; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HeLRARWoyg8Va94Z6ay+CauVKyiZ38OKRS9Vk7LJXu0=; b=ovqtWQgakTlmHLAjr9pE1TxF0446f5IBmYek+UlpjwVTwIz7CvQoajVzciRatU2QYP 3R+n2xIFCjDgTcQmBoqvlgDjMY4js4Rbo2EpHbPFaKx0yqjvU3zzOhYWa7axOaiaCmj1 Yp40TBFi8Ght7AzgEdNLTlZBlK6t441GsiCOB+OsnD/xcBSpiATJcibIbc7RyFknRUTa N2UlN0rwN4AkFQo8fQLZuk5LQTjt3B1LGc+JkoRqe3cvewSasu6Jv6XbytxrvgJ3rD3a npAgrTsW5k/aBFNFAOzm+BVO4D23A1TuADw+GeOXIM3912w49TPH14ZYxmCD1O6wfUhK 0SLg== X-Forwarded-Encrypted: i=1; AJvYcCV/ht6XZAQIp+3fQvJJW9Wg2TZIc4TdYdr/HmOskHPOpt8IMkhU3xPnPWuLZq3A51L/ZcxVI3RrZmqt@vger.kernel.org, AJvYcCVD7gxjCoiG/P0s50R9X/AX25tpA6s+NdI6w1g5jwD22ou+wyTjCO6vk2rHejCNNyliIfGLtTLuUr3lVw93@vger.kernel.org, AJvYcCX2Uoq514azb78WJLFDdn3vThqUFJ0NK3cNsr269vKyDaxn2FJYcWKf7vsYudrAjadQ/u2l3EzVIsfdvw==@vger.kernel.org X-Gm-Message-State: AOJu0Yw0ogHDYNOaAtHR/fMhhNAxAjfxFvy5f018SBcwnLpc9emObRGb cyVK+ZmS+15dUbA+9l8IWq/LFSt+16u5nJT7xV8fJtwvEGdPKEzk X-Google-Smtp-Source: AGHT+IFqysuG3AXUKoxFaw5qrYiIo8V/qjCFFeXyI/1B97ZSodG1BX51263mSjk6qGH57Ms5vz5GhA== X-Received: by 2002:a17:907:eaa:b0:a99:f496:794c with SMTP id a640c23a62f3a-a9de632bf9bmr736810966b.52.1730138389055; Mon, 28 Oct 2024 10:59:49 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:48 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 05/13] clocksource: mips-gic-timer: Always use cluster 0 counter as clocksource Date: Mon, 28 Oct 2024 18:59:27 +0100 Message-Id: <20241028175935.51250-6-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paul Burton In a multi-cluster MIPS system, there are multiple GICs - one in each cluster - each of which has its independent counter. The counters in each GIC are not synchronized in any way, so they can drift relative to one another through the lifetime of the system. This is problematic for a clock source which ought to be global. Avoid problems by always accessing cluster 0's counter, using cross-cluster register access. This adds overhead so it is applied only on multi-cluster systems. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- drivers/clocksource/mips-gic-timer.c | 39 +++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mip= s-gic-timer.c index 110347707ff9..7907b740497a 100644 --- a/drivers/clocksource/mips-gic-timer.c +++ b/drivers/clocksource/mips-gic-timer.c @@ -166,6 +166,37 @@ static u64 gic_hpt_read(struct clocksource *cs) return gic_read_count(); } =20 +static u64 gic_hpt_read_multicluster(struct clocksource *cs) +{ + unsigned int hi, hi2, lo; + u64 count; + + mips_cm_lock_other(0, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + if (mips_cm_is64) { + count =3D read_gic_redir_counter(); + goto out; + } + + hi =3D read_gic_redir_counter_32h(); + while (true) { + lo =3D read_gic_redir_counter_32l(); + + /* If hi didn't change then lo didn't wrap & we're done */ + hi2 =3D read_gic_redir_counter_32h(); + if (hi2 =3D=3D hi) + break; + + /* Otherwise, repeat with the latest hi value */ + hi =3D hi2; + } + + count =3D (((u64)hi) << 32) + lo; +out: + mips_cm_unlock_other(); + return count; +} + static struct clocksource gic_clocksource =3D { .name =3D "GIC", .read =3D gic_hpt_read, @@ -203,6 +234,11 @@ static int __init __gic_clocksource_init(void) gic_clocksource.rating =3D 200; gic_clocksource.rating +=3D clamp(gic_frequency / 10000000, 0, 99); =20 + if (mips_cps_multicluster_cpus()) { + gic_clocksource.read =3D &gic_hpt_read_multicluster; + gic_clocksource.vdso_clock_mode =3D VDSO_CLOCKMODE_NONE; + } + ret =3D clocksource_register_hz(&gic_clocksource, gic_frequency); if (ret < 0) pr_warn("Unable to register clocksource\n"); @@ -261,7 +297,8 @@ static int __init gic_clocksource_of_init(struct device= _node *node) * stable CPU frequency or on the platforms with CM3 and CPU frequency * change performed by the CPC core clocks divider. */ - if (mips_cm_revision() >=3D CM_REV_CM3 || !IS_ENABLED(CONFIG_CPU_FREQ)) { + if ((mips_cm_revision() >=3D CM_REV_CM3 || !IS_ENABLED(CONFIG_CPU_FREQ)) = && + !mips_cps_multicluster_cpus()) { sched_clock_register(mips_cm_is64 ? gic_read_count_64 : gic_read_count_2x32, gic_count_width, gic_frequency); --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA28E1DFD90; Mon, 28 Oct 2024 17:59:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138395; cv=none; b=pCLLVCrqS9X2n6YslHHmqnTLQvD0MnqKNyau/sIMeJNYTtNgk0ywGTdfxpydS4EJsZhWc5mWqTTKwlYeFZ4BOVVbyeoMigNHGOWUkN5eDhK6vCFvllr07DiUDXdAPwDWHIioV6iYJP1dzzWvOsftsBSTjTLDtnEiqEIGrDtbf0I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138395; c=relaxed/simple; bh=4JnwAaxzzTAViWg7ct3Xj13ZKq9MDJz4VfUrOXaNL+Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=XVcjyDkzIk3n54rHiHRJnJi/f57cbImtuvjn+/RACiAihkXZk4w8BbUKlfJWUcA7PKQxsqFEVRZXXKWAbtadROxcAkVqWDTsSXBDx4/YttiHrlnu0X6giVhw+23q7C+z8YF/4UCCs3EaaYwpPuaPP9tEyaICE6ZmiEs9OnkLqCc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gMTzjD1z; arc=none smtp.client-ip=209.85.218.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gMTzjD1z" Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a9a0472306cso630900866b.3; Mon, 28 Oct 2024 10:59:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138391; x=1730743191; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TRZGYpywOr8SfCIjJp+/4G1Vc/5w1pIrXams/TeKR5k=; b=gMTzjD1zpezRWpzhwxe2hQM8neyMeczSqdGq1mDrGdjXqb6pfxF/12pHgs8vi1Wkbd x3zzLn1p4/NbKl1BIoN2PUROS5i1nyCk9t+KIuAcc7Q8lovOwmbaePEXZDiNgQkwq/gS UihkGnDxrwWB8aSOpbGYfxwRXzh8qWDk3G5bLUf9H7e4ApIMzXqmf7qPvxd41dO5qxmk s0/LosaC3eNYP4DXEpn8gxNjjIhIqh6IxTQWzBBMaXZYRb7Qu1J/CmyStdS6p7qTqTtI 03QvdyA0N8Zuokf5aXyzQzWMlNiu8nQI0JX58X5PvTe7UzAqQls/L7mnJs1KzEb7UvjV Bd3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138391; x=1730743191; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TRZGYpywOr8SfCIjJp+/4G1Vc/5w1pIrXams/TeKR5k=; b=qb9L3K3TmJRHIBjuiKWu1Dz98+uhVuXhQTwMUmQIZI0ExwpR2wZn7nQ8XzwE1BRMc5 u5nIHjrzMPJVHu27pdEdU3wxbm5RoKGSr9k9m3W7jPeZ0RM9B1qva75U2P6Rm4F3hjl9 1OyGQHdByK3QlghW3uD5ZKNpC9qu4LkR9+y4Kwj5TwcPa0o8Nit9nNP+pvHh02scj4Rc BF7M2pGbmVyY9vVkH79MQDOwrtN8sCzS7MWwCddRJ/GF2GfyQCymgPYmoOXBBHG80AyK H877h0tumUkC9Tl4Np94WR0l9Jv9Vq2/NFkrj6Ml4UwpRQyQzGCBHKcf3baQqzpv2V8O IQeg== X-Forwarded-Encrypted: i=1; AJvYcCUF7UFHooxpaHx0VzbMGe14OKk4aIzMgmquS5RwTgHPtS1JmleIag5eI0Tp9nYuEX8NH56xZWqs07TO@vger.kernel.org, AJvYcCW9yDrzeSou8toq2XUNHBSqv1LfqFLLa2/agbClY+D0aTOq2uKIuvm88wDGvNJHSjDE5km78tBAtgxQCVCx@vger.kernel.org, AJvYcCWa6Tttuv3zKz8b2eNbzVXlfv++6ySWgI+GkJ6G/sfEtz2Z2FbPqNW9yeGtFLNGPozDboNojHCFDwb4kQ==@vger.kernel.org X-Gm-Message-State: AOJu0YyYgUNh100CCv4q2yZv1gNutrUUhW4TCTjeKcp/+YWWhItQG8XE bv1Nz6H+Oe75T0mqy8gwIKmLdhyaQ/day/0dbGjORLh9iX82zMaC X-Google-Smtp-Source: AGHT+IG5CeFXBy+88t4/mlcYBWdzsqbqzkUPO/IOnAI5iiWjhEbd5sESo8/SN+xxquLLJjOSBHxmhw== X-Received: by 2002:a17:906:da86:b0:a9a:49a8:f1fa with SMTP id a640c23a62f3a-a9de5db9e9bmr813270066b.23.1730138391023; Mon, 28 Oct 2024 10:59:51 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:50 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 06/13] clocksource: mips-gic-timer: Enable counter when CPUs start Date: Mon, 28 Oct 2024 18:59:28 +0100 Message-Id: <20241028175935.51250-7-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Paul Burton In multi-cluster MIPS I6500 systems there is a GIC in each cluster, each with its own counter. When a cluster powers up the counter will be stopped, with the COUNTSTOP bit set in the GIC_CONFIG register. In single cluster systems, it has been fine to clear COUNTSTOP once in gic_clocksource_of_init() to start the counter. In multi-cluster systems, this will only have started the counter in the boot cluster, and any CPUs in other clusters will find their counter stopped which will break the GIC clock_event_device. Resolve this by having CPUs clear the COUNTSTOP bit when they come online, using the existing gic_starting_cpu() CPU hotplug callback. This will allow CPUs in secondary clusters to ensure that the cluster's GIC counter is running as expected. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- drivers/clocksource/mips-gic-timer.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mip= s-gic-timer.c index 7907b740497a..abb685a080a5 100644 --- a/drivers/clocksource/mips-gic-timer.c +++ b/drivers/clocksource/mips-gic-timer.c @@ -115,6 +115,9 @@ static void gic_update_frequency(void *data) =20 static int gic_starting_cpu(unsigned int cpu) { + /* Ensure the GIC counter is running */ + clear_gic_config(GIC_CONFIG_COUNTSTOP); + gic_clockevent_cpu_init(cpu, this_cpu_ptr(&gic_clockevent_device)); return 0; } @@ -288,9 +291,6 @@ static int __init gic_clocksource_of_init(struct device= _node *node) pr_warn("Unable to register clock notifier\n"); } =20 - /* And finally start the counter */ - clear_gic_config(GIC_CONFIG_COUNTSTOP); - /* * It's safe to use the MIPS GIC timer as a sched clock source only if * its ticks are stable, which is true on either the platforms with --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE0B01DF976; Mon, 28 Oct 2024 17:59:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138397; cv=none; b=jq1rhcwgThzS9ru1t8p9DfgtxPWGl++ZhW4qKcYDZsdGgpaS5hc1SgZ9QK7ujIklv88BePClQQTIOFkVNJDtTPoERsI5qxPYc5fbNkjNJllruv8yP5ESb4J02JgYQcNQ7Ccbq3XU/yzi/lGPaTckrmoKtUwesT8D6GY+hpIpPPA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138397; c=relaxed/simple; bh=juegPF0QAfCy/64jfsXI53FQb/EgNPwFWBJGG+Zvlro=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nDdiOuc3LC/w2JdV/jHxI0yYjlolC1AZrK2aV8nlOnJ5xvFBLmdu5eW8O868B84pqJ5gAMs4TER7ZY9bklqShDLBgvKPGOK6ZNsCw8LEbhvW+Gi4KELu69RMMnsp9x+HSi0/PICUPTnUpFThdFnBKPbSD8Inq1PspC9uECDog3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aQhJmqHw; arc=none smtp.client-ip=209.85.208.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aQhJmqHw" Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-5c935d99dc5so5240350a12.1; Mon, 28 Oct 2024 10:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138393; x=1730743193; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qxVQmw22U/pUgtkzMWu3ru0gpeZOa1eNAUYZCGmjfkg=; b=aQhJmqHwHOsCNfGMOQ6d7YgekMkcmQNyUrPBq0FB0UHh4jujrBPRpcDhlqEn6Tf8dr /D1GFObo09bVxiB4WVUcNXRyqH4/YqHkFBPyXgUGK8Q34xLDKkJnRDBNRiY2ovyUO35E u/WCBT6YF1Ft1YRYbmWkNRTYHCJyhTt5gKynpd6toK4tB71viMwLqMOUa7sLIojV95OQ dUtGn4YdC0wDaOIPUVrrYCT7pNw/aGn3Q95V6dcDeYq6LB7X2C+UnAa36I3BURP1i+iL nfmW+juvvuJhGtHR5E+9cHvrO2YB5o3Td6Rh4HmlWNOXvMEP+B50hqEwalvpt+Akd3cF fZkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138393; x=1730743193; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qxVQmw22U/pUgtkzMWu3ru0gpeZOa1eNAUYZCGmjfkg=; b=QBvSS7jT43O+PMO9+x+40CrJ039uvdrTKyYgXA8MPl7NarWE3kHMQXhUPSerurJaUu +NHk5q6/iyYJ6iIeOZYnExOHH056JHOT3qG9AswzqVDq4snIM6hOQqo2m/E0emuQ0UFw d14jFfUsaTJopENEQrxv2oNZ/c/d1u5IPAmNAxRkNUHSlPCoCM1ARzHkDmyebKSjMjgy V5LftuaE5+uDpHED3OF15H+Oysh3loKcLlY6zRCl4/BpU+vsj+92w6ji5CW274+BFtap TCvCAziAir3mwcJTpj1CAvHjsIVuFjzOPDveVhnlaYuiILTLpv1YGvy3hlQQleanPvs7 FI9Q== X-Forwarded-Encrypted: i=1; AJvYcCVMMMA4pjr+hx+aPC/C3y2Buaf1p9LaDcOzmUnkuE0GUh235FJxiTVC644PSe0o8mo9ghimMereDYyuFiRv@vger.kernel.org, AJvYcCW458K65kN38YIrBD7QDBb/B6RdA5saZWgDA/3G7orQyNOD5lRSogw/9w2vcq2dFB4G8e8IKmCcCxJ7@vger.kernel.org, AJvYcCXvcH25KvSLd8MTH7+xMlTMGs70PRcT9wYZNqpk3sPe6bTMauKkyjPdJuFDyQnsG5ctT0l8sltDZyTsIw==@vger.kernel.org X-Gm-Message-State: AOJu0YxO0S0L57Fbwvf7H1rh+ZCSQkF3x3aoSatVnfnOIUgAvDGUGs0P Xkx7lkEpdTpfwMynXSU1WTCMQpYm7b2skDq6if/pmH3kCs96X7xB X-Google-Smtp-Source: AGHT+IFskJ3dWZJhyzh76NiXogiRVmfnB0os/ljT/42MYxlNH3EIlxWwM5k43kzT7XbpQeNd+0qfnw== X-Received: by 2002:a17:907:7e8e:b0:a99:e939:d69e with SMTP id a640c23a62f3a-a9de61d1a8cmr674720266b.51.1730138392955; Mon, 28 Oct 2024 10:59:52 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:52 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 07/13] MIPS: pm-cps: Use per-CPU variables as per-CPU, not per-core Date: Mon, 28 Oct 2024 18:59:29 +0100 Message-Id: <20241028175935.51250-8-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paul Burton The pm-cps code has up until now used per-CPU variables indexed by core, rather than CPU number, in order to share data amongst sibling CPUs (ie. VPs/threads in a core). This works fine for single cluster systems, but with multi-cluster systems a core number is no longer unique in the system, leading to sharing between CPUs that are not actually siblings. Avoid this issue by using per-CPU variables as they are more generally used - ie. access them using CPU numbers rather than core numbers. Sharing between siblings is then accomplished by: - Assigning the same pointer to entries for each sibling CPU for the nc_asm_enter & ready_count variables, which allow this by virtue of being per-CPU pointers. - Indexing by the first CPU set in a CPUs cpu_sibling_map in the case of pm_barrier, for which we can't use the previous approach because the per-CPU variable is not a pointer. Signed-off-by: Paul Burton Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- arch/mips/kernel/pm-cps.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index d09ca77e624d..9369a8dc385e 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -57,10 +57,7 @@ static DEFINE_PER_CPU_ALIGNED(u32*, ready_count); /* Indicates online CPUs coupled with the current CPU */ static DEFINE_PER_CPU_ALIGNED(cpumask_t, online_coupled); =20 -/* - * Used to synchronize entry to deep idle states. Actually per-core rather - * than per-CPU. - */ +/* Used to synchronize entry to deep idle states */ static DEFINE_PER_CPU_ALIGNED(atomic_t, pm_barrier); =20 /* Saved CPU state across the CPS_PM_POWER_GATED state */ @@ -112,9 +109,10 @@ int cps_pm_enter_state(enum cps_pm_state state) cps_nc_entry_fn entry; struct core_boot_config *core_cfg; struct vpe_boot_config *vpe_cfg; + atomic_t *barrier; =20 /* Check that there is an entry function for this state */ - entry =3D per_cpu(nc_asm_enter, core)[state]; + entry =3D per_cpu(nc_asm_enter, cpu)[state]; if (!entry) return -EINVAL; =20 @@ -150,7 +148,7 @@ int cps_pm_enter_state(enum cps_pm_state state) smp_mb__after_atomic(); =20 /* Create a non-coherent mapping of the core ready_count */ - core_ready_count =3D per_cpu(ready_count, core); + core_ready_count =3D per_cpu(ready_count, cpu); nc_addr =3D kmap_noncoherent(virt_to_page(core_ready_count), (unsigned long)core_ready_count); nc_addr +=3D ((unsigned long)core_ready_count & ~PAGE_MASK); @@ -158,7 +156,8 @@ int cps_pm_enter_state(enum cps_pm_state state) =20 /* Ensure ready_count is zero-initialised before the assembly runs */ WRITE_ONCE(*nc_core_ready_count, 0); - coupled_barrier(&per_cpu(pm_barrier, core), online); + barrier =3D &per_cpu(pm_barrier, cpumask_first(&cpu_sibling_map[cpu])); + coupled_barrier(barrier, online); =20 /* Run the generated entry code */ left =3D entry(online, nc_core_ready_count); @@ -629,12 +628,14 @@ static void *cps_gen_entry_code(unsigned cpu, enum cp= s_pm_state state) =20 static int cps_pm_online_cpu(unsigned int cpu) { - enum cps_pm_state state; - unsigned core =3D cpu_core(&cpu_data[cpu]); + unsigned int sibling, core; void *entry_fn, *core_rc; + enum cps_pm_state state; + + core =3D cpu_core(&cpu_data[cpu]); =20 for (state =3D CPS_PM_NC_WAIT; state < CPS_PM_STATE_COUNT; state++) { - if (per_cpu(nc_asm_enter, core)[state]) + if (per_cpu(nc_asm_enter, cpu)[state]) continue; if (!test_bit(state, state_support)) continue; @@ -646,16 +647,19 @@ static int cps_pm_online_cpu(unsigned int cpu) clear_bit(state, state_support); } =20 - per_cpu(nc_asm_enter, core)[state] =3D entry_fn; + for_each_cpu(sibling, &cpu_sibling_map[cpu]) + per_cpu(nc_asm_enter, sibling)[state] =3D entry_fn; } =20 - if (!per_cpu(ready_count, core)) { + if (!per_cpu(ready_count, cpu)) { core_rc =3D kmalloc(sizeof(u32), GFP_KERNEL); if (!core_rc) { pr_err("Failed allocate core %u ready_count\n", core); return -ENOMEM; } - per_cpu(ready_count, core) =3D core_rc; + + for_each_cpu(sibling, &cpu_sibling_map[cpu]) + per_cpu(ready_count, sibling) =3D core_rc; } =20 return 0; --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C97B91E0B61; Mon, 28 Oct 2024 17:59:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138399; cv=none; b=n7RrlII9euzpYn6abIzzlEgEk//cmV3Mj0jWXHx9Y4cnq+u2DxbjgqCF8Fb789A2vGdk8HKVTtVD7TmTOBqJiX2F20kr0gWUjpz+cICSwS1aXKiRNBawbo09wT0pSBF91l1GVZPpYlVdlP9d+ENupoVVuvhP+PKt+FnMByURTUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138399; c=relaxed/simple; bh=GOD8XfkJz4zsO80sDemTkSTBWYgvVnNxTFCnWXZ0gII=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TT4bHmBX+7n8jcZ81901Ow7EkM8DmybXGtsVThA2plRXaIyMwYW/OpRkNm9NhIfpjYnqYavwexS95XYt8QJLdKBug1RFnDxqNyActkUrAyfafd9aolV0Dsz52PlB4CaiGnXdxI31D035KGrSVyTDkJolfyaXdtdky54L9rEVK0c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mDcvWb1K; arc=none smtp.client-ip=209.85.208.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mDcvWb1K" Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-5c935d99dc5so5240379a12.1; Mon, 28 Oct 2024 10:59:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138395; x=1730743195; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4jHqaxSvoWLgnCFeLMCaBBBf/bAjTs2yGSSBO7ssnOk=; b=mDcvWb1K5JZYShm5ymJ2Nno1ysF/igrAOJoXtqYyy7e9q8oU3HfkifN/G+2WsSTaEM q90o7jx2dXKSS83flbyOcpz2ysbk3gBWgFUaMouooMQd3K580ugDELi37cfZWaaNdtjw uXU4FUglGcqF9ecAW2gX7Cxg5NmicZvVdyadqvhziZqvmYKYCUoCDJGeo8gHOtaQDAjs QQEDGfv0IA5BBTQGBdfjza5hhD/SoMB+0v9G/R6wabT1dywCioaGXT7dZiVAr+KnfHTD uETuhPD6A52SMN9praHYMgeTE9+yspUmE6fwe1J7NRAaq4aw7RQfSt0IjFPUlK9qA8CA zLYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138395; x=1730743195; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4jHqaxSvoWLgnCFeLMCaBBBf/bAjTs2yGSSBO7ssnOk=; b=bWMZIc1woW2La9dWJb3QsPTm7vAdLOcF0J3eNeiyhvX1HUh5Fn+TLBeHcJq2Khyt4p av77O8uexoJBf3OXi5ycoHApVqV9sLSHexE3K+008LD6T0q8UjWB8KjtjDj+WSJA7SH7 EuO0M0OqXoGTwnqOq9J0em7E9DkQ/9smT/vFCy78RVDObPyM3vjBmzD25oGN23HFkF6/ T3sQHpdXYpuCBVOr3FSfXW5XseFud8UrKKtCsG5KkgQTbJSZ64KMPvFvcOaNckx+xa7M jAB8iES+FQqBYlXajSd9oTX+XI2CTuOZ/a5rgDHrKrUkx3VNRKzKsCsSJsbs62IMtYwy ZuRA== X-Forwarded-Encrypted: i=1; AJvYcCWsbCB3bJpaypcCRH4BkDGiuvmkqrxMXtb4wyq/bbrVKEHKc9Dj7B+ol6srI2WbeUXIepYGrH9YCoOdAfPT@vger.kernel.org, AJvYcCX83N7kDMuOBTpdjG5IccP+auF8yXJtmK9Gk/h0eqpt2RwrwkE3yAA3k0+cF3OHrwbG/JqJMyn/7bCKRg==@vger.kernel.org, AJvYcCXLDNlJ4RZkPwNhJCzrAmXumkTCXhSPKJxQa2+aTZwjdDsDGiaVUP0qKTNMRJz1BioJdDsBpbPLkKay@vger.kernel.org X-Gm-Message-State: AOJu0YzFMQTZRDCxGN7ChmCs+yZrO0UTwth2hKTSlOSde7lzXWIGqHod 5htuiKp1VP+V2BnB0Cuyx9RHCUURe1tw0OdwZorH+HZEcN7PY7ae X-Google-Smtp-Source: AGHT+IEmeSOtc9Iyr4lHeC6Hz0t0cL8os8+bzqNv8I7gV7skDce9pH2TrwhZJlfkeIGSdqRoh0xSHA== X-Received: by 2002:a17:907:6d0d:b0:a9a:147d:fe9c with SMTP id a640c23a62f3a-a9de61d5d59mr865717766b.43.1730138394960; Mon, 28 Oct 2024 10:59:54 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:54 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 08/13] MIPS: CPS: Introduce struct cluster_boot_config Date: Mon, 28 Oct 2024 18:59:30 +0100 Message-Id: <20241028175935.51250-9-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paul Burton In preparation for supporting multi-cluster systems, introduce a struct cluster_boot_config as an extra layer in the boot configuration maintained by the MIPS Coherent Processing System (CPS) SMP implementation. For now only one struct cluster_boot_config will be allocated & we'll simply defererence its core_config field to find the struct core_boot_config array which can be used to boot as usual. Signed-off-by: Paul Burton Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- arch/mips/include/asm/smp-cps.h | 6 ++- arch/mips/kernel/asm-offsets.c | 3 ++ arch/mips/kernel/cps-vec.S | 19 ++++++-- arch/mips/kernel/pm-cps.c | 5 +- arch/mips/kernel/smp-cps.c | 82 +++++++++++++++++++++------------ 5 files changed, 81 insertions(+), 34 deletions(-) diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cp= s.h index ab94e50f62b8..a629e948a6fd 100644 --- a/arch/mips/include/asm/smp-cps.h +++ b/arch/mips/include/asm/smp-cps.h @@ -22,7 +22,11 @@ struct core_boot_config { struct vpe_boot_config *vpe_config; }; =20 -extern struct core_boot_config *mips_cps_core_bootcfg; +struct cluster_boot_config { + struct core_boot_config *core_config; +}; + +extern struct cluster_boot_config *mips_cps_cluster_bootcfg; =20 extern void mips_cps_core_boot(int cca, void __iomem *gcr_base); extern void mips_cps_core_init(void); diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c index cb1045ebab06..b29944160b28 100644 --- a/arch/mips/kernel/asm-offsets.c +++ b/arch/mips/kernel/asm-offsets.c @@ -404,6 +404,9 @@ void output_cps_defines(void) { COMMENT(" MIPS CPS offsets. "); =20 + OFFSET(CLUSTERBOOTCFG_CORECONFIG, cluster_boot_config, core_config); + DEFINE(CLUSTERBOOTCFG_SIZE, sizeof(struct cluster_boot_config)); + OFFSET(COREBOOTCFG_VPEMASK, core_boot_config, vpe_mask); OFFSET(COREBOOTCFG_VPECONFIG, core_boot_config, vpe_config); DEFINE(COREBOOTCFG_SIZE, sizeof(struct core_boot_config)); diff --git a/arch/mips/kernel/cps-vec.S b/arch/mips/kernel/cps-vec.S index f876309130ad..2ae7034a3d5c 100644 --- a/arch/mips/kernel/cps-vec.S +++ b/arch/mips/kernel/cps-vec.S @@ -19,6 +19,10 @@ #define GCR_CPC_BASE_OFS 0x0088 #define GCR_CL_COHERENCE_OFS 0x2008 #define GCR_CL_ID_OFS 0x2028 +#define CM3_GCR_Cx_ID_CLUSTER_SHF 8 +#define CM3_GCR_Cx_ID_CLUSTER_MSK (0xff << 8) +#define CM3_GCR_Cx_ID_CORENUM_SHF 0 +#define CM3_GCR_Cx_ID_CORENUM_MSK (0xff << 0) =20 #define CPC_CL_VC_STOP_OFS 0x2020 #define CPC_CL_VC_RUN_OFS 0x2028 @@ -271,12 +275,21 @@ LEAF(mips_cps_core_init) */ LEAF(mips_cps_get_bootcfg) /* Calculate a pointer to this cores struct core_boot_config */ + PTR_LA v0, mips_cps_cluster_bootcfg + PTR_L v0, 0(v0) lw t0, GCR_CL_ID_OFS(s1) +#ifdef CONFIG_CPU_MIPSR6 + ext t1, t0, CM3_GCR_Cx_ID_CLUSTER_SHF, 8 + li t2, CLUSTERBOOTCFG_SIZE + mul t1, t1, t2 + PTR_ADDU \ + v0, v0, t1 +#endif + PTR_L v0, CLUSTERBOOTCFG_CORECONFIG(v0) + andi t0, t0, CM3_GCR_Cx_ID_CORENUM_MSK li t1, COREBOOTCFG_SIZE mul t0, t0, t1 - PTR_LA t1, mips_cps_core_bootcfg - PTR_L t1, 0(t1) - PTR_ADDU v0, t0, t1 + PTR_ADDU v0, v0, t0 =20 /* Calculate this VPEs ID. If the core doesn't support MT use 0 */ li t9, 0 diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index 9369a8dc385e..3de0e05e0511 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -101,12 +101,14 @@ static void coupled_barrier(atomic_t *a, unsigned onl= ine) int cps_pm_enter_state(enum cps_pm_state state) { unsigned cpu =3D smp_processor_id(); + unsigned int cluster =3D cpu_cluster(¤t_cpu_data); unsigned core =3D cpu_core(¤t_cpu_data); unsigned online, left; cpumask_t *coupled_mask =3D this_cpu_ptr(&online_coupled); u32 *core_ready_count, *nc_core_ready_count; void *nc_addr; cps_nc_entry_fn entry; + struct cluster_boot_config *cluster_cfg; struct core_boot_config *core_cfg; struct vpe_boot_config *vpe_cfg; atomic_t *barrier; @@ -136,7 +138,8 @@ int cps_pm_enter_state(enum cps_pm_state state) if (!mips_cps_smp_in_use()) return -EINVAL; =20 - core_cfg =3D &mips_cps_core_bootcfg[core]; + cluster_cfg =3D &mips_cps_cluster_bootcfg[cluster]; + core_cfg =3D &cluster_cfg->core_config[core]; vpe_cfg =3D &core_cfg->vpe_config[cpu_vpe_id(¤t_cpu_data)]; vpe_cfg->pc =3D (unsigned long)mips_cps_pm_restore; vpe_cfg->gp =3D (unsigned long)current_thread_info(); diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c index 395622c37325..f71e2bb58318 100644 --- a/arch/mips/kernel/smp-cps.c +++ b/arch/mips/kernel/smp-cps.c @@ -40,7 +40,7 @@ static DECLARE_BITMAP(core_power, NR_CPUS); static uint32_t core_entry_reg; static phys_addr_t cps_vec_pa; =20 -struct core_boot_config *mips_cps_core_bootcfg; +struct cluster_boot_config *mips_cps_cluster_bootcfg; =20 static unsigned __init core_vpe_count(unsigned int cluster, unsigned core) { @@ -212,8 +212,10 @@ static void __init cps_smp_setup(void) =20 static void __init cps_prepare_cpus(unsigned int max_cpus) { - unsigned ncores, core_vpes, c, cca; + unsigned int nclusters, ncores, core_vpes, c, cl, cca; bool cca_unsuitable, cores_limited; + struct cluster_boot_config *cluster_bootcfg; + struct core_boot_config *core_bootcfg; =20 mips_mt_set_cpuoptions(); =20 @@ -255,40 +257,54 @@ static void __init cps_prepare_cpus(unsigned int max_= cpus) =20 setup_cps_vecs(); =20 - /* Allocate core boot configuration structs */ - ncores =3D mips_cps_numcores(0); - mips_cps_core_bootcfg =3D kcalloc(ncores, sizeof(*mips_cps_core_bootcfg), - GFP_KERNEL); - if (!mips_cps_core_bootcfg) { - pr_err("Failed to allocate boot config for %u cores\n", ncores); - goto err_out; - } + /* Allocate cluster boot configuration structs */ + nclusters =3D mips_cps_numclusters(); + mips_cps_cluster_bootcfg =3D kcalloc(nclusters, + sizeof(*mips_cps_cluster_bootcfg), + GFP_KERNEL); =20 - /* Allocate VPE boot configuration structs */ - for (c =3D 0; c < ncores; c++) { - core_vpes =3D core_vpe_count(0, c); - mips_cps_core_bootcfg[c].vpe_config =3D kcalloc(core_vpes, - sizeof(*mips_cps_core_bootcfg[c].vpe_config), - GFP_KERNEL); - if (!mips_cps_core_bootcfg[c].vpe_config) { - pr_err("Failed to allocate %u VPE boot configs\n", - core_vpes); + for (cl =3D 0; cl < nclusters; cl++) { + /* Allocate core boot configuration structs */ + ncores =3D mips_cps_numcores(cl); + core_bootcfg =3D kcalloc(ncores, sizeof(*core_bootcfg), + GFP_KERNEL); + if (!core_bootcfg) goto err_out; + mips_cps_cluster_bootcfg[cl].core_config =3D core_bootcfg; + + /* Allocate VPE boot configuration structs */ + for (c =3D 0; c < ncores; c++) { + core_vpes =3D core_vpe_count(cl, c); + core_bootcfg[c].vpe_config =3D kcalloc(core_vpes, + sizeof(*core_bootcfg[c].vpe_config), + GFP_KERNEL); + if (!core_bootcfg[c].vpe_config) + goto err_out; } } =20 /* Mark this CPU as booted */ - atomic_set(&mips_cps_core_bootcfg[cpu_core(¤t_cpu_data)].vpe_mask, - 1 << cpu_vpe_id(¤t_cpu_data)); + cl =3D cpu_cluster(¤t_cpu_data); + c =3D cpu_core(¤t_cpu_data); + cluster_bootcfg =3D &mips_cps_cluster_bootcfg[cl]; + core_bootcfg =3D &cluster_bootcfg->core_config[c]; + atomic_set(&core_bootcfg->vpe_mask, 1 << cpu_vpe_id(¤t_cpu_data)); =20 return; err_out: /* Clean up allocations */ - if (mips_cps_core_bootcfg) { - for (c =3D 0; c < ncores; c++) - kfree(mips_cps_core_bootcfg[c].vpe_config); - kfree(mips_cps_core_bootcfg); - mips_cps_core_bootcfg =3D NULL; + if (mips_cps_cluster_bootcfg) { + for (cl =3D 0; cl < nclusters; cl++) { + cluster_bootcfg =3D &mips_cps_cluster_bootcfg[cl]; + ncores =3D mips_cps_numcores(cl); + for (c =3D 0; c < ncores; c++) { + core_bootcfg =3D &cluster_bootcfg->core_config[c]; + kfree(core_bootcfg->vpe_config); + } + kfree(mips_cps_cluster_bootcfg[c].core_config); + } + kfree(mips_cps_cluster_bootcfg); + mips_cps_cluster_bootcfg =3D NULL; } =20 /* Effectively disable SMP by declaring CPUs not present */ @@ -376,17 +392,23 @@ static void boot_core(unsigned int core, unsigned int= vpe_id) =20 static void remote_vpe_boot(void *dummy) { + unsigned int cluster =3D cpu_cluster(¤t_cpu_data); unsigned core =3D cpu_core(¤t_cpu_data); - struct core_boot_config *core_cfg =3D &mips_cps_core_bootcfg[core]; + struct cluster_boot_config *cluster_cfg =3D + &mips_cps_cluster_bootcfg[cluster]; + struct core_boot_config *core_cfg =3D &cluster_cfg->core_config[core]; =20 mips_cps_boot_vpes(core_cfg, cpu_vpe_id(¤t_cpu_data)); } =20 static int cps_boot_secondary(int cpu, struct task_struct *idle) { + unsigned int cluster =3D cpu_cluster(&cpu_data[cpu]); unsigned core =3D cpu_core(&cpu_data[cpu]); unsigned vpe_id =3D cpu_vpe_id(&cpu_data[cpu]); - struct core_boot_config *core_cfg =3D &mips_cps_core_bootcfg[core]; + struct cluster_boot_config *cluster_cfg =3D + &mips_cps_cluster_bootcfg[cluster]; + struct core_boot_config *core_cfg =3D &cluster_cfg->core_config[core]; struct vpe_boot_config *vpe_cfg =3D &core_cfg->vpe_config[vpe_id]; unsigned int remote; int err; @@ -544,12 +566,14 @@ static void cps_kexec_nonboot_cpu(void) static int cps_cpu_disable(void) { unsigned cpu =3D smp_processor_id(); + struct cluster_boot_config *cluster_cfg; struct core_boot_config *core_cfg; =20 if (!cps_pm_support_state(CPS_PM_POWER_GATED)) return -EINVAL; =20 - core_cfg =3D &mips_cps_core_bootcfg[cpu_core(¤t_cpu_data)]; + cluster_cfg =3D &mips_cps_cluster_bootcfg[cpu_cluster(¤t_cpu_data)]; + core_cfg =3D &cluster_cfg->core_config[cpu_core(¤t_cpu_data)]; atomic_sub(1 << cpu_vpe_id(¤t_cpu_data), &core_cfg->vpe_mask); smp_mb__after_atomic(); set_cpu_online(cpu, false); --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7B1D1E0DB5; Mon, 28 Oct 2024 17:59:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138401; cv=none; b=DdnZuLJRrYvxvC6KhHaiN8V9iPSJffGVpTuRYbzG23hOJrVz7Qsf4bSRzRrhmsWHtDHVLNkl9RFSnbgAw5uXgPGFAdqb2Ct1GsXt1CURrJxIISaLM5Qlqjhhr5b1YNwaXe0pVPlJmZgIOdShdE04CkTUoh3YgH1xjE5wBG+nFoE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138401; c=relaxed/simple; bh=wiHGtSxfl3n0xkDe/1B57xJyBQzlskJcWg8mMgof1LQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qTIEvel47vsPk+Rghv2ELkrtZo4X0ZMvMireQ/HRGmImpWal/zenWFD2y+OMKoX/C9CzkMpWRRb0CH+LaE1ivhkXFrGHXVCeIMgump9sqipIVQQLejVaAucwKOCYyF+Rtppz74vK3WCZL9qHrPU92/EdTjVwYvOzYjVHCgcc+EI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WgWzMqmo; arc=none smtp.client-ip=209.85.218.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WgWzMqmo" Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-a9a1b71d7ffso695142466b.1; Mon, 28 Oct 2024 10:59:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138397; x=1730743197; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cuV9PDzEHrhcFYzAlFi29DYFvb0ar6cl1lK3srcUt1I=; b=WgWzMqmoGH7AhKFtr6MbNBcCNuOLas8XjYPIkYEhyzSzVoGgcIKfbCGL2XFPiN91gD TSTTTk+E9/wOg8dixoOZ7zOwS9hGbbfathbpzdYSyuhfkTNZ6Ygedy85g5gTAla84qFP iMIoJZmCZnrC+VrRDoxd0qVv8uggZOg7NfWG0+7AULlXIqoZUQDWfjP9Sq0e0N/vbk3B 4YhLyF7y2MZpmUdb18nzqcoxH4Ge5n9kLDrS3FIZn6dA3HBstPpKwFQmE0cwkxW+JVPN rCLvX1j+fk07wggRygIy69MM7HHKimuynwFd8LBwe3ToK7+gbswbVvP+ul4qhzCcwZ20 RDOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138397; x=1730743197; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cuV9PDzEHrhcFYzAlFi29DYFvb0ar6cl1lK3srcUt1I=; b=eTP+Ue4DCMjTE7A464+R5Dihl7RQFSuCwReX8YnFXhIEnGi2r1sNB/E0psnSWbU1n8 IlEd+pMSaG3ZFp6nMCyE1T5hLd8IdSf1br9+y/YrIhRNihRIlwiLSqkM2VeCb37APga1 z9vJFjoq7pjlr8Rl43TpGBM9CzOvI1ewiT20Iv8OrXxYg7epf1DB2UjhmZjnJKQ5Exjb Q0xgQWtuabz4y8d3bNiR3owF7+jBgkaWJHtcuRqoziYaPlFkCrAZP8W52pahPXykIFXC BVcVhJp+fR1U+nboUtIh54iltIMZvt2vdP5/HCBhLi2FZZpntVwK8yoY50Nm7P2L9K4C BAvA== X-Forwarded-Encrypted: i=1; AJvYcCVOWrXLbXHPRklkZyKBG3HHbs1vjQABtHQN5opHIDVVioZLUl9mhAJiNvYAq5sM7uPYmwVZhX6Fn0lu/fOt@vger.kernel.org, AJvYcCXTWP2U/35Kb2ALRXL7QBtdP5zCiLSLkuZ4oUfuhhcWkyPGkj50MZJQ3zHD6jfja6bDqsrpKiUqiIlO@vger.kernel.org, AJvYcCXzn/ERo3ARw44gRBYl8k4KVhiEbEaSoYHe8Bcbputz2abqaTpDQY9jvMGANzgeFsendiG4gQCvXJZmFg==@vger.kernel.org X-Gm-Message-State: AOJu0YzRQPCRnCQZHCSjT+EBFqhCpRsL87FyP72fwrXx/IGbHyr1cQfr 7TzDuwpe5OW8QuUgeIshT/XeFN268PdA6fwpSSMGtfYF1Y+Rz3Kf X-Google-Smtp-Source: AGHT+IF6PpweLBWPFmSMHCgyDq/AB+g9BAzY6m5iUbJ6saY0o+mz14J/YtCK+tskBjH5wauHcZKvQQ== X-Received: by 2002:a17:906:7949:b0:a9a:80cc:d7b0 with SMTP id a640c23a62f3a-a9de6157a62mr674908966b.44.1730138396874; Mon, 28 Oct 2024 10:59:56 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:56 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 09/13] MIPS: CPS: Boot CPUs in secondary clusters Date: Mon, 28 Oct 2024 18:59:31 +0100 Message-Id: <20241028175935.51250-10-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Paul Burton Probe for & boot CPUs (cores & VPs) in secondary clusters (ie. not the cluster that began booting Linux) when they are present in systems with CM 3.5 or higher. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- arch/mips/include/asm/mips-cm.h | 18 +++ arch/mips/include/asm/smp-cps.h | 1 + arch/mips/kernel/mips-cm.c | 4 +- arch/mips/kernel/smp-cps.c | 205 ++++++++++++++++++++++++++++---- 4 files changed, 207 insertions(+), 21 deletions(-) diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-c= m.h index 1e782275850a..4d47163647dd 100644 --- a/arch/mips/include/asm/mips-cm.h +++ b/arch/mips/include/asm/mips-cm.h @@ -255,6 +255,12 @@ GCR_ACCESSOR_RW(32, 0x130, l2_config) GCR_ACCESSOR_RO(32, 0x150, sys_config2) #define CM_GCR_SYS_CONFIG2_MAXVPW GENMASK(3, 0) =20 +/* GCR_L2-RAM_CONFIG - Configuration & status of L2 cache RAMs */ +GCR_ACCESSOR_RW(64, 0x240, l2_ram_config) +#define CM_GCR_L2_RAM_CONFIG_PRESENT BIT(31) +#define CM_GCR_L2_RAM_CONFIG_HCI_DONE BIT(30) +#define CM_GCR_L2_RAM_CONFIG_HCI_SUPPORTED BIT(29) + /* GCR_L2_PFT_CONTROL - Controls hardware L2 prefetching */ GCR_ACCESSOR_RW(32, 0x300, l2_pft_control) #define CM_GCR_L2_PFT_CONTROL_PAGEMASK GENMASK(31, 12) @@ -266,6 +272,18 @@ GCR_ACCESSOR_RW(32, 0x308, l2_pft_control_b) #define CM_GCR_L2_PFT_CONTROL_B_CEN BIT(8) #define CM_GCR_L2_PFT_CONTROL_B_PORTID GENMASK(7, 0) =20 +/* GCR_L2_TAG_ADDR - Access addresses in L2 cache tags */ +GCR_ACCESSOR_RW(64, 0x600, l2_tag_addr) + +/* GCR_L2_TAG_STATE - Access L2 cache tag state */ +GCR_ACCESSOR_RW(64, 0x608, l2_tag_state) + +/* GCR_L2_DATA - Access data in L2 cache lines */ +GCR_ACCESSOR_RW(64, 0x610, l2_data) + +/* GCR_L2_ECC - Access ECC information from L2 cache lines */ +GCR_ACCESSOR_RW(64, 0x618, l2_ecc) + /* GCR_L2SM_COP - L2 cache op state machine control */ GCR_ACCESSOR_RW(32, 0x620, l2sm_cop) #define CM_GCR_L2SM_COP_PRESENT BIT(31) diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cp= s.h index a629e948a6fd..10d3ebd890cb 100644 --- a/arch/mips/include/asm/smp-cps.h +++ b/arch/mips/include/asm/smp-cps.h @@ -23,6 +23,7 @@ struct core_boot_config { }; =20 struct cluster_boot_config { + unsigned long *core_power; struct core_boot_config *core_config; }; =20 diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c index 3eb2cfb893e1..9854bc2b6895 100644 --- a/arch/mips/kernel/mips-cm.c +++ b/arch/mips/kernel/mips-cm.c @@ -308,7 +308,9 @@ void mips_cm_lock_other(unsigned int cluster, unsigned = int core, FIELD_PREP(CM3_GCR_Cx_OTHER_VP, vp); =20 if (cm_rev >=3D CM_REV_CM3_5) { - val |=3D CM_GCR_Cx_OTHER_CLUSTER_EN; + if (cluster !=3D cpu_cluster(¤t_cpu_data)) + val |=3D CM_GCR_Cx_OTHER_CLUSTER_EN; + val |=3D CM_GCR_Cx_OTHER_GIC_EN; val |=3D FIELD_PREP(CM_GCR_Cx_OTHER_CLUSTER, cluster); val |=3D FIELD_PREP(CM_GCR_Cx_OTHER_BLOCK, block); } else { diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c index f71e2bb58318..4f344c890a23 100644 --- a/arch/mips/kernel/smp-cps.c +++ b/arch/mips/kernel/smp-cps.c @@ -36,12 +36,56 @@ enum label_id { =20 UASM_L_LA(_not_nmi) =20 -static DECLARE_BITMAP(core_power, NR_CPUS); static uint32_t core_entry_reg; static phys_addr_t cps_vec_pa; =20 struct cluster_boot_config *mips_cps_cluster_bootcfg; =20 +static void power_up_other_cluster(unsigned int cluster) +{ + u32 stat, seq_state; + unsigned int timeout; + + mips_cm_lock_other(cluster, CM_GCR_Cx_OTHER_CORE_CM, 0, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); + stat =3D read_cpc_co_stat_conf(); + mips_cm_unlock_other(); + + seq_state =3D stat & CPC_Cx_STAT_CONF_SEQSTATE; + seq_state >>=3D __ffs(CPC_Cx_STAT_CONF_SEQSTATE); + if (seq_state =3D=3D CPC_Cx_STAT_CONF_SEQSTATE_U5) + return; + + /* Set endianness & power up the CM */ + mips_cm_lock_other(cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + write_cpc_redir_sys_config(IS_ENABLED(CONFIG_CPU_BIG_ENDIAN)); + write_cpc_redir_pwrup_ctl(1); + mips_cm_unlock_other(); + + /* Wait for the CM to start up */ + timeout =3D 1000; + mips_cm_lock_other(cluster, CM_GCR_Cx_OTHER_CORE_CM, 0, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); + while (1) { + stat =3D read_cpc_co_stat_conf(); + seq_state =3D stat & CPC_Cx_STAT_CONF_SEQSTATE; + seq_state >>=3D __ffs(CPC_Cx_STAT_CONF_SEQSTATE); + if (seq_state =3D=3D CPC_Cx_STAT_CONF_SEQSTATE_U5) + break; + + if (timeout) { + mdelay(1); + timeout--; + } else { + pr_warn("Waiting for cluster %u CM to power up... STAT_CONF=3D0x%x\n", + cluster, stat); + mdelay(1000); + } + } + + mips_cm_unlock_other(); +} + static unsigned __init core_vpe_count(unsigned int cluster, unsigned core) { return min(smp_max_threads, mips_cps_numvps(cluster, core)); @@ -152,6 +196,9 @@ static void __init cps_smp_setup(void) pr_cont(","); pr_cont("{"); =20 + if (mips_cm_revision() >=3D CM_REV_CM3_5) + power_up_other_cluster(cl); + ncores =3D mips_cps_numcores(cl); for (c =3D 0; c < ncores; c++) { core_vpes =3D core_vpe_count(cl, c); @@ -179,8 +226,8 @@ static void __init cps_smp_setup(void) =20 /* Indicate present CPUs (CPU being synonymous with VPE) */ for (v =3D 0; v < min_t(unsigned, nvpes, NR_CPUS); v++) { - set_cpu_possible(v, cpu_cluster(&cpu_data[v]) =3D=3D 0); - set_cpu_present(v, cpu_cluster(&cpu_data[v]) =3D=3D 0); + set_cpu_possible(v, true); + set_cpu_present(v, true); __cpu_number_map[v] =3D v; __cpu_logical_map[v] =3D v; } @@ -188,9 +235,6 @@ static void __init cps_smp_setup(void) /* Set a coherent default CCA (CWB) */ change_c0_config(CONF_CM_CMASK, 0x5); =20 - /* Core 0 is powered up (we're running on it) */ - bitmap_set(core_power, 0, 1); - /* Initialise core 0 */ mips_cps_core_init(); =20 @@ -272,6 +316,10 @@ static void __init cps_prepare_cpus(unsigned int max_c= pus) goto err_out; mips_cps_cluster_bootcfg[cl].core_config =3D core_bootcfg; =20 + mips_cps_cluster_bootcfg[cl].core_power =3D + kcalloc(BITS_TO_LONGS(ncores), sizeof(unsigned long), + GFP_KERNEL); + /* Allocate VPE boot configuration structs */ for (c =3D 0; c < ncores; c++) { core_vpes =3D core_vpe_count(cl, c); @@ -283,11 +331,12 @@ static void __init cps_prepare_cpus(unsigned int max_= cpus) } } =20 - /* Mark this CPU as booted */ + /* Mark this CPU as powered up & booted */ cl =3D cpu_cluster(¤t_cpu_data); c =3D cpu_core(¤t_cpu_data); cluster_bootcfg =3D &mips_cps_cluster_bootcfg[cl]; core_bootcfg =3D &cluster_bootcfg->core_config[c]; + bitmap_set(cluster_bootcfg->core_power, cpu_core(¤t_cpu_data), 1); atomic_set(&core_bootcfg->vpe_mask, 1 << cpu_vpe_id(¤t_cpu_data)); =20 return; @@ -315,13 +364,118 @@ static void __init cps_prepare_cpus(unsigned int max= _cpus) } } =20 -static void boot_core(unsigned int core, unsigned int vpe_id) +static void init_cluster_l2(void) { - u32 stat, seq_state; - unsigned timeout; + u32 l2_cfg, l2sm_cop, result; + + while (1) { + l2_cfg =3D read_gcr_redir_l2_ram_config(); + + /* If HCI is not supported, use the state machine below */ + if (!(l2_cfg & CM_GCR_L2_RAM_CONFIG_PRESENT)) + break; + if (!(l2_cfg & CM_GCR_L2_RAM_CONFIG_HCI_SUPPORTED)) + break; + + /* If the HCI_DONE bit is set, we're finished */ + if (l2_cfg & CM_GCR_L2_RAM_CONFIG_HCI_DONE) + return; + } + + l2sm_cop =3D read_gcr_redir_l2sm_cop(); + if (WARN(!(l2sm_cop & CM_GCR_L2SM_COP_PRESENT), + "L2 init not supported on this system yet")) + return; + + /* Clear L2 tag registers */ + write_gcr_redir_l2_tag_state(0); + write_gcr_redir_l2_ecc(0); + + /* Ensure the L2 tag writes complete before the state machine starts */ + mb(); + + /* Wait for the L2 state machine to be idle */ + do { + l2sm_cop =3D read_gcr_redir_l2sm_cop(); + } while (l2sm_cop & CM_GCR_L2SM_COP_RUNNING); + + /* Start a store tag operation */ + l2sm_cop =3D CM_GCR_L2SM_COP_TYPE_IDX_STORETAG; + l2sm_cop <<=3D __ffs(CM_GCR_L2SM_COP_TYPE); + l2sm_cop |=3D CM_GCR_L2SM_COP_CMD_START; + write_gcr_redir_l2sm_cop(l2sm_cop); + + /* Ensure the state machine starts before we poll for completion */ + mb(); + + /* Wait for the operation to be complete */ + do { + l2sm_cop =3D read_gcr_redir_l2sm_cop(); + result =3D l2sm_cop & CM_GCR_L2SM_COP_RESULT; + result >>=3D __ffs(CM_GCR_L2SM_COP_RESULT); + } while (!result); + + WARN(result !=3D CM_GCR_L2SM_COP_RESULT_DONE_OK, + "L2 state machine failed cache init with error %u\n", result); +} + +static void boot_core(unsigned int cluster, unsigned int core, + unsigned int vpe_id) +{ + struct cluster_boot_config *cluster_cfg; + u32 access, stat, seq_state; + unsigned int timeout, ncores; + + cluster_cfg =3D &mips_cps_cluster_bootcfg[cluster]; + ncores =3D mips_cps_numcores(cluster); + + if ((cluster !=3D cpu_cluster(¤t_cpu_data)) && + bitmap_empty(cluster_cfg->core_power, ncores)) { + power_up_other_cluster(cluster); + + mips_cm_lock_other(cluster, core, 0, + CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + /* Ensure cluster GCRs are where we expect */ + write_gcr_redir_base(read_gcr_base()); + write_gcr_redir_cpc_base(read_gcr_cpc_base()); + write_gcr_redir_gic_base(read_gcr_gic_base()); + + init_cluster_l2(); + + /* Mirror L2 configuration */ + write_gcr_redir_l2_only_sync_base(read_gcr_l2_only_sync_base()); + write_gcr_redir_l2_pft_control(read_gcr_l2_pft_control()); + write_gcr_redir_l2_pft_control_b(read_gcr_l2_pft_control_b()); + + /* Mirror ECC/parity setup */ + write_gcr_redir_err_control(read_gcr_err_control()); + + /* Set BEV base */ + write_gcr_redir_bev_base(core_entry_reg); + + mips_cm_unlock_other(); + } + + if (cluster !=3D cpu_cluster(¤t_cpu_data)) { + mips_cm_lock_other(cluster, core, 0, + CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + /* Ensure the core can access the GCRs */ + access =3D read_gcr_redir_access(); + access |=3D BIT(core); + write_gcr_redir_access(access); + + mips_cm_unlock_other(); + } else { + /* Ensure the core can access the GCRs */ + access =3D read_gcr_access(); + access |=3D BIT(core); + write_gcr_access(access); + } =20 /* Select the appropriate core */ - mips_cm_lock_other(0, core, 0, CM_GCR_Cx_OTHER_BLOCK_LOCAL); + mips_cm_lock_other(cluster, core, 0, CM_GCR_Cx_OTHER_BLOCK_LOCAL); =20 /* Set its reset vector */ write_gcr_co_reset_base(core_entry_reg); @@ -387,7 +541,17 @@ static void boot_core(unsigned int core, unsigned int = vpe_id) mips_cm_unlock_other(); =20 /* The core is now powered up */ - bitmap_set(core_power, core, 1); + bitmap_set(cluster_cfg->core_power, core, 1); + + /* + * Restore CM_PWRUP=3D0 so that the CM can power down if all the cores in + * the cluster do (eg. if they're all removed via hotplug. + */ + if (mips_cm_revision() >=3D CM_REV_CM3_5) { + mips_cm_lock_other(cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + write_cpc_redir_pwrup_ctl(0); + mips_cm_unlock_other(); + } } =20 static void remote_vpe_boot(void *dummy) @@ -413,10 +577,6 @@ static int cps_boot_secondary(int cpu, struct task_str= uct *idle) unsigned int remote; int err; =20 - /* We don't yet support booting CPUs in other clusters */ - if (cpu_cluster(&cpu_data[cpu]) !=3D cpu_cluster(&raw_current_cpu_data)) - return -ENOSYS; - vpe_cfg->pc =3D (unsigned long)&smp_bootstrap; vpe_cfg->sp =3D __KSTK_TOS(idle); vpe_cfg->gp =3D (unsigned long)task_thread_info(idle); @@ -425,14 +585,15 @@ static int cps_boot_secondary(int cpu, struct task_st= ruct *idle) =20 preempt_disable(); =20 - if (!test_bit(core, core_power)) { + if (!test_bit(core, cluster_cfg->core_power)) { /* Boot a VPE on a powered down core */ - boot_core(core, vpe_id); + boot_core(cluster, core, vpe_id); goto out; } =20 if (cpu_has_vp) { - mips_cm_lock_other(0, core, vpe_id, CM_GCR_Cx_OTHER_BLOCK_LOCAL); + mips_cm_lock_other(cluster, core, vpe_id, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); write_gcr_co_reset_base(core_entry_reg); mips_cm_unlock_other(); } @@ -639,11 +800,15 @@ static void cps_cpu_die(unsigned int cpu) { } =20 static void cps_cleanup_dead_cpu(unsigned cpu) { + unsigned int cluster =3D cpu_cluster(&cpu_data[cpu]); unsigned core =3D cpu_core(&cpu_data[cpu]); unsigned int vpe_id =3D cpu_vpe_id(&cpu_data[cpu]); ktime_t fail_time; unsigned stat; int err; + struct cluster_boot_config *cluster_cfg; + + cluster_cfg =3D &mips_cps_cluster_bootcfg[cluster]; =20 /* * Now wait for the CPU to actually offline. Without doing this that @@ -695,7 +860,7 @@ static void cps_cleanup_dead_cpu(unsigned cpu) } while (1); =20 /* Indicate the core is powered off */ - bitmap_clear(core_power, core, 1); + bitmap_clear(cluster_cfg->core_power, core, 1); } else if (cpu_has_mipsmt) { /* * Have a CPU with access to the offlined CPUs registers wait --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B660B1E0DEF; Mon, 28 Oct 2024 18:00:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138403; cv=none; b=Wl2uO3MIvKXBKhQ1WM8fbHmB5cSVSDcuQKT1gX4STt6LrqT4hf566M5iHLjlrJGgpaTgy984j96Dy8Q1neemAPqURRhgTijS0KJt+3xjKXzQZbcdy6PsoCKYBFMfwN/cGOZS3I2dZG6Fia2Cc2VU6OVbtHxzG6ubaRRpyAWztIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138403; c=relaxed/simple; bh=rS0N/3KIrMgeRTv6NPz2ZD31BGLzdZCfMlctjmDcBHs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EWRJ5bXEcMEUywS6pOoKCH5BXaaXV+gijx+xFW+V2LABdY+aGBnIc/w7myrS0S439Fcun8han1j8lHqQ4Uq5NHUi5MJ8h5YvkCl4WdaKd15iNACOKoHv668MCpeeLHXhz1pQbyj09Ld1GjBqRaFGYlqh9NxsQVNDYjxQCvzz+cE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fQkuq3h2; arc=none smtp.client-ip=209.85.208.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fQkuq3h2" Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-5cbb719839eso4345746a12.2; Mon, 28 Oct 2024 11:00:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138399; x=1730743199; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sHycpaE7Vy70YH+3wi7D/PPKiZa8yZVDKmv2FNPd7q8=; b=fQkuq3h2g+LxCveKmcTA5V7hu2YfkxXS3TxseDOL0bxSPBEZGC7Ak6BYNPUMAVHPkI 9vgbA0BLcqwVVbfwrUT9BHjhGIzgr5qwbP2BxxxQEzEEH0QUzNnDqqj1NJ3Il4CmqxRL TL31+sL6SoTN8Vaxj9o4WwisKQkzISID0iMQw+NuXbJlye8m0ydXHmYKBUuXrVulnAGq hyldy47LLPwam1KgjJi4CE13oRjCX+XNFt07CAAEYeRsMPOq5IbLPl6LOD5AiY09s0xd 6Wukh9pn06thFNsGdSklfylPstaoNOC4TEdrzEfoO4p5P7bALp63k6oUvkQjdWssgiyX CzXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138399; x=1730743199; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sHycpaE7Vy70YH+3wi7D/PPKiZa8yZVDKmv2FNPd7q8=; b=MAtkx0iNvtsFV0Wh/mqJIyZypY2fIeQf9+nkziR7PUGMXn0sU2oZOPOfn16vPzvg5y f5+D4SkjhyKA47QBmgz0feWS4HnnxBAohsFcckvPB41ymUvDFkVfxN6DuuClNR4JTaCW II6biZVf2HRvMx4LmRb8G0Ewa+4KzI8ybM1zhVXYMDb0pebP5xw1lx0mFU7WZQBfqoAa 5i+BA0akWsTllOES23QRnSdzAbZIM4PECCHKC88aFwGTn842eeCsnMFR15Q9082pz/cR 0cqVkSpSh+vZheui/blisOrtLZ2HALWmnuUrYPQopk9hHfjrgzEVyUm6d3MhbXGUz9XE PY0g== X-Forwarded-Encrypted: i=1; AJvYcCVyokoKd5CU00a8Ew9brbdpzAHPrba0cJBRcxVnfHnkkJiJkEl6CuBAPNc/SHLKogF2+Uh933lmoqwl7Q==@vger.kernel.org, AJvYcCW6TSfqqv8NAPVW/4cTf8u4fpip5+cN5x9UT0FVAjzZrosdDedqnfGdkAPEZ6P9UlJq+a0YkcqyU7Hp@vger.kernel.org, AJvYcCWA5EtoRnCm62B6nuTd7NdYJ37ErtxEw8iSwT6O9kFGdxdwrvotzHtYBtPmJNSh2Pa+5tu1lghvGWs3paJ3@vger.kernel.org X-Gm-Message-State: AOJu0YyfUW12JoX2huILuEOgu6+A8YWzvsr1XmvJZzqGW0ZB8oW5XGhY QVZDPmXN+qVTEEeGh1eaKHKBo+nxHYbqFmdNX0tghDaOabOPgvnf X-Google-Smtp-Source: AGHT+IFS31eKhtPRXSznANdPJXdFgnfjzo4LzQPOkHa4E4uNuYZrMxJvWY4hb9slKmh/+HVD1uKd1Q== X-Received: by 2002:a17:907:9486:b0:a99:f975:9fb8 with SMTP id a640c23a62f3a-a9de632ea47mr770978166b.53.1730138398883; Mon, 28 Oct 2024 10:59:58 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 10:59:58 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 10/13] dt-bindings: mips: cpu: Add property for broken HCI information Date: Mon, 28 Oct 2024 18:59:32 +0100 Message-Id: <20241028175935.51250-11-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Gregory CLEMENT Some CM3.5 reports show that Hardware Cache Initialization is complete, but in reality it's not the case. They also incorrectly indicate that Hardware Cache Initialization is supported. This optional property allows warning about this broken feature that cannot be detected at runtime. Signed-off-by: Gregory CLEMENT Signed-off-by: Aleksandar Rikalo Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- Documentation/devicetree/bindings/mips/cpus.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/Documentation/devicetree/bindings/mips/cpus.yaml b/Documentati= on/devicetree/bindings/mips/cpus.yaml index a85137add668..57e93c07ab1b 100644 --- a/Documentation/devicetree/bindings/mips/cpus.yaml +++ b/Documentation/devicetree/bindings/mips/cpus.yaml @@ -47,6 +47,12 @@ properties: clocks: maxItems: 1 =20 + cm3-l2-config-hci-broken: + type: boolean + description: + If present, indicates that the HCI (Hardware Cache Initialization) + information for the L2 cache in multi-cluster configuration is broke= n. + device_type: true =20 allOf: --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C32EC1E103C; Mon, 28 Oct 2024 18:00:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138405; cv=none; b=fu7hSTwVkjfPVNVPOHlsz/QdTppvXfXCoDefr9Hag8RuvHdP5gxdRqQVGaQnSiE+nrwMzsWxJwMffB7Y1Uz/pWNubwdsV+DJqSriWPl8QgC7eG0r73MSJ6ar6UndGLzaWi9HHfk3sAdZgGVuSs46IAa3Vpj/SXxaetD3V+fky7A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138405; c=relaxed/simple; bh=kIF9957r5L4wQ3ncmcvDTVAMQbE/zBtqFZPtCQuRNUQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ahAYcaraiYCGLRGoo9j3Aeug+6sQ7k6PmpUPRvL11WZ6C4zeMINeTxA57lwt9GJdEAVAriujvk16Lue6cnHB/ScWOlgcAQezVmtI24GPi2IkHd88DCUbENqTAQq/HH41KVnnvA7aHwK1tttk0FxWXzbx0KVWcIp/wUiIS5ZEgj8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lregVJzZ; arc=none smtp.client-ip=209.85.208.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lregVJzZ" Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-5cb72918bddso6100427a12.3; Mon, 28 Oct 2024 11:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138401; x=1730743201; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0zQMaFDQJWlsPtf7l7ch6SDTyUDA1/vrM9zQnzmMI/Q=; b=lregVJzZ0Z+5wUhHgCDlNEIDThTfxMhQ5/CTihGnQsLZ4y0oRJ38WDOKjNohU3mFG5 EhzhOlBWsP4/4NvxZ+yxuriOBGoAFVOaUJvtZyVkOtJ7Au1cNcW1vcjl+RZj+p/HEyqL P2njiXMB9fN+StfmXoj1Vt/PlTlAMMZEbjtNi4f/G1jXAkHRQsGcucEETg7lk6BUdKbd 6pZ75mG2QsBKJHwZv5arGBBbQSBJjlnS7Mhu8WIKOhjeIyHwzuKJUs7uh/r9SXR5OJQB m3/U/yJRRM4VrYpfylo2mFdFR/pcQxOhprDI6RtHD04QH2y4iUL3D2ZFeHHsXVVcT9Jg tCCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138401; x=1730743201; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0zQMaFDQJWlsPtf7l7ch6SDTyUDA1/vrM9zQnzmMI/Q=; b=PMOZ+m3Isk+Y3ehpb40xdRiIOoeyXh2X6DTwRbcv53zY+9emiKKP2QXEMa0YgXRtin 4doOlYEtWqx7RQchuuArYLrAvnkVQLTeCyLmd0tHfiZtR635+l/ScBTNU8IIKJDqTLe0 27gb5iZL+WuRj+zzjnxyr4Arfz40/Kfk/SAtNoUhgFgEinjCgZ6iCUHSltYJq3STu90i l+p0YPX0pGCPK+z+fRcnJyt9wtd3TMluvHlnE6NmkUY4o6rhMO/kJWvH2VWIpfFhLjiT Wc5ynUnfAX+Np4xB91uaNND2kMYuIaezJcKL/wvylbK1xT4uinZvVU7/Rgz+zVIONviV +yoA== X-Forwarded-Encrypted: i=1; AJvYcCVZ2VGvoYlKKAxkm5U8EISfVhdbbENyMzUmO6pL15Pz/0EVes7jHtAO2o+804mJhl9k22zOAaUVlMC4+A==@vger.kernel.org, AJvYcCX0Tva98/hAoDuDd1vFOqGnfYHCj9cqD3FP3rJbsF3u+Dfeor+M388wHB2jbFzqGl3GP4XqdsOyp6q3@vger.kernel.org, AJvYcCXqTl/ZA2+/WdR0SE6wsF2HjVcSNbmlM/mLJPM0exAoyw6DEqnD+DJvuuB6l+8z7eOYkOI/mRy9R5LTqaGi@vger.kernel.org X-Gm-Message-State: AOJu0Yweaud5KyLvb+p3s9TOflo6O71gse16JMFwegGxywpipSL6Ybvt vOBLISVG2GonF38yMzYt7pjjSUPb6SPrmqhFl1lCFLM7K/lhUitE X-Google-Smtp-Source: AGHT+IFqnX5pduxZANYQv//6T59Mipqh6SvMZuDqlQSJ+Vw7ZNoz6+YT7RHKAvn+U4U/L1CUj2lg/g== X-Received: by 2002:a17:907:9495:b0:a9a:5d15:26c2 with SMTP id a640c23a62f3a-a9de619c888mr874683166b.45.1730138400924; Mon, 28 Oct 2024 11:00:00 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.10.59.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 11:00:00 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 11/13] MIPS: CPS: Support broken HCI for multicluster Date: Mon, 28 Oct 2024 18:59:33 +0100 Message-Id: <20241028175935.51250-12-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Gregory CLEMENT Some CM3.5 devices incorrectly report that hardware cache initialization has completed, and also claim to support hardware cache initialization when they don't actually do so. This commit fixes this issue by retrieving the correct information from the device tree and allowing the system to bypass the hardware cache initialization step. Instead, it relies on manual operation. As a result, multi-user support is now possible for these CPUs. Signed-off-by: Gregory CLEMENT Signed-off-by: Aleksandar Rikalo Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- arch/mips/kernel/smp-cps.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c index 4f344c890a23..265cf52c0dd1 100644 --- a/arch/mips/kernel/smp-cps.c +++ b/arch/mips/kernel/smp-cps.c @@ -39,6 +39,7 @@ UASM_L_LA(_not_nmi) static uint32_t core_entry_reg; static phys_addr_t cps_vec_pa; =20 +static bool l2_hci_broken; struct cluster_boot_config *mips_cps_cluster_bootcfg; =20 static void power_up_other_cluster(unsigned int cluster) @@ -254,6 +255,22 @@ static void __init cps_smp_setup(void) #endif /* CONFIG_MIPS_MT_FPAFF */ } =20 +static void __init check_hci_quirk(void) +{ + struct device_node *np; + + np =3D of_cpu_device_node_get(0); + if (!np) { + pr_debug("%s: No cpu node in the device tree\n", __func__); + return; + } + + if (of_property_read_bool(np, "cm3-l2-config-hci-broken")) { + pr_info("HCI (Hardware Cache Init for the L2 cache) in GCR_L2_RAM_CONFIG= from the CM3 is broken"); + l2_hci_broken =3D true; + } +} + static void __init cps_prepare_cpus(unsigned int max_cpus) { unsigned int nclusters, ncores, core_vpes, c, cl, cca; @@ -307,6 +324,9 @@ static void __init cps_prepare_cpus(unsigned int max_cp= us) sizeof(*mips_cps_cluster_bootcfg), GFP_KERNEL); =20 + if (nclusters > 1) + check_hci_quirk(); + for (cl =3D 0; cl < nclusters; cl++) { /* Allocate core boot configuration structs */ ncores =3D mips_cps_numcores(cl); @@ -368,7 +388,7 @@ static void init_cluster_l2(void) { u32 l2_cfg, l2sm_cop, result; =20 - while (1) { + while (!l2_hci_broken) { l2_cfg =3D read_gcr_redir_l2_ram_config(); =20 /* If HCI is not supported, use the state machine below */ --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7EA91E32DD; Mon, 28 Oct 2024 18:00:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138407; cv=none; b=XHg7kYa0hBkKq256cPhptz92c38+BztG4K0FMDnBFY9j5vtq4xKVnbUO3TQO/jLa5eQv671kFG4UTiqnnGgsJEGHtjnfVbfnOpeU2BrzsIuhHlxWCphNLGB3sGlZquLz5DV0ITFk1bQuRuaLlr1a2la0MW2bgyS7XSHKQB5p+4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138407; c=relaxed/simple; bh=1R/Tb0SJiU1keUkgHz9X53MLpIuXfiBUmnoI6VhnQpU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RvKvYTZagiWQ3TSj2CCHM11n4SXTjcls50K4/yH5Ai9Z+cNFIIyGnapWvprT0HE63Hdlg7t0YAzjen4QHbxeo1bEcRcFW8znnY1EGsjXG4snAeEe6Fw14xHEEgAGNqrw2SAYs7db3lxkr0aNvcQFd/HaHpH6MkfA3SSAVq9uzNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Ko7JtBla; arc=none smtp.client-ip=209.85.218.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ko7JtBla" Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-a9a0c40849cso752288366b.3; Mon, 28 Oct 2024 11:00:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138404; x=1730743204; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kAbmYZ1mhTULYI3vw1kfmqedtmD2E3ytfSgsYzBMC3Y=; b=Ko7JtBlaFfUc7rXLaBC3mha4ho332XwwYzid8f4B4qtsw1U0KxzXT2lr2XcURFrLza SeAvKAHuRzue1eIjCVjTVGC85aaXyqztmpyRGjEfEAHnnQrIRy+DU3+/OYqDGlnRK4E5 INjN4ctaFWOBmvGvoLE0XnCBWDa3FCgre3d+iGVeivjGkzDXosQLqUAIGlBm/PcCpHEu nrB2khT3kmKK9ytOGQWRiKOCICZQ09MT9gOZ34bPTkmN1qV0O3NUhnoZ7M0Rfvh+T2eY WcfW/LdRGVPSZIcZ1CuFVIufcobgQ2od4eCJiQrxBb5bwTTm+6T3QBO8mxAWbKCxNr3x mU8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138404; x=1730743204; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kAbmYZ1mhTULYI3vw1kfmqedtmD2E3ytfSgsYzBMC3Y=; b=fDn02kNXhUfA8m9ryIPinLek/Gqvme9JCM4XBBUAiSXVN10ycld5Q0qw7fwE6rd7yH qumINbXLhHlx+fsF8G1P/OEl+8ze6qPwRSSbwavxB7CqNWpIhKyYAL8LpxMrYKum9UdO 8rKLjcCvZ/FDtaAW+768aBWGGrgalIhkc18/l2g4FwzA58akCias4lARDTCBq+XbDHrH O3JTT7MAOlC50eZow7bt7Y3WMwk4BOmoDhPITTfdCofYSHzIgVhkg1pwQNkeD+OLcgNL GJWmckBAt7q5owNqMR9aNMqdQiO4/hFfaXSVQ7vULTStt9OETK55DglqQE9jK/MeC6kn EWxQ== X-Forwarded-Encrypted: i=1; AJvYcCWXQhgqXDbHAWRxDqv+DkdAgRQxLacKepxjCqX7aEcerdt0aV0bAZNUotOAjeJ3dVRnb4AMUaAJq95wzg==@vger.kernel.org, AJvYcCWZ1MNje70s1vzZSzzkUPIOpdkTm5bCqmBJHfZtw6SPyYFWCzkKeCsL1PiIrcvCRNKgu+5LT9JqUMdGKue7@vger.kernel.org, AJvYcCXDlnDqvoNKtbPN/NGMz+2MJ2pUw434fvM4SP3XnHzcdJnhvYm1qMbOWKz4j5RiABOkf7lxEwq3uzro@vger.kernel.org X-Gm-Message-State: AOJu0YxRrt0wMccEwkQfbx+ZPagr1hXRJQkKlbKPkXuwUK0+orKg/1Ed 0QCI1ffDFpV/JWjwglPKIjMH7Xii8rx3pnGybtYK0GtWxSlIEXD+ X-Google-Smtp-Source: AGHT+IE9G5kZr2VnQdi3oJ5fK9NfyoYvvRFAnLjyDDKECkhSiYMPXb7MozsNPe9M9cfg9spN/dDNoQ== X-Received: by 2002:a17:906:794f:b0:a9a:4597:a7f3 with SMTP id a640c23a62f3a-a9de619876amr929630166b.62.1730138404031; Mon, 28 Oct 2024 11:00:04 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.11.00.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 11:00:02 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 12/13] MIPS: mobileye: dts: eyeq6h: Enable cluster support Date: Mon, 28 Oct 2024 18:59:34 +0100 Message-Id: <20241028175935.51250-13-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Gregory CLEMENT The CM3.5 device used in EyeQ6H SoCs incorrectly reports the status for Hardware Cache Initialization (HCI). This commit adds a property to acknowledge this issue, which enables the use of the second CPU cluster. Signed-off-by: Gregory CLEMENT Signed-off-by: Aleksandar Rikalo Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- arch/mips/boot/dts/mobileye/eyeq6h.dtsi | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/mips/boot/dts/mobileye/eyeq6h.dtsi b/arch/mips/boot/dts/m= obileye/eyeq6h.dtsi index 1db3c3cda2e3..4ea85dfd4eed 100644 --- a/arch/mips/boot/dts/mobileye/eyeq6h.dtsi +++ b/arch/mips/boot/dts/mobileye/eyeq6h.dtsi @@ -18,6 +18,7 @@ cpu@0 { compatible =3D "img,i6500"; reg =3D <0>; clocks =3D <&occ_cpu>; + cm3-l2-config-hci-broken; }; }; =20 --=20 2.25.1 From nobody Mon Nov 25 09:27:43 2024 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 027A21EE008; Mon, 28 Oct 2024 18:00:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138410; cv=none; b=BlYB6GFRlLgNc6o36gNkKAsvb8Hr0//FM7ej9NUQyn/qxoLiyE9rvH784atfSk5jYx+GRHNJ2VnNN2qztLNVrbv4dwE+j3Wo9gC63Bn/IG4fUwR+MmyuH0PFb4eSeBCcUpWg46puUTA1lN7sLGmt4g6lQTkcoAA6Xv9QYxLvp+4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730138410; c=relaxed/simple; bh=r8fWALxY9pnx1sb7A9xCP2KVXvLRIDr6Sm4j2n3XmZs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iEfRc7BgAtHVx62J08KT9hvtk4LTTo5lAF67GOd0Y2jcHIC8GrIiZUl+dstIWek7iB/jj3ae8Dl/azFAFrUqsI3yH7UhmdEEIM2GzTgU3b1vo0VjVVN5OLuFiH1Iotp932/vtodsAErmw5J5dw1PZLhkYxBjbajt6o0qxLnw+UI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Tt5C+odd; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Tt5C+odd" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a9a3dc089d8so682637866b.3; Mon, 28 Oct 2024 11:00:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730138406; x=1730743206; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RiQZgL3ro4zAwZa43jZc1RKKTs9QphATYHJZ+duswLY=; b=Tt5C+oddz+yIEHUtQJuYnHhkrR5oWQiTfuF1awuJifWj0+Ou2YUDt5J0K8K/ZQCY0U 3WtfcJ5rpXZKRHMEnZPNN/n17aJw/aZKsuGc0rIz1YkwMf7L3a31aNqRbCC3droZ50Ox Dy0kim7GWgkRUnZmlbE3JylZzD9QpFFxa3gN6oLWYq+wHTvSDSaviqtqzTjE4S2Daxsg fHUjozIKBBMSHPxZmVum/zJYb+XiALMLtBHf3/bTQ/r6ZeHP6H2EypK4BYtVz8tugEUm NkUs8l6ss/VAjHhW7IokIPlPgflIvGwLoh5iXTMYXQq71hygrFjGI61K538iAqssn/+C wHSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730138406; x=1730743206; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RiQZgL3ro4zAwZa43jZc1RKKTs9QphATYHJZ+duswLY=; b=CmuiFxXdEFP7gwvv7M19KTSArQgKkF1AePeEX4dV4YUM9jKTfHfJUwakSlC2MXFCAW WrNCMQiW3o2lG57DoojJdFOpgPZaXhB5WRgnsfPVsxCT7aRBchxKk+7zDV7HwSFJ3nPB 8Zkc9VJ8XgJNR055ROo9jiiE+rrVXbHl6oKFEQ/pjGKkznAhqWbKU3BZSziZ1gdGgcf1 sqX0rHhNi6+KupNF9GccH5p0JFi/6QGmmw0vKC5OcsCG4JIb6KC7A2Sqg3m95YBRRvGg 9VHe7x7SqhgcV6gfjuBKi4KAQ+2OfwY1RUHsvfoXWvm5mm7tbwS/GZiKFi381j3dPdO3 BOdw== X-Forwarded-Encrypted: i=1; AJvYcCVXgndti/bBcheuXi2+o2wT8+E/g6jSSV+y/NujS3pSZTA+hCUBhKDX8pyBsM4OZZv45Cr0uR6U0L45mHQn@vger.kernel.org, AJvYcCVuES4+QKzJiwTueLiRYgUdGPrN9DkpU01t6VqWIhG1vW3DVo6djLyx+cw/sRopF9f3AKc2MWmiixWskw==@vger.kernel.org, AJvYcCW7G0HGJWkRiL1EeRRer3IXW2yp5MjFseTaPH7K7Tg4GNQUjc2Vp7B6S8hS+0E4PzTKlYIAj6EJwWbL@vger.kernel.org X-Gm-Message-State: AOJu0Yz/qVyQMp7m4t9CkuiouKe2FXu4QtrUt//Mkpz5CBRnm0FKeo2j /KZ6rUT2ZsEeYbGxH6k71YR4KEed0nlJpEtF5gsrptDQI3ucW+GN X-Google-Smtp-Source: AGHT+IG5jA+Nssk1Re12RA2qe936eHRH5gV4JTqDwHRurOHr80Tqiif6nLNwRMr7EsOl7pxdldGEAQ== X-Received: by 2002:a17:907:7e86:b0:a99:77f0:51f7 with SMTP id a640c23a62f3a-a9de62ec48amr694120066b.61.1730138406055; Mon, 28 Oct 2024 11:00:06 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9b1dfbdfe2sm396990766b.36.2024.10.28.11.00.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2024 11:00:05 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Rob Herring , Krzysztof Kozlowski , Conor Dooley , Vladimir Kondratiev , Gregory CLEMENT , Theo Lebrun , Arnd Bergmann , devicetree@vger.kernel.org, Djordje Todorovic , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Tiezhu Yang , Aleksandar Rikalo Subject: [PATCH v8 13/13] irqchip: mips-gic: Handle case with cluster without CPU cores Date: Mon, 28 Oct 2024 18:59:35 +0100 Message-Id: <20241028175935.51250-14-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241028175935.51250-1-arikalo@gmail.com> References: <20241028175935.51250-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Gregory CLEMENT It is possible to have no CPU cores in a cluster; in such cases, it is not possible to access the GIC, and any indirect access leads to an exception. This patch dynamically skips the indirect access in such situations. Signed-off-by: Gregory CLEMENT Signed-off-by: Aleksandar Rikalo Tested-by: Gregory CLEMENT Tested-by: Jiaxun Yang # Single cluster I6500 --- drivers/irqchip/irq-mips-gic.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index f42f69bbd6fb..bca8053864b2 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -141,7 +141,8 @@ static bool gic_irq_lock_cluster(struct irq_data *d) cl =3D cpu_cluster(&cpu_data[cpu]); if (cl =3D=3D cpu_cluster(¤t_cpu_data)) return false; - + if (mips_cps_numcores(cl) =3D=3D 0) + return false; mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); return true; } @@ -507,6 +508,9 @@ static void gic_mask_local_irq_all_vpes(struct irq_data= *d) struct gic_all_vpes_chip_data *cd; int intr, cpu; =20 + if (!mips_cps_multicluster_cpus()) + return; + intr =3D GIC_HWIRQ_TO_LOCAL(d->hwirq); cd =3D irq_data_get_irq_chip_data(d); cd->mask =3D false; @@ -520,6 +524,9 @@ static void gic_unmask_local_irq_all_vpes(struct irq_da= ta *d) struct gic_all_vpes_chip_data *cd; int intr, cpu; =20 + if (!mips_cps_multicluster_cpus()) + return; + intr =3D GIC_HWIRQ_TO_LOCAL(d->hwirq); cd =3D irq_data_get_irq_chip_data(d); cd->mask =3D true; @@ -687,8 +694,10 @@ static int gic_irq_domain_map(struct irq_domain *d, un= signed int virq, if (!gic_local_irq_is_routable(intr)) return -EPERM; =20 - for_each_online_cpu_gic(cpu, &gic_lock) - write_gic_vo_map(mips_gic_vx_map_reg(intr), map); + if (mips_cps_multicluster_cpus()) { + for_each_online_cpu_gic(cpu, &gic_lock) + write_gic_vo_map(mips_gic_vx_map_reg(intr), map); + } =20 return 0; } @@ -982,7 +991,7 @@ static int __init gic_of_init(struct device_node *node, change_gic_trig(i, GIC_TRIG_LEVEL); write_gic_rmask(i); } - } else { + } else if (mips_cps_numcores(cl) !=3D 0) { mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); for (i =3D 0; i < gic_shared_intrs; i++) { change_gic_redir_pol(i, GIC_POL_ACTIVE_HIGH); @@ -990,6 +999,9 @@ static int __init gic_of_init(struct device_node *node, write_gic_redir_rmask(i); } mips_cm_unlock_other(); + + } else { + pr_warn("No CPU cores on the cluster %d skip it\n", cl); } } =20 --=20 2.25.1