From nobody Fri Sep 20 23:49:02 2024 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB11A19309B for ; Tue, 20 Aug 2024 15:24:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167477; cv=none; b=LFGusLbbnbBEOdhIvFm0/gxM2XuO621ga14tH/hNtM36dJe9rLGslT8+KK2GQph3wJvQh3iH/gyQaActjhqfynmz/a7vjZ37oMVuMR8g4/vqPZyhQbyNE0tG57SlozY6P+K9S5x/0zrHSSipQeAYWHMDGi0k9M4v8aKMpwwtgdQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167477; c=relaxed/simple; bh=eAta5sI3YfIDzmAhM8a/NQDHd/CVyWeV0xGX51ip71g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GIlN9/VubGqxTPjkEGJb6Io53Pqp1GwaqvwONfd/OekQCcNUuLYE6ODG3q5skqIDH+5mBnQQFWSo+wgOZQzXhnWYcyVJHm/R7Ofj33VX+KI8OcL3ZJcPyeAXzmC8NGy+OQUzLqMRWfK4YEOE6P/+YrbCBQBEyIaUO4iVbGflfLI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=mGAWdLob; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="mGAWdLob" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2025031eb60so17084135ad.3 for ; Tue, 20 Aug 2024 08:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1724167474; x=1724772274; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L2IZDhIIluWTl5Gv1nsT36X2FhV2tW66iYvT1Y/MHGQ=; b=mGAWdLobiuhgaHQiUapDOER8VtN3lI9TkOXra0Q9oexhHnVwfiA87zxGxmvJCFF0Uh 9Xny3IwChkRBgCcOOHISaRo4OfuvL/+a681J3z8fJ6VpSBFEDZ1tqP6z6XGr8qAvElKi Arco2bPKZP4C/Y5v+AulOBYFRxvTqRyy+x34PQ0Z6m74/yau2kfRbSqdnMN74FcaLthr O32UpYPngqTvi0x9KusLoGQLH54BzxCoputUXwrEAKVTmQO+M/PuhBZ5zZ/cDknaEfQl Vb/5sCenjEPAXza09+VLw+IRgeAr4gDeeWqRPC7GFHz7WAdBxA+LdKcI4Zc5dT7NwrRN TPQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724167474; x=1724772274; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L2IZDhIIluWTl5Gv1nsT36X2FhV2tW66iYvT1Y/MHGQ=; b=Mg9Iin2jPZlr3mjeQlwpmv+Vk7/vQTYJkIjUPHOX6O1qQXksZeCyXIS5mshv6Z7dcC wupdGVVDB6iJKYhlezTYK5WZIDl4A/MydDPw/WY+fiHmtqcLSmecLrNEkQvkWtwNXHKf eE43vdjNquc0vp7gH86W/b9K5/lg3bsmjuuqDvfKPrTjhCayc/SSik56W34w91l3hLcY D9dNMByUcPa0Oro+cAW7vvSD5cPAvg5XgId+XOstBGy8FgEUTi6C/PPIgl9HmLGExkFv Ae/nGP6Y59xRENQmbPKnqh+nBo562Urv0mEGAKOdxSf7UQAhPT/kfsSH1WbySGnKgiCB 3JVQ== X-Forwarded-Encrypted: i=1; AJvYcCVG6juUuSm+SLw+MZaG7lCGoLEkUnKRc+Yy+mxz77g3imgjyv5Gt4pB63z791jjvl/RYy0+Y28T0XP9B8+OVJn0cfc3ZVXsSrsfOCwO X-Gm-Message-State: AOJu0YzW/Z4kNW4w69CSOXK6BLQsGN8n91Llz+VkzfJSgE41oMQ8KQPm 060EW4Vrpa3sGj19qjAUPIWtpt2g2bwCsvrBcM/B6/S5eNnl+WxjjL+cD59AjCQ= X-Google-Smtp-Source: AGHT+IFsrRGrirP49Nv3Fw0WBLsW834HqqZUUD2gt3If73tNPml67l6zbySpa0AxpE2qC16p0ViYLg== X-Received: by 2002:a17:90a:b108:b0:2c9:90f5:cfca with SMTP id 98e67ed59e1d1-2d5c0ef60e9mr2639905a91.42.1724167474424; Tue, 20 Aug 2024 08:24:34 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d40bea7cb3sm7258157a91.25.2024.08.20.08.24.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 08:24:34 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH v9 1/6] RISC-V: Check scalar unaligned access on all CPUs Date: Tue, 20 Aug 2024 11:24:19 -0400 Message-ID: <20240820152424.1973078-2-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240820152424.1973078-1-jesse@rivosinc.com> References: <20240820152424.1973078-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Originally, the check_unaligned_access_emulated_all_cpus function only checked the boot hart. This fixes the function to check all harts. Fixes: 71c54b3d169d ("riscv: report misaligned accesses emulation to hwprob= e") Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins Reviewed-by: Evan Green Cc: stable@vger.kernel.org --- V1 -> V2: - New patch V2 -> V3: - Split patch V3 -> V4: - Re-add check for a system where a heterogeneous CPU is hotplugged into a previously homogenous system. V4 -> V5: - Change work_struct *unused to work_struct *work __always_unused V5 -> V6: - Change check_unaligned_access_emulated to extern V6 -> V7: - No changes V7 -> V8: - Rebase onto fixes V8 -> V9: - No changes --- arch/riscv/include/asm/cpufeature.h | 2 ++ arch/riscv/kernel/traps_misaligned.c | 14 +++++++------- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 45f9c1171a48..dfa5cdddd367 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -8,6 +8,7 @@ =20 #include #include +#include #include #include #include @@ -60,6 +61,7 @@ void riscv_user_isa_enable(void); =20 #if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed); void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); DECLARE_PER_CPU(long, misaligned_access_speed); diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps= _misaligned.c index 192cd5603e95..1ad981b2c7a3 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -526,11 +526,11 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } =20 -static bool check_unaligned_access_emulated(int cpu) +void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed) { + int cpu =3D smp_processor_id(); long *mas_ptr =3D per_cpu_ptr(&misaligned_access_speed, cpu); unsigned long tmp_var, tmp_val; - bool misaligned_emu_detected; =20 *mas_ptr =3D RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN; =20 @@ -538,19 +538,16 @@ static bool check_unaligned_access_emulated(int cpu) " "REG_L" %[tmp], 1(%[ptr])\n" : [tmp] "=3Dr" (tmp_val) : [ptr] "r" (&tmp_var) : "memory"); =20 - misaligned_emu_detected =3D (*mas_ptr =3D=3D RISCV_HWPROBE_MISALIGNED_SCA= LAR_EMULATED); /* * If unaligned_ctl is already set, this means that we detected that all * CPUS uses emulated misaligned access at boot time. If that changed * when hotplugging the new cpu, this is something we don't handle. */ - if (unlikely(unaligned_ctl && !misaligned_emu_detected)) { + if (unlikely(unaligned_ctl && (*mas_ptr !=3D RISCV_HWPROBE_MISALIGNED_SCA= LAR_EMULATED))) { pr_crit("CPU misaligned accesses non homogeneous (expected all emulated)= \n"); while (true) cpu_relax(); } - - return misaligned_emu_detected; } =20 bool check_unaligned_access_emulated_all_cpus(void) @@ -562,8 +559,11 @@ bool check_unaligned_access_emulated_all_cpus(void) * accesses emulated since tasks requesting such control can run on any * CPU. */ + schedule_on_each_cpu(check_unaligned_access_emulated); + for_each_online_cpu(cpu) - if (!check_unaligned_access_emulated(cpu)) + if (per_cpu(misaligned_access_speed, cpu) + !=3D RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED) return false; =20 unaligned_ctl =3D true; --=20 2.45.2 From nobody Fri Sep 20 23:49:02 2024 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3E3719412E for ; Tue, 20 Aug 2024 15:24:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167481; cv=none; b=h2VDKZqbV2Kks6QSSRNceVPuHlMlGdnwiQzGLNbJvAZUytwx7AJ2OPydz3L7mSgx1thRPbnYmd1Mia52vp1Bo2gUcVTi4PaClGm4fpWTj282f2pGrOKpasCqDrjeDGctTM4btX5K36lTrAf+8WPFov5ENeAv+L4IrxE54QNyrZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167481; c=relaxed/simple; bh=2rc8FU9M+fSW+QkKpagTjuE9uoduYcxat2+eo3FYBq8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fImxHklPR6oKIkTcCvHYjw98vits3lgLVhz7ZR3uYaEDvhPA4QthMmez4vANCSYWok/tNZ7u2ir+GPEXPwQIe4yPKzTED9MEexEvNjp7PESI3nvIjb+7IJ8h9IIQOfpgB77hfQA3HtCzf1MZxdwHYuGjasvCeEj5H9sZl058KXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=lRXLgSCF; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="lRXLgSCF" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2d3da054f7cso3393978a91.1 for ; Tue, 20 Aug 2024 08:24:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1724167479; x=1724772279; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pcWMRJ0Y65fLMG5U9Pko1FotKijofEtRgUtr9N2ekns=; b=lRXLgSCFkVpYU5kVoYMWnzwyodepKebc5VKCtoyHZnaxDhkVBOjQp9TiDFr3gcNo1X Lbnvvv3Pw5lG//OG4+70k++UQis0Y7nrJoOj44i+CCOC7aBv0A8EXgMQzWKW44sD5NS1 VSTRQN4/RMQeLrXb+fpxmEvzUCqtYgzb7dWT2WIC22o+F7kkH1fFovXJ/obrtkj+fGkn Azv/AB17QAs6WRgpTKRzFUW+/zMqj1jdP+xMkwMsJ67z6mF/oriQybqxal9PXyS+6qbt icAo6uSt4Eq74vmSsw/zW0G+CcbqigK+Rxja3+wWKWkbZxpGsiDy4NnISvjfMPxSyGv6 3tmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724167479; x=1724772279; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pcWMRJ0Y65fLMG5U9Pko1FotKijofEtRgUtr9N2ekns=; b=DCg1OAW82bNotycO3+XD0gEf6CFrpOKONhdgwe9oTN82XrdfhabSSukvpLxhB66FbR pqASGDyxWNfSt0MXL3xmsELnQZinaO0kxEs1RUVF0tf6Cq3sLtRkfpcfrMflIt9F9P9F 9zslS8Adq6eoiuA174MhbscgYLb162H1b1PSRQgzrnVsjcwhKIgncRWeq/vAbPtebU0o Ba1gqOjsp8Tc5eRRAJx/HLhd+OylEGkUoT6CSQi202SsvsPzMOaQQb9ky2Qc5grUiiFz KFGO6iv1ldtmprTD/kMb/6ug6u6D/yyzWx4w73tffFqQbJkcmqoSlLejNZhjMp8IIseP lNaA== X-Forwarded-Encrypted: i=1; AJvYcCXU4jncdJ2QJu/c3mdpG3EwLFqshwsBZCKrZom0ooM3I1holCeqPm7T9YZSJ/USCvAnHeahat+JPMn7fm5/G7qBJkBuWrfVM8ssVJQT X-Gm-Message-State: AOJu0YzQXuuNTE6irbihDHtsE8tfN3ltPzFYksYTpH/6zSlo0IsYYqAO rs+WLXAQAq9wPV72S6umzZdlyTrPglp8GJOvkm913ZI6xXqACm5qt2zbf2+fXcI= X-Google-Smtp-Source: AGHT+IE+Km4Z4Uzp0Ic/89irlMkCcLxwcNYIyvvdXIzqeBq5tarmpBAMzPn+ZBc205ATxzcUSAGINA== X-Received: by 2002:a17:90b:1086:b0:2cd:4100:ef17 with SMTP id 98e67ed59e1d1-2d3dfff6674mr13170727a91.31.1724167478774; Tue, 20 Aug 2024 08:24:38 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d40bea7cb3sm7258157a91.25.2024.08.20.08.24.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 08:24:38 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH v9 2/6] RISC-V: Scalar unaligned access emulated on hotplug CPUs Date: Tue, 20 Aug 2024 11:24:20 -0400 Message-ID: <20240820152424.1973078-3-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240820152424.1973078-1-jesse@rivosinc.com> References: <20240820152424.1973078-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The check_unaligned_access_emulated() function should have been called during CPU hotplug to ensure that if all CPUs had emulated unaligned accesses, the new CPU also does. This patch adds the call to check_unaligned_access_emulated() in the hotplug path. Fixes: 55e0bf49a0d0 ("RISC-V: Probe misaligned access speed in parallel") Signed-off-by: Jesse Taube Reviewed-by: Evan Green Cc: stable@vger.kernel.org --- V5 -> V6: - New patch V6 -> V7: - No changes V7 -> V8: - Rebase onto fixes V8 -> V9: - No changes --- arch/riscv/kernel/unaligned_access_speed.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel= /unaligned_access_speed.c index 160628a2116d..f3508cc54f91 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -191,6 +191,7 @@ static int riscv_online_cpu(unsigned int cpu) if (per_cpu(misaligned_access_speed, cpu) !=3D RISCV_HWPROBE_MISALIGNED_S= CALAR_UNKNOWN) goto exit; =20 + check_unaligned_access_emulated(NULL); buf =3D alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); if (!buf) { pr_warn("Allocation failure, not measuring misaligned performance\n"); --=20 2.45.2 From nobody Fri Sep 20 23:49:02 2024 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFA8C194A5C for ; Tue, 20 Aug 2024 15:24:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167485; cv=none; b=CZJwrZhxjDU0DmoZz/j+Nfq8JOlJRg/9VI2RlIce0+IOz5Isqdh0U88aUurIKPexiX9+j1CABrLZOQJV3xROx/0Q8KWqqbBxmyZX/KtKF4K/kRrTiuJx3wizhA1dHatyplh6JxaGPmEdg5pb37DXTR3iRwkOkoj8Vlp5cFp6s3w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167485; c=relaxed/simple; bh=hu9iNDPYCG2l1/0+La0H2CMxvE1QMiMSu720ozrvTZY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XDae96gOMvT3f58RcflsZmWDNxeXwQbS0EmI+Yfnhuqgw3J3/jmjC1SkjS6rUgQGSOhwU7wn5slphlaGX+KZS729CRP1JgKFK0Z9G0du4MaoMJDlaLrleMRmH79p5jkoY0oeswiqAftxCXZv3T8hFRBLzxQIBTY4VR5vTm2tM1o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=pxM/7lmu; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="pxM/7lmu" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2d3eda6603cso3263291a91.3 for ; Tue, 20 Aug 2024 08:24:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1724167483; x=1724772283; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kr28cnDKR2hf/rAx6taoMNDeMkJdk5A7shPnifntETA=; b=pxM/7lmuiGqzr/zUshtnGFfQkyvLuVaz7yPjhaJEmd8uuRkgNSwT7fMfCdOIErGam/ PZorWFMMbtH9qEWaQaLEsmIn+gJTj2z+T4Lzo1kQvY/5wb2ytOZW+DxVUJdy74zkotob hJO5z+T3LytlnOxaEtxv6jt5RdTMlpSjCuRCQn4paQnFzlRIOGAKGMnbfAUnfJt3P8VF AMJFFtgT/1OIcXL6IrSHWsw6Zlo8bmCuAddIUCnIproMBOmeg5W0kff96AZ7En71H8N3 kFFnTG27KjX0Z31r5txLa2oOOFmtFe87JqtFLqRuw+AOnDAtOPglQV4zx9WCX8uYo90J 1NZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724167483; x=1724772283; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kr28cnDKR2hf/rAx6taoMNDeMkJdk5A7shPnifntETA=; b=HX1527uqfdmbocrQWC1i7SHWof6V7/krfH9wXJ+3sOr9wqVhfCQW0ZQGUJjQ+aesAw JtbSTTA2kaPIKt9PUkgWDQBe7ojv6yT8CkjJ7vKaXrFICG5G2xBrRtRXUm3fy5kNDfUm fNQAhgobYVXk3WMw6WYmbYL8oXhwyQ9YNI4NRB0bxtH3VUD7XpK9AIsB8QWS8jji+MFr NpNFTK9VRFZmdCWSmRJbwdzGRuIZvmi/wheqQ/JLkwrNYUC/YQFofiiHPh71LSO/DlOF d8ADw6iLCS11UyeEPixOvOi0MmpsRQlGlR9RSeR6uBR3ACXEBv3Hu8MHIV9x8vCxaadv wp1Q== X-Forwarded-Encrypted: i=1; AJvYcCXsVPC7/ipJWKxngjcOqOdWxI6WhZu/EH8BGAKnjmTj3ukUd1ubxhqyRAxxcJfj1o95PmPwN2wq4xCTWHw=@vger.kernel.org X-Gm-Message-State: AOJu0YyOPYt3Nzsno6+9L43ZkxIe4P3zfNIAwGCdAkoGaL43HfAE64Tl UMDTJZ20pYXCbmk4kjoL+6fptM5AoOV4o+CQCGgYSn5G/TwATgOmO5tbP6304HU= X-Google-Smtp-Source: AGHT+IEs+fOb8xDUxhVr88hKMN1WaP41Ow4SfW38HuzsBc7du8pwa4fzv0vfZ+kCiB5hdUxcmpZEpA== X-Received: by 2002:a17:90b:4b47:b0:2d3:ce98:9dd5 with SMTP id 98e67ed59e1d1-2d3e00f1bb8mr15020623a91.43.1724167483047; Tue, 20 Aug 2024 08:24:43 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d40bea7cb3sm7258157a91.25.2024.08.20.08.24.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 08:24:42 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Conor Dooley Subject: [PATCH v9 3/6] RISC-V: Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED Date: Tue, 20 Aug 2024 11:24:21 -0400 Message-ID: <20240820152424.1973078-4-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240820152424.1973078-1-jesse@rivosinc.com> References: <20240820152424.1973078-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED to allow for the addition of RISCV_VECTOR_MISALIGNED in a later patch. Signed-off-by: Jesse Taube Reviewed-by: Conor Dooley Reviewed-by: Charlie Jenkins Reviewed-by: Evan Green --- V2 -> V3: - New patch V3 -> V4: - No changes V4 -> V5: - No changes V5 -> V6: - fix accidental moving of check_unaligned_access_emulated_all_cpus out of= the #ifdef V6 -> V7: - No changes V7 -> V8: - Rebase onto fixes V8 -> V9: - No changes --- arch/riscv/Kconfig | 6 +++--- arch/riscv/include/asm/cpufeature.h | 2 +- arch/riscv/include/asm/entry-common.h | 2 +- arch/riscv/kernel/Makefile | 4 ++-- arch/riscv/kernel/fpu.S | 4 ++-- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 0f3cd7c3a436..e9295a56b3a5 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -765,7 +765,7 @@ config THREAD_SIZE_ORDER Specify the Pages of thread stack size (from 4KB to 64KB), which also affects irq stack size, which is equal to thread stack size. =20 -config RISCV_MISALIGNED +config RISCV_SCALAR_MISALIGNED bool select SYSCTL_ARCH_UNALIGN_ALLOW help @@ -782,7 +782,7 @@ choice =20 config RISCV_PROBE_UNALIGNED_ACCESS bool "Probe for hardware unaligned access support" - select RISCV_MISALIGNED + select RISCV_SCALAR_MISALIGNED help During boot, the kernel will run a series of tests to determine the speed of unaligned accesses. This probing will dynamically determine @@ -793,7 +793,7 @@ config RISCV_PROBE_UNALIGNED_ACCESS =20 config RISCV_EMULATED_UNALIGNED_ACCESS bool "Emulate unaligned access where system support is missing" - select RISCV_MISALIGNED + select RISCV_SCALAR_MISALIGNED help If unaligned memory accesses trap into the kernel as they are not supported by the system, the kernel will emulate the unaligned diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index dfa5cdddd367..ccc6cf141c20 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -59,7 +59,7 @@ void riscv_user_isa_enable(void); #define __RISCV_ISA_EXT_SUPERSET_VALIDATE(_name, _id, _sub_exts, _validate= ) \ _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts), _valida= te) =20 -#if defined(CONFIG_RISCV_MISALIGNED) +#if defined(CONFIG_RISCV_SCALAR_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed); void unaligned_emulation_finish(void); diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm= /entry-common.h index 2293e535f865..0a4e3544c877 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,7 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct = pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); =20 -#ifdef CONFIG_RISCV_MISALIGNED +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); #else diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 06d407f1b30b..71442b22efc8 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -64,8 +64,8 @@ obj-y +=3D probes/ obj-y +=3D tests/ obj-$(CONFIG_MMU) +=3D vdso.o vdso/ =20 -obj-$(CONFIG_RISCV_MISALIGNED) +=3D traps_misaligned.o -obj-$(CONFIG_RISCV_MISALIGNED) +=3D unaligned_access_speed.o +obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D traps_misaligned.o +obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o =20 obj-$(CONFIG_FPU) +=3D fpu.o diff --git a/arch/riscv/kernel/fpu.S b/arch/riscv/kernel/fpu.S index 327cf527dd7e..f74f6b60e347 100644 --- a/arch/riscv/kernel/fpu.S +++ b/arch/riscv/kernel/fpu.S @@ -170,7 +170,7 @@ SYM_FUNC_END(__fstate_restore) __access_func(f31) =20 =20 -#ifdef CONFIG_RISCV_MISALIGNED +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED =20 /* * Disable compressed instructions set to keep a constant offset between FP @@ -224,4 +224,4 @@ SYM_FUNC_START(get_f64_reg) fp_access_epilogue SYM_FUNC_END(get_f64_reg) =20 -#endif /* CONFIG_RISCV_MISALIGNED */ +#endif /* CONFIG_RISCV_SCALAR_MISALIGNED */ --=20 2.45.2 From nobody Fri Sep 20 23:49:02 2024 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16961194ADB for ; Tue, 20 Aug 2024 15:24:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167489; cv=none; b=cbTStb/doZ1mFYbvFzMqvGMxLoNovpHR29pR2GZNiUDEnnHZiKumpBxmLVUY/E1VdqQ5ZiWYgDYOs7RlN/zE3R3m/ZTw3FPjzi859At5P4/kI6zC4ep6I+usZIq4vM07iNK6E7nmREiZq+1s2Mq6rwqMJcSs30gwLTLOZ9V5HLY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167489; c=relaxed/simple; bh=mP6Z8b3C9V0QxXqOAVoQw3zlF9CCPFEMPK3wDnGFD+w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sKWT5eQeD9GmGFY9DL+5ScUSwzuQ4q6Vdlfm5KSt9jf5RJ23kEzq1uWxsVs5slrF5sU/ZFfZfO1GQhENQKv7tbxXDiyfLJpcXJw/zbU+G2fLJDES9mX5oO6r68kdJrp3z5bowBJUEDUKhwuA2lxIOsoY+6sfBoKROwxxJkZ3+Xg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=HWs7pTOS; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="HWs7pTOS" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2d3c5f769d6so3912493a91.3 for ; Tue, 20 Aug 2024 08:24:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1724167487; x=1724772287; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uzTbK/13n367xJxI170oejVIhD/YTTvLfyt3vR+BENU=; b=HWs7pTOSwQAUjyMc3wbNwmm8POPMjtSIo64dtBzGjf1kIbSCqaPupSSaeKl/9L8cNe shyJroMW3kw2N7diuLooE5kT9e1YaEXKrxfg7GQArWRZI+pOEASF/gjGFB+8wgHJU8ps 5M+zlLM6/E0ZTRv0c5uwZdoo+CvgjX2LT8UD46aX73Iy/f7m7l7KHx+5IYwjE5M69LJb JQIePUO6fWLjR7XX9pTBuufXU/pJhE9uGD0DxQiiI/IUuMicY/E5PrJYF8NL/0yQNVIp CpiZgphIi7syO6Bm8izuFhl3FKdahZaP0bkxAcRjLDwOxasvDEc9yJAOi+Cl7i5HID3Q UA+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724167487; x=1724772287; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uzTbK/13n367xJxI170oejVIhD/YTTvLfyt3vR+BENU=; b=KF2g5K4pfMc1VXoul0NBdSk8SojxgdoMxkqyFhyb+3BDNU8gHq9wbNrt1Rw6JEfWPU TCVxGVtWVZ7IdiZ6c518R8hRym6dwepjCu0l35ZqZQTTOvJHDA+aejcFsXSmp0L1ruVm qJ9gtoPAM/KvmNhUptt9NVcvd5pngZqEvVs4xr5r8X6lr8r2iEwpz3U5MmzM+vhXkuDY d03Mk6VRT0QgxcUoiUQdusfasEWMwibmFb/IzASaEOpLpckvp09fh4vBL7T291K43ctQ OtBwrwqeiUjotEIYpHnaN4eK78LPqBGjywoci1xC7d2fEaP6BgRvZC9NrF6rB0fY8NRn V3NQ== X-Forwarded-Encrypted: i=1; AJvYcCXStc8FOTbfuugqrLjNHxX79qHjAbSyFl+J8AmUkpxHmeRTEg+fKbII/E42p7djR5x4r8zHx6zQahCWbp7ugMyBV2son5zseK5RPsXK X-Gm-Message-State: AOJu0YyVferant+2xKr0nOzNqnf87ilh4xZcRq71BQLcgZo2/9jfYmpA 74ScIvfY2mzEYRoKDZ3BPws9RyxQe9mnDGK50FM3cRQkaS+dxxoNteUtlpopVpQ= X-Google-Smtp-Source: AGHT+IFZ7vKdClNICnDbWCbYSIskNYci/VBNV/AotQthjAWMlsczkTtL87aYdvNmEwxiQiLCgfDQYA== X-Received: by 2002:a17:90a:744a:b0:2c9:61ad:dcd9 with SMTP id 98e67ed59e1d1-2d3e00ec17dmr13527898a91.27.1724167487226; Tue, 20 Aug 2024 08:24:47 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d40bea7cb3sm7258157a91.25.2024.08.20.08.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 08:24:46 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v9 4/6] RISC-V: Detect unaligned vector accesses supported Date: Tue, 20 Aug 2024 11:24:22 -0400 Message-ID: <20240820152424.1973078-5-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240820152424.1973078-1-jesse@rivosinc.com> References: <20240820152424.1973078-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Run a unaligned vector access to test if the system supports vector unaligned access. Add the result to a new key in hwprobe. This is useful for usermode to know if vector misaligned accesses are supported and if they are faster or slower than equivalent byte accesses. Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins --- V1 -> V2: - Add Kconfig options - Add insn_is_vector - Add handle_vector_misaligned_load - Fix build - Seperate vector from scalar misaligned access - This patch was almost completely rewritten V2 -> V3: - Fixed CONFIG_ in Kconfig - Fixed check_vector_unaligned_access_emulated leaving vector_misaligned_access as unknown. - Remove local_irq_enable - Remove RISCV_DETECT_VECTOR_UNALIGNED_ACCESS - Remove RISCV_VEC_UNALIGNED_ACCESS_UNSUPPORTED V3 -> V4: - Spell out _VECTOR_ in macros V4 -> V5: - Change work_struct *unused to work_struct *work __always_unused - Add insn_is_vector definition to vector.h when V is not defined V5 -> V6: - Change check_vector_unaligned_access_emulated to extern - Move check_unaligned_access_emulated_all_cpus out of the #ifdef see last commit V6 -> V7: - Change SLOW to UNKNOWN when used as a placeholder V7 -> V8: - Rebase onto fixes - s/RISCV_HWPROBE_VECTOR_MISALIGNED/RISCV_HWPROBE_MISALIGNED_VECTOR/g V8 -> V9: - Missed one RISCV_HWPROBE_VECTOR_MISALIGNED... --- arch/riscv/Kconfig | 35 ++++++ arch/riscv/include/asm/cpufeature.h | 8 +- arch/riscv/include/asm/entry-common.h | 11 -- arch/riscv/include/asm/hwprobe.h | 2 +- arch/riscv/include/asm/vector.h | 2 + arch/riscv/include/uapi/asm/hwprobe.h | 5 + arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/sys_hwprobe.c | 35 ++++++ arch/riscv/kernel/traps_misaligned.c | 117 ++++++++++++++++++++- arch/riscv/kernel/unaligned_access_speed.c | 22 ++-- arch/riscv/kernel/vector.c | 2 +- 11 files changed, 213 insertions(+), 30 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index e9295a56b3a5..3bb7bf0e9ddc 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -765,12 +765,26 @@ config THREAD_SIZE_ORDER Specify the Pages of thread stack size (from 4KB to 64KB), which also affects irq stack size, which is equal to thread stack size. =20 +config RISCV_MISALIGNED + bool + help + Embed support for detecting and emulating misaligned + scalar or vector loads and stores. + config RISCV_SCALAR_MISALIGNED bool + select RISCV_MISALIGNED select SYSCTL_ARCH_UNALIGN_ALLOW help Embed support for emulating misaligned loads and stores. =20 +config RISCV_VECTOR_MISALIGNED + bool + select RISCV_MISALIGNED + depends on RISCV_ISA_V + help + Enable detecting support for vector misaligned loads and stores. + choice prompt "Unaligned Accesses Support" default RISCV_PROBE_UNALIGNED_ACCESS @@ -822,6 +836,27 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS =20 endchoice =20 +choice + prompt "Vector unaligned Accesses Support" + depends on RISCV_ISA_V + default RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + help + This determines the level of support for vector unaligned accesses. This + information is used by the kernel to perform optimizations. It is also + exposed to user space via the hwprobe syscall. The hardware will be + probed at boot by default. + +config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + bool "Probe speed of vector unaligned accesses" + select RISCV_VECTOR_MISALIGNED + help + During boot, the kernel will run a series of tests to determine the + speed of vector unaligned accesses if they are supported. This probing + will dynamically determine the speed of vector unaligned accesses on + the underlying system if they are supported. + +endchoice + source "arch/riscv/Kconfig.vendor" =20 endmenu # "Platform type" diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index ccc6cf141c20..85bf1bce51e6 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -59,8 +59,8 @@ void riscv_user_isa_enable(void); #define __RISCV_ISA_EXT_SUPERSET_VALIDATE(_name, _id, _sub_exts, _validate= ) \ _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts), _valida= te) =20 -#if defined(CONFIG_RISCV_SCALAR_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_SCALAR_MISALIGNED) void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed); void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); @@ -72,6 +72,12 @@ static inline bool unaligned_ctl_available(void) } #endif =20 +bool check_vector_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_VECTOR_MISALIGNED) +void check_vector_unaligned_access_emulated(struct work_struct *work __alw= ays_unused); +DECLARE_PER_CPU(long, vector_misaligned_access); +#endif + #if defined(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) DECLARE_STATIC_KEY_FALSE(fast_unaligned_access_speed_key); =20 diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm= /entry-common.h index 0a4e3544c877..7b32d2b08bb6 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,18 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct= pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); =20 -#ifdef CONFIG_RISCV_SCALAR_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); -#else -static inline int handle_misaligned_load(struct pt_regs *regs) -{ - return -1; -} -static inline int handle_misaligned_store(struct pt_regs *regs) -{ - return -1; -} -#endif =20 #endif /* _ASM_RISCV_ENTRY_COMMON_H */ diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwpr= obe.h index ffb9484531af..1ce1df6d0ff3 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -8,7 +8,7 @@ =20 #include =20 -#define RISCV_HWPROBE_MAX_KEY 9 +#define RISCV_HWPROBE_MAX_KEY 10 =20 static inline bool riscv_hwprobe_key_is_valid(__s64 key) { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vecto= r.h index be7d309cca8a..c7c023afbacd 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -21,6 +21,7 @@ =20 extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); +bool insn_is_vector(u32 insn_buf); bool riscv_v_first_use_handler(struct pt_regs *regs); void kernel_vector_begin(void); void kernel_vector_end(void); @@ -268,6 +269,7 @@ struct pt_regs; =20 static inline int riscv_v_setup_vsize(void) { return -EOPNOTSUPP; } static __always_inline bool has_vector(void) { return false; } +static __always_inline bool insn_is_vector(u32 insn_buf) { return false; } static inline bool riscv_v_first_use_handler(struct pt_regs *regs) { retur= n false; } static inline bool riscv_v_vstate_query(struct pt_regs *regs) { return fal= se; } static inline bool riscv_v_vstate_ctrl_user_allowed(void) { return false; } diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uap= i/asm/hwprobe.h index 1e153cda57db..34c88c15322c 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -88,6 +88,11 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW 2 #define RISCV_HWPROBE_MISALIGNED_SCALAR_FAST 3 #define RISCV_HWPROBE_MISALIGNED_SCALAR_UNSUPPORTED 4 +#define RISCV_HWPROBE_KEY_MISALIGNED_VECTOR_PERF 10 +#define RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN 0 +#define RISCV_HWPROBE_MISALIGNED_VECTOR_SLOW 2 +#define RISCV_HWPROBE_MISALIGNED_VECTOR_FAST 3 +#define RISCV_HWPROBE_MISALIGNED_VECTOR_UNSUPPORTED 4 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ =20 /* Flags */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 71442b22efc8..06d407f1b30b 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -64,8 +64,8 @@ obj-y +=3D probes/ obj-y +=3D tests/ obj-$(CONFIG_MMU) +=3D vdso.o vdso/ =20 -obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D traps_misaligned.o -obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D unaligned_access_speed.o +obj-$(CONFIG_RISCV_MISALIGNED) +=3D traps_misaligned.o +obj-$(CONFIG_RISCV_MISALIGNED) +=3D unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o =20 obj-$(CONFIG_FPU) +=3D fpu.o diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprob= e.c index cea0ca2bf2a2..6441baada36b 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -201,6 +201,37 @@ static u64 hwprobe_misaligned(const struct cpumask *cp= us) } #endif =20 +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + int cpu; + u64 perf =3D -1ULL; + + /* Return if supported or not even if speed wasn't probed */ + for_each_cpu(cpu, cpus) { + int this_perf =3D per_cpu(vector_misaligned_access, cpu); + + if (perf =3D=3D -1ULL) + perf =3D this_perf; + + if (perf !=3D this_perf) { + perf =3D RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN; + break; + } + } + + if (perf =3D=3D -1ULL) + return RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN; + + return perf; +} +#else +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + return RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN; +} +#endif + static void hwprobe_one_pair(struct riscv_hwprobe *pair, const struct cpumask *cpus) { @@ -229,6 +260,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pai= r, pair->value =3D hwprobe_misaligned(cpus); break; =20 + case RISCV_HWPROBE_KEY_MISALIGNED_VECTOR_PERF: + pair->value =3D hwprobe_vec_misaligned(cpus); + break; + case RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE: pair->value =3D 0; if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOZ)) diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps= _misaligned.c index 1ad981b2c7a3..6af73012d9df 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -16,6 +16,7 @@ #include #include #include +#include =20 #define INSN_MATCH_LB 0x3 #define INSN_MASK_LB 0x707f @@ -322,12 +323,37 @@ union reg_data { u64 data_u64; }; =20 -static bool unaligned_ctl __read_mostly; - /* sysctl hooks */ int unaligned_enabled __read_mostly =3D 1; /* Enabled by default */ =20 -int handle_misaligned_load(struct pt_regs *regs) +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc =3D regs->epc; + unsigned long insn; + + if (get_insn(regs, epc, &insn)) + return -1; + + /* Only return 0 when in check_vector_unaligned_access_emulated */ + if (*this_cpu_ptr(&vector_misaligned_access) =3D=3D RISCV_HWPROBE_MISALIG= NED_VECTOR_UNKNOWN) { + *this_cpu_ptr(&vector_misaligned_access) =3D RISCV_HWPROBE_MISALIGNED_VE= CTOR_UNSUPPORTED; + regs->epc =3D epc + INSN_LEN(insn); + return 0; + } + + /* If vector instruction we don't emulate it yet */ + regs->epc =3D epc; + return -1; +} +#else +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + return -1; +} +#endif + +static int handle_scalar_misaligned_load(struct pt_regs *regs) { union reg_data val; unsigned long epc =3D regs->epc; @@ -435,7 +461,7 @@ int handle_misaligned_load(struct pt_regs *regs) return 0; } =20 -int handle_misaligned_store(struct pt_regs *regs) +static int handle_scalar_misaligned_store(struct pt_regs *regs) { union reg_data val; unsigned long epc =3D regs->epc; @@ -526,6 +552,83 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } =20 +int handle_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc =3D regs->epc; + unsigned long insn; + + if (IS_ENABLED(CONFIG_RISCV_VECTOR_MISALIGNED)) { + if (get_insn(regs, epc, &insn)) + return -1; + + if (insn_is_vector(insn)) + return handle_vector_misaligned_load(regs); + } + + if (IS_ENABLED(CONFIG_RISCV_SCALAR_MISALIGNED)) + return handle_scalar_misaligned_load(regs); + + return -1; +} + +int handle_misaligned_store(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_RISCV_SCALAR_MISALIGNED)) + return handle_scalar_misaligned_store(regs); + + return -1; +} + +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +void check_vector_unaligned_access_emulated(struct work_struct *work __alw= ays_unused) +{ + long *mas_ptr =3D this_cpu_ptr(&vector_misaligned_access); + unsigned long tmp_var; + + *mas_ptr =3D RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN; + + kernel_vector_begin(); + __asm__ __volatile__ ( + ".balign 4\n\t" + ".option push\n\t" + ".option arch, +zve32x\n\t" + " vsetivli zero, 1, e16, m1, ta, ma\n\t" // Vectors of 16b + " vle16.v v0, (%[ptr])\n\t" // Load bytes + ".option pop\n\t" + : : [ptr] "r" ((u8 *)&tmp_var + 1) : "v0"); + kernel_vector_end(); +} + +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + int cpu; + + if (!has_vector()) { + for_each_online_cpu(cpu) + per_cpu(vector_misaligned_access, cpu) =3D RISCV_HWPROBE_MISALIGNED_VEC= TOR_UNSUPPORTED; + return false; + } + + schedule_on_each_cpu(check_vector_unaligned_access_emulated); + + for_each_online_cpu(cpu) + if (per_cpu(vector_misaligned_access, cpu) + =3D=3D RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN) + return false; + + return true; +} +#else +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif + +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED + +static bool unaligned_ctl __read_mostly; + void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed) { int cpu =3D smp_processor_id(); @@ -574,3 +677,9 @@ bool unaligned_ctl_available(void) { return unaligned_ctl; } +#else +bool check_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel= /unaligned_access_speed.c index f3508cc54f91..0b8b5e17453a 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -19,7 +19,8 @@ #define MISALIGNED_BUFFER_ORDER get_order(MISALIGNED_BUFFER_SIZE) #define MISALIGNED_COPY_SIZE ((MISALIGNED_BUFFER_SIZE / 2) - 0x80) =20 -DEFINE_PER_CPU(long, misaligned_access_speed); +DEFINE_PER_CPU(long, misaligned_access_speed) =3D RISCV_HWPROBE_MISALIGNED= _SCALAR_UNKNOWN; +DEFINE_PER_CPU(long, vector_misaligned_access) =3D RISCV_HWPROBE_MISALIGNE= D_VECTOR_UNSUPPORTED; =20 #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS static cpumask_t fast_misaligned_access; @@ -260,23 +261,24 @@ static int check_unaligned_access_speed_all_cpus(void) kfree(bufs); return 0; } +#else /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ +static int check_unaligned_access_speed_all_cpus(void) +{ + return 0; +} +#endif =20 static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated =3D check_unaligned_access_emulated_all_cpus(); + bool all_cpus_emulated; + + all_cpus_emulated =3D check_unaligned_access_emulated_all_cpus(); + check_vector_unaligned_access_emulated_all_cpus(); =20 if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); =20 return 0; } -#else /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ -static int check_unaligned_access_all_cpus(void) -{ - check_unaligned_access_emulated_all_cpus(); - - return 0; -} -#endif =20 arch_initcall(check_unaligned_access_all_cpus); diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 682b3feee451..821818886fab 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -66,7 +66,7 @@ void __init riscv_v_setup_ctx_cache(void) #endif } =20 -static bool insn_is_vector(u32 insn_buf) +bool insn_is_vector(u32 insn_buf) { u32 opcode =3D insn_buf & __INSN_OPCODE_MASK; u32 width, csr; --=20 2.45.2 From nobody Fri Sep 20 23:49:02 2024 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 542561953A1 for ; Tue, 20 Aug 2024 15:24:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167494; cv=none; b=LE+ycBTSiiRYAAMgei/sryk1JnS4628Bel6pIT+CppXtWQawzXVvnAAFB7eKKzkJbxaX3srKyPkkJdsbfVyF5mMyj6UtyIZWuIjIUA7sG6BoPHSt0+V8MJoWnV8DnAuQ9wDSnx3obLVRQwWY0v9yTnAVvdAW8go0bur8t5dI0EA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167494; c=relaxed/simple; bh=DbOHrPe1mr3cFnNHX7wKeibCUMzw9ermJ59o2oTKbLg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uZV9vpkSacXHsGn9pwDMag7g8nDvs7uTozTfloTqQ/nj5tk1n9kH/IdcsVqUBpuSSgHX2k1XnOBNw4KkMf+9CV7UoeudhINdnvSqATba7ECi1Ix4toA9ZNZ9+Wx8Oo+lol77fMPhhFkIL81sDvGTYIyxN7Cmz00qLdYlsb3pUXI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=nd+aNgiC; arc=none smtp.client-ip=209.85.216.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="nd+aNgiC" Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2d3da054f7cso3394201a91.1 for ; Tue, 20 Aug 2024 08:24:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1724167491; x=1724772291; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rJi2ebhbP9zlbLGPWODvcw9blh6x5jyJs+zGiuzhdIE=; b=nd+aNgiCRCSkClk9kSxttzpDw3RAPzLAJmKMHVyi3Lv06kwvH8qKzmY4EsKu1nOgg/ 8T2zFxhZRKL8UhqhN9bPYosDvZRG7eFqCAf/0SlslPMnVsIc8tQkhAm9lYbHyS8G1QRQ SWD9FeBjtWWng6VEzap4ljwsBKjeH52gOr+ePE7uUcQd5vrcUO13BgS6RpOxBWsfgAvk Yj8IYLmH1FlJeEGmlUqWb8hm/epbSgCD8CzNtHAqpj2vv1CPpYUxRKkfigUHJPO75G6w 55wL9rYkPPKc2IWrK7GD7WEZPL8yKhRtrQp5IZk+zEXVFTsmJc96yCWlE7yp1ddg0moe nJ/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724167491; x=1724772291; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rJi2ebhbP9zlbLGPWODvcw9blh6x5jyJs+zGiuzhdIE=; b=cs0LbogakQNbgcRkXON8JIEa+DQtojQNVfHKTL4Qf0BjvYWAM4KpnxNfrgbZJneIAj p1GLeMCUL0e3g9/xYozCJ8C1T6bVScVPtHHsUyd7E+FZhhpEa7lyVD1oCB7H8JO98uzZ 8swql6CJHSH6riAx5MMP+aNYxus/MrJCaxI5mosF/dV153tKNXnMBp1iddtrU5Ggjfau NC2shEBeCsboTtmB2MsALjva5ytp2pgHD5npVQ/egMBgU6cz8lLajWjwwIeGgctdG3rk FtJC26OgYgq/cl6abdYVMMmx2n3X9PQk7TE4iYfCS1GUnOWk467RS9YujWWY4xKy+K9q Q05Q== X-Forwarded-Encrypted: i=1; AJvYcCU0S/6YYdWid0NIoB4AGF6QyHxjH0Mwxw15BLyzRF31C29MSphk+hfN9rl19q8GCTVPAOp/BGM3SVNGClw=@vger.kernel.org X-Gm-Message-State: AOJu0YzviQm1ycRMXCFjn607HnWdc3EV0XgNn4ZgotNGEFGk4MtyEoVG a6BB/VQ8AZKgmP6uZQ53ErnypbUlkRbaymqPoSRR+yCBmvg40uHYsXF5HLXXvK4= X-Google-Smtp-Source: AGHT+IEclpseZglRZhbnRuBkQilDDbPp9TBiZLIEf2DItUIwluK/RElu63vQyqZY6QZ4MDhVhG0uhw== X-Received: by 2002:a17:90a:8c94:b0:2d1:ca16:5555 with SMTP id 98e67ed59e1d1-2d3e03e83e5mr13286356a91.37.1724167491452; Tue, 20 Aug 2024 08:24:51 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d40bea7cb3sm7258157a91.25.2024.08.20.08.24.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 08:24:51 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v9 5/6] RISC-V: Report vector unaligned access speed hwprobe Date: Tue, 20 Aug 2024 11:24:23 -0400 Message-ID: <20240820152424.1973078-6-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240820152424.1973078-1-jesse@rivosinc.com> References: <20240820152424.1973078-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Detect if vector misaligned accesses are faster or slower than equivalent vector byte accesses. This is useful for usermode to know whether vector byte accesses or vector misaligned accesses have a better bandwidth for operations like memcpy. Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins --- V1 -> V2: - Add Kconfig options - Add WORD_EEW to vec-copy-unaligned.S V2 -> V3: - Remove unnecessary comment - Remove local_irq_enable V3 -> V4: - Add preempt_disable/enable - Alphabetize includes in vec-copy-unaligned.S and unaligned_access_speed.c - Add duplicate comments above mb() to please checkpatch - change all_cpus_vec_supported to all_cpus_vec_unsupported so speed is tested if any cpus support unaligned vector accesses - Spell out _VECTOR_ in macros V4 -> V5: - Change void *unused to void *unused __always_unused V5 -> V6: - Check for vector misaligned access support in hotplug V6 -> V7: - Change SLOW to UNKNOWN when used as a placeholder V7 -> V8: - Rebase onto fixes - s/RISCV_HWPROBE_VECTOR_MISALIGNED/RISCV_HWPROBE_MISALIGNED_VECTOR/g V8 -> V9: - No changes --- arch/riscv/Kconfig | 18 +++ arch/riscv/kernel/Makefile | 3 +- arch/riscv/kernel/copy-unaligned.h | 5 + arch/riscv/kernel/sys_hwprobe.c | 6 + arch/riscv/kernel/unaligned_access_speed.c | 141 ++++++++++++++++++++- arch/riscv/kernel/vec-copy-unaligned.S | 58 +++++++++ 6 files changed, 228 insertions(+), 3 deletions(-) create mode 100644 arch/riscv/kernel/vec-copy-unaligned.S diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 3bb7bf0e9ddc..db1393ba5258 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -855,6 +855,24 @@ config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS will dynamically determine the speed of vector unaligned accesses on the underlying system if they are supported. =20 +config RISCV_SLOW_VECTOR_UNALIGNED_ACCESS + bool "Assume the system supports slow vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports slow vector unaligned memory accesses. = The + kernel and userspace programs may not be able to run at all on systems + that do not support unaligned memory accesses. + +config RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS + bool "Assume the system supports fast vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports fast vector unaligned memory accesses. = When + enabled, this option improves the performance of the kernel on such + systems. However, the kernel and userspace programs will run much more + slowly, or will not be able to run at all, on systems that do not + support efficient unaligned memory accesses. + endchoice =20 source "arch/riscv/Kconfig.vendor" diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 06d407f1b30b..c72ca98a1a1c 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -66,7 +66,8 @@ obj-$(CONFIG_MMU) +=3D vdso.o vdso/ =20 obj-$(CONFIG_RISCV_MISALIGNED) +=3D traps_misaligned.o obj-$(CONFIG_RISCV_MISALIGNED) +=3D unaligned_access_speed.o -obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS) +=3D vec-copy-unaligned.o =20 obj-$(CONFIG_FPU) +=3D fpu.o obj-$(CONFIG_FPU) +=3D kernel_mode_fpu.o diff --git a/arch/riscv/kernel/copy-unaligned.h b/arch/riscv/kernel/copy-un= aligned.h index e3d70d35b708..85d4d11450cb 100644 --- a/arch/riscv/kernel/copy-unaligned.h +++ b/arch/riscv/kernel/copy-unaligned.h @@ -10,4 +10,9 @@ void __riscv_copy_words_unaligned(void *dst, const void *src, size_t size); void __riscv_copy_bytes_unaligned(void *dst, const void *src, size_t size); =20 +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +void __riscv_copy_vec_words_unaligned(void *dst, const void *src, size_t s= ize); +void __riscv_copy_vec_bytes_unaligned(void *dst, const void *src, size_t s= ize); +#endif + #endif /* __RISCV_KERNEL_COPY_UNALIGNED_H */ diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprob= e.c index 6441baada36b..6673278e84d5 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -228,6 +228,12 @@ static u64 hwprobe_vec_misaligned(const struct cpumask= *cpus) #else static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) { + if (IS_ENABLED(CONFIG_RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_MISALIGNED_VECTOR_FAST; + + if (IS_ENABLED(CONFIG_RISCV_SLOW_VECTOR_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_MISALIGNED_VECTOR_SLOW; + return RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN; } #endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel= /unaligned_access_speed.c index 0b8b5e17453a..91f189cf1611 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -6,11 +6,13 @@ #include #include #include +#include #include #include #include #include #include +#include =20 #include "copy-unaligned.h" =20 @@ -268,12 +270,147 @@ static int check_unaligned_access_speed_all_cpus(voi= d) } #endif =20 +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +static void check_vector_unaligned_access(struct work_struct *work __alway= s_unused) +{ + int cpu =3D smp_processor_id(); + u64 start_cycles, end_cycles; + u64 word_cycles; + u64 byte_cycles; + int ratio; + unsigned long start_jiffies, now; + struct page *page; + void *dst; + void *src; + long speed =3D RISCV_HWPROBE_MISALIGNED_VECTOR_SLOW; + + if (per_cpu(vector_misaligned_access, cpu) !=3D RISCV_HWPROBE_MISALIGNED_= VECTOR_UNKNOWN) + return; + + page =3D alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); + if (!page) { + pr_warn("Allocation failure, not measuring vector misaligned performance= \n"); + return; + } + + /* Make an unaligned destination buffer. */ + dst =3D (void *)((unsigned long)page_address(page) | 0x1); + /* Unalign src as well, but differently (off by 1 + 2 =3D 3). */ + src =3D dst + (MISALIGNED_BUFFER_SIZE / 2); + src +=3D 2; + word_cycles =3D -1ULL; + + /* Do a warmup. */ + kernel_vector_begin(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + + start_jiffies =3D jiffies; + while ((now =3D jiffies) =3D=3D start_jiffies) + cpu_relax(); + + /* + * For a fixed amount of time, repeatedly try the function, and take + * the best time in cycles as the measurement. + */ + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles =3D get_cycles64(); + /* Ensure the CSR read can't reorder WRT to the copy. */ + mb(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + /* Ensure the copy ends before the end time is snapped. */ + mb(); + end_cycles =3D get_cycles64(); + if ((end_cycles - start_cycles) < word_cycles) + word_cycles =3D end_cycles - start_cycles; + } + + byte_cycles =3D -1ULL; + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + start_jiffies =3D jiffies; + while ((now =3D jiffies) =3D=3D start_jiffies) + cpu_relax(); + + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles =3D get_cycles64(); + /* Ensure the CSR read can't reorder WRT to the copy. */ + mb(); + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + /* Ensure the copy ends before the end time is snapped. */ + mb(); + end_cycles =3D get_cycles64(); + if ((end_cycles - start_cycles) < byte_cycles) + byte_cycles =3D end_cycles - start_cycles; + } + + kernel_vector_end(); + + /* Don't divide by zero. */ + if (!word_cycles || !byte_cycles) { + pr_warn("cpu%d: rdtime lacks granularity needed to measure unaligned vec= tor access speed\n", + cpu); + + return; + } + + if (word_cycles < byte_cycles) + speed =3D RISCV_HWPROBE_MISALIGNED_VECTOR_FAST; + + ratio =3D div_u64((byte_cycles * 100), word_cycles); + pr_info("cpu%d: Ratio of vector byte access time to vector unaligned word= access is %d.%02d, unaligned accesses are %s\n", + cpu, + ratio / 100, + ratio % 100, + (speed =3D=3D RISCV_HWPROBE_MISALIGNED_VECTOR_FAST) ? "fast" : "slow"); + + per_cpu(vector_misaligned_access, cpu) =3D speed; +} + +static int riscv_online_cpu_vec(unsigned int cpu) +{ + if (!has_vector()) + return 0; + + if (per_cpu(vector_misaligned_access, cpu) !=3D RISCV_HWPROBE_MISALIGNED_= VECTOR_UNSUPPORTED) + return 0; + + check_vector_unaligned_access_emulated(NULL); + check_vector_unaligned_access(NULL); + return 0; +} + +/* Measure unaligned access speed on all CPUs present at boot in parallel.= */ +static int vec_check_unaligned_access_speed_all_cpus(void *unused __always= _unused) +{ + schedule_on_each_cpu(check_vector_unaligned_access); + + /* + * Setup hotplug callbacks for any new CPUs that come online or go + * offline. + */ + cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "riscv:online", + riscv_online_cpu_vec, NULL); + + return 0; +} +#else /* CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS */ +static int vec_check_unaligned_access_speed_all_cpus(void *unused __always= _unused) +{ + return 0; +} +#endif + static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated; + bool all_cpus_emulated, all_cpus_vec_unsupported; =20 all_cpus_emulated =3D check_unaligned_access_emulated_all_cpus(); - check_vector_unaligned_access_emulated_all_cpus(); + all_cpus_vec_unsupported =3D check_vector_unaligned_access_emulated_all_c= pus(); + + if (!all_cpus_vec_unsupported && + IS_ENABLED(CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS)) { + kthread_run(vec_check_unaligned_access_speed_all_cpus, + NULL, "vec_check_unaligned_access_speed_all_cpus"); + } =20 if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); diff --git a/arch/riscv/kernel/vec-copy-unaligned.S b/arch/riscv/kernel/vec= -copy-unaligned.S new file mode 100644 index 000000000000..d16f19f1b3b6 --- /dev/null +++ b/arch/riscv/kernel/vec-copy-unaligned.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2024 Rivos Inc. */ + +#include +#include +#include + + .text + +#define WORD_EEW 32 + +#define WORD_SEW CONCATENATE(e, WORD_EEW) +#define VEC_L CONCATENATE(vle, WORD_EEW).v +#define VEC_S CONCATENATE(vle, WORD_EEW).v + +/* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using word loads and stores= . */ +/* Note: The size is truncated to a multiple of WORD_EEW */ +SYM_FUNC_START(__riscv_copy_vec_words_unaligned) + andi a4, a2, ~(WORD_EEW-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, WORD_SEW, m8, ta, ma + VEC_L v0, (a1) + VEC_S v0, (a0) + addi a0, a0, WORD_EEW + addi a1, a1, WORD_EEW + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_words_unaligned) + +/* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using only byte accesses. */ +/* Note: The size is truncated to a multiple of 8 */ +SYM_FUNC_START(__riscv_copy_vec_bytes_unaligned) + andi a4, a2, ~(8-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, e8, m8, ta, ma + vle8.v v0, (a1) + vse8.v v0, (a0) + addi a0, a0, 8 + addi a1, a1, 8 + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_bytes_unaligned) --=20 2.45.2 From nobody Fri Sep 20 23:49:02 2024 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63A4A195B08 for ; Tue, 20 Aug 2024 15:24:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167497; cv=none; b=aWs4/UWGH6lx+UIOpgR1Wav36RKtfp0E6rCAM/Rl+Tt0mqqTEX4MUpj3DoafJdASlglYBqm6YWPXry41cALPmxAkA7lBpMZryunB36TLi+owyXMRlLb9E1qL5jIFE46pmT8E3pzd3QqQP/j5XD8WjF4eqQkZjveWL8u4o09+G24= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724167497; c=relaxed/simple; bh=5IrSzO4NK/egoPWEpij4VPue9Dvq4+IjNrpVVyKy6PA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hH3OCeGVasjQRFC9iRlZ4zLmJvlqSXkjYq75N9znsWcDsiy0kBer4OaKGQ77GWHDkOb8e/3c8UMKjbrktoyik+sPY0B6jjqHk3eUQK3zu7xAqT3zxHY2a7GHe5AWHyH+x0CtnrzTqfZ0rhfT2gmuks9w9G2TRW8h34B+oQqDv0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=vE4hmdCw; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="vE4hmdCw" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2d3c5f769d6so3912600a91.3 for ; Tue, 20 Aug 2024 08:24:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1724167496; x=1724772296; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K9yIy6nc3JckPfgq1SVS1dt/ubbPwao76sjUpFJbUU0=; b=vE4hmdCw6asCkj/eoSUDZyyql82iSLSrYE4zYJZZIfB30hIweA1oom04xEUSd4RgMI 6nG1SBqpcyY5Aatcrwkmx00UIJdn8ckGJorSmOKxhX9QZW4Ic7hIslvLgxBd2U5pLmga 3LpQoLsdMMzx/PXEwcU4zAtKy2IvYsCgWiRPYf5OSwP8zLOx6GWhB7b4uFecemYwFUHx In+urh3VA558N+DdSA0pvoMUktpUG+84SJ1lLJTvODK+LLncTsKCCGJVX7yq1swiqV6I kL/6ZwfyRkKM/cKmeZWp6JXLQYHEHbJBGCcqiTpxvkazNyrLpjEpQecctFFEhM+qbgYR rjPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724167496; x=1724772296; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K9yIy6nc3JckPfgq1SVS1dt/ubbPwao76sjUpFJbUU0=; b=JTEPqIvfzJgk/pkk+C4v5fD8A8HsGZsa/8sNC96diS73DCA4qii3jGExT0Jp+iyu3O gBIWZK/btoYT0fQ/40P8vp1KtKt2Mz3kQaiw3SDf0YUkTFMP85Feua/cGsKSOu94xiwU Kwbzj5qhgbSks51K9BQs2zvvksdG/lXXo/+OLe7pm3ZNcX9AaAH4vyx39uciWcy55US9 xNH3bz6k/wBhzvPlgN69fUaHn/Ki3u9rrGBM/v6VGY18FRiPkCaaa6ezlii+gzatf8fR je2A5n++eZ6CHIUz7xqp2obP8AnwAID86Sxx4XUuDVzp3NCyZ95DnXr/QMs34YNAtnGD Unfg== X-Forwarded-Encrypted: i=1; AJvYcCV14x6mrF35IvJu5GMEQtmxLJl9SWsJOqBEuKW3FYw+UsniPrE30xQk2Y+NBIH61qf6GSq+NMq22p1TBbh13IxjGx1/c31viyQMIkrm X-Gm-Message-State: AOJu0YyK4O+WR/prb5oXGXYBoadthK+AqMD2AXnMshU/W1jEc8cbBxFb NpxwOXvJCjSCej2bM1hHBSPLvXuYGbV35wIiMCL9PBuOq7nI2OOQv4r1UXu8THOfuiPN/juRsGl D X-Google-Smtp-Source: AGHT+IFjpFHeANOi3RNiE4tvCPPb8ENoVMI5hfDn6rIWfrZrc5je7gBHKyKa1cngmecZurzkCscwrA== X-Received: by 2002:a17:90a:1bcf:b0:2ca:f39c:8d76 with SMTP id 98e67ed59e1d1-2d3e03e8cb9mr14580242a91.39.1724167495634; Tue, 20 Aug 2024 08:24:55 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d40bea7cb3sm7258157a91.25.2024.08.20.08.24.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 08:24:55 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v9 6/6] RISC-V: hwprobe: Document unaligned vector perf key Date: Tue, 20 Aug 2024 11:24:24 -0400 Message-ID: <20240820152424.1973078-7-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240820152424.1973078-1-jesse@rivosinc.com> References: <20240820152424.1973078-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Document key for reporting the speed of unaligned vector accesses. The descriptions are the same as the scalar equivalent values. Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins --- V1 -> V2: - New patch V2 -> V3: - Specify access width V3 -> V4: - Clarify we're talking about byte accesses using vector registers - Spell out _VECTOR_ in macros V4 -> V5: - No changes V5 -> V6: - No changes V6 -> V7: - No changes V7 -> V8: - Rebase onto fixes - s/RISCV_HWPROBE_VECTOR_MISALIGNED/RISCV_HWPROBE_MISALIGNED_VECTOR/g V8 -> V9: - No changes --- Documentation/arch/riscv/hwprobe.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/risc= v/hwprobe.rst index 85b709257918..ea4e0b9c73e7 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -274,3 +274,19 @@ The following keys are defined: represent the highest userspace virtual address usable. =20 * :c:macro:`RISCV_HWPROBE_KEY_TIME_CSR_FREQ`: Frequency (in Hz) of `time C= SR`. + +* :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_VECTOR_PERF`: An enum value descr= ibing the + performance of misaligned vector accesses on the selected set of proc= essors. + + * :c:macro:`RISCV_HWPROBE_MISALIGNED_VECTOR_UNKNOWN`: The performance of= misaligned + vector accesses is unknown. + + * :c:macro:`RISCV_HWPROBE_MISALIGNED_VECTOR_SLOW`: 32-bit misaligned acc= esses using vector + registers are slower than the equivalent quantity of byte accesses via= vector registers. + Misaligned accesses may be supported directly in hardware, or trapped = and emulated by software. + + * :c:macro:`RISCV_HWPROBE_MISALIGNED_VECTOR_FAST`: 32-bit misaligned acc= esses using vector + registers are faster than the equivalent quantity of byte accesses via= vector registers. + + * :c:macro:`RISCV_HWPROBE_MISALIGNED_VECTOR_UNSUPPORTED`: Misaligned vec= tor accesses are + not supported at all and will generate a misaligned address fault. --=20 2.45.2