From nobody Thu Feb 12 10:55:44 2026 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6A5B14D6FB for ; Thu, 13 Jun 2024 19:17:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718306257; cv=none; b=N+8y1lUjb+6/8GRhTc6CVNVFWZmcaYwjQxLOuDlTt7IlIcpds4oKu05c5cmLRwKK6NqJQxY+qSkZrP8PC1NLFbt2hjMIZBVBncWvs8wexjvYOoKS0SlhJ0ZRGbvg7g92uYc51MXtmXHA/qUMeXC0ntQHOfI9ehYATfq2eaStv7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718306257; c=relaxed/simple; bh=0cPUkspGSjjMOzYvSZ14kgZhWCkJU0pTLCeIJz1tZ+U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BtqCScfUqMmIEhF4MUsmuFDa+IGqRFa15ED+GS3Kg8/4ZGEUr6rZfw5C7Bst4e78ChjjF1snzz4F3MU1pMDp5RwhxOI3QuKTOwZz8pyfxltxCZxR7YhYJysaCdqkd2RZRXnM4f94HtYVkPy6CXnWjsoQ6VGpSyhTaj+Tsjt1O6A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=it1LhTqn; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="it1LhTqn" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1f4c7b022f8so13447745ad.1 for ; Thu, 13 Jun 2024 12:17:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1718306254; x=1718911054; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=muXFJgsYoKCYgzsWRlHOJmTR9Kg/zixWqbAUEZb+IAk=; b=it1LhTqn/mihHquNoDLYiaqR7jVtmYItwSSR2hKXLyHEnw2aUemVDPxWd2hG5BpLCC U4gODAHZvhCjcElL/qsrJQsZiaZ2vPV9TBucisXkh5PLvOcDMmlFUX3ZAKgWtUXu1cqg yRJ14ZYELrvRiaFiI+fXtN/fUqUQDWVHRkrssF2v+NpArXyFGEC/5kRmJn2Kf3Q+s4Ya HLzjxHQEyR6ol1zyLdqr7qFzU+Om9i20K834TF6VxuSSZjrpYNM0d0OK/mG/0+4a8Ti+ c/yuT2ZoWGgvCCBUc/F7Ey4IFG1FhI1HdY9TBb8Yjq2mqFeWEiN/VmBJW4gjz+p4lWwO ePQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718306254; x=1718911054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=muXFJgsYoKCYgzsWRlHOJmTR9Kg/zixWqbAUEZb+IAk=; b=Ajtp3x6zKzM4RsMRy+CKYWtf4EzsPgmCA4QUBppN9PSSZneKBTzgLPECmOk5XvCKTp wW8MzxAysP4hiNISTu2PV6cz4T+WQ+0loXIlZAFQrrTDwW30BUM4lR99tVIjPsVnXF81 g23TOp+nduUuhFjeks8Vf2IULFcox/+3EI/nbzJHotU2gRVDX5GibX0iJfp78sDPK6ir 1DwA5lqIgxbnN+rfVtxGp+y2LHeT6mWd24rAyjSOhEDksfK6qW4/wa95joiZW+1S4lm/ tGjEswH3Yb9yFuS95pY40y77Jn84NLdMuF3R7RxB/zvgpti00u6DckeECsgEm5fcgA0k TR/Q== X-Forwarded-Encrypted: i=1; AJvYcCUu/dg5zB9txYVSNlxZ3+T9qGOidzUnl+qiV5ZYhHvVUTTKuXTd0qYUCS5M/yqZhG8+opVtiRKWdm9jj8B366Q6gIZP+XI54Bgx/dsY X-Gm-Message-State: AOJu0YzsdIdN4skU81ZKzn/cal5rZ7/peRpaXoJme0aNs4qgXFGD+x7X LO24hrVg5+vxRDlLaiHtMCCQwB7hkXrCafIfWuRaL7kfXGnKa1+oVQd9oH+F3qA= X-Google-Smtp-Source: AGHT+IE5SiAOXuRQi90VkoN3+jqAIhtZnqDv6P9KtRxeggeK3SAZ6fJEsAO5NNsLNSZ2/jrdFBDNcg== X-Received: by 2002:a17:902:d2d0:b0:1f4:ac10:3ede with SMTP id d9443c01a7336-1f8627d1668mr5735685ad.21.1718306253854; Thu, 13 Jun 2024 12:17:33 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e7ca78sm17471015ad.106.2024.06.13.12.17.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 12:17:33 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v2 4/6] RISC-V: Detect unaligned vector accesses supported. Date: Thu, 13 Jun 2024 15:16:13 -0400 Message-ID: <20240613191616.2101821-5-jesse@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240613191616.2101821-1-jesse@rivosinc.com> References: <20240613191616.2101821-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Run a unaligned vector access to test if the system supports vector unaligned access. Add the result to a new key in hwprobe. This is useful for usermode to know if vector misaligned accesses are supported and if they are faster or slower than equivalent byte accesses. Signed-off-by: Jesse Taube --- V1 -> V2: - Add Kconfig options - Add insn_is_vector - Add handle_vector_misaligned_load - Fix build - Seperate vector from scalar misaligned access - This patch was almost completely rewritten --- arch/riscv/Kconfig | 41 +++++++ arch/riscv/include/asm/cpufeature.h | 7 +- arch/riscv/include/asm/entry-common.h | 11 -- arch/riscv/include/asm/hwprobe.h | 2 +- arch/riscv/include/asm/vector.h | 1 + arch/riscv/include/uapi/asm/hwprobe.h | 5 + arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/sys_hwprobe.c | 41 +++++++ arch/riscv/kernel/traps_misaligned.c | 119 ++++++++++++++++++++- arch/riscv/kernel/unaligned_access_speed.c | 9 +- arch/riscv/kernel/vector.c | 2 +- 11 files changed, 221 insertions(+), 21 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index b94176e25be1..f12df0ca6c18 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -723,6 +723,12 @@ config RISCV_MISALIGNED help Embed support for emulating misaligned loads and stores. =20 +config RISCV_VECTOR_MISALIGNED + bool + depends on RISCV_ISA_V + help + Enable detecting support for vector misaligned loads and stores. + choice prompt "Unaligned Accesses Support" default RISCV_PROBE_UNALIGNED_ACCESS @@ -774,6 +780,41 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS =20 endchoice =20 +choice + prompt "Vector unaligned Accesses Support" + depends on RISCV_ISA_V + default RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + help + This determines the level of support for vector unaligned accesses. This + information is used by the kernel to perform optimizations. It is also + exposed to user space via the hwprobe syscall. The hardware will be + probed at boot by default. + +config RISCV_DETECT_VECTOR_UNALIGNED_ACCESS + bool "Detect support for vector unaligned accesses" + select RISCV_VECTOR_MISALIGNED + help + During boot, the kernel will detect if the system supports vector + unaligned accesses. + +config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + bool "Probe speed of vector unaligned accesses" + select RISCV_VECTOR_MISALIGNED + help + During boot, the kernel will run a series of tests to determine the + speed of vector unaligned accesses if they are supported. This probing + will dynamically determine the speed of vector unaligned accesses on + the underlying system if they are supported. + +config CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED + bool "Assume the system does not support vector unaligned memory accesses" + help + Assume that the system does not support vector unaligned memory accesse= s. + The kernel and userspace programs may run them successfully on systems + that do support vector unaligned memory accesses. + +endchoice + endmenu # "Platform type" =20 menu "Kernel features" diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 347805446151..d0ea5921ab20 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -33,8 +33,8 @@ extern struct riscv_isainfo hart_isa[NR_CPUS]; =20 void riscv_user_isa_enable(void); =20 -#if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_MISALIGNED) void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); DECLARE_PER_CPU(long, misaligned_access_speed); @@ -45,6 +45,11 @@ static inline bool unaligned_ctl_available(void) } #endif =20 +bool check_vector_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_VECTOR_MISALIGNED) +DECLARE_PER_CPU(long, vector_misaligned_access); +#endif + #if defined(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) DECLARE_STATIC_KEY_FALSE(fast_unaligned_access_speed_key); =20 diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm= /entry-common.h index 2293e535f865..7b32d2b08bb6 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,18 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct= pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); =20 -#ifdef CONFIG_RISCV_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); -#else -static inline int handle_misaligned_load(struct pt_regs *regs) -{ - return -1; -} -static inline int handle_misaligned_store(struct pt_regs *regs) -{ - return -1; -} -#endif =20 #endif /* _ASM_RISCV_ENTRY_COMMON_H */ diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwpr= obe.h index 150a9877b0af..ef01c182af2b 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -8,7 +8,7 @@ =20 #include =20 -#define RISCV_HWPROBE_MAX_KEY 7 +#define RISCV_HWPROBE_MAX_KEY 8 =20 static inline bool riscv_hwprobe_key_is_valid(__s64 key) { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vecto= r.h index be7d309cca8a..99b0f91db9ee 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -21,6 +21,7 @@ =20 extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); +bool insn_is_vector(u32 insn_buf); bool riscv_v_first_use_handler(struct pt_regs *regs); void kernel_vector_begin(void); void kernel_vector_end(void); diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uap= i/asm/hwprobe.h index 023b7771d1b7..2fee870e41bb 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -75,6 +75,11 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 #define RISCV_HWPROBE_KEY_MISALIGNED_PERF 7 +#define RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF 8 +#define RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN 0 +#define RISCV_HWPROBE_VEC_MISALIGNED_SLOW 2 +#define RISCV_HWPROBE_VEC_MISALIGNED_FAST 3 +#define RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED 4 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ =20 /* Flags */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 5b243d46f4b1..62ac19c029f1 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -62,8 +62,8 @@ obj-y +=3D probes/ obj-y +=3D tests/ obj-$(CONFIG_MMU) +=3D vdso.o vdso/ =20 -obj-$(CONFIG_RISCV_MISALIGNED) +=3D traps_misaligned.o -obj-$(CONFIG_RISCV_MISALIGNED) +=3D unaligned_access_speed.o +obj-y +=3D traps_misaligned.o +obj-y +=3D unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o =20 obj-$(CONFIG_FPU) +=3D fpu.o diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprob= e.c index e910e2971984..c40df314058b 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -194,6 +194,43 @@ static u64 hwprobe_misaligned(const struct cpumask *cp= us) } #endif =20 +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + int cpu; + u64 perf =3D -1ULL; + + if (IS_ENABLED(CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED)) + return RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + + /* Return if supported or not even if speed wasn't probed */ + for_each_cpu(cpu, cpus) { + int this_perf =3D per_cpu(vector_misaligned_access, cpu); + + if (perf =3D=3D -1ULL) + perf =3D this_perf; + + if (perf !=3D this_perf) { + perf =3D RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + break; + } + } + + if (perf =3D=3D -1ULL) + return RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + + return perf; +} +#else +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + if (IS_ENABLED(CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED)) + return RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + + return RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; +} +#endif + static void hwprobe_one_pair(struct riscv_hwprobe *pair, const struct cpumask *cpus) { @@ -222,6 +259,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pai= r, pair->value =3D hwprobe_misaligned(cpus); break; =20 + case RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF: + pair->value =3D hwprobe_vec_misaligned(cpus); + break; + case RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE: pair->value =3D 0; if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOZ)) diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps= _misaligned.c index 8fadbe00dd62..6f0264a8c9de 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -16,6 +16,7 @@ #include #include #include +#include =20 #define INSN_MATCH_LB 0x3 #define INSN_MASK_LB 0x707f @@ -322,12 +323,37 @@ union reg_data { u64 data_u64; }; =20 -static bool unaligned_ctl __read_mostly; - /* sysctl hooks */ int unaligned_enabled __read_mostly =3D 1; /* Enabled by default */ =20 -int handle_misaligned_load(struct pt_regs *regs) +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc =3D regs->epc; + unsigned long insn; + + if (get_insn(regs, epc, &insn)) + return -1; + + /* Only return 0 when in check_vector_unaligned_access_emulated */ + if (*this_cpu_ptr(&vector_misaligned_access) =3D=3D RISCV_HWPROBE_VEC_MIS= ALIGNED_UNKNOWN) { + *this_cpu_ptr(&vector_misaligned_access) =3D RISCV_HWPROBE_VEC_MISALIGNE= D_UNSUPPORTED; + regs->epc =3D epc + INSN_LEN(insn); + return 0; + } + + /* If vector instruction we don't emulate it yet */ + regs->epc =3D epc; + return -1; +} +#else +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + return -1; +} +#endif + +static int handle_scalar_misaligned_load(struct pt_regs *regs) { union reg_data val; unsigned long epc =3D regs->epc; @@ -435,7 +461,7 @@ int handle_misaligned_load(struct pt_regs *regs) return 0; } =20 -int handle_misaligned_store(struct pt_regs *regs) +static int handle_scalar_misaligned_store(struct pt_regs *regs) { union reg_data val; unsigned long epc =3D regs->epc; @@ -526,6 +552,85 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } =20 +int handle_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc =3D regs->epc; + unsigned long insn; + + if (IS_ENABLED(CONFIG_RISCV_VECTOR_MISALIGNED)) { + if (get_insn(regs, epc, &insn)) + return -1; + + if (insn_is_vector(insn)) + return handle_vector_misaligned_load(regs); + } + + if (IS_ENABLED(CONFIG_RISCV_MISALIGNED)) + return handle_scalar_misaligned_load(regs); + + return -1; +} + +int handle_misaligned_store(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_RISCV_MISALIGNED)) + return handle_scalar_misaligned_store(regs); + + return -1; +} + +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static void check_vector_unaligned_access_emulated(struct work_struct *unu= sed) +{ + long *mas_ptr =3D this_cpu_ptr(&vector_misaligned_access); + unsigned long tmp_var; + + *mas_ptr =3D RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + + local_irq_enable(); + kernel_vector_begin(); + __asm__ __volatile__ ( + ".balign 4\n\t" + ".option push\n\t" + ".option arch, +zve32x\n\t" + " vsetivli zero, 1, e16, m1, ta, ma\n\t" // Vectors of 16b + " vle16.v v0, (%[ptr])\n\t" // Load bytes + ".option pop\n\t" + : : [ptr] "r" ((u8 *)&tmp_var + 1) : "v0"); + kernel_vector_end(); +} + +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + int cpu; + bool ret =3D true; + + if (!has_vector()) { + for_each_online_cpu(cpu) + per_cpu(vector_misaligned_access, cpu) =3D RISCV_HWPROBE_VEC_MISALIGNED= _UNSUPPORTED; + return false; + } + + schedule_on_each_cpu(check_vector_unaligned_access_emulated); + + for_each_online_cpu(cpu) + if (per_cpu(vector_misaligned_access, cpu) + !=3D RISCV_HWPROBE_VEC_MISALIGNED_SLOW) + return false; + + return ret; +} +#else +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif + +#ifdef CONFIG_RISCV_MISALIGNED + +static bool unaligned_ctl __read_mostly; + static void check_unaligned_access_emulated(struct work_struct *unused) { int cpu =3D smp_processor_id(); @@ -563,3 +668,9 @@ bool unaligned_ctl_available(void) { return unaligned_ctl; } +#else +bool check_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel= /unaligned_access_speed.c index 70c1588fc353..c6106bd4a25a 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -19,7 +19,8 @@ #define MISALIGNED_BUFFER_ORDER get_order(MISALIGNED_BUFFER_SIZE) #define MISALIGNED_COPY_SIZE ((MISALIGNED_BUFFER_SIZE / 2) - 0x80) =20 -DEFINE_PER_CPU(long, misaligned_access_speed); +DEFINE_PER_CPU(long, misaligned_access_speed) =3D RISCV_HWPROBE_MISALIGNED= _UNKNOWN; +DEFINE_PER_CPU(long, vector_misaligned_access) =3D RISCV_HWPROBE_VEC_MISAL= IGNED_UNSUPPORTED; =20 #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS static cpumask_t fast_misaligned_access; @@ -268,12 +269,18 @@ static int check_unaligned_access_all_cpus(void) =20 if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICCLSM)) { for_each_online_cpu(cpu) { +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED + per_cpu(vector_misaligned_access, cpu) =3D RISCV_HWPROBE_VEC_MISALIGNED= _FAST; +#endif +#ifdef CONFIG_RISCV_MISALIGNED per_cpu(misaligned_access_speed, cpu) =3D RISCV_HWPROBE_MISALIGNED_FAST; +#endif } return 0; } =20 all_cpus_emulated =3D check_unaligned_access_emulated_all_cpus(); + check_vector_unaligned_access_emulated_all_cpus(); =20 #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS if (!all_cpus_emulated) diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 682b3feee451..821818886fab 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -66,7 +66,7 @@ void __init riscv_v_setup_ctx_cache(void) #endif } =20 -static bool insn_is_vector(u32 insn_buf) +bool insn_is_vector(u32 insn_buf) { u32 opcode =3D insn_buf & __INSN_OPCODE_MASK; u32 width, csr; --=20 2.43.0