From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A92DC17C7D7 for ; Fri, 26 Jul 2024 16:37:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011851; cv=none; b=kHFASuWQNM9g/22ds/DS++4ahsDW+k7jUdB1YZ0vkaMWs0FZ+Sj+y+0ihz3vLWqdpqkLb/kKquoJocGyBcE/fJxdqmusmaUFAPhEWmFSdVxayK/Aqu2C/uXRGwbF1r9WikRXLuGH1OSZ5lSWBBb2Sen4UFNqctzOe1uo9/qfPB8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011851; c=relaxed/simple; bh=WXm9/HAXk5Lab1ONciXViU9JUFRbDlVwDeuyb8DjUoQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cKXlI9L/S5/0PIn44xuQkMSPKIxOKttJ83uYryoTzWTf/c1SIOd62BOwiNZBUN9q9TT30dsOezhmPychLx4n3YuY6CqytT2WphDYzp7uxeGudJS7klEEZy3pCz4oE6/l/eLorUTJKpFMFIMeRJbH3tpHObXd0GxoNLG2CwqKhig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=GbPk1VnS; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="GbPk1VnS" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-70d2b921cdfso1051448b3a.0 for ; Fri, 26 Jul 2024 09:37:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011849; x=1722616649; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FV9ltTuKAXhQPitmW4cZ8QxbMM7L9R/Whun6TjodNdQ=; b=GbPk1VnS5ELBO44VFihW30svqRWXgmMqfhDu8Dd3z0ymPL9Dy9P/SuZOeD28P8AwXz uarGu//K7VFgTHh2UxEVtkDuTj3RUbzTAaRNBE7Ku3ogXLMSBbg1rmCd11TXpPha/mX1 /6aKbtlFDixcm4SK+K1PDOvGFTQaDdP+Z5hQzCCRjEVLi108p9O9Wah91SFqlNSkAMFU c44+dMA1Fe3ur7LRwcmLJX+xRCLkQhnEX8x800IeD6UfgyIySnJxdl1A7bDF8Yt6XlTH 4ezE+YmXlVM/+2BXh1joFSXQt4JDWH6ZeulJ9forNZkaaYnZFKAM2yFuenpV4FBUlAct PU5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011849; x=1722616649; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FV9ltTuKAXhQPitmW4cZ8QxbMM7L9R/Whun6TjodNdQ=; b=gIjmOKdV5AEcF8cUUIz398/xILpzpK8s6UzI/bfNGr1HdSnfdOZ+C1hoW9xgUySdLA tQWWMf0up+fbFxDus1aOT5HNUBUHFspnS60ze9aHncB5UnAO5cOlnBZOp3LTK5jBsD97 EWaPRADDZwkkxfl9oNaSuRFtNVvK84G1TRZl5zqOLahF3GKaDegY+5jlkP4Ac4AIlOtH gYps3LL0Md2xyqECxmct2348Grbq2C8S0gtEvA8LVgD5Xu71IxRVi6slaByCIPjBq9o+ zoOo+XUMn+z+TWxDEPyeHo7QHjefvXL+YLYXrP7fK+2vTc9ti4ksP0umCy6O/Pl1kQgx SqHg== X-Forwarded-Encrypted: i=1; AJvYcCU18nK3/aU1rVtceu62EWvWRwjhFctW27fCyN4rL7u2yKLpwJrDcMRLMEYhh2MonDSZY04sj3ZOaK4ArY7Ws/K+qgGVKalDy3McuPaX X-Gm-Message-State: AOJu0YxR/Ny9zMDcGTSN2QNu8YemPZxk4PVEbK6ybD1lWF/E68+ahJT3 dtE6QYvljLCyxDmIEcMmZzh+yk4BPvdYn58YxN+N/YT0tIKvJBp8p8i5JjWLtMI= X-Google-Smtp-Source: AGHT+IG4pJEcmV6d6Swzvphs9nnzJgXZ5aoQKqB0B10DiV8FlpfaCXhCy2PLo2Umh1yZ10URzDEtKw== X-Received: by 2002:a17:90a:fb81:b0:2c9:66d3:4663 with SMTP id 98e67ed59e1d1-2cf7e720c12mr19937a91.43.1722011849015; Fri, 26 Jul 2024 09:37:29 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:28 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Conor Dooley Subject: [PATCH v7 1/8] RISC-V: Add Zicclsm to cpufeature and hwprobe Date: Fri, 26 Jul 2024 12:37:12 -0400 Message-ID: <20240726163719.1667923-2-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" > Zicclsm Misaligned loads and stores to main memory regions with both > the cacheability and coherence PMAs must be supported. > Note: > This introduces a new extension name for this feature. > This requires misaligned support for all regular load and store > instructions (including scalar and vector) but not AMOs or other > specialized forms of memory access. Even though mandated, misaligned > loads and stores might execute extremely slowly. Standard software > distributions should assume their existence only for correctness, > not for performance. Detecing zicclsm allows the kernel to report if the hardware supports misaligned accesses even if support wasn't probed. This is useful for usermode to know if vector misaligned accesses are supported. Signed-off-by: Jesse Taube Reviewed-by: Conor Dooley Reviewed-by: Andy Chiu --- V1 -> V2: - Add documentation for Zicclsm - Move Zicclsm to correct location V2 -> V3: - No changes V3 -> V4: - Add definitions to hwprobe.rst V4 -> V5: - No changes V5 -> V6: - No changes --- Documentation/arch/riscv/hwprobe.rst | 5 +++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/uapi/asm/hwprobe.h | 1 + arch/riscv/kernel/cpufeature.c | 1 + arch/riscv/kernel/sys_hwprobe.c | 1 + 5 files changed, 9 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/risc= v/hwprobe.rst index df5045103e73..78acd37b6477 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -207,6 +207,11 @@ The following keys are defined: * :c:macro:`RISCV_HWPROBE_EXT_ZVE64D`: The Vector sub-extension Zve64d is supported, as defined by version 1.0 of the RISC-V Vector extension ma= nual. =20 + * :c:macro:`RISCV_HWPROBE_EXT_ZICCLSM`: The Zicclsm extension is suppor= ted as + defined in the RISC-V RVA Profiles Specification. Misaligned suppo= rt for + all regular load and store instructions (including scalar and vect= or) but + not AMOs or other specialized forms of memory access. + * :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: Deprecated. Returns similar valu= es to :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_PERF`, but the key was mistake= nly classified as a bitmask rather than a value. diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index f64d4e98e67c..0b3bd8885a2b 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -86,6 +86,7 @@ #define RISCV_ISA_EXT_ZVE64X 77 #define RISCV_ISA_EXT_ZVE64F 78 #define RISCV_ISA_EXT_ZVE64D 79 +#define RISCV_ISA_EXT_ZICCLSM 80 =20 #define RISCV_ISA_EXT_XLINUXENVCFG 127 =20 diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uap= i/asm/hwprobe.h index 2fb8a8185e7a..023b7771d1b7 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -65,6 +65,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_EXT_ZVE64X (1ULL << 39) #define RISCV_HWPROBE_EXT_ZVE64F (1ULL << 40) #define RISCV_HWPROBE_EXT_ZVE64D (1ULL << 41) +#define RISCV_HWPROBE_EXT_ZICCLSM (1ULL << 42) #define RISCV_HWPROBE_KEY_CPUPERF_0 5 #define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) #define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 1d6e4fda00f8..83c5ae16ad5e 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -283,6 +283,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] =3D { __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h), __RISCV_ISA_EXT_SUPERSET(zicbom, RISCV_ISA_EXT_ZICBOM, riscv_xlinuxenvcfg= _exts), __RISCV_ISA_EXT_SUPERSET(zicboz, RISCV_ISA_EXT_ZICBOZ, riscv_xlinuxenvcfg= _exts), + __RISCV_ISA_EXT_DATA(zicclsm, RISCV_ISA_EXT_ZICCLSM), __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR), __RISCV_ISA_EXT_DATA(zicond, RISCV_ISA_EXT_ZICOND), __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR), diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprob= e.c index e4ec9166339f..e910e2971984 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -96,6 +96,7 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair, EXT_KEY(ZBB); EXT_KEY(ZBS); EXT_KEY(ZICBOZ); + EXT_KEY(ZICCLSM); EXT_KEY(ZBC); =20 EXT_KEY(ZBKB); --=20 2.45.2 From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF35717CA16 for ; Fri, 26 Jul 2024 16:37:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011855; cv=none; b=FoeSSaWt1zF8kIcTou4GIjMGqJe5s1NvMvv4/w/6NWzXvH9e2wm3lQ0rEccDxned3rkcB3KE/5RaDxll6Pk6oGMabQ5YRdH5j9xqZHMrMlZbKo3XddSRcJSVWAAUpxcHK4+A1VTGs0xXp8r65JzLHvkGzy8WQ2bhr/LFDizNvGU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011855; c=relaxed/simple; bh=vNIHtugGXysUf1ONK+0MY3ajgJQFX1BdZc09x5lwA3E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pYdsoAw6JCHQ3leCs4JZMBu7ERjFmBbmKIv8EQvK7V+ncjCVJPy5AyugekdZqk83Mx2jftaO0fUmJXNmGX48bzPPNaemfbtGhKgXrg+KC51IAvk+gkqxwe+FV/TH/VItyWDrhsICa8T1fN/ZUhSmM1b/T+f57abnBE7+ugCJ7R4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=WOshvIcX; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="WOshvIcX" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2c964f5a037so866974a91.0 for ; Fri, 26 Jul 2024 09:37:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011853; x=1722616653; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ct5IHVvBReq+T2pk7qCX19ZuVPxoUlBznhISu+pLlbA=; b=WOshvIcXhaNIRbhzn+8Vgi3rv3gzDyfsonqKa0oWEXIiNSLQqk3VQauy5HCBDVYJis r5S8va6YyMRQf+3LdfahRrCWTWPbe7ZDwlvgOHA7K9WeYv3xofCQ29ph16BTdxHLppoU 4x/Z0nbFDFB5s8zWDDfD5nI6F+YHQdmZcgy/6jph40fzzIbv692NlI7ckitY9IenCH9g konSbNtA64Rq9vxNLwquShSr2MCt4eUOUghqZ3IvlZGf2/CjkyORkZxJmy09IpMHQn4Z nr1LHB9uAWefC1oeZvwvu0BeFBldIDYe/M+YKUhHJgMa+yg6SjqXhJVvdPMmqkuUq45c iGRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011853; x=1722616653; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ct5IHVvBReq+T2pk7qCX19ZuVPxoUlBznhISu+pLlbA=; b=CiNA3icbu6NpxEVJCI25b8nJqw9VLj4yeudfA81NYoGbpdx/AtJb41ps44HsDs5mYM mrmhIX9bctk2sZF/p1VFTGhHs/bI/DZFrgWIbedYsv7aZRWd9BzWlDjtQaadRpz7fn2t reLpIqCrw7FUeN1pyaeipxrznZBq6yOlh/15gWM7M+BVzs1Wl49ZgCfGkdYBAGT0kgDr 5k34+YLX4uYB7yY5ordtrkj0qw30HjWoiW8GcuC/mPXgj/xXjh/YexQP39LHHm/DAhMd Gk2g6mOVWdGSXNlE9MUDNyE4cSPVm9DkDmZDbC4Q17xTeoDiw2GVWrZWp6z2vICeYpj8 KG0g== X-Forwarded-Encrypted: i=1; AJvYcCVk0Qg/FVCd8eKBZNw0/qj5WKubJomOeYp5e6xlPkux4BTx6/391UdMYiRPT0m2w/sCe3THaW7uCXSW7wCvcMSHvvVbFUZv+Twm1hXK X-Gm-Message-State: AOJu0YzYi3MGQez7SGVsxn1bvrem7CX1I68XYbNz3GbOnQbN3s9auwy0 V3d7N0yj5QkV7OfOB+zBTWYmd0+daHu5he1sTWQ2CHBzoxs/iL331DIBSmrHmbQ= X-Google-Smtp-Source: AGHT+IGzZDQaijmw3sqbbu/8le81Z5VtH5YaKfaTBBpqys+glA9U5Y2/yFGQbXUxzMVFyUnSDk99Uw== X-Received: by 2002:a17:90a:a018:b0:2c9:359c:b0c with SMTP id 98e67ed59e1d1-2cf7e82fdf9mr16585a91.28.1722011853207; Fri, 26 Jul 2024 09:37:33 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:32 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Conor Dooley Subject: [PATCH v7 2/8] dt-bindings: riscv: Add Zicclsm ISA extension description. Date: Fri, 26 Jul 2024 12:37:13 -0400 Message-ID: <20240726163719.1667923-3-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add description for Zicclsm ISA extension. Signed-off-by: Jesse Taube Acked-by: Conor Dooley --- V1 -> V2: - New patch V2 -> V3: - No changes V3 -> V4: - No changes V4 -> V5: - No changes V5 -> V6: - No changes V6 -> V7: - No changes --- Documentation/devicetree/bindings/riscv/extensions.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/devicetree/bindings/riscv/extensions.yaml b/Docu= mentation/devicetree/bindings/riscv/extensions.yaml index cfed80ad5540..9f6aae1f5b65 100644 --- a/Documentation/devicetree/bindings/riscv/extensions.yaml +++ b/Documentation/devicetree/bindings/riscv/extensions.yaml @@ -317,6 +317,13 @@ properties: The standard Zicboz extension for cache-block zeroing as ratif= ied in commit 3dd606f ("Create cmobase-v1.0.pdf") of riscv-CMOs. =20 + - const: zicclsm + description: + The standard Zicclsm extension for misaligned support for all = regular + load and store instructions (including scalar and vector) but = not AMOs + or other specialized forms of memory access. Defined in the + RISC-V RVA Profiles Specification. + - const: zicntr description: The standard Zicntr extension for base counters and timers, as --=20 2.45.2 From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD2BE17C7BF for ; Fri, 26 Jul 2024 16:37:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011859; cv=none; b=kDUuL/clzGIL0CkLSq93IQnSX6YOAZX9aMghAOvfkzizSiK3gbxXKnUKLgaou5GnNAIFBC4G1cerfCHkw13XLJqY0BDAdnN1MCuUasldZI0fCzXJr5EX5ACiLjfICda6H3//BHfOp0p7zqXVHl3KIdfBEUm6eCLJ4ilzRwRH0Ig= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011859; c=relaxed/simple; bh=Ts9RMTg8E4vUO5l2oJxTGuV2knkgDFBFZAmV+CJYwtU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MgEKbtNAZ24JvjJQ2KTrpskeWUwa4QzslH4t3VlnWWNJSyla/qaBpBghvDoLc/iodoTW3uKYvpGvXLbFExKm+kaveIhZFl+NFiA6Afhgix/IJJ6OvygPnKdBuYDV3zbHf7wLT842VOXFNLkoE2sRqqhpsymAq3VOYwqGoFCq8lw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=S0+p8fss; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="S0+p8fss" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2cf78366187so138546a91.3 for ; Fri, 26 Jul 2024 09:37:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011857; x=1722616657; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VTR8dGx/lbWJSWUsaeW/F3DPGM3HgfXNV2wpAfhqAQs=; b=S0+p8fssDmAf9LRXawkUFHSfGzF/F86+xaHmiMT3oHv7XjAabgvO95Swj/tl2qaQVB /4hGRjcxJPRU6d7LBpzD6mtEIHlMdsJojPf6vk0BvWNHcGq1C9DBiqDWYjawepQd7/fc Cy9Gqd5eGLpHCxLiSZkyBq/31Eiu5exEPAlnLZuxE/AIsXiyNg059rk4p0YyOKO/3sTx NHSgwCA1pWjRsup4+wvQeBXOjIxmGG/VcsptPrM0aJ06lUeWDZE2LcJ1mnT6eQfskwqW pX8S1qX6IrAUWyw8RWQdgnTXLcTNUFgXwnPyaFUKbXhn5AJ06P9ECd8Ce8O2h3V/bt4+ ddkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011857; x=1722616657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VTR8dGx/lbWJSWUsaeW/F3DPGM3HgfXNV2wpAfhqAQs=; b=A5A+PS7XJ/tEhfGlW+zsKgyGFBfbPejeMxmmtTQWsqvhdKn2xql6KfGIWm14DsdASK KSHDISedrzogy7Qu0j9Nhh6+dTqGKH12qggNuFCKnF8MpXA0BXTm+mUrEIo38rIKSsfi 75KXEf86vIaIfm0cDLffspVO95zpcrMZJnJoyvbQwHM2N3VKHsferJxN8nUMrUkjaeLZ BzGMVL4ZBe5jzRgMQodSI7HP9B4o1me8C90zTYvPQEoi9ai7zSgjw1DaXrlanSdJZy0V XULL09MEYVuP0pXcWRlwFo6RLX6Ji/ehq7ZdoprIPhnHDGy0yIvNjWTEjClHqCA72Ob2 zd6A== X-Forwarded-Encrypted: i=1; AJvYcCWCpS0icHKlPux7oa2LhshWF/KIj4WY1nFXKFDNFJ12nl1uqpq89YkGkzZyDBOPNphG77reYs+557wp1zDzhCRgpYSleDYzgWe0YPrC X-Gm-Message-State: AOJu0YzcU/XDOY/oRl0bQfo7F1GY4HHTjOGO4gDizK4+EISH1pPLQIeE 0kDS+kyO+FMU6ucnlLpDBr9gq8S1wlnnkyWTCliND92UwIRugEjaN+IlpDBHQW8= X-Google-Smtp-Source: AGHT+IGGkDNdhZ23Ek6RTThUGQibiW/JUfd83zkY/iy7TIRyI5XilQkJJhS2kwmHeXFgHj6b/FVkug== X-Received: by 2002:a17:90a:4305:b0:2c9:65f5:5f5b with SMTP id 98e67ed59e1d1-2cf7e21702cmr42488a91.24.1722011857425; Fri, 26 Jul 2024 09:37:37 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:37 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH v7 3/8] RISC-V: Check scalar unaligned access on all CPUs Date: Fri, 26 Jul 2024 12:37:14 -0400 Message-ID: <20240726163719.1667923-4-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Originally, the check_unaligned_access_emulated_all_cpus function only checked the boot hart. This fixes the function to check all harts. Fixes: 71c54b3d169d ("riscv: report misaligned accesses emulation to hwprob= e") Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins Reviewed-by: Evan Green Cc: stable@vger.kernel.org --- V1 -> V2: - New patch V2 -> V3: - Split patch V3 -> V4: - Re-add check for a system where a heterogeneous CPU is hotplugged into a previously homogenous system. V4 -> V5: - Change work_struct *unused to work_struct *work __always_unused V5 -> V6: - Change check_unaligned_access_emulated to extern V6 -> V7: - No changes --- arch/riscv/include/asm/cpufeature.h | 2 ++ arch/riscv/kernel/traps_misaligned.c | 14 +++++++------- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 347805446151..3b24342c7d2a 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -8,6 +8,7 @@ =20 #include #include +#include #include #include #include @@ -35,6 +36,7 @@ void riscv_user_isa_enable(void); =20 #if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed); void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); DECLARE_PER_CPU(long, misaligned_access_speed); diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps= _misaligned.c index b62d5a2f4541..bb09357778c5 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -526,11 +526,11 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } =20 -static bool check_unaligned_access_emulated(int cpu) +void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed) { + int cpu =3D smp_processor_id(); long *mas_ptr =3D per_cpu_ptr(&misaligned_access_speed, cpu); unsigned long tmp_var, tmp_val; - bool misaligned_emu_detected; =20 *mas_ptr =3D RISCV_HWPROBE_MISALIGNED_UNKNOWN; =20 @@ -538,19 +538,16 @@ static bool check_unaligned_access_emulated(int cpu) " "REG_L" %[tmp], 1(%[ptr])\n" : [tmp] "=3Dr" (tmp_val) : [ptr] "r" (&tmp_var) : "memory"); =20 - misaligned_emu_detected =3D (*mas_ptr =3D=3D RISCV_HWPROBE_MISALIGNED_EMU= LATED); /* * If unaligned_ctl is already set, this means that we detected that all * CPUS uses emulated misaligned access at boot time. If that changed * when hotplugging the new cpu, this is something we don't handle. */ - if (unlikely(unaligned_ctl && !misaligned_emu_detected)) { + if (unlikely(unaligned_ctl && (*mas_ptr !=3D RISCV_HWPROBE_MISALIGNED_EMU= LATED))) { pr_crit("CPU misaligned accesses non homogeneous (expected all emulated)= \n"); while (true) cpu_relax(); } - - return misaligned_emu_detected; } =20 bool check_unaligned_access_emulated_all_cpus(void) @@ -562,8 +559,11 @@ bool check_unaligned_access_emulated_all_cpus(void) * accesses emulated since tasks requesting such control can run on any * CPU. */ + schedule_on_each_cpu(check_unaligned_access_emulated); + for_each_online_cpu(cpu) - if (!check_unaligned_access_emulated(cpu)) + if (per_cpu(misaligned_access_speed, cpu) + !=3D RISCV_HWPROBE_MISALIGNED_EMULATED) return false; =20 unaligned_ctl =3D true; --=20 2.45.2 From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C04317D89D for ; Fri, 26 Jul 2024 16:37:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011863; cv=none; b=lqKCEa9ppYoin76nZsbKO8veOg3xBR5UaU3AUEePvyAVBWRwz4pnS/f5QHdZHApTDLU1Q5zFpRVcxvXzAwjDnNR9gbbn01byzZSb8VKu0cChqBFG+aqCY37gzX4ept/jqxriaJY9j36mBH2mlebgwa8RG34J0J0HpMsMLZXh25Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011863; c=relaxed/simple; bh=F9TUuRxlIYugU1/usYl0Y6QWEsQM7GZcfyLclleEylQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hTg7QEiuTIvCFyMYxTnyiMBBWN1CdgiQ5L2O5QK2/m21Y+2O7wb1NgPX15iR8xDxSARdSBjiwQLnJvBnSyY4H28CzpnW5lL8m7gHF8XFYid/TVL/vVYPLJ7n5AsPUu2osWGFJeEKelXJu0DOytp/P4FwOYq7gL9IRv2iSXFlCVs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=Ylp7A1zn; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="Ylp7A1zn" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2cb55418470so833803a91.1 for ; Fri, 26 Jul 2024 09:37:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011861; x=1722616661; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o0rt4u3JgYQdmYK/5XCmCvbMmsxncqWCmcN6vbrwaOA=; b=Ylp7A1zngH6RlgwwDxueNwnrfdZOJB1xgvITv5VuLskUrJyN4K9j+R+0Ksoqwgtj8c 2qSuy6c+4Z+1fDnVE/X7JFDSvwdMH00s7R30QlgVjPf3CHdZmw/K1SP4YdSdPDO8aRj1 7/dZ5g7x2hj0gvnmavHxBrASDnb2O019fjphHV1QYIBcLzdqkhaJyqr87AMshaLxnF0H clKjFCGV327LtsxvUFkeS7b7ZVrGvNFJlJz18ffFVrJ5PPcSFCWr3LxfXtRWlgkQXubs iedK5qkSRLhKIAZoPWQ8v9ZTLJqWDxAAWLdN22WCdjG6Hg6NTA2uT8qOAs6rqJjMut/2 hWLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011861; x=1722616661; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o0rt4u3JgYQdmYK/5XCmCvbMmsxncqWCmcN6vbrwaOA=; b=A3YI8CMfj9UqRzJstzW+7sGMQ4BjerjHajuHj/PSekJIvrLzjEGVQHKBFLtTs29nsp mpVgDjBzQ2Nzap6ZRT/bYanJV3EIG6o/rIDk3U4qKu6WpSzdAIWPfXUS5umukIjT4mZ7 Y0GLjKAPba1ZFl20DIBWTfOkEWHSJ1d2Lx+dMlUNR1oPfL3WofDJeabmunFPersQZ4ll 5wnEgVGREejylgeRvP4nfflDbhVfOIsHOOWXzREcmmYQJOfsbCYu5Md3EWDKT3XgOFET d7P2I+C7mQ7vTUSKm9iUgEw7Icyf6qH564uSBdx3pNGSqPPIZl0eyucSwtQzdRhPLat3 tFiw== X-Forwarded-Encrypted: i=1; AJvYcCU/2q9lZ0JckPo/iPL5FQAaC6DVRjtS2WElH9a9EJyeufWaJuZZB/wWu4YByDQeclv2c+zrXar7jedOr1uLYgNnkOMZPzcqBSH9br+M X-Gm-Message-State: AOJu0YxyDJ20MLKooP2gEm3oF1ZE4Eho13SWKfJDbGZUXUkYymVM780a jCaE7MeU48QlVKz4aANa/AuW9QQSTXrGvhRmG9PQfexD4BU96suVp0rTMqJwI/w= X-Google-Smtp-Source: AGHT+IFt0Hx7tCHd8rmEPH3r9WQzWMpJRolI2HaFqal33DHnd4BM9lIVbdq7BXaVWa8aOT/1lsYEQA== X-Received: by 2002:a17:90b:2552:b0:2c9:62be:292a with SMTP id 98e67ed59e1d1-2cf7e1fba30mr38548a91.21.1722011861534; Fri, 26 Jul 2024 09:37:41 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:41 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH v7 4/8] RISC-V: Scalar unaligned access emulated on hotplug CPUs Date: Fri, 26 Jul 2024 12:37:15 -0400 Message-ID: <20240726163719.1667923-5-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The check_unaligned_access_emulated() function should have been called during CPU hotplug to ensure that if all CPUs had emulated unaligned accesses, the new CPU also does. This patch adds the call to check_unaligned_access_emulated() in the hotplug path. Fixes: 55e0bf49a0d0 ("RISC-V: Probe misaligned access speed in parallel") Signed-off-by: Jesse Taube Reviewed-by: Evan Green Cc: stable@vger.kernel.org --- V5 -> V6: - New patch V6 -> V7: - No changes --- arch/riscv/kernel/unaligned_access_speed.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel= /unaligned_access_speed.c index a9a6bcb02acf..b67db1fc3740 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -191,6 +191,7 @@ static int riscv_online_cpu(unsigned int cpu) if (per_cpu(misaligned_access_speed, cpu) !=3D RISCV_HWPROBE_MISALIGNED_U= NKNOWN) goto exit; =20 + check_unaligned_access_emulated(NULL); buf =3D alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); if (!buf) { pr_warn("Allocation failure, not measuring misaligned performance\n"); --=20 2.45.2 From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D85317C9EE for ; Fri, 26 Jul 2024 16:37:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011868; cv=none; b=E3ttS3UvJaESNqkvKzEJP9cvys+yhGvLvWIoGdzs9jg0ltOQNF4DNwP9cdCPenGo3BF0tVbGKKjjyeJPDY2igqWFLBUQHI+78jaRBZqXPj/EjZwe9UFiSJkljgkJrGUC2qaAAB8ecy1pAYEpi+5LBR4T/CH7htz8jf3Qd5weaOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011868; c=relaxed/simple; bh=KPzo1t8aNGcupG0Pg2gWOUn9JiWdbpJchlMkHseo+HY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uCpFukDRPAS5XV79da88LsHQcCCjsKVcZyG9Kh/kFFAHjlLnC3A02ZCAw8kGLSr0Mic1rIV4BoLyNxjYPop6BbdmDvOjZEM1EOJJNlIYJ5I0/7DL2ZFo2ONxkYKwRqyd4iSZPCuLxYygaoNmhopnuHdGxeGhNaIdg60KktoURNk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=VMQVmofq; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="VMQVmofq" Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-7a0b2924e52so698672a12.2 for ; Fri, 26 Jul 2024 09:37:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011866; x=1722616666; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jL/Wcr3kp2TFFG3UjxVeBtOA+tWOHsY6KHzkM6T1E70=; b=VMQVmofqYjH4tN8mJ1bzS+n3abMfA87C4ngdNJLlq9bMxgBHgfJmnqW1Uo8n9q5+Xh GjAF73LEYZopwqLkWNSYS0hxwY4JoSK8mXvx+2aNq9V5aYecph67lnIHXRdy09E1RrLQ +jdNy5g12Ny0LtEL4qIc4pP/HYPwO7UP+GfLDqmn34P/J6T3JMV3nJHYB2mcXRXVOjgs rpJgiTDx9gS4ONp92adTHa7it+hA1t78/R7wSJXuKQJK+B/zhaveusKN/kIWe+2ZYPA1 XxE+clgn9d3O816S+zJBNo52qnqkHj8LsYj4nYqaCXbPJbvYINBzwOgT7bQ8RDiAkO9s 4L7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011866; x=1722616666; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jL/Wcr3kp2TFFG3UjxVeBtOA+tWOHsY6KHzkM6T1E70=; b=A8FlidP5B6hVGzBvbMZE6vL7PYIAJG6jhfhAI7f29ruKBvS6+LMTY+JOMW7zdD6z9a 6ocsXs9GE+ddgpY7gOr9q6pcyXF6AhwR2vcqiONqD2p1WHAjj7Nhswfel1GrUKIGRY0e A6e/y9t+jytqp9WZbNPoEr4kegZL4kvEIZGreSMrY49NXmo0gCCOqo7PwCVgYXbwQSkj 0pWM59+oE746nc8zHXdappdAPlMdHksLxn8Fsg1rIz2HztK9dvUjgXxjPM5WEjLQLOGU 2CoJlbR4BMCxORCysk2zkbnKIZY4VA7bgPYxpsmaUcak5xuhJSsFlbQXyXFj7OxF7XkQ tLlA== X-Forwarded-Encrypted: i=1; AJvYcCVNJ4Xim8oMqlvvJoCgGm+bLkD7v0Igkxf0IkT3tk+1D9uZy9UJWg1c/JTtofHd9TONKZ3dZr1zQJkOxbg4sBi94zWDUJYrcrbwHIJz X-Gm-Message-State: AOJu0Ywfx7y/Mz4fvmwS7GP7HJ4t9bo0RCI07T5oQ5fqBRXyPfFdN6rA QaTc2+jlURgcVNOjB3ay4edzhSWnE8ZPePpZ8xIIPZKRtKcpPrJ8MUQrt+RH+ww= X-Google-Smtp-Source: AGHT+IGeDGfTKa28zuUIyGNAc40tXgpN6lNbD7TWL4IPjzeIquVunAJ7rdtjdUVk2ZZZ/Wul3+wjfQ== X-Received: by 2002:a17:90a:750:b0:2c9:64fb:1c7e with SMTP id 98e67ed59e1d1-2cf7e1c0e3emr50338a91.14.1722011865651; Fri, 26 Jul 2024 09:37:45 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:45 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Conor Dooley Subject: [PATCH v7 5/8] RISC-V: Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED Date: Fri, 26 Jul 2024 12:37:16 -0400 Message-ID: <20240726163719.1667923-6-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED to allow for the addition of RISCV_VECTOR_MISALIGNED in a later patch. Signed-off-by: Jesse Taube Reviewed-by: Conor Dooley Reviewed-by: Charlie Jenkins Reviewed-by: Evan Green --- V2 -> V3: - New patch V3 -> V4: - No changes V4 -> V5: - No changes V5 -> V6: - fix accidental moving of check_unaligned_access_emulated_all_cpus out of= the #ifdef V6 -> V7: - No changes --- arch/riscv/Kconfig | 6 +++--- arch/riscv/include/asm/cpufeature.h | 2 +- arch/riscv/include/asm/entry-common.h | 2 +- arch/riscv/kernel/Makefile | 4 ++-- arch/riscv/kernel/fpu.S | 4 ++-- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index b94176e25be1..34d24242e37a 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -717,7 +717,7 @@ config THREAD_SIZE_ORDER Specify the Pages of thread stack size (from 4KB to 64KB), which also affects irq stack size, which is equal to thread stack size. =20 -config RISCV_MISALIGNED +config RISCV_SCALAR_MISALIGNED bool select SYSCTL_ARCH_UNALIGN_ALLOW help @@ -734,7 +734,7 @@ choice =20 config RISCV_PROBE_UNALIGNED_ACCESS bool "Probe for hardware unaligned access support" - select RISCV_MISALIGNED + select RISCV_SCALAR_MISALIGNED help During boot, the kernel will run a series of tests to determine the speed of unaligned accesses. This probing will dynamically determine @@ -745,7 +745,7 @@ config RISCV_PROBE_UNALIGNED_ACCESS =20 config RISCV_EMULATED_UNALIGNED_ACCESS bool "Emulate unaligned access where system support is missing" - select RISCV_MISALIGNED + select RISCV_SCALAR_MISALIGNED help If unaligned memory accesses trap into the kernel as they are not supported by the system, the kernel will emulate the unaligned diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 3b24342c7d2a..4ade9f87fc55 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -34,7 +34,7 @@ extern struct riscv_isainfo hart_isa[NR_CPUS]; =20 void riscv_user_isa_enable(void); =20 -#if defined(CONFIG_RISCV_MISALIGNED) +#if defined(CONFIG_RISCV_SCALAR_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed); void unaligned_emulation_finish(void); diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm= /entry-common.h index 2293e535f865..0a4e3544c877 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,7 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct = pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); =20 -#ifdef CONFIG_RISCV_MISALIGNED +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); #else diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 5b243d46f4b1..8d4e7d40e42f 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -62,8 +62,8 @@ obj-y +=3D probes/ obj-y +=3D tests/ obj-$(CONFIG_MMU) +=3D vdso.o vdso/ =20 -obj-$(CONFIG_RISCV_MISALIGNED) +=3D traps_misaligned.o -obj-$(CONFIG_RISCV_MISALIGNED) +=3D unaligned_access_speed.o +obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D traps_misaligned.o +obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o =20 obj-$(CONFIG_FPU) +=3D fpu.o diff --git a/arch/riscv/kernel/fpu.S b/arch/riscv/kernel/fpu.S index 327cf527dd7e..f74f6b60e347 100644 --- a/arch/riscv/kernel/fpu.S +++ b/arch/riscv/kernel/fpu.S @@ -170,7 +170,7 @@ SYM_FUNC_END(__fstate_restore) __access_func(f31) =20 =20 -#ifdef CONFIG_RISCV_MISALIGNED +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED =20 /* * Disable compressed instructions set to keep a constant offset between FP @@ -224,4 +224,4 @@ SYM_FUNC_START(get_f64_reg) fp_access_epilogue SYM_FUNC_END(get_f64_reg) =20 -#endif /* CONFIG_RISCV_MISALIGNED */ +#endif /* CONFIG_RISCV_SCALAR_MISALIGNED */ --=20 2.45.2 From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D32617C7D3 for ; Fri, 26 Jul 2024 16:37:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011872; cv=none; b=YEeB1ejIG++KM06YAgQfEIICKqH6C+UjkuEFtqjlAmnCdF+8K6sI/puy0YgEqWveZw3t/b0DeQKkk1CyE4ku21/WEKoMFwJmic22e3LxvFKtRnJRoRpdAWI1ddqetc9TWAsosb6DO3nb0K5NR7EFjFKU6JMt6XG33zNc1EJ7iBQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011872; c=relaxed/simple; bh=uFQPx/qiXC042DZnAvnPXkBi/tiSEefmyQmjqsVytxw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aJELLd6Ow3Q9bI4nyTSCQrDz9ej1SVZLuHgtvqGs00tX1pM5nbkrAxL/MLr3Vu+rPb4N33xwU2chvxbAP8FBtWWmkQ6VBeUvaybGPYOovRFLcxMIgvjQk9CnjqLgh+/lZqAfagS3vdC5IWIEizUrJeej0XXC6+o3fItSSogEdVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=zAn+0XDM; arc=none smtp.client-ip=209.85.215.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="zAn+0XDM" Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-656d8b346d2so805546a12.2 for ; Fri, 26 Jul 2024 09:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011870; x=1722616670; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+P4RX7d/L3FZGBDsK0s3on5T+V7dOJXGyuIWGUB7I7M=; b=zAn+0XDM6hddO8k+GAnjBZuvP2fcRgNpchKkU+TpiJi2vZ3+vbJ5+0AEJROodz1LXl o4ELB5/Jya2YUAkEmF8Gkd3ByqJO7mbwoyaf2mpGRP0wKtQb6GfExiDrbmSYliRPWgn3 OtD5UVB/UIEyYzUOgGbSzKfDSBOyzf63mQB9pkJo7owbZ40bRLcZKFnsyeXzZkNOLEwn 5AeufOy+urq8VGewKhrQjuFU51zrRm3VsJgMqeRCEJf9q+5uuusUACVyytU1dSWJy9wC sVQ4YYL9CQ4SJHG2FjmEW6hN7gbqdsKruD9nzQqipXyZ3GFUIzTsBmZdvJiep9EwzYaJ Nbzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011870; x=1722616670; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+P4RX7d/L3FZGBDsK0s3on5T+V7dOJXGyuIWGUB7I7M=; b=HZaKXpULOLbtuoOn4I3+jG4OBW1U/M8UGNGuJKhKmLDp6nq0El5lA/7gMQuq5aQ47z iagZB2o0Pr6J7vc0/oc+VwZ0P804D/6sIDTAAz6DMrvTnnzbOM63QZF7MNVLHUvoUxWc mWAHstI2GyjWezNqAwnQ7x6Qa1emRsGzRrQi62Kp6AJa+Vkpw6cr2Z9aX2CwBHvcKJhE 50F8bw0UvqvG+ROZj1c7TksAkI94WfJXfkN2852wQH2rbpZueQPkG4TM9PKyHiWOZyDE nKKw5VZu2YDNSWezcD8D6gK0rTkhGQKg8IrtHG6REjC8MAsiASMYtBuZQksNh5Rd5eCf kE1g== X-Forwarded-Encrypted: i=1; AJvYcCXZ15l+931nlGuhf5akTed6ZmhDbiuN+yuYl3LvL2mvyufDHLry5jTVBMSjHBVTYp1G9ZZdeDE87T4yqa6xK41DmuFNEB54E8ZSzhIi X-Gm-Message-State: AOJu0YxMvdcmhzD4fzu7t3isDiI1qiWEo43R/BaPOq2yNaJkNjVNsAg2 7Lu2O/e+2/xQuBmwJTukDaD99+9EUQTmEmm4/LkTr9XPDK3eho1mFaTK7SH48E8= X-Google-Smtp-Source: AGHT+IGBvtWOsApLYZ5Cg4zDJSY8O9sopdqryjBRpwYPtGpRpK8eIHoDZC32WCL8ldM8mc0H4Du81A== X-Received: by 2002:a17:90a:5105:b0:2c9:74cf:7dd6 with SMTP id 98e67ed59e1d1-2cf7e719fdcmr21007a91.35.1722011869758; Fri, 26 Jul 2024 09:37:49 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:49 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v7 6/8] RISC-V: Detect unaligned vector accesses supported Date: Fri, 26 Jul 2024 12:37:17 -0400 Message-ID: <20240726163719.1667923-7-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Run a unaligned vector access to test if the system supports vector unaligned access. Add the result to a new key in hwprobe. This is useful for usermode to know if vector misaligned accesses are supported and if they are faster or slower than equivalent byte accesses. Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins --- V1 -> V2: - Add Kconfig options - Add insn_is_vector - Add handle_vector_misaligned_load - Fix build - Seperate vector from scalar misaligned access - This patch was almost completely rewritten V2 -> V3: - Fixed CONFIG_ in Kconfig - Fixed check_vector_unaligned_access_emulated leaving vector_misaligned_access as unknown. - Remove local_irq_enable - Remove RISCV_DETECT_VECTOR_UNALIGNED_ACCESS - Remove RISCV_VEC_UNALIGNED_ACCESS_UNSUPPORTED V3 -> V4: - Spell out _VECTOR_ in macros V4 -> V5: - Change work_struct *unused to work_struct *work __always_unused - Add insn_is_vector definition to vector.h when V is not defined V5 -> V6: - Change check_vector_unaligned_access_emulated to extern - Move check_unaligned_access_emulated_all_cpus out of the #ifdef see last commit V6 -> V7: - Change SLOW to UNKNOWN when used as a placeholder --- arch/riscv/Kconfig | 35 ++++++ arch/riscv/include/asm/cpufeature.h | 8 +- arch/riscv/include/asm/entry-common.h | 11 -- arch/riscv/include/asm/hwprobe.h | 2 +- arch/riscv/include/asm/vector.h | 2 + arch/riscv/include/uapi/asm/hwprobe.h | 5 + arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/sys_hwprobe.c | 35 ++++++ arch/riscv/kernel/traps_misaligned.c | 117 ++++++++++++++++++++- arch/riscv/kernel/unaligned_access_speed.c | 22 ++-- arch/riscv/kernel/vector.c | 2 +- 11 files changed, 213 insertions(+), 30 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 34d24242e37a..ffbe0fdd7fb3 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -717,12 +717,26 @@ config THREAD_SIZE_ORDER Specify the Pages of thread stack size (from 4KB to 64KB), which also affects irq stack size, which is equal to thread stack size. =20 +config RISCV_MISALIGNED + bool + help + Embed support for detecting and emulating misaligned + scalar or vector loads and stores. + config RISCV_SCALAR_MISALIGNED bool + select RISCV_MISALIGNED select SYSCTL_ARCH_UNALIGN_ALLOW help Embed support for emulating misaligned loads and stores. =20 +config RISCV_VECTOR_MISALIGNED + bool + select RISCV_MISALIGNED + depends on RISCV_ISA_V + help + Enable detecting support for vector misaligned loads and stores. + choice prompt "Unaligned Accesses Support" default RISCV_PROBE_UNALIGNED_ACCESS @@ -774,6 +788,27 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS =20 endchoice =20 +choice + prompt "Vector unaligned Accesses Support" + depends on RISCV_ISA_V + default RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + help + This determines the level of support for vector unaligned accesses. This + information is used by the kernel to perform optimizations. It is also + exposed to user space via the hwprobe syscall. The hardware will be + probed at boot by default. + +config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + bool "Probe speed of vector unaligned accesses" + select RISCV_VECTOR_MISALIGNED + help + During boot, the kernel will run a series of tests to determine the + speed of vector unaligned accesses if they are supported. This probing + will dynamically determine the speed of vector unaligned accesses on + the underlying system if they are supported. + +endchoice + endmenu # "Platform type" =20 menu "Kernel features" diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 4ade9f87fc55..5634f702f3fe 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -34,8 +34,8 @@ extern struct riscv_isainfo hart_isa[NR_CPUS]; =20 void riscv_user_isa_enable(void); =20 -#if defined(CONFIG_RISCV_SCALAR_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_SCALAR_MISALIGNED) void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed); void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); @@ -47,6 +47,12 @@ static inline bool unaligned_ctl_available(void) } #endif =20 +bool check_vector_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_VECTOR_MISALIGNED) +void check_vector_unaligned_access_emulated(struct work_struct *work __alw= ays_unused); +DECLARE_PER_CPU(long, vector_misaligned_access); +#endif + #if defined(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) DECLARE_STATIC_KEY_FALSE(fast_unaligned_access_speed_key); =20 diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm= /entry-common.h index 0a4e3544c877..7b32d2b08bb6 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,18 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct= pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); =20 -#ifdef CONFIG_RISCV_SCALAR_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); -#else -static inline int handle_misaligned_load(struct pt_regs *regs) -{ - return -1; -} -static inline int handle_misaligned_store(struct pt_regs *regs) -{ - return -1; -} -#endif =20 #endif /* _ASM_RISCV_ENTRY_COMMON_H */ diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwpr= obe.h index 150a9877b0af..ef01c182af2b 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -8,7 +8,7 @@ =20 #include =20 -#define RISCV_HWPROBE_MAX_KEY 7 +#define RISCV_HWPROBE_MAX_KEY 8 =20 static inline bool riscv_hwprobe_key_is_valid(__s64 key) { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vecto= r.h index be7d309cca8a..c7c023afbacd 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -21,6 +21,7 @@ =20 extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); +bool insn_is_vector(u32 insn_buf); bool riscv_v_first_use_handler(struct pt_regs *regs); void kernel_vector_begin(void); void kernel_vector_end(void); @@ -268,6 +269,7 @@ struct pt_regs; =20 static inline int riscv_v_setup_vsize(void) { return -EOPNOTSUPP; } static __always_inline bool has_vector(void) { return false; } +static __always_inline bool insn_is_vector(u32 insn_buf) { return false; } static inline bool riscv_v_first_use_handler(struct pt_regs *regs) { retur= n false; } static inline bool riscv_v_vstate_query(struct pt_regs *regs) { return fal= se; } static inline bool riscv_v_vstate_ctrl_user_allowed(void) { return false; } diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uap= i/asm/hwprobe.h index 023b7771d1b7..48b92fb07edf 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -75,6 +75,11 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 #define RISCV_HWPROBE_KEY_MISALIGNED_PERF 7 +#define RISCV_HWPROBE_KEY_VECTOR_MISALIGNED_PERF 8 +#define RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN 0 +#define RISCV_HWPROBE_VECTOR_MISALIGNED_SLOW 2 +#define RISCV_HWPROBE_VECTOR_MISALIGNED_FAST 3 +#define RISCV_HWPROBE_VECTOR_MISALIGNED_UNSUPPORTED 4 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ =20 /* Flags */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 8d4e7d40e42f..5b243d46f4b1 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -62,8 +62,8 @@ obj-y +=3D probes/ obj-y +=3D tests/ obj-$(CONFIG_MMU) +=3D vdso.o vdso/ =20 -obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D traps_misaligned.o -obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) +=3D unaligned_access_speed.o +obj-$(CONFIG_RISCV_MISALIGNED) +=3D traps_misaligned.o +obj-$(CONFIG_RISCV_MISALIGNED) +=3D unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o =20 obj-$(CONFIG_FPU) +=3D fpu.o diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprob= e.c index e910e2971984..2e3e324bad38 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -194,6 +194,37 @@ static u64 hwprobe_misaligned(const struct cpumask *cp= us) } #endif =20 +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + int cpu; + u64 perf =3D -1ULL; + + /* Return if supported or not even if speed wasn't probed */ + for_each_cpu(cpu, cpus) { + int this_perf =3D per_cpu(vector_misaligned_access, cpu); + + if (perf =3D=3D -1ULL) + perf =3D this_perf; + + if (perf !=3D this_perf) { + perf =3D RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN; + break; + } + } + + if (perf =3D=3D -1ULL) + return RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN; + + return perf; +} +#else +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + return RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN; +} +#endif + static void hwprobe_one_pair(struct riscv_hwprobe *pair, const struct cpumask *cpus) { @@ -222,6 +253,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pai= r, pair->value =3D hwprobe_misaligned(cpus); break; =20 + case RISCV_HWPROBE_KEY_VECTOR_MISALIGNED_PERF: + pair->value =3D hwprobe_vec_misaligned(cpus); + break; + case RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE: pair->value =3D 0; if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOZ)) diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps= _misaligned.c index bb09357778c5..19947cd0160c 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -16,6 +16,7 @@ #include #include #include +#include =20 #define INSN_MATCH_LB 0x3 #define INSN_MASK_LB 0x707f @@ -322,12 +323,37 @@ union reg_data { u64 data_u64; }; =20 -static bool unaligned_ctl __read_mostly; - /* sysctl hooks */ int unaligned_enabled __read_mostly =3D 1; /* Enabled by default */ =20 -int handle_misaligned_load(struct pt_regs *regs) +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc =3D regs->epc; + unsigned long insn; + + if (get_insn(regs, epc, &insn)) + return -1; + + /* Only return 0 when in check_vector_unaligned_access_emulated */ + if (*this_cpu_ptr(&vector_misaligned_access) =3D=3D RISCV_HWPROBE_VECTOR_= MISALIGNED_UNKNOWN) { + *this_cpu_ptr(&vector_misaligned_access) =3D RISCV_HWPROBE_VECTOR_MISALI= GNED_UNSUPPORTED; + regs->epc =3D epc + INSN_LEN(insn); + return 0; + } + + /* If vector instruction we don't emulate it yet */ + regs->epc =3D epc; + return -1; +} +#else +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + return -1; +} +#endif + +static int handle_scalar_misaligned_load(struct pt_regs *regs) { union reg_data val; unsigned long epc =3D regs->epc; @@ -435,7 +461,7 @@ int handle_misaligned_load(struct pt_regs *regs) return 0; } =20 -int handle_misaligned_store(struct pt_regs *regs) +static int handle_scalar_misaligned_store(struct pt_regs *regs) { union reg_data val; unsigned long epc =3D regs->epc; @@ -526,6 +552,83 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } =20 +int handle_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc =3D regs->epc; + unsigned long insn; + + if (IS_ENABLED(CONFIG_RISCV_VECTOR_MISALIGNED)) { + if (get_insn(regs, epc, &insn)) + return -1; + + if (insn_is_vector(insn)) + return handle_vector_misaligned_load(regs); + } + + if (IS_ENABLED(CONFIG_RISCV_SCALAR_MISALIGNED)) + return handle_scalar_misaligned_load(regs); + + return -1; +} + +int handle_misaligned_store(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_RISCV_SCALAR_MISALIGNED)) + return handle_scalar_misaligned_store(regs); + + return -1; +} + +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +void check_vector_unaligned_access_emulated(struct work_struct *work __alw= ays_unused) +{ + long *mas_ptr =3D this_cpu_ptr(&vector_misaligned_access); + unsigned long tmp_var; + + *mas_ptr =3D RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN; + + kernel_vector_begin(); + __asm__ __volatile__ ( + ".balign 4\n\t" + ".option push\n\t" + ".option arch, +zve32x\n\t" + " vsetivli zero, 1, e16, m1, ta, ma\n\t" // Vectors of 16b + " vle16.v v0, (%[ptr])\n\t" // Load bytes + ".option pop\n\t" + : : [ptr] "r" ((u8 *)&tmp_var + 1) : "v0"); + kernel_vector_end(); +} + +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + int cpu; + + if (!has_vector()) { + for_each_online_cpu(cpu) + per_cpu(vector_misaligned_access, cpu) =3D RISCV_HWPROBE_VECTOR_MISALIG= NED_UNSUPPORTED; + return false; + } + + schedule_on_each_cpu(check_vector_unaligned_access_emulated); + + for_each_online_cpu(cpu) + if (per_cpu(vector_misaligned_access, cpu) + =3D=3D RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN) + return false; + + return true; +} +#else +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif + +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED + +static bool unaligned_ctl __read_mostly; + void check_unaligned_access_emulated(struct work_struct *work __always_unu= sed) { int cpu =3D smp_processor_id(); @@ -574,3 +677,9 @@ bool unaligned_ctl_available(void) { return unaligned_ctl; } +#else +bool check_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel= /unaligned_access_speed.c index b67db1fc3740..1a87dfe9cb87 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -19,7 +19,8 @@ #define MISALIGNED_BUFFER_ORDER get_order(MISALIGNED_BUFFER_SIZE) #define MISALIGNED_COPY_SIZE ((MISALIGNED_BUFFER_SIZE / 2) - 0x80) =20 -DEFINE_PER_CPU(long, misaligned_access_speed); +DEFINE_PER_CPU(long, misaligned_access_speed) =3D RISCV_HWPROBE_MISALIGNED= _UNKNOWN; +DEFINE_PER_CPU(long, vector_misaligned_access) =3D RISCV_HWPROBE_VECTOR_MI= SALIGNED_UNSUPPORTED; =20 #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS static cpumask_t fast_misaligned_access; @@ -260,23 +261,24 @@ static int check_unaligned_access_speed_all_cpus(void) kfree(bufs); return 0; } +#else /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ +static int check_unaligned_access_speed_all_cpus(void) +{ + return 0; +} +#endif =20 static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated =3D check_unaligned_access_emulated_all_cpus(); + bool all_cpus_emulated; + + all_cpus_emulated =3D check_unaligned_access_emulated_all_cpus(); + check_vector_unaligned_access_emulated_all_cpus(); =20 if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); =20 return 0; } -#else /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ -static int check_unaligned_access_all_cpus(void) -{ - check_unaligned_access_emulated_all_cpus(); - - return 0; -} -#endif =20 arch_initcall(check_unaligned_access_all_cpus); diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 682b3feee451..821818886fab 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -66,7 +66,7 @@ void __init riscv_v_setup_ctx_cache(void) #endif } =20 -static bool insn_is_vector(u32 insn_buf) +bool insn_is_vector(u32 insn_buf) { u32 opcode =3D insn_buf & __INSN_OPCODE_MASK; u32 width, csr; --=20 2.45.2 From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97DA217BB1F for ; Fri, 26 Jul 2024 16:37:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011876; cv=none; b=IAfpBouvb1SxSNyaKUQUfcl7SXS6PAzLJ09cP60ELYieh4UHTSsWdUw46ak7jK1ZjNSC1eVxYRVE+BU2EuvsDfhtn2FyABfkWKxpjEKHFKerPwkgEs4vrxRJelsTbMaetlFB0dAs08LQj8IX+GDjTIRgD37gYbId7gqZQnntZGU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011876; c=relaxed/simple; bh=T6IEvmOxWiyxDd5tgupdBFgmzQCERKnF5V4aRN+NuzU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YlSKKZXWat5C7lBgWotsMQRBu0PPjN2Lmmf5g/pXO+3/J70yg9/3xl3Qy0UYrk6zO9981/ye4M4NYN9Ireb6xlQBG5rdFBwUL5coNxcuS/d4FYaPTtCkbfIczBrT71RYExdTnrfV0ngni2pj0rWKaPKQRPdtMRHVo2yb1Uzjoak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=N9CW348q; arc=none smtp.client-ip=209.85.216.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="N9CW348q" Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-2cf213128a1so791914a91.2 for ; Fri, 26 Jul 2024 09:37:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011874; x=1722616674; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xgNvoZZjYGSQoQk8wgRdXSUA1h+KM6ckJH/M9L6ssNQ=; b=N9CW348qqLXIpVfKiIcNJubTgoODzrouvd80wRduT91Ok9/Ad3Dn3pppB+LVETEu0j EU4zztKsT/3CBkjQE5UlRcsvYfiFqQhhycKc11T288WJwl1oHFzQYooB/XaGX793a8Bt ZBRMgkgno0qJLuYpLQf8Z48F+SanuKeQJNThOIEZe4ye+Sj2joUeXA0qT5NaM0AR6tEY QrTzHnVDOXMfQ5BybhpETzVf8O1vLLNsIZfN9A0XEft+I0dTPrrB4EaauwVRJbcKb9wA vj+o7YG67XAKwpjCSe9f2Lhs23e2pLF7teoZwKN7ich/sE3OgsW9ob+eoZYg6hCTyRLu CN9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011874; x=1722616674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xgNvoZZjYGSQoQk8wgRdXSUA1h+KM6ckJH/M9L6ssNQ=; b=Q+zVlGlRGcKYetO6i9FNEyTDBNNJueG7pmdCpsuJw1kPcJg/QFLQm9Skz7Z0oSnl/B QKxVcjVCJrzjYOmWEWTBqkTioFmktpHmbDrEtkd34SKigdHOskqAvx8jzP6FtBrKbfEW EKiHJZqyb8+upJsyI6RopFFkbEsRt1nDg6hncl7LT0Kaq9igg1ONAYcRbCdT94IvXRDt hqMSi9BwMWZts0u0cYbIMczkj7U7C7pbnzA++SdViN2gDwjPjuWqFBdPJfTNNw1RfLI1 PHYledC2ElGl7i6840OxObJ/yqGqmE5oCNbFcIBqe1b9+YhVXKHEQVL9wv2YeUyVXXAk lh9w== X-Forwarded-Encrypted: i=1; AJvYcCUTvev2k/thC+5eIxkHlQ1TlBbXRft6IlvcSyG2C/TKwdYA3xGjnVYvB5ysltX8idwi/LxUay7XwXOxhmuUlg6E3h9uTMBb/FF+921a X-Gm-Message-State: AOJu0YwWr+R+ZShJfru5lGVxMS3YmA27he/PPaili704XnxolWtAd0YX fg2iNo3T4XoY3gIKCeUnrAb/pl2Of5qBZLVtfr9TeRZY5SjhNqEJ3iEyGMcT6/M= X-Google-Smtp-Source: AGHT+IFwaBOvRsjbQX7EWJtYGpRcesMfm22BYAZsC53JwBs6H3g+yumJhw4KF69PwuJ3umcfwBWGXg== X-Received: by 2002:a17:90a:a014:b0:2cb:1563:ccd1 with SMTP id 98e67ed59e1d1-2cf7e73094bmr20170a91.38.1722011873910; Fri, 26 Jul 2024 09:37:53 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:53 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v7 7/8] RISC-V: Report vector unaligned access speed hwprobe Date: Fri, 26 Jul 2024 12:37:18 -0400 Message-ID: <20240726163719.1667923-8-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Detect if vector misaligned accesses are faster or slower than equivalent vector byte accesses. This is useful for usermode to know whether vector byte accesses or vector misaligned accesses have a better bandwidth for operations like memcpy. Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins --- V1 -> V2: - Add Kconfig options - Add WORD_EEW to vec-copy-unaligned.S V2 -> V3: - Remove unnecessary comment - Remove local_irq_enable V3 -> V4: - Add preempt_disable/enable - Alphabetize includes in vec-copy-unaligned.S and unaligned_access_speed.c - Add duplicate comments above mb() to please checkpatch - change all_cpus_vec_supported to all_cpus_vec_unsupported so speed is tested if any cpus support unaligned vector accesses - Spell out _VECTOR_ in macros V4 -> V5: - Change void *unused to void *unused __always_unused V5 -> V6: - Check for vector misaligned access support in hotplug V6 -> V7: - Change SLOW to UNKNOWN when used as a placeholder --- arch/riscv/Kconfig | 18 +++ arch/riscv/kernel/Makefile | 3 +- arch/riscv/kernel/copy-unaligned.h | 5 + arch/riscv/kernel/sys_hwprobe.c | 6 + arch/riscv/kernel/unaligned_access_speed.c | 141 ++++++++++++++++++++- arch/riscv/kernel/vec-copy-unaligned.S | 58 +++++++++ 6 files changed, 228 insertions(+), 3 deletions(-) create mode 100644 arch/riscv/kernel/vec-copy-unaligned.S diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index ffbe0fdd7fb3..e98139340c1c 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -807,6 +807,24 @@ config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS will dynamically determine the speed of vector unaligned accesses on the underlying system if they are supported. =20 +config RISCV_SLOW_VECTOR_UNALIGNED_ACCESS + bool "Assume the system supports slow vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports slow vector unaligned memory accesses. = The + kernel and userspace programs may not be able to run at all on systems + that do not support unaligned memory accesses. + +config RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS + bool "Assume the system supports fast vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports fast vector unaligned memory accesses. = When + enabled, this option improves the performance of the kernel on such + systems. However, the kernel and userspace programs will run much more + slowly, or will not be able to run at all, on systems that do not + support efficient unaligned memory accesses. + endchoice =20 endmenu # "Platform type" diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 5b243d46f4b1..291935a084d5 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -64,7 +64,8 @@ obj-$(CONFIG_MMU) +=3D vdso.o vdso/ =20 obj-$(CONFIG_RISCV_MISALIGNED) +=3D traps_misaligned.o obj-$(CONFIG_RISCV_MISALIGNED) +=3D unaligned_access_speed.o -obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) +=3D copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS) +=3D vec-copy-unaligned.o =20 obj-$(CONFIG_FPU) +=3D fpu.o obj-$(CONFIG_FPU) +=3D kernel_mode_fpu.o diff --git a/arch/riscv/kernel/copy-unaligned.h b/arch/riscv/kernel/copy-un= aligned.h index e3d70d35b708..85d4d11450cb 100644 --- a/arch/riscv/kernel/copy-unaligned.h +++ b/arch/riscv/kernel/copy-unaligned.h @@ -10,4 +10,9 @@ void __riscv_copy_words_unaligned(void *dst, const void *src, size_t size); void __riscv_copy_bytes_unaligned(void *dst, const void *src, size_t size); =20 +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +void __riscv_copy_vec_words_unaligned(void *dst, const void *src, size_t s= ize); +void __riscv_copy_vec_bytes_unaligned(void *dst, const void *src, size_t s= ize); +#endif + #endif /* __RISCV_KERNEL_COPY_UNALIGNED_H */ diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprob= e.c index 2e3e324bad38..99d3e63d0444 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -221,6 +221,12 @@ static u64 hwprobe_vec_misaligned(const struct cpumask= *cpus) #else static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) { + if (IS_ENABLED(CONFIG_RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_VECTOR_MISALIGNED_FAST; + + if (IS_ENABLED(CONFIG_RISCV_SLOW_VECTOR_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_VECTOR_MISALIGNED_SLOW; + return RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN; } #endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel= /unaligned_access_speed.c index 1a87dfe9cb87..1cc35be0aaaa 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -6,11 +6,13 @@ #include #include #include +#include #include #include #include #include #include +#include =20 #include "copy-unaligned.h" =20 @@ -268,12 +270,147 @@ static int check_unaligned_access_speed_all_cpus(voi= d) } #endif =20 +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +static void check_vector_unaligned_access(struct work_struct *work __alway= s_unused) +{ + int cpu =3D smp_processor_id(); + u64 start_cycles, end_cycles; + u64 word_cycles; + u64 byte_cycles; + int ratio; + unsigned long start_jiffies, now; + struct page *page; + void *dst; + void *src; + long speed =3D RISCV_HWPROBE_VECTOR_MISALIGNED_SLOW; + + if (per_cpu(vector_misaligned_access, cpu) !=3D RISCV_HWPROBE_VECTOR_MISA= LIGNED_UNKNOWN) + return; + + page =3D alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); + if (!page) { + pr_warn("Allocation failure, not measuring vector misaligned performance= \n"); + return; + } + + /* Make an unaligned destination buffer. */ + dst =3D (void *)((unsigned long)page_address(page) | 0x1); + /* Unalign src as well, but differently (off by 1 + 2 =3D 3). */ + src =3D dst + (MISALIGNED_BUFFER_SIZE / 2); + src +=3D 2; + word_cycles =3D -1ULL; + + /* Do a warmup. */ + kernel_vector_begin(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + + start_jiffies =3D jiffies; + while ((now =3D jiffies) =3D=3D start_jiffies) + cpu_relax(); + + /* + * For a fixed amount of time, repeatedly try the function, and take + * the best time in cycles as the measurement. + */ + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles =3D get_cycles64(); + /* Ensure the CSR read can't reorder WRT to the copy. */ + mb(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + /* Ensure the copy ends before the end time is snapped. */ + mb(); + end_cycles =3D get_cycles64(); + if ((end_cycles - start_cycles) < word_cycles) + word_cycles =3D end_cycles - start_cycles; + } + + byte_cycles =3D -1ULL; + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + start_jiffies =3D jiffies; + while ((now =3D jiffies) =3D=3D start_jiffies) + cpu_relax(); + + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles =3D get_cycles64(); + /* Ensure the CSR read can't reorder WRT to the copy. */ + mb(); + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + /* Ensure the copy ends before the end time is snapped. */ + mb(); + end_cycles =3D get_cycles64(); + if ((end_cycles - start_cycles) < byte_cycles) + byte_cycles =3D end_cycles - start_cycles; + } + + kernel_vector_end(); + + /* Don't divide by zero. */ + if (!word_cycles || !byte_cycles) { + pr_warn("cpu%d: rdtime lacks granularity needed to measure unaligned vec= tor access speed\n", + cpu); + + return; + } + + if (word_cycles < byte_cycles) + speed =3D RISCV_HWPROBE_VECTOR_MISALIGNED_FAST; + + ratio =3D div_u64((byte_cycles * 100), word_cycles); + pr_info("cpu%d: Ratio of vector byte access time to vector unaligned word= access is %d.%02d, unaligned accesses are %s\n", + cpu, + ratio / 100, + ratio % 100, + (speed =3D=3D RISCV_HWPROBE_VECTOR_MISALIGNED_FAST) ? "fast" : "slow"); + + per_cpu(vector_misaligned_access, cpu) =3D speed; +} + +static int riscv_online_cpu_vec(unsigned int cpu) +{ + if (!has_vector()) + return 0; + + if (per_cpu(vector_misaligned_access, cpu) !=3D RISCV_HWPROBE_VECTOR_MISA= LIGNED_UNSUPPORTED) + return 0; + + check_vector_unaligned_access_emulated(NULL); + check_vector_unaligned_access(NULL); + return 0; +} + +/* Measure unaligned access speed on all CPUs present at boot in parallel.= */ +static int vec_check_unaligned_access_speed_all_cpus(void *unused __always= _unused) +{ + schedule_on_each_cpu(check_vector_unaligned_access); + + /* + * Setup hotplug callbacks for any new CPUs that come online or go + * offline. + */ + cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "riscv:online", + riscv_online_cpu_vec, NULL); + + return 0; +} +#else /* CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS */ +static int vec_check_unaligned_access_speed_all_cpus(void *unused __always= _unused) +{ + return 0; +} +#endif + static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated; + bool all_cpus_emulated, all_cpus_vec_unsupported; =20 all_cpus_emulated =3D check_unaligned_access_emulated_all_cpus(); - check_vector_unaligned_access_emulated_all_cpus(); + all_cpus_vec_unsupported =3D check_vector_unaligned_access_emulated_all_c= pus(); + + if (!all_cpus_vec_unsupported && + IS_ENABLED(CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS)) { + kthread_run(vec_check_unaligned_access_speed_all_cpus, + NULL, "vec_check_unaligned_access_speed_all_cpus"); + } =20 if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); diff --git a/arch/riscv/kernel/vec-copy-unaligned.S b/arch/riscv/kernel/vec= -copy-unaligned.S new file mode 100644 index 000000000000..d16f19f1b3b6 --- /dev/null +++ b/arch/riscv/kernel/vec-copy-unaligned.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2024 Rivos Inc. */ + +#include +#include +#include + + .text + +#define WORD_EEW 32 + +#define WORD_SEW CONCATENATE(e, WORD_EEW) +#define VEC_L CONCATENATE(vle, WORD_EEW).v +#define VEC_S CONCATENATE(vle, WORD_EEW).v + +/* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using word loads and stores= . */ +/* Note: The size is truncated to a multiple of WORD_EEW */ +SYM_FUNC_START(__riscv_copy_vec_words_unaligned) + andi a4, a2, ~(WORD_EEW-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, WORD_SEW, m8, ta, ma + VEC_L v0, (a1) + VEC_S v0, (a0) + addi a0, a0, WORD_EEW + addi a1, a1, WORD_EEW + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_words_unaligned) + +/* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using only byte accesses. */ +/* Note: The size is truncated to a multiple of 8 */ +SYM_FUNC_START(__riscv_copy_vec_bytes_unaligned) + andi a4, a2, ~(8-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, e8, m8, ta, ma + vle8.v v0, (a1) + vse8.v v0, (a0) + addi a0, a0, 8 + addi a1, a1, 8 + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_bytes_unaligned) --=20 2.45.2 From nobody Mon Sep 16 19:18:06 2024 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA4D917E443 for ; Fri, 26 Jul 2024 16:37:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011880; cv=none; b=UUwXdw1wwuFtJ60yIuV9ZpwLRMMiM+EU4XPNrQMkP4xzdJu44oSlqrDXlxZlx5rKXeosmyJj6WBFX0ILZRBsAA623DptXModEM+TOaoSCQZSMm1AaZEZ+caB6A0X82qtVlpMyIEo/W7ave7qL1d2yat95QuNzqD77smug79tzGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722011880; c=relaxed/simple; bh=wJlwKvHnHsmxr8ol1Mky5rqH/81g2E3nBO2+CHtrVdo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hS3yCPvM5nY91B3hHywLrBY0ESA0PAKvRwvaXcp1aoVYI7CMZ1/Yp63t1k/5g2FeUhA31FFn4M5DvfsBAx/VV4sUsB/1UPsNkhxzhaPTenygYfjfiU/hSKh68VolcqGbEROX2RG4nZcoiG5hqnM0bFXQgIAmxHfiq8Bawkxc8QI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=za7UJh5e; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="za7UJh5e" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2cb57e25387so901131a91.3 for ; Fri, 26 Jul 2024 09:37:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1722011878; x=1722616678; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cCBQVSPRYPBIeQ5EoC+hl2LHpusqOR8yThszxhSpMg8=; b=za7UJh5e5Y3eX4YWKOA56x87J3jAN9WbzToyvaj/bw1FEqzyYeHeZvN/dzpHSE1w0I jKYTMSkl50oPUmpW9+oV4GVbgbtNJEgq0oWFI7Jy9k1tESFSwJ78f+NwTTlHwl9OaMTm ZJh06JXzNdUYljZcsC1/Qc9Zxt953dIkQ32xbyR+sz/CHhqSS9oG9CMx4ruoeBLs9K99 m1hkrZ5QhwkM06FLGttWuPlfEP3KRszeul5xAVzeZQe6BbCODrCHa0CQSnMllGDmWHj7 lA5Qw3W9v1r41Tc9bOencUMeSWekDHc+8cG7MdTKoclxGWN4uaQ67nOHQasdInhKKFAa KgsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722011878; x=1722616678; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cCBQVSPRYPBIeQ5EoC+hl2LHpusqOR8yThszxhSpMg8=; b=UFt8odYx2D7ZEw0XtfCGJmCEXZT+G/PEh4pQxxJ5Hrh+7q5byJAf1MFmELDNoeumed t9CS0jh7DewbQlrW5Wva40XOImupq14Swal2tm0CWPsDPuELsdP+ZfWozZHU5RLj64DS doJFhtTXd6HKMFsMmfAJ+B2VEAigSIED46JTVmz1OvKzWG8+YWruE38e/X3wUeHK4D02 yqVfGgYHMLwjwvEI6D7yvltaH8ZHiOD1sYSa4bByA7o2fSWLMmqoDSb9I2sr+fjNErje zzmoCw7tYnkJtNtuGpNXACwTrzFP1bnUNZWjVjvKofwmikkB+sK9rg7j6VoghFZ0Wa0X ibBg== X-Forwarded-Encrypted: i=1; AJvYcCU0d8F4RhYepZuCM3d23MG41YZPhCsLSYfnOmkRdwvamcfKbO590l+hehadFEv3KjWnwBC1kORHDRwWTO0rIuWP93hEWlP667uac58e X-Gm-Message-State: AOJu0YxdV4H+fBFDTUbiVmgK9I+ukso5/TUomHzJXPxQMVa0rZKyB+mR E2A8PkUfGP1WFNSBCPKULp2DTJfLbqHweGsHEvkxxVeOF+uKx9SV8BYfwx9aY1g= X-Google-Smtp-Source: AGHT+IEXYMNp7r6J/EYXhRPF4svQfnFw/pxaQt/HnMEC+56U/jYOIE7D/L1AibRAOg2CvPdeVu0dyA== X-Received: by 2002:a17:90a:d98b:b0:2bd:d42b:22dc with SMTP id 98e67ed59e1d1-2cf7e868df4mr14654a91.43.1722011877991; Fri, 26 Jul 2024 09:37:57 -0700 (PDT) Received: from jesse-desktop.ba.rivosinc.com (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cf28c55a2dsm3676619a91.7.2024.07.26.09.37.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jul 2024 09:37:57 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v7 8/8] RISC-V: hwprobe: Document unaligned vector perf key Date: Fri, 26 Jul 2024 12:37:19 -0400 Message-ID: <20240726163719.1667923-9-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240726163719.1667923-1-jesse@rivosinc.com> References: <20240726163719.1667923-1-jesse@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Document key for reporting the speed of unaligned vector accesses. The descriptions are the same as the scalar equivalent values. Signed-off-by: Jesse Taube Reviewed-by: Charlie Jenkins --- V1 -> V2: - New patch V2 -> V3: - Specify access width V3 -> V4: - Clarify we're talking about byte accesses using vector registers - Spell out _VECTOR_ in macros V4 -> V5: - No changes V5 -> V6: - No changes V6 -> V7: - No changes --- Documentation/arch/riscv/hwprobe.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/risc= v/hwprobe.rst index 78acd37b6477..f83a13dc4cbc 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -238,3 +238,19 @@ The following keys are defined: =20 * :c:macro:`RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE`: An unsigned int which represents the size of the Zicboz block in bytes. + +* :c:macro:`RISCV_HWPROBE_KEY_VECTOR_MISALIGNED_PERF`: An enum value descr= ibing the + performance of misaligned vector accesses on the selected set of proc= essors. + + * :c:macro:`RISCV_HWPROBE_VECTOR_MISALIGNED_UNKNOWN`: The performance of= misaligned + vector accesses is unknown. + + * :c:macro:`RISCV_HWPROBE_VECTOR_MISALIGNED_SLOW`: 32-bit misaligned acc= esses using vector + registers are slower than the equivalent quantity of byte accesses via= vector registers. + Misaligned accesses may be supported directly in hardware, or trapped = and emulated by software. + + * :c:macro:`RISCV_HWPROBE_VECTOR_MISALIGNED_FAST`: 32-bit misaligned acc= esses using vector + registers are faster than the equivalent quantity of byte accesses via= vector registers. + + * :c:macro:`RISCV_HWPROBE_VECTOR_MISALIGNED_UNSUPPORTED`: Misaligned vec= tor accesses are + not supported at all and will generate a misaligned address fault. --=20 2.45.2