From nobody Sun Apr 26 09:36:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0019CC433EF for ; Sun, 19 Jun 2022 20:32:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237150AbiFSUck (ORCPT ); Sun, 19 Jun 2022 16:32:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231547AbiFSUch (ORCPT ); Sun, 19 Jun 2022 16:32:37 -0400 Received: from gloria.sntech.de (gloria.sntech.de [185.11.138.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50558B841; Sun, 19 Jun 2022 13:32:35 -0700 (PDT) Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o31an-0000EW-AZ; Sun, 19 Jun 2022 22:32:21 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner Subject: [PATCH 1/4] of: also handle dma-noncoherent in of_dma_is_coherent() Date: Sun, 19 Jun 2022 22:32:09 +0200 Message-Id: <20220619203212.3604485-2-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220619203212.3604485-1-heiko@sntech.de> References: <20220619203212.3604485-1-heiko@sntech.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" of_dma_is_coherent() currently expects the architecture to be non-coherent and some devices being coherent getting marked as such with the dma-coherent devicetree property. For PowerPC CONFIG_OF_DMA_DEFAULT_COHERENT was added which currently makes of_dma_is_coherent() always return true but doesn't handle the case of the architecture being coherent but some devices not. So modify the function to also check for dma-noncoherent and set a suitable default return value. If CONFIG_OF_DMA_DEFAULT_COHERENT is set the value starts with true and finding dma-noncoherent will set it to false and without CONFIG_OF_DMA_DEFAULT_COHERENT, the behaviour is reversed. Signed-off-by: Heiko Stuebner --- drivers/of/address.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/of/address.c b/drivers/of/address.c index 94f017d808c4..1c362d09983d 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -1045,26 +1045,32 @@ phys_addr_t __init of_dma_get_max_cpu_address(struc= t device_node *np) * * It returns true if "dma-coherent" property was found * for this device in the DT, or if DMA is coherent by - * default for OF devices on the current platform. + * default for OF devices on the current platform and no + * "dma-noncoherent" property was found for this device. */ bool of_dma_is_coherent(struct device_node *np) { struct device_node *node; + bool ret =3D false; =20 if (IS_ENABLED(CONFIG_OF_DMA_DEFAULT_COHERENT)) - return true; + ret =3D true; =20 node =3D of_node_get(np); =20 while (node) { if (of_property_read_bool(node, "dma-coherent")) { - of_node_put(node); - return true; + ret =3D true; + break; + } + if (of_property_read_bool(node, "dma-noncoherent")) { + ret =3D false; + break; } node =3D of_get_next_dma_parent(node); } of_node_put(node); - return false; + return ret; } EXPORT_SYMBOL_GPL(of_dma_is_coherent); =20 --=20 2.35.1 From nobody Sun Apr 26 09:36:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DF65CCA480 for ; Sun, 19 Jun 2022 20:32:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235253AbiFSUci (ORCPT ); Sun, 19 Jun 2022 16:32:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231753AbiFSUch (ORCPT ); Sun, 19 Jun 2022 16:32:37 -0400 Received: from gloria.sntech.de (gloria.sntech.de [185.11.138.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 437D965C5; Sun, 19 Jun 2022 13:32:34 -0700 (PDT) Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o31an-0000EW-Va; Sun, 19 Jun 2022 22:32:22 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner , Rob Herring Subject: [PATCH 2/4] dt-bindings: riscv: document cbom-block-size Date: Sun, 19 Jun 2022 22:32:10 +0200 Message-Id: <20220619203212.3604485-3-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220619203212.3604485-1-heiko@sntech.de> References: <20220619203212.3604485-1-heiko@sntech.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The Zicbom operates on a block-size defined for the cpu-core, which does not necessarily match other cache-sizes used. So add the necessary property for the system to know the core's block-size. Signed-off-by: Heiko Stuebner Reviewed-by: Anup Patel Acked-by: Rob Herring --- Documentation/devicetree/bindings/riscv/cpus.yaml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentat= ion/devicetree/bindings/riscv/cpus.yaml index d632ac76532e..873dd12f6e89 100644 --- a/Documentation/devicetree/bindings/riscv/cpus.yaml +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml @@ -63,6 +63,11 @@ properties: - riscv,sv48 - riscv,none =20 + riscv,cbom-block-size: + $ref: /schemas/types.yaml#/definitions/uint32 + description: + The blocksize in bytes for the Zicbom cache operations. + riscv,isa: description: Identifies the specific RISC-V instruction set architecture --=20 2.35.1 From nobody Sun Apr 26 09:36:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED63FC433EF for ; Sun, 19 Jun 2022 20:33:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237270AbiFSUdE (ORCPT ); Sun, 19 Jun 2022 16:33:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237259AbiFSUcv (ORCPT ); Sun, 19 Jun 2022 16:32:51 -0400 Received: from gloria.sntech.de (gloria.sntech.de [185.11.138.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4BEF3237; Sun, 19 Jun 2022 13:32:48 -0700 (PDT) Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o31ao-0000EW-Kk; Sun, 19 Jun 2022 22:32:22 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner , Atish Patra Subject: [PATCH 3/4] riscv: Implement Zicbom-based cache management operations Date: Sun, 19 Jun 2022 22:32:11 +0200 Message-Id: <20220619203212.3604485-4-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220619203212.3604485-1-heiko@sntech.de> References: <20220619203212.3604485-1-heiko@sntech.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The Zicbom ISA-extension was ratified in november 2021 and introduces instructions for dcache invalidate, clean and flush operations. Implement cache management operations based on them. Of course not all cores will support this, so implement an alternative-based mechanism that replaces empty instructions with ones done around Zicbom instructions. As discussed in previous versions, assume the platform being coherent by default so that non-coherent devices need to get marked accordingly by firmware. Signed-off-by: Heiko Stuebner Cc: Christoph Hellwig Cc: Atish Patra Cc: Guo Ren Cc: Anup Patel --- arch/riscv/Kconfig | 31 +++++++++ arch/riscv/Makefile | 4 ++ arch/riscv/include/asm/cache.h | 4 ++ arch/riscv/include/asm/cacheflush.h | 6 ++ arch/riscv/include/asm/errata_list.h | 19 +++++- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 18 ++++++ arch/riscv/kernel/setup.c | 2 + arch/riscv/mm/Makefile | 1 + arch/riscv/mm/dma-noncoherent.c | 96 ++++++++++++++++++++++++++++ 11 files changed, 182 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/mm/dma-noncoherent.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 32ffef9f6e5b..f7b2b3a4b7f1 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -113,6 +113,7 @@ config RISCV select MODULES_USE_ELF_RELA if MODULES select MODULE_SECTIONS if MODULES select OF + select OF_DMA_DEFAULT_COHERENT select OF_EARLY_FLATTREE select OF_IRQ select PCI_DOMAINS_GENERIC if PCI @@ -218,6 +219,14 @@ config PGTABLE_LEVELS config LOCKDEP_SUPPORT def_bool y =20 +config RISCV_DMA_NONCOHERENT + bool + select ARCH_HAS_DMA_PREP_COHERENT + select ARCH_HAS_SYNC_DMA_FOR_DEVICE + select ARCH_HAS_SYNC_DMA_FOR_CPU + select ARCH_HAS_SETUP_DMA_OPS + select DMA_DIRECT_REMAP + source "arch/riscv/Kconfig.socs" source "arch/riscv/Kconfig.erratas" =20 @@ -376,6 +385,28 @@ config RISCV_ISA_SVPBMT =20 If you don't know what to do here, say Y. =20 +config CC_HAS_ZICBOM + bool + default y if 64BIT && $(cc-option,-mabi=3Dlp64 -march=3Drv64ima_zicbom) + default y if 32BIT && $(cc-option,-mabi=3Dlp64 -march=3Drv32ima_zicbom) + +config RISCV_ISA_ZICBOM + bool "Zicbom extension support for non-coherent DMA operation" + depends on CC_HAS_ZICBOM + depends on !XIP_KERNEL + select RISCV_DMA_NONCOHERENT + select RISCV_ALTERNATIVE + default y + help + Adds support to dynamically detect the presence of the ZICBOM + extension (Cache Block Management Operations) and enable its + usage. + + The Zicbom extension can be used to handle for example + non-coherent DMA support on devices that need it. + + If you don't know what to do here, say Y. + config FPU bool "FPU support" default y diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 34cf8a598617..fbaabc98b3d2 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -56,6 +56,10 @@ riscv-march-$(CONFIG_RISCV_ISA_C) :=3D $(riscv-march-y)c toolchain-need-zicsr-zifencei :=3D $(call cc-option-yn, -march=3D$(riscv-m= arch-y)_zicsr_zifencei) riscv-march-$(toolchain-need-zicsr-zifencei) :=3D $(riscv-march-y)_zicsr_z= ifencei =20 +# Check if the toolchain supports Zicbom extension +toolchain-supports-zicbom :=3D $(call cc-option-yn, -march=3D$(riscv-march= -y)_zicbom) +riscv-march-$(toolchain-supports-zicbom) :=3D $(riscv-march-y)_zicbom + KBUILD_CFLAGS +=3D -march=3D$(subst fd,,$(riscv-march-y)) KBUILD_AFLAGS +=3D -march=3D$(riscv-march-y) =20 diff --git a/arch/riscv/include/asm/cache.h b/arch/riscv/include/asm/cache.h index 9b58b104559e..d3036df23ccb 100644 --- a/arch/riscv/include/asm/cache.h +++ b/arch/riscv/include/asm/cache.h @@ -11,6 +11,10 @@ =20 #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) =20 +#ifdef CONFIG_RISCV_DMA_NONCOHERENT +#define ARCH_DMA_MINALIGN L1_CACHE_BYTES +#endif + /* * RISC-V requires the stack pointer to be 16-byte aligned, so ensure that * the flat loader aligns it accordingly. diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/c= acheflush.h index 23ff70350992..eb12d014b158 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -42,6 +42,12 @@ void flush_icache_mm(struct mm_struct *mm, bool local); =20 #endif /* CONFIG_SMP */ =20 +#ifdef CONFIG_RISCV_ISA_ZICBOM +void riscv_init_cbom_blocksize(void); +#else +static inline void riscv_init_cbom_blocksize(void) { } +#endif + /* * Bits in sys_riscv_flush_icache()'s flags argument. */ diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/= errata_list.h index 398e351e7002..79d89aeeaa6c 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -20,7 +20,8 @@ #endif =20 #define CPUFEATURE_SVPBMT 0 -#define CPUFEATURE_NUMBER 1 +#define CPUFEATURE_ZICBOM 1 +#define CPUFEATURE_NUMBER 2 =20 #ifdef __ASSEMBLY__ =20 @@ -87,6 +88,22 @@ asm volatile(ALTERNATIVE( \ #define ALT_THEAD_PMA(_val) #endif =20 +#define ALT_CMO_OP(_op, _start, _size, _cachesize) \ +asm volatile(ALTERNATIVE( \ + __nops(5), \ + "mv a0, %1\n\t" \ + "j 2f\n\t" \ + "3:\n\t" \ + "cbo." __stringify(_op) " (a0)\n\t" \ + "add a0, a0, %0\n\t" \ + "2:\n\t" \ + "bltu a0, %2, 3b\n\t", 0, \ + CPUFEATURE_ZICBOM, CONFIG_RISCV_ISA_ZICBOM) \ + : : "r"(_cachesize), \ + "r"((unsigned long)(_start) & ~((_cachesize) - 1UL)), \ + "r"((unsigned long)(_start) + (_size)) \ + : "a0") + #endif /* __ASSEMBLY__ */ =20 #endif diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 4e2486881840..6044e402003d 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -53,6 +53,7 @@ extern unsigned long elf_hwcap; enum riscv_isa_ext_id { RISCV_ISA_EXT_SSCOFPMF =3D RISCV_ISA_EXT_BASE, RISCV_ISA_EXT_SVPBMT, + RISCV_ISA_EXT_ZICBOM, RISCV_ISA_EXT_ID_MAX =3D RISCV_ISA_EXT_MAX, }; =20 diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index fba9e9f46a8c..0365557f7122 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -89,6 +89,7 @@ int riscv_of_parent_hartid(struct device_node *node) static struct riscv_isa_ext_data isa_ext_arr[] =3D { __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), + __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), __RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX), }; =20 diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 6a40cb8134bd..c66bf5c33cc3 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -199,6 +199,7 @@ void __init riscv_fill_hwcap(void) } else { SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF); SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT); + SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); } #undef SET_ISA_EXT_MAP } @@ -259,6 +260,20 @@ static bool __init_or_module cpufeature_probe_svpbmt(u= nsigned int stage) return false; } =20 +static bool __init_or_module cpufeature_probe_zicbom(unsigned int stage) +{ +#ifdef CONFIG_RISCV_ISA_ZICBOM + switch (stage) { + case RISCV_ALTERNATIVES_EARLY_BOOT: + return false; + default: + return riscv_isa_extension_available(NULL, ZICBOM); + } +#endif + + return false; +} + /* * Probe presence of individual extensions. * @@ -273,6 +288,9 @@ static u32 __init_or_module cpufeature_probe(unsigned i= nt stage) if (cpufeature_probe_svpbmt(stage)) cpu_req_feature |=3D (1U << CPUFEATURE_SVPBMT); =20 + if (cpufeature_probe_zicbom(stage)) + cpu_req_feature |=3D (1U << CPUFEATURE_ZICBOM); + return cpu_req_feature; } =20 diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index f0f36a4a0e9b..95ef6e2bf45c 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -22,6 +22,7 @@ #include =20 #include +#include #include #include #include @@ -296,6 +297,7 @@ void __init setup_arch(char **cmdline_p) #endif =20 riscv_fill_hwcap(); + riscv_init_cbom_blocksize(); apply_boot_alternatives(); } =20 diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index ac7a25298a04..d76aabf4b94d 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -30,3 +30,4 @@ endif endif =20 obj-$(CONFIG_DEBUG_VIRTUAL) +=3D physaddr.o +obj-$(CONFIG_RISCV_DMA_NONCOHERENT) +=3D dma-noncoherent.o diff --git a/arch/riscv/mm/dma-noncoherent.c b/arch/riscv/mm/dma-noncoheren= t.c new file mode 100644 index 000000000000..6be6ff2cd5d8 --- /dev/null +++ b/arch/riscv/mm/dma-noncoherent.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RISC-V specific functions to support DMA for non-coherent devices + * + * Copyright (c) 2021 Western Digital Corporation or its affiliates. + */ + +#include +#include +#include +#include +#include +#include + +static unsigned int riscv_cbom_block_size =3D L1_CACHE_BYTES; + +void arch_sync_dma_for_device(phys_addr_t paddr, size_t size, + enum dma_data_direction dir) +{ + void *vaddr =3D phys_to_virt(paddr); + + switch (dir) { + case DMA_TO_DEVICE: + ALT_CMO_OP(clean, vaddr, size, riscv_cbom_block_size); + break; + case DMA_FROM_DEVICE: + ALT_CMO_OP(inval, vaddr, size, riscv_cbom_block_size); + break; + case DMA_BIDIRECTIONAL: + ALT_CMO_OP(flush, vaddr, size, riscv_cbom_block_size); + break; + default: + break; + } +} + +void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size, + enum dma_data_direction dir) +{ + void *vaddr =3D phys_to_virt(paddr); + + switch (dir) { + case DMA_TO_DEVICE: + break; + case DMA_FROM_DEVICE: + case DMA_BIDIRECTIONAL: + ALT_CMO_OP(flush, vaddr, size, riscv_cbom_block_size); + break; + default: + break; + } +} + +void arch_dma_prep_coherent(struct page *page, size_t size) +{ + void *flush_addr =3D page_address(page); + + ALT_CMO_OP(flush, flush_addr, size, riscv_cbom_block_size); +} + +void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, + const struct iommu_ops *iommu, bool coherent) +{ + dev->dma_coherent =3D coherent; +} + +#ifdef CONFIG_RISCV_ISA_ZICBOM +void riscv_init_cbom_blocksize(void) +{ + struct device_node *node; + int ret; + u32 val; + + for_each_of_cpu_node(node) { + int hartid =3D riscv_of_processor_hartid(node); + int cbom_hartid; + + if (hartid < 0) + continue; + + /* set block-size for cbom extension if available */ + ret =3D of_property_read_u32(node, "riscv,cbom-block-size", &val); + if (ret) + continue; + + if (!riscv_cbom_block_size) { + riscv_cbom_block_size =3D val; + cbom_hartid =3D hartid; + } else { + if (riscv_cbom_block_size !=3D val) + pr_warn("cbom-block-size mismatched between harts %d and %d\n", + cbom_hartid, hartid); + } + } +} +#endif --=20 2.35.1 From nobody Sun Apr 26 09:36:29 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82BF0C433EF for ; Sun, 19 Jun 2022 20:33:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237511AbiFSUdM (ORCPT ); Sun, 19 Jun 2022 16:33:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235930AbiFSUcv (ORCPT ); Sun, 19 Jun 2022 16:32:51 -0400 Received: from gloria.sntech.de (gloria.sntech.de [185.11.138.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35F2711B; Sun, 19 Jun 2022 13:32:47 -0700 (PDT) Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o31ap-0000EW-AQ; Sun, 19 Jun 2022 22:32:23 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner Subject: [PATCH 4/4] riscv: implement cache-management errata for T-Head SoCs Date: Sun, 19 Jun 2022 22:32:12 +0200 Message-Id: <20220619203212.3604485-5-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220619203212.3604485-1-heiko@sntech.de> References: <20220619203212.3604485-1-heiko@sntech.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The T-Head C906 and C910 implement a scheme for handling cache operations different from the generic Zicbom extension. Add an errata for it next to the generic dma coherency ops. Signed-off-by: Heiko Stuebner Reviewed-by: Samuel Holland Tested-by: Samuel Holland --- arch/riscv/Kconfig.erratas | 11 +++++++ arch/riscv/errata/thead/errata.c | 15 +++++++++ arch/riscv/include/asm/errata_list.h | 48 +++++++++++++++++++++++++--- 3 files changed, 69 insertions(+), 5 deletions(-) diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas index 457ac72c9b36..3223e533fd87 100644 --- a/arch/riscv/Kconfig.erratas +++ b/arch/riscv/Kconfig.erratas @@ -55,4 +55,15 @@ config ERRATA_THEAD_PBMT =20 If you don't know what to do here, say "Y". =20 +config ERRATA_THEAD_CMO + bool "Apply T-Head cache management errata" + depends on ERRATA_THEAD + select RISCV_DMA_NONCOHERENT + default y + help + This will apply the cache management errata to handle the + non-standard handling on non-coherent operations on T-Head SoCs. + + If you don't know what to do here, say "Y". + endmenu diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/err= ata.c index b37b6fedd53b..c97e59ae1b79 100644 --- a/arch/riscv/errata/thead/errata.c +++ b/arch/riscv/errata/thead/errata.c @@ -27,6 +27,18 @@ static bool errata_probe_pbmt(unsigned int stage, return false; } =20 +static bool errata_probe_cmo(unsigned int stage, + unsigned long arch_id, unsigned long impid) +{ + if (arch_id !=3D 0 || impid !=3D 0) + return false; + + if (stage =3D=3D RISCV_ALTERNATIVES_EARLY_BOOT) + return false; + + return true; +} + static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid) { @@ -35,6 +47,9 @@ static u32 thead_errata_probe(unsigned int stage, if (errata_probe_pbmt(stage, archid, impid)) cpu_req_errata |=3D (1U << ERRATA_THEAD_PBMT); =20 + if (errata_probe_cmo(stage, archid, impid)) + cpu_req_errata |=3D (1U << ERRATA_THEAD_CMO); + return cpu_req_errata; } =20 diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/= errata_list.h index 79d89aeeaa6c..19a771085781 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -16,7 +16,8 @@ =20 #ifdef CONFIG_ERRATA_THEAD #define ERRATA_THEAD_PBMT 0 -#define ERRATA_THEAD_NUMBER 1 +#define ERRATA_THEAD_CMO 1 +#define ERRATA_THEAD_NUMBER 2 #endif =20 #define CPUFEATURE_SVPBMT 0 @@ -88,17 +89,54 @@ asm volatile(ALTERNATIVE( \ #define ALT_THEAD_PMA(_val) #endif =20 +/* + * dcache.ipa rs1 (invalidate, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01010 rs1 000 00000 0001011 + * dache.iva rs1 (invalida, virtual address) + * 0000001 00110 rs1 000 00000 0001011 + * + * dcache.cpa rs1 (clean, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01001 rs1 000 00000 0001011 + * dcache.cva rs1 (clean, virtual address) + * 0000001 00100 rs1 000 00000 0001011 + * + * dcache.cipa rs1 (clean then invalidate, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01011 rs1 000 00000 0001011 + * dcache.civa rs1 (... virtual address) + * 0000001 00111 rs1 000 00000 0001011 + * + * sync.s (make sure all cache operations finished) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000000 11001 00000 000 00000 0001011 + */ +#define THEAD_inval_A0 ".long 0x0265000b" +#define THEAD_clean_A0 ".long 0x0245000b" +#define THEAD_flush_A0 ".long 0x0275000b" +#define THEAD_SYNC_S ".long 0x0190000b" + #define ALT_CMO_OP(_op, _start, _size, _cachesize) \ -asm volatile(ALTERNATIVE( \ - __nops(5), \ +asm volatile(ALTERNATIVE_2( \ + __nops(6), \ "mv a0, %1\n\t" \ "j 2f\n\t" \ "3:\n\t" \ "cbo." __stringify(_op) " (a0)\n\t" \ "add a0, a0, %0\n\t" \ "2:\n\t" \ - "bltu a0, %2, 3b\n\t", 0, \ - CPUFEATURE_ZICBOM, CONFIG_RISCV_ISA_ZICBOM) \ + "bltu a0, %2, 3b\n\t" \ + "nop", 0, CPUFEATURE_ZICBOM, CONFIG_RISCV_ISA_ZICBOM, \ + "mv a0, %1\n\t" \ + "j 2f\n\t" \ + "3:\n\t" \ + THEAD_##_op##_A0 "\n\t" \ + "add a0, a0, %0\n\t" \ + "2:\n\t" \ + "bltu a0, %2, 3b\n\t" \ + THEAD_SYNC_S, THEAD_VENDOR_ID, \ + ERRATA_THEAD_CMO, CONFIG_ERRATA_THEAD_CMO) \ : : "r"(_cachesize), \ "r"((unsigned long)(_start) & ~((_cachesize) - 1UL)), \ "r"((unsigned long)(_start) + (_size)) \ --=20 2.35.1