From nobody Mon Jun 15 09:08:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F365C433EF for ; Sat, 30 Apr 2022 15:38:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1382946AbiD3PlY (ORCPT ); Sat, 30 Apr 2022 11:41:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232786AbiD3PlQ (ORCPT ); Sat, 30 Apr 2022 11:41:16 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A396CA0BD2 for ; Sat, 30 Apr 2022 08:37:54 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id p8so9215843pfh.8 for ; Sat, 30 Apr 2022 08:37:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=sMDsvqsINC/1HzV0XVeIiP6qJbruGiMokRnjzI8Iijs=; b=XwrBMV84AyVfrW6L5mcdJTABJJHR5Cc7I+uxzoBIZkBekPAupOfEZ/k+xOdbEKEife EuQaMP+UWNSV75iAffC+dhjvPA1NaL2+4Vv99GAnx8rRzw+i1YjF/iXl/h5HIlGA1BJP kJYDzMEeqLhKYrTdf84LxohlkUiVLM0+wsskIQ1ffeDS9wpyvOSTjwuvmTWf7zojGnh5 0gtpJ582CRTl1aBaRbQCKZoLO1qLrMnoMtwzFYd7gX9j2nxRyz4PTc/UBcirCJrUc/9M 8CzT7gASjopV5S44exy8VEtWKH/asaKYNKyStQHWara8f1zP/z+5LHZgqMC4RrdIfMr4 IKcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=sMDsvqsINC/1HzV0XVeIiP6qJbruGiMokRnjzI8Iijs=; b=aa+2lrcjAfOAPc9odtgd52p3xM0pVJ+d3kmEaEZcb2cymTkhEofiqnWTbr3wyyUlF8 05XPSB3Bsj0d9SOCKdHj/vDvjgjLv42MCmOxXBkLQtrsUcRKY//mRDcv86k79TTVZeKx vLy4SV6zrpDQR/LlyjdYKUa7OnG5y4qURPvLiDKH/K9iR8i9pgeutVw7mTktyYAcj9GD B5m+ljOlCMK9aN1JMztDGxwiDdnJn2ADF3W+RrT69ez88oUcpZk+d29cWbvbiVwuWNrE krP1u4kBQSe4IK2x3QSOh0lFbRHF305oSm6/2zdRxq7QRK8wAAaDiyyhhY0Rij9Mm/Iw JpCg== X-Gm-Message-State: AOAM5333qwzAiG7wDU/ZDRJpztPEEu5NMM641jTZ/GSgPa73ZmK4VYgl GdveT5cVbSEoGQ3//e/Ftu5G0g== X-Google-Smtp-Source: ABdhPJxnHiq/z/AiUlN9BinSyVfPTpl48VBq4+Usu5YRZ0auqyHrkcbtSZj+xWHAH6hS6v1DaYkGEA== X-Received: by 2002:a63:38e:0:b0:3ab:ada6:a219 with SMTP id 136-20020a63038e000000b003abada6a219mr3560920pgd.140.1651333074093; Sat, 30 Apr 2022 08:37:54 -0700 (PDT) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id d3-20020a170902854300b0015e8d4eb26esm1626226plo.184.2022.04.30.08.37.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 08:37:53 -0700 (PDT) Subject: [PATCH v4 1/7] asm-generic: ticket-lock: New generic ticket-based spinlock Date: Sat, 30 Apr 2022 08:36:20 -0700 Message-Id: <20220430153626.30660-2-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220430153626.30660-1-palmer@rivosinc.com> References: <20220430153626.30660-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: guoren@kernel.org, peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , Greg KH , sudipm.mukherjee@gmail.com, macro@orcam.me.uk, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra This is a simple, fair spinlock. Specifically it doesn't have all the subtle memory model dependencies that qspinlock has, which makes it more suitable for simple systems as it is more likely to be correct. It is implemented entirely in terms of standard atomics and thus works fine without any arch-specific code. This replaces the existing asm-generic/spinlock.h, which just errored out on SMP systems. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Palmer Dabbelt Reviewed-by: Guo Ren Tested-by: Heiko Stuebner --- include/asm-generic/spinlock.h | 94 +++++++++++++++++++++++++--- include/asm-generic/spinlock_types.h | 17 +++++ 2 files changed, 104 insertions(+), 7 deletions(-) create mode 100644 include/asm-generic/spinlock_types.h diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h index adaf6acab172..fdfebcb050f4 100644 --- a/include/asm-generic/spinlock.h +++ b/include/asm-generic/spinlock.h @@ -1,12 +1,92 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __ASM_GENERIC_SPINLOCK_H -#define __ASM_GENERIC_SPINLOCK_H + /* - * You need to implement asm/spinlock.h for SMP support. The generic - * version does not handle SMP. + * 'Generic' ticket-lock implementation. + * + * It relies on atomic_fetch_add() having well defined forward progress + * guarantees under contention. If your architecture cannot provide this, = stick + * to a test-and-set lock. + * + * It also relies on atomic_fetch_add() being safe vs smp_store_release() = on a + * sub-word of the value. This is generally true for anything LL/SC althou= gh + * you'd be hard pressed to find anything useful in architecture specifica= tions + * about this. If your architecture cannot do this you might be better off= with + * a test-and-set. + * + * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and = hence + * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along= with + * a full fence after the spin to upgrade the otherwise-RCpc + * atomic_cond_read_acquire(). + * + * The implementation uses smp_cond_load_acquire() to spin, so if the + * architecture has WFE like instructions to sleep instead of poll for word + * modifications be sure to implement that (see ARM64 for example). + * */ -#ifdef CONFIG_SMP -#error need an architecture specific asm/spinlock.h -#endif + +#ifndef __ASM_GENERIC_SPINLOCK_H +#define __ASM_GENERIC_SPINLOCK_H + +#include +#include + +static __always_inline void arch_spin_lock(arch_spinlock_t *lock) +{ + u32 val =3D atomic_fetch_add(1<<16, lock); + u16 ticket =3D val >> 16; + + if (ticket =3D=3D (u16)val) + return; + + /* + * atomic_cond_read_acquire() is RCpc, but rather than defining a + * custom cond_read_rcsc() here we just emit a full fence. We only + * need the prior reads before subsequent writes ordering from + * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we + * have no outstanding writes due to the atomic_fetch_add() the extra + * orderings are free. + */ + atomic_cond_read_acquire(lock, ticket =3D=3D (u16)VAL); + smp_mb(); +} + +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock) +{ + u32 old =3D atomic_read(lock); + + if ((old >> 16) !=3D (old & 0xffff)) + return false; + + return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */ +} + +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock) +{ + u16 *ptr =3D (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN); + u32 val =3D atomic_read(lock); + + smp_store_release(ptr, (u16)val + 1); +} + +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock) +{ + u32 val =3D atomic_read(lock); + + return ((val >> 16) !=3D (val & 0xffff)); +} + +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock) +{ + u32 val =3D atomic_read(lock); + + return (s16)((val >> 16) - (val & 0xffff)) > 1; +} + +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock) +{ + return !arch_spin_is_locked(&lock); +} + +#include =20 #endif /* __ASM_GENERIC_SPINLOCK_H */ diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spi= nlock_types.h new file mode 100644 index 000000000000..8962bb730945 --- /dev/null +++ b/include/asm-generic/spinlock_types.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_GENERIC_SPINLOCK_TYPES_H +#define __ASM_GENERIC_SPINLOCK_TYPES_H + +#include +typedef atomic_t arch_spinlock_t; + +/* + * qrwlock_types depends on arch_spinlock_t, so we must typedef that befor= e the + * include. + */ +#include + +#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0) + +#endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */ --=20 2.34.1 From nobody Mon Jun 15 09:08:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D33A2C433EF for ; Sat, 30 Apr 2022 15:38:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1383004AbiD3Plp (ORCPT ); Sat, 30 Apr 2022 11:41:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242874AbiD3PlT (ORCPT ); Sat, 30 Apr 2022 11:41:19 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0443BA0BC8 for ; Sat, 30 Apr 2022 08:37:56 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id qe3-20020a17090b4f8300b001dc24e4da73so1284788pjb.1 for ; Sat, 30 Apr 2022 08:37:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=z18FUB9s7KBEskqVTf7aP0q/WSmapl+AGV7tm/Rzx+w=; b=FFgRoVbZwMv/agjgfBQeiD4RduxnvL5RXSepip5M3kO2MagsWbSqQqxN8cL5qUbeSF BYxou0ljjXmk//t8pH9P0AP/HLxtAE0BLwIcoy+Ou0Lpi2MlCzGQ6f682gOG6CuxBpM2 nwCLJBaCh5RV2r+qT1H5KRlCTIZUXo7CJ4l12KZINBErQtyavfgirI9l4JGBZus3gpLe R/LbIsa9B1VFw3791kqpL2EF7PWqMzHmRevq+xETQxk+DF6ipj782xSdFBUvyauIwqQ5 aCr8I9PL5iINM/w5oq+qM/NT//BBL45/kvYgK8Ddw1j+NpxzLAepCkGEW3FID5OiPVqO qAwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=z18FUB9s7KBEskqVTf7aP0q/WSmapl+AGV7tm/Rzx+w=; b=fLWgyhOU2KYQkTFFPbDhQ/bsBotEZEjkh9haBe8dXpHF0qU/WRG60TWo43HmfcRrHF cABLbZfLNeSXh0mLsdqIPC0454Up71P4leWJDy1GCjodbKeU9iHhPlLnZtJYVRa7d8a6 YVu43N+qdjfC4wqXIRMtTT0mXffK8UJ6cdiwNoTq0nA31X6bA3YeM2CRAzI+4qECeBK5 LTKcWFWSdKn8mL2wWCcQJNKmJuwMcMi2PqCnSkGyViL+3SgZyej8/rS0CpPq/LcOKALq rrImCGLuJ/pgrCsV3yUBCRqUyWcJR4FjMmMay/+PGhrA8vwQuGJKv/eRVQ2PjNxpBv+O fnBA== X-Gm-Message-State: AOAM533W9sXKg4j1V3J7RZUgLcN22UXfx8xmxlfaEbMHQ1D5/6ss42qt JkZ+KWU/Rah7D1ii81qfmfKyeg== X-Google-Smtp-Source: ABdhPJzOcQtcQr8kDnX5vfoKXrEejVF7HXmixwlBQw/BjGtWP2W5ppTBL/zygWmxy79MN6V3JkKj7Q== X-Received: by 2002:a17:90b:688:b0:1d9:9ddd:1f71 with SMTP id m8-20020a17090b068800b001d99ddd1f71mr9480283pjz.207.1651333075503; Sat, 30 Apr 2022 08:37:55 -0700 (PDT) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id y14-20020a62b50e000000b0050dc7628165sm1669724pfe.63.2022.04.30.08.37.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 08:37:55 -0700 (PDT) Subject: [PATCH v4 2/7] asm-generic: qspinlock: Indicate the use of mixed-size atomics Date: Sat, 30 Apr 2022 08:36:21 -0700 Message-Id: <20220430153626.30660-3-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220430153626.30660-1-palmer@rivosinc.com> References: <20220430153626.30660-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: guoren@kernel.org, peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , Greg KH , sudipm.mukherjee@gmail.com, macro@orcam.me.uk, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra The qspinlock implementation depends on having well behaved mixed-size atomics. This is true on the more widely-used platforms, but these requirements are somewhat subtle and may not be satisfied by all the platforms that qspinlock is used on. Document these requirements, so ports that use qspinlock can more easily determine if they meet these requirements. Signed-off-by: Peter Zijlstra (Intel) Acked-by: Waiman Long Signed-off-by: Palmer Dabbelt --- include/asm-generic/qspinlock.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinloc= k.h index d74b13825501..95be3f3c28b5 100644 --- a/include/asm-generic/qspinlock.h +++ b/include/asm-generic/qspinlock.h @@ -2,6 +2,37 @@ /* * Queued spinlock * + * A 'generic' spinlock implementation that is based on MCS locks. An + * architecture that's looking for a 'generic' spinlock, please first cons= ider + * ticket-lock.h and only come looking here when you've considered all the + * constraints below and can show your hardware does actually perform bett= er + * with qspinlock. + * + * + * It relies on atomic_*_release()/atomic_*_acquire() to be RCsc (or no we= aker + * than RCtso if you're power), where regular code only expects atomic_t t= o be + * RCpc. + * + * It relies on a far greater (compared to asm-generic/spinlock.h) set of + * atomic operations to behave well together, please audit them carefully = to + * ensure they all have forward progress. Many atomic operations may defau= lt to + * cmpxchg() loops which will not have good forward progress properties on + * LL/SC architectures. + * + * One notable example is atomic_fetch_or_acquire(), which x86 cannot (che= aply) + * do. Carefully read the patches that introduced + * queued_fetch_set_pending_acquire(). + * + * It also heavily relies on mixed size atomic operations, in specific it + * requires architectures to have xchg16; something which many LL/SC + * architectures need to implement as a 32bit and+or in order to satisfy t= he + * forward progress guarantees mentioned above. + * + * Further reading on mixed size atomics that might be relevant: + * + * http://www.cl.cam.ac.uk/~pes20/popl17/mixed-size.pdf + * + * * (C) Copyright 2013-2015 Hewlett-Packard Development Company, L.P. * (C) Copyright 2015 Hewlett-Packard Enterprise Development LP * --=20 2.34.1 From nobody Mon Jun 15 09:08:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B2F0C433EF for ; Sat, 30 Apr 2022 15:38:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1383029AbiD3Pl7 (ORCPT ); Sat, 30 Apr 2022 11:41:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1382955AbiD3PlY (ORCPT ); Sat, 30 Apr 2022 11:41:24 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CF1BA0BDF for ; Sat, 30 Apr 2022 08:37:57 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id gj17-20020a17090b109100b001d8b390f77bso12922607pjb.1 for ; Sat, 30 Apr 2022 08:37:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=GsO1g7Ro+rsf0io9vopg1aQfP2P2g91oLlrkEyQDXiM=; b=kF7TN9WBue7gogtLcWCSwM/BfmlmiiSeL5p6P+r9TFf7mJTPo7V03Vb2pERNLvb7OE aRiiolMYojHxd4x5cQCGpG9H2wT2LX9AGbU77WuAKd7VD9xTc7yVRQuxr9CmreoFTg1r PvNjg/njaujbrqAe+WzlaHXAa3YGjcCyPClqxCArzRUK1f4wnu8wIGOKQzuuPux3fanL dNudXAVOCI0w1bdRz0Bat3QH8ewK8iAbnP04kEkDwytD4R2TvR69JqSwtluXppkeLWp+ iJKg4ylgLJ4xSXkYkJB/5TkTjMKtqArr+XkBvDHr/ZIlgr8RD4A5+MLPtRKEzg0pyefW MJ6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=GsO1g7Ro+rsf0io9vopg1aQfP2P2g91oLlrkEyQDXiM=; b=SztZCg7Irw9DEZqONvxSKesn6z0ZcEF6APu61/lpWngVny6pAg1mr9gHQBjLrhZOdB 7QL7/GXN24KtnfS3LCjCiYGqT5pN73PUYKEbW2jp5hRXly9fLl0oUEyYc9np8hzR9WTM 3X9WbEX4zmrBJA8YRGrZBKTjcmFO0m0kdpU+7QOyDsavpsH0+zn5wbzkAjwWwHjb34Iv fY4fWzRtOUraGFgCfixPMiVu1BBBHemlhrwj3rcITri3JRVcTDIXX3MERqeuL9Gh3E7k GuhLiCEGt/iQLn3ZnTfrTLaD0ljFgHXpY67ILP3p7c4nu4ysvJo9IauuwSTC9A87gb7k BTYA== X-Gm-Message-State: AOAM5308ViM4rX0k2TCv3lgDAXYZtin/wPBjqDvxDwKONS80D0U01bxv qDLA161WEpm/VXzo4ZqYABpuh2cuDln5yQ== X-Google-Smtp-Source: ABdhPJzNtj8d/a+lfHQeb2reqO0YCZeJo7LPF+70BVvrDTXjKfwmfYC5tWlm1kzXph5Ui9RlcPxgHQ== X-Received: by 2002:a17:902:8d8e:b0:159:4f6:c4aa with SMTP id v14-20020a1709028d8e00b0015904f6c4aamr4142558plo.115.1651333076544; Sat, 30 Apr 2022 08:37:56 -0700 (PDT) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id m2-20020a17090a2c0200b001d7761ee6fcsm12923373pjd.3.2022.04.30.08.37.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 08:37:56 -0700 (PDT) Subject: [PATCH v4 3/7] asm-generic: qrwlock: Document the spinlock fairness requirements Date: Sat, 30 Apr 2022 08:36:22 -0700 Message-Id: <20220430153626.30660-4-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220430153626.30660-1-palmer@rivosinc.com> References: <20220430153626.30660-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: guoren@kernel.org, peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , Greg KH , sudipm.mukherjee@gmail.com, macro@orcam.me.uk, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt I could only find the fairness requirements documented as the C code, this calls them out in a comment just to be a bit more explicit. Signed-off-by: Palmer Dabbelt --- include/asm-generic/qrwlock.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/asm-generic/qrwlock.h b/include/asm-generic/qrwlock.h index 7ae0ece07b4e..24ae09c1db9f 100644 --- a/include/asm-generic/qrwlock.h +++ b/include/asm-generic/qrwlock.h @@ -2,6 +2,10 @@ /* * Queue read/write lock * + * These use generic atomic and locking routines, but depend on a fair spi= nlock + * implementation in order to be fair themselves. The implementation in + * asm-generic/spinlock.h meets these requirements. + * * (C) Copyright 2013-2014 Hewlett-Packard Development Company, L.P. * * Authors: Waiman Long --=20 2.34.1 From nobody Mon Jun 15 09:08:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6BFEC433EF for ; Sat, 30 Apr 2022 15:38:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1382963AbiD3Plt (ORCPT ); Sat, 30 Apr 2022 11:41:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242932AbiD3Plg (ORCPT ); Sat, 30 Apr 2022 11:41:36 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D1A8A0BE6 for ; Sat, 30 Apr 2022 08:37:58 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id 7so4774475pga.12 for ; Sat, 30 Apr 2022 08:37:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=z3KLQgMj1TGNiy68N/+4cjWSftL0fQ46WdCXDumEo+g=; b=OIjS69Rysux2zOuPu+QGyQyEdlRWBp2cAHY90hMkRDcU+740wg96FctIHu6gimhe37 lz3jiYHZzNFTdx32AfsrWEbeDRWge1y2JeufrIPVxrl4eUYhH7+j85QUjV4yTlpew1Qz O/Lwal75dK7+QounVjK9C7dYqAFShV4gJMNyJ1gWPoAIdrZQYBITK18BbXnd0idplft1 QHNO3WQkZIuBSzo2GgKyrMyVdDvBNibHbD/fYMkpOV4gf5DDYvB1VFIuM6r8Hp6RMADH N/4tYcc94hfUa/k8OhPS1Z3ZmLWbtys1gNxrDGeNmBnciOp1mBgdO5PLtUf77tbM7NSE nxrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=z3KLQgMj1TGNiy68N/+4cjWSftL0fQ46WdCXDumEo+g=; b=U6oNp0KI4hJs5wfTk7ikJlMuqd4O8xbqTXCT0g9I+PTb0P+2gN+TqIdJOwRSKk299z 2bAzTCL2/C8k0Swm0Pg0gVgmSyJoVcKLw3vvIVl05CcNCK60p4zUHdmPMZXYfPzvozBl Q+TrwHPoLWDSDYfRh26Ro3iVNXBx1fRVxFOSQ8NQ6HF7qHh+ju7q6J414OvhujDkuCaP 0walsvR9GoL5WaxwlRduKVOiRv/Mn3ixp7JfgS6RBmkKMTdSkIRwdYccYx3DabLOKXu3 Xa0x1ndrwx6Ncn8NW7hzyfbjle+xNUQGGayt60ZQm6Oice56Iq87p/F3Hld3kMPe4UHu j/PQ== X-Gm-Message-State: AOAM530Xg6GZ0r1E89RfkAvsKyhueh3Z6R+l4d1DEg2RxKGg/ciAvAx4 1zJ0bvcV5PD+XGnXL5cCSPCXxA== X-Google-Smtp-Source: ABdhPJwQ5vPL3rfJKlBBkun+N1EkTvaAg1pnc1AebUDuMRn0c3hata7ORS3NhPRk+anvBJZz91htfQ== X-Received: by 2002:a63:81c8:0:b0:3ab:6025:f43c with SMTP id t191-20020a6381c8000000b003ab6025f43cmr3472832pgd.189.1651333077608; Sat, 30 Apr 2022 08:37:57 -0700 (PDT) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id g11-20020a63110b000000b003c14af50614sm8108129pgl.44.2022.04.30.08.37.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 08:37:57 -0700 (PDT) Subject: [PATCH v4 4/7] openrisc: Move to ticket-spinlock Date: Sat, 30 Apr 2022 08:36:23 -0700 Message-Id: <20220430153626.30660-5-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220430153626.30660-1-palmer@rivosinc.com> References: <20220430153626.30660-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: guoren@kernel.org, peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , Greg KH , sudipm.mukherjee@gmail.com, macro@orcam.me.uk, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra We have no indications that openrisc meets the qspinlock requirements, so move to ticket-spinlock as that is more likey to be correct. Signed-off-by: Peter Zijlstra (Intel) Acked-by: Stafford Horne Signed-off-by: Palmer Dabbelt --- arch/openrisc/Kconfig | 1 - arch/openrisc/include/asm/Kbuild | 5 ++-- arch/openrisc/include/asm/spinlock.h | 27 ---------------------- arch/openrisc/include/asm/spinlock_types.h | 7 ------ 4 files changed, 2 insertions(+), 38 deletions(-) delete mode 100644 arch/openrisc/include/asm/spinlock.h delete mode 100644 arch/openrisc/include/asm/spinlock_types.h diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig index 0d68adf6e02b..99f0e4a4cbbd 100644 --- a/arch/openrisc/Kconfig +++ b/arch/openrisc/Kconfig @@ -30,7 +30,6 @@ config OPENRISC select HAVE_DEBUG_STACKOVERFLOW select OR1K_PIC select CPU_NO_EFFICIENT_FFS if !OPENRISC_HAVE_INST_FF1 - select ARCH_USE_QUEUED_SPINLOCKS select ARCH_USE_QUEUED_RWLOCKS select OMPIC if SMP select ARCH_WANT_FRAME_POINTERS diff --git a/arch/openrisc/include/asm/Kbuild b/arch/openrisc/include/asm/K= build index ca5987e11053..3386b9c1c073 100644 --- a/arch/openrisc/include/asm/Kbuild +++ b/arch/openrisc/include/asm/Kbuild @@ -1,9 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 generic-y +=3D extable.h generic-y +=3D kvm_para.h -generic-y +=3D mcs_spinlock.h -generic-y +=3D qspinlock_types.h -generic-y +=3D qspinlock.h +generic-y +=3D spinlock_types.h +generic-y +=3D spinlock.h generic-y +=3D qrwlock_types.h generic-y +=3D qrwlock.h generic-y +=3D user.h diff --git a/arch/openrisc/include/asm/spinlock.h b/arch/openrisc/include/a= sm/spinlock.h deleted file mode 100644 index 264944a71535..000000000000 --- a/arch/openrisc/include/asm/spinlock.h +++ /dev/null @@ -1,27 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-or-later */ -/* - * OpenRISC Linux - * - * Linux architectural port borrowing liberally from similar works of - * others. All original copyrights apply as per the original source - * declaration. - * - * OpenRISC implementation: - * Copyright (C) 2003 Matjaz Breskvar - * Copyright (C) 2010-2011 Jonas Bonn - * et al. - */ - -#ifndef __ASM_OPENRISC_SPINLOCK_H -#define __ASM_OPENRISC_SPINLOCK_H - -#include - -#include - -#define arch_spin_relax(lock) cpu_relax() -#define arch_read_relax(lock) cpu_relax() -#define arch_write_relax(lock) cpu_relax() - - -#endif diff --git a/arch/openrisc/include/asm/spinlock_types.h b/arch/openrisc/inc= lude/asm/spinlock_types.h deleted file mode 100644 index 7c6fb1208c88..000000000000 --- a/arch/openrisc/include/asm/spinlock_types.h +++ /dev/null @@ -1,7 +0,0 @@ -#ifndef _ASM_OPENRISC_SPINLOCK_TYPES_H -#define _ASM_OPENRISC_SPINLOCK_TYPES_H - -#include -#include - -#endif /* _ASM_OPENRISC_SPINLOCK_TYPES_H */ --=20 2.34.1 From nobody Mon Jun 15 09:08:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1748EC433EF for ; Sat, 30 Apr 2022 15:38:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1383021AbiD3Pl4 (ORCPT ); Sat, 30 Apr 2022 11:41:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1382943AbiD3Plh (ORCPT ); Sat, 30 Apr 2022 11:41:37 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A911A0BF3 for ; Sat, 30 Apr 2022 08:37:59 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id fv2so9440178pjb.4 for ; Sat, 30 Apr 2022 08:37:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=vsFDZUB/84NzhX1Z1IP0W5vQn6O0PaUxSIdB0QjeUDc=; b=r9PDVKcgm65Yi4vII/nk4Hbo/0VBC0YV7rdeo8wYExc1mL5i9iTQrvolkp1GVFLke7 +vCF/A5uqEZvh2csjfLPm8RWQsuHbKJX5kwLzm78bw0kFy42B6J4nK8MD/fKN48dI4bG 523x+G8tMmvMFwvBTnJ9CHVBhnmsDko/gzea7i1d/e4C2uq1pHJ2kE0F23EmSkojAL3L fd201kWeiUl5qiV4l0f1gNZUR0QSWEbHO12tcQN0BazqIjtOLG62PJBWmQZP4OcuE4Eb RxvgdLj+WdYFMnxwj+PMOBWi7eBUZ7BBl2ku0KO40Vcvdzu19+9wALmnOJee6SEHdHxN toNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=vsFDZUB/84NzhX1Z1IP0W5vQn6O0PaUxSIdB0QjeUDc=; b=OSrDLy8r4s5N3csAnCsTe3CuZkyutzdV7yBHciYs+/NHh+f9WEptN1lnhGbyEd+8da nNJa0CYjROLvHG86Mlrhx9M7odBOx3txx/roRCpsp9YYV8CnwIo4qFNWRu3MHR85sDNk VKtFHLvR+3ca6jImRwkBPBAjE0Oz/vXkb1U8AbhqXJh350reiVmjEEiSSpH+Xk+M9ijz mGTQ6M+mHpxFb25J4DEASmWLiVrQ/JLocS9CTAmjnot+fqMsW5ot6iypXn2a3O60gBt3 9WrQk2cI5T02bS7irRriQIewCgEwKtXiyiFPk0qjgawTdnAxN7SHt9neOSCLigO7WBbs oemA== X-Gm-Message-State: AOAM532OGiqbcZzYzsfyizoEthPGSlfHt3ftMzrkPZlBqgAUDt5cibDk H6+BtZH1KA4LJ2IRYOyO2j7EQw== X-Google-Smtp-Source: ABdhPJw/hFbzhIhDKerSo/xxSkM2N2Ke32oM0UbAy8C7RthY2xSvx/irL3sI6rRGhR1JZDHiupHHjA== X-Received: by 2002:a17:902:8b8a:b0:158:fbd0:45ab with SMTP id ay10-20020a1709028b8a00b00158fbd045abmr4283114plb.110.1651333078732; Sat, 30 Apr 2022 08:37:58 -0700 (PDT) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id i1-20020a17090332c100b0015e8d4eb1f4sm1630983plr.62.2022.04.30.08.37.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 08:37:58 -0700 (PDT) Subject: [PATCH v4 5/7] RISC-V: Move to generic spinlocks Date: Sat, 30 Apr 2022 08:36:24 -0700 Message-Id: <20220430153626.30660-6-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220430153626.30660-1-palmer@rivosinc.com> References: <20220430153626.30660-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: guoren@kernel.org, peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , Greg KH , sudipm.mukherjee@gmail.com, macro@orcam.me.uk, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt Our existing spinlocks aren't fair and replacing them has been on the TODO list for a long time. This moves to the recently-introduced ticket spinlocks, which are simple enough that they are likely to be correct and fast on the vast majority of extant implementations. This introduces a horrible hack that allows us to split out the spinlock conversion from the rwlock conversion. We have to do the spinlocks first because qrwlock needs fair spinlocks, but we don't want to pollute the asm-generic code to support the generic spinlocks without qrwlocks. Thus we pollute the RISC-V code, but just until the next commit as it's all going away. Signed-off-by: Palmer Dabbelt Reviewed-by: Guo Ren Tested-by: Conor Dooley Tested-by: Heiko Stuebner --- arch/riscv/include/asm/Kbuild | 2 ++ arch/riscv/include/asm/spinlock.h | 44 +++---------------------- arch/riscv/include/asm/spinlock_types.h | 9 +++-- 3 files changed, 10 insertions(+), 45 deletions(-) diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 5edf5b8587e7..c3f229ae8033 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -3,5 +3,7 @@ generic-y +=3D early_ioremap.h generic-y +=3D flat.h generic-y +=3D kvm_para.h generic-y +=3D parport.h +generic-y +=3D qrwlock.h +generic-y +=3D qrwlock_types.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spi= nlock.h index f4f7fa1b7ca8..88a4d5d0d98a 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -7,49 +7,13 @@ #ifndef _ASM_RISCV_SPINLOCK_H #define _ASM_RISCV_SPINLOCK_H =20 +/* This is horible, but the whole file is going away in the next commit. */ +#define __ASM_GENERIC_QRWLOCK_H + #include #include #include - -/* - * Simple spin lock operations. These provide no fairness guarantees. - */ - -/* FIXME: Replace this with a ticket lock, like MIPS. */ - -#define arch_spin_is_locked(x) (READ_ONCE((x)->lock) !=3D 0) - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - int tmp =3D 1, busy; - - __asm__ __volatile__ ( - " amoswap.w %0, %2, %1\n" - RISCV_ACQUIRE_BARRIER - : "=3Dr" (busy), "+A" (lock->lock) - : "r" (tmp) - : "memory"); - - return !busy; -} - -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - while (1) { - if (arch_spin_is_locked(lock)) - continue; - - if (arch_spin_trylock(lock)) - break; - } -} - -/***********************************************************/ +#include =20 static inline void arch_read_lock(arch_rwlock_t *lock) { diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/a= sm/spinlock_types.h index 5a35a49505da..f2f9b5d7120d 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -6,15 +6,14 @@ #ifndef _ASM_RISCV_SPINLOCK_TYPES_H #define _ASM_RISCV_SPINLOCK_TYPES_H =20 +/* This is horible, but the whole file is going away in the next commit. */ +#define __ASM_GENERIC_QRWLOCK_TYPES_H + #ifndef __LINUX_SPINLOCK_TYPES_RAW_H # error "please don't include this file directly" #endif =20 -typedef struct { - volatile unsigned int lock; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { 0 } +#include =20 typedef struct { volatile unsigned int lock; --=20 2.34.1 From nobody Mon Jun 15 09:08:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C15DBC433F5 for ; Sat, 30 Apr 2022 15:38:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1382984AbiD3PmR (ORCPT ); Sat, 30 Apr 2022 11:42:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1382964AbiD3Plh (ORCPT ); Sat, 30 Apr 2022 11:41:37 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E428A144C for ; Sat, 30 Apr 2022 08:38:00 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id o69so8082224pjo.3 for ; Sat, 30 Apr 2022 08:38:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=PhfY9SC+wYhs3i/pCnS1F0qe7XGea8rOjY/kEhC+LrE=; b=sCDHVOiLVN32Y8ICH+ZkCKw/InoCrY43xa27irUukeSvGPghb/JBYOw6oTCkeIBkNa +H/XBYvBx7CV0nBuiGA7CSeqpBzMXbIkhHwzR01gcHQLnmqthIK6PG5GbP5QbH/LT2af 9Em8Bhs6r45/fJ462zBxoDeJfmyc3zQOXCJQOLKUa7NZyWoNpE+p5rq+evLX48Wxqz7Z 8QuGzNeewDVFwR94Xv+ACR39i3XWraOwk4MmDN7l7tVP+OXZvX8sxAPK4yys2j11psjV Aa1jdjYTmV+9Er+vvhIB/vTozu0gkGHHg2IByGGcQHC5KnFPIj4uZM2dHaqz/yWVmr+t 6dBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=PhfY9SC+wYhs3i/pCnS1F0qe7XGea8rOjY/kEhC+LrE=; b=tWamvwph9pbHaBdnmWZK1Iwwmtl9B4H7VvW8Al5rv1sMB6+wJqHH6hssQ8xGFGOxl+ pvnZZGCrt8JOPy0dWcOk/oisFDX70sWWGXOfb/wKOWIcD7Q6D8yTm6YU1C66XlUAC5m5 yBbb88lezh4rgBHXczvnouCRlHJ45tT6bktnYY6i2+WzeHUOJPdNei4SR9F2wIlY3pzR DRgaUOW60e3IgxjxQtu3p6IeYpEs0p5bZ5j7t8UuhpSlL+Gdj8l1aNjd6VcX4cGO0CFK /kOFmZVhBXpqZNJbFR4jOknx+5VHM3P3J98JJknPtBK9iCwr8S7pnvXeMM/+U7LZOMb3 dAyg== X-Gm-Message-State: AOAM530Rdsjsgu/6S90w9KTQ+k7BsDx36tlfb4rZgr/HKnRIRlSM/Ooj BKrfqg/ucVGAG+yqMvWWOLaqsw== X-Google-Smtp-Source: ABdhPJyQzZr23BkCwG1+xSArqcEmCk72rbg9rU3h9fy701WXzCXMfDWtm+qk2uXLdC5yg4y6n4z3Nw== X-Received: by 2002:a17:902:b789:b0:15b:5d52:7542 with SMTP id e9-20020a170902b78900b0015b5d527542mr4396814pls.26.1651333079791; Sat, 30 Apr 2022 08:37:59 -0700 (PDT) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id x5-20020aa793a5000000b0050dc7628201sm1677478pff.219.2022.04.30.08.37.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 08:37:59 -0700 (PDT) Subject: [PATCH v4 6/7] RISC-V: Move to queued RW locks Date: Sat, 30 Apr 2022 08:36:25 -0700 Message-Id: <20220430153626.30660-7-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220430153626.30660-1-palmer@rivosinc.com> References: <20220430153626.30660-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: guoren@kernel.org, peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , Greg KH , sudipm.mukherjee@gmail.com, macro@orcam.me.uk, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt Now that we have fair spinlocks we can use the generic queued rwlocks, so we might as well do so. Signed-off-by: Palmer Dabbelt Reviewed-by: Guo Ren Tested-by: Conor Dooley Tested-by: Heiko Stuebner --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/Kbuild | 2 + arch/riscv/include/asm/spinlock.h | 99 ------------------------- arch/riscv/include/asm/spinlock_types.h | 24 ------ 4 files changed, 3 insertions(+), 123 deletions(-) delete mode 100644 arch/riscv/include/asm/spinlock.h delete mode 100644 arch/riscv/include/asm/spinlock_types.h diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 00fd9c548f26..f8a55d94016d 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -39,6 +39,7 @@ config RISCV select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU select ARCH_SUPPORTS_HUGETLBFS if MMU select ARCH_USE_MEMTEST + select ARCH_USE_QUEUED_RWLOCKS select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index c3f229ae8033..504f8b7e72d4 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -3,6 +3,8 @@ generic-y +=3D early_ioremap.h generic-y +=3D flat.h generic-y +=3D kvm_para.h generic-y +=3D parport.h +generic-y +=3D spinlock.h +generic-y +=3D spinlock_types.h generic-y +=3D qrwlock.h generic-y +=3D qrwlock_types.h generic-y +=3D user.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spi= nlock.h deleted file mode 100644 index 88a4d5d0d98a..000000000000 --- a/arch/riscv/include/asm/spinlock.h +++ /dev/null @@ -1,99 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (C) 2015 Regents of the University of California - * Copyright (C) 2017 SiFive - */ - -#ifndef _ASM_RISCV_SPINLOCK_H -#define _ASM_RISCV_SPINLOCK_H - -/* This is horible, but the whole file is going away in the next commit. */ -#define __ASM_GENERIC_QRWLOCK_H - -#include -#include -#include -#include - -static inline void arch_read_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1b\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=3D&r" (tmp) - :: "memory"); -} - -static inline void arch_write_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1b\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=3D&r" (tmp) - :: "memory"); -} - -static inline int arch_read_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1f\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=3D&r" (busy) - :: "memory"); - - return !busy; -} - -static inline int arch_write_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1f\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=3D&r" (busy) - :: "memory"); - - return !busy; -} - -static inline void arch_read_unlock(arch_rwlock_t *lock) -{ - __asm__ __volatile__( - RISCV_RELEASE_BARRIER - " amoadd.w x0, %1, %0\n" - : "+A" (lock->lock) - : "r" (-1) - : "memory"); -} - -static inline void arch_write_unlock(arch_rwlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -#endif /* _ASM_RISCV_SPINLOCK_H */ diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/a= sm/spinlock_types.h deleted file mode 100644 index f2f9b5d7120d..000000000000 --- a/arch/riscv/include/asm/spinlock_types.h +++ /dev/null @@ -1,24 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (C) 2015 Regents of the University of California - */ - -#ifndef _ASM_RISCV_SPINLOCK_TYPES_H -#define _ASM_RISCV_SPINLOCK_TYPES_H - -/* This is horible, but the whole file is going away in the next commit. */ -#define __ASM_GENERIC_QRWLOCK_TYPES_H - -#ifndef __LINUX_SPINLOCK_TYPES_RAW_H -# error "please don't include this file directly" -#endif - -#include - -typedef struct { - volatile unsigned int lock; -} arch_rwlock_t; - -#define __ARCH_RW_LOCK_UNLOCKED { 0 } - -#endif /* _ASM_RISCV_SPINLOCK_TYPES_H */ --=20 2.34.1 From nobody Mon Jun 15 09:08:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56EE5C433EF for ; Sat, 30 Apr 2022 15:38:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1383035AbiD3PmJ (ORCPT ); Sat, 30 Apr 2022 11:42:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1382970AbiD3Plh (ORCPT ); Sat, 30 Apr 2022 11:41:37 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6995DA1462 for ; Sat, 30 Apr 2022 08:38:01 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id r83so8669082pgr.2 for ; Sat, 30 Apr 2022 08:38:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=uLn9nEAVfqfhF99bZmCoDyMs8/O9emD7jJiMXxYFAyY=; b=jB62pUCVrsBBWgk3g/p2juBmwhI8a4Jh1x7TlcBNmDmJGAG6PaxZE+M3PZcJj+jg9T 0iwrwbwp72h6V88rQPMbzIZsP4IpGTz5q9yERP4YLef+W+tIHP7/gntK1Jk0sJFq+YCB NJbWXx+W69U1T0Z2EjSYZTt6T15hvtMQcnch/kf+XW1iP0aNrMhACS24jp2yFH8lyeI3 nkc+Wa+2NkrmLILV8dAzs1Z7Wxk4Irsxgf+XGdhN3xHKnqEGkwvBzBsU7nN9UMGk9AA4 AikUbAz22IfR5jO2puqZ66QF+AD8umrShdTRxEgul/QR7LqlgZSw01y2sJ4BW9oAvI9o FFLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=uLn9nEAVfqfhF99bZmCoDyMs8/O9emD7jJiMXxYFAyY=; b=kkjNEEe9G27sFk3A/BzR8YEmaWjefW7af30gjbU84iRVli6lXyhpFtTje3XsWKybXX uoKMTFvla3+1HgeHi/JgJJDfStb41TkXZio4t+gerSStdSC0c8Wfp+IcXaWUlGK0GI54 5syZK2JWLg1K5G6JWbPHJFQc9MHPEi7RWar1MQ9x/wXPXMXdCbL+6ZG1q9hXMSulRquC x528E2MXNWO+FJ9qQEZ/TR/UFynSEyIONBU/Blj2YppqX2aH2Nf/93GlnhVAZNyCNTVy XOb7edIk+yA3ij02oJ12ZN/2wtxIc/I9gFPtjYsOVmrwwAwUr2gniEWXIb8ScyJlSL8N Biag== X-Gm-Message-State: AOAM5328e5ylC/Gtva6ZQ7KrpW+ynGCuCq/nrolKddNjVYz+NdzhgWj6 jFKL3QwuvB5ChGMeWsia6PxYLw== X-Google-Smtp-Source: ABdhPJx/DqzRzvSgQ995MUeH3WVVmaYceu4MzC5GgHW89dbtErwZa8oZe5QDvyTn1qGcu3QALKuXVQ== X-Received: by 2002:a05:6a00:98b:b0:50d:ac70:c39a with SMTP id u11-20020a056a00098b00b0050dac70c39amr4083386pfg.25.1651333080906; Sat, 30 Apr 2022 08:38:00 -0700 (PDT) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id j4-20020a62b604000000b0050dc7628184sm1716696pff.94.2022.04.30.08.38.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 08:38:00 -0700 (PDT) Subject: [PATCH v4 7/7] csky: Move to generic ticket-spinlock Date: Sat, 30 Apr 2022 08:36:26 -0700 Message-Id: <20220430153626.30660-8-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220430153626.30660-1-palmer@rivosinc.com> References: <20220430153626.30660-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: guoren@kernel.org, peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , Greg KH , sudipm.mukherjee@gmail.com, macro@orcam.me.uk, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren , Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Guo Ren There is no benefit from custom implementation for ticket-spinlock, so move to generic ticket-spinlock for easy maintenance. Signed-off-by: Guo Ren Signed-off-by: Palmer Dabbelt --- arch/csky/include/asm/Kbuild | 3 + arch/csky/include/asm/spinlock.h | 89 -------------------------- arch/csky/include/asm/spinlock_types.h | 27 -------- 3 files changed, 3 insertions(+), 116 deletions(-) delete mode 100644 arch/csky/include/asm/spinlock.h delete mode 100644 arch/csky/include/asm/spinlock_types.h diff --git a/arch/csky/include/asm/Kbuild b/arch/csky/include/asm/Kbuild index 888248235c23..103207a58f97 100644 --- a/arch/csky/include/asm/Kbuild +++ b/arch/csky/include/asm/Kbuild @@ -3,7 +3,10 @@ generic-y +=3D asm-offsets.h generic-y +=3D extable.h generic-y +=3D gpio.h generic-y +=3D kvm_para.h +generic-y +=3D spinlock.h +generic-y +=3D spinlock_types.h generic-y +=3D qrwlock.h +generic-y +=3D qrwlock_types.h generic-y +=3D parport.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/csky/include/asm/spinlock.h b/arch/csky/include/asm/spinl= ock.h deleted file mode 100644 index 69f5aa249c5f..000000000000 --- a/arch/csky/include/asm/spinlock.h +++ /dev/null @@ -1,89 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ - -#ifndef __ASM_CSKY_SPINLOCK_H -#define __ASM_CSKY_SPINLOCK_H - -#include -#include - -/* - * Ticket-based spin-locking. - */ -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - arch_spinlock_t lockval; - u32 ticket_next =3D 1 << TICKET_NEXT; - u32 *p =3D &lock->lock; - u32 tmp; - - asm volatile ( - "1: ldex.w %0, (%2) \n" - " mov %1, %0 \n" - " add %0, %3 \n" - " stex.w %0, (%2) \n" - " bez %0, 1b \n" - : "=3D&r" (tmp), "=3D&r" (lockval) - : "r"(p), "r"(ticket_next) - : "cc"); - - while (lockval.tickets.next !=3D lockval.tickets.owner) - lockval.tickets.owner =3D READ_ONCE(lock->tickets.owner); - - smp_mb(); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - u32 tmp, contended, res; - u32 ticket_next =3D 1 << TICKET_NEXT; - u32 *p =3D &lock->lock; - - do { - asm volatile ( - " ldex.w %0, (%3) \n" - " movi %2, 1 \n" - " rotli %1, %0, 16 \n" - " cmpne %1, %0 \n" - " bt 1f \n" - " movi %2, 0 \n" - " add %0, %0, %4 \n" - " stex.w %0, (%3) \n" - "1: \n" - : "=3D&r" (res), "=3D&r" (tmp), "=3D&r" (contended) - : "r"(p), "r"(ticket_next) - : "cc"); - } while (!res); - - if (!contended) - smp_mb(); - - return !contended; -} - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_mb(); - WRITE_ONCE(lock->tickets.owner, lock->tickets.owner + 1); -} - -static inline int arch_spin_value_unlocked(arch_spinlock_t lock) -{ - return lock.tickets.owner =3D=3D lock.tickets.next; -} - -static inline int arch_spin_is_locked(arch_spinlock_t *lock) -{ - return !arch_spin_value_unlocked(READ_ONCE(*lock)); -} - -static inline int arch_spin_is_contended(arch_spinlock_t *lock) -{ - struct __raw_tickets tickets =3D READ_ONCE(lock->tickets); - - return (tickets.next - tickets.owner) > 1; -} -#define arch_spin_is_contended arch_spin_is_contended - -#include - -#endif /* __ASM_CSKY_SPINLOCK_H */ diff --git a/arch/csky/include/asm/spinlock_types.h b/arch/csky/include/asm= /spinlock_types.h deleted file mode 100644 index db87a12c3827..000000000000 --- a/arch/csky/include/asm/spinlock_types.h +++ /dev/null @@ -1,27 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ - -#ifndef __ASM_CSKY_SPINLOCK_TYPES_H -#define __ASM_CSKY_SPINLOCK_TYPES_H - -#ifndef __LINUX_SPINLOCK_TYPES_RAW_H -# error "please don't include this file directly" -#endif - -#define TICKET_NEXT 16 - -typedef struct { - union { - u32 lock; - struct __raw_tickets { - /* little endian */ - u16 owner; - u16 next; - } tickets; - }; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { { 0 } } - -#include - -#endif /* __ASM_CSKY_SPINLOCK_TYPES_H */ --=20 2.34.1