From nobody Mon Jun 22 21:20:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 812D3C433EF for ; Wed, 16 Mar 2022 23:28:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347429AbiCPX3v (ORCPT ); Wed, 16 Mar 2022 19:29:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346164AbiCPX3m (ORCPT ); Wed, 16 Mar 2022 19:29:42 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20640167CA for ; Wed, 16 Mar 2022 16:28:26 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id z16so5409261pfh.3 for ; Wed, 16 Mar 2022 16:28:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=ofUFlPn1fFr6Pz05IWJLvQM8LyheEnwZgHIcbcC4aPw=; b=S5EoqvKolpNViVFVC6Nzsj4aBgAWTlxXUWEEMY7p3wEb0057ehZpzaUnP8f4GDTFJs zse0vx0vaol845GXW+EPh58H5hafan3CLwS2NebIMBjef8U0HNWEx3yRz3jPiV3uJlEC YXWJrMiwVe+oiaYZPhnxkOPG0gNmZ72sB2JgKC7u3KkkHP6Mrza8LqpBn+wgIhU0+wXO eiGMxzeY0dkF1fMYunyzzQlM6whMlxgT8yXppR7Shy3VF4INxmMTGDdTecXUZiBbWpJi KxadBRMOObu8VavJ+IpQtZPAu2dQ5o1nwyDfes8oJeqdvEAFgSJuP+TqCD4jDajWtIKl w7BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=ofUFlPn1fFr6Pz05IWJLvQM8LyheEnwZgHIcbcC4aPw=; b=BL22UWWYq/R1FVs33xjBhAAQnjSTPTpeQdrWG5KZmMXmZYZD/IIV3h2OesYq5Bu84c L8aLDtKxMTctvwXH7OkiJyK5ehrsngCPHlqStn8w6k5f4quxcGsmdYXvVAYKMsHLAHp7 ajBvJ8pUrPlgqIRpZoMScMmu5tOdzDR1IM5DJLuNzI10EB3efrQp+M0hpwyqgBPe1W4p Oo07hpu8WEZREk8GhjR1le/whZneu9vHfo011RfMJS2RlT/+n0xoY5dtUDSvyiLqn+VL FHR+bY/YkkR6LyGRhmO6LBY74Ift/tTzdkiy8bSFS2ch42Uj/UbsKjfkF/KejnzxyHrg VdLA== X-Gm-Message-State: AOAM531UEYfomkb3TMjnc0ZjVOBpW6YTHbQfSudxGRSNUCWR1YvLv6Og fq7FYR1OANm9T1avUABo9eKjtA== X-Google-Smtp-Source: ABdhPJxYRrhM5pW5WVasmxAVWyXeEsoX5VSiHkFO1pWtO6NNC1CxKjdOixv6tMKvEgnHK4ZptvG9zQ== X-Received: by 2002:a63:5855:0:b0:380:a9f7:e373 with SMTP id i21-20020a635855000000b00380a9f7e373mr1418979pgm.557.1647473305604; Wed, 16 Mar 2022 16:28:25 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id l186-20020a633ec3000000b003820485172asm145763pga.65.2022.03.16.16.28.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:25 -0700 (PDT) Subject: [PATCH 1/5] asm-generic: qspinlock: Indicate the use of mixed-size atomics Date: Wed, 16 Mar 2022 16:25:56 -0700 Message-Id: <20220316232600.20419-2-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra The qspinlock implementation depends on having well behaved mixed-size atomics. This is true on the more widely-used platforms, but these requirements are somewhat subtle and may not be satisfied by all the platforms that qspinlock is used on. Document these requirements, so ports that use qspinlock can more easily determine if they meet these requirements. Signed-off-by: Palmer Dabbelt Acked-by: Waiman Long Tested-by: Guo Ren Tested-by: Heiko Stuebner --- I have specifically not included Peter's SOB on this, as he sent his original patch without one. --- include/asm-generic/qspinlock.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinloc= k.h index d74b13825501..a7a1296b0b4d 100644 --- a/include/asm-generic/qspinlock.h +++ b/include/asm-generic/qspinlock.h @@ -2,6 +2,36 @@ /* * Queued spinlock * + * A 'generic' spinlock implementation that is based on MCS locks. An + * architecture that's looking for a 'generic' spinlock, please first cons= ider + * ticket-lock.h and only come looking here when you've considered all the + * constraints below and can show your hardware does actually perform bett= er + * with qspinlock. + * + * + * It relies on atomic_*_release()/atomic_*_acquire() to be RCsc (or no we= aker + * than RCtso if you're power), where regular code only expects atomic_t t= o be + * RCpc. + * + * It relies on a far greater (compared to ticket-lock.h) set of atomic + * operations to behave well together, please audit them carefully to ensu= re + * they all have forward progress. Many atomic operations may default to + * cmpxchg() loops which will not have good forward progress properties on + * LL/SC architectures. + * + * One notable example is atomic_fetch_or_acquire(), which x86 cannot (che= aply) + * do. Carefully read the patches that introduced queued_fetch_set_pending= _acquire(). + * + * It also heavily relies on mixed size atomic operations, in specific it + * requires architectures to have xchg16; something which many LL/SC + * architectures need to implement as a 32bit and+or in order to satisfy t= he + * forward progress guarantees mentioned above. + * + * Further reading on mixed size atomics that might be relevant: + * + * http://www.cl.cam.ac.uk/~pes20/popl17/mixed-size.pdf + * + * * (C) Copyright 2013-2015 Hewlett-Packard Development Company, L.P. * (C) Copyright 2015 Hewlett-Packard Enterprise Development LP * --=20 2.34.1 From nobody Mon Jun 22 21:20:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2617C433EF for ; Wed, 16 Mar 2022 23:28:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348143AbiCPX3z (ORCPT ); Wed, 16 Mar 2022 19:29:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346296AbiCPX3m (ORCPT ); Wed, 16 Mar 2022 19:29:42 -0400 Received: from mail-pg1-x52a.google.com (mail-pg1-x52a.google.com [IPv6:2607:f8b0:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7010D167CC for ; Wed, 16 Mar 2022 16:28:27 -0700 (PDT) Received: by mail-pg1-x52a.google.com with SMTP id e6so1310528pgn.2 for ; Wed, 16 Mar 2022 16:28:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=FX0vp498CDh+Q8VYRYD8Az++zOHOkwOZmBFCOJ/MPQk=; b=g7RGlpUtsYp0dL6ZURcJ9XqT0FTjuYFUg3R+dEpF965ZohDKCeoeGHfjtjiVSkGxEj lANdfMQl9/IBkJ/9UMg+SPHYCtMWSo9c4xaW9U1+sxAE128S6sObkonPfJPoKIzymiIw 09sK8rm+D+QlLwCSg7j5kwRZZ5Ze8MI5Jek627pUrimcrgKq4K2jJlUa/Pj8a2SElTV9 xl4ffwNapdts7992kA3feRtQzz/jnL60ZUCdKR4LQ7AoV/Y7AfVaxzeYDr7myiwH00f4 SRKTk+WG9MddPkkdOopQzO/1xBaFNEzmYY2D9huskq0oyJ4kYPwaGuIBtGRfGZgtAFr0 gBJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=FX0vp498CDh+Q8VYRYD8Az++zOHOkwOZmBFCOJ/MPQk=; b=uuwZ9zRYcUGFDPJ0diuyzpV0BnvR1TaXD7gTlFkTDXJhvet77rJdi08aNjzcKsFIbp W7uv28svj6yrPc6nciI57yUpMM1t1/mlEMcdftUmPMnidPpOWd3T1mXMpR1ONU8Oe5+u P35zVwmivUALfXnVk0X4eawLerXEBm9/UBxQ5ERj39GfRANy4mgCztQFpNWyBR+cAInb uxIpIWlvlJR/1wnMvGxrDZgeF1LKXxfPSGy+msgdMD5m7eSZOh/bLeFhVLA+UmNWAaL/ CHOb9wL+TDHZI45+Rd4hlmLQX1QDMiCqf2agORsS6EGNNbs2zlwo/C2J1W3N5QDqn2UQ qxMw== X-Gm-Message-State: AOAM533Jl4XsU40MpFKFjYPCe8C7QeyLU3prD7+wa5UB27aPy3juprck BVhJ3JI9ugdod+4rp5kp8N4qag== X-Google-Smtp-Source: ABdhPJxPEnRTV51QRb/xegbfDS8UXc6ttxfzGlax57AVGxnNF0PWp6AH0Fu+hZMGJACkhYsvidNMYQ== X-Received: by 2002:a05:6a00:729:b0:4f7:77ed:c256 with SMTP id 9-20020a056a00072900b004f777edc256mr2013933pfm.1.1647473306932; Wed, 16 Mar 2022 16:28:26 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id ce3-20020a17090aff0300b001c51f47840csm3560019pjb.0.2022.03.16.16.28.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:26 -0700 (PDT) Subject: [PATCH 2/5] asm-generic: ticket-lock: New generic ticket-based spinlock Date: Wed, 16 Mar 2022 16:25:57 -0700 Message-Id: <20220316232600.20419-3-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra This is a simple, fair spinlock. Specifically it doesn't have all the subtle memory model dependencies that qspinlock has, which makes it more suitable for simple systems as it is more likely to be correct. [Palmer: commit text] Signed-off-by: Palmer Dabbelt -- I have specifically not included Peter's SOB on this, as he sent his original patch without one. Acked-by: Waiman Long Tested-by: Guo Ren Tested-by: Heiko Stuebner --- include/asm-generic/ticket-lock-types.h | 11 ++++ include/asm-generic/ticket-lock.h | 86 +++++++++++++++++++++++++ 2 files changed, 97 insertions(+) create mode 100644 include/asm-generic/ticket-lock-types.h create mode 100644 include/asm-generic/ticket-lock.h diff --git a/include/asm-generic/ticket-lock-types.h b/include/asm-generic/= ticket-lock-types.h new file mode 100644 index 000000000000..829759aedda8 --- /dev/null +++ b/include/asm-generic/ticket-lock-types.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_GENERIC_TICKET_LOCK_TYPES_H +#define __ASM_GENERIC_TICKET_LOCK_TYPES_H + +#include +typedef atomic_t arch_spinlock_t; + +#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0) + +#endif /* __ASM_GENERIC_TICKET_LOCK_TYPES_H */ diff --git a/include/asm-generic/ticket-lock.h b/include/asm-generic/ticket= -lock.h new file mode 100644 index 000000000000..3f0d53e21a37 --- /dev/null +++ b/include/asm-generic/ticket-lock.h @@ -0,0 +1,86 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * 'Generic' ticket-lock implementation. + * + * It relies on atomic_fetch_add() having well defined forward progress + * guarantees under contention. If your architecture cannot provide this, = stick + * to a test-and-set lock. + * + * It also relies on atomic_fetch_add() being safe vs smp_store_release() = on a + * sub-word of the value. This is generally true for anything LL/SC althou= gh + * you'd be hard pressed to find anything useful in architecture specifica= tions + * about this. If your architecture cannot do this you might be better off= with + * a test-and-set. + * + * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and = hence + * uses atomic_fetch_add() which is SC to create an RCsc lock. + * + * The implementation uses smp_cond_load_acquire() to spin, so if the + * architecture has WFE like instructions to sleep instead of poll for word + * modifications be sure to implement that (see ARM64 for example). + * + */ + +#ifndef __ASM_GENERIC_TICKET_LOCK_H +#define __ASM_GENERIC_TICKET_LOCK_H + +#include +#include + +static __always_inline void ticket_lock(arch_spinlock_t *lock) +{ + u32 val =3D atomic_fetch_add(1<<16, lock); /* SC, gives us RCsc */ + u16 ticket =3D val >> 16; + + if (ticket =3D=3D (u16)val) + return; + + atomic_cond_read_acquire(lock, ticket =3D=3D (u16)VAL); +} + +static __always_inline bool ticket_trylock(arch_spinlock_t *lock) +{ + u32 old =3D atomic_read(lock); + + if ((old >> 16) !=3D (old & 0xffff)) + return false; + + return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */ +} + +static __always_inline void ticket_unlock(arch_spinlock_t *lock) +{ + u16 *ptr =3D (u16 *)lock + __is_defined(__BIG_ENDIAN); + u32 val =3D atomic_read(lock); + + smp_store_release(ptr, (u16)val + 1); +} + +static __always_inline int ticket_is_locked(arch_spinlock_t *lock) +{ + u32 val =3D atomic_read(lock); + + return ((val >> 16) !=3D (val & 0xffff)); +} + +static __always_inline int ticket_is_contended(arch_spinlock_t *lock) +{ + u32 val =3D atomic_read(lock); + + return (s16)((val >> 16) - (val & 0xffff)) > 1; +} + +static __always_inline int ticket_value_unlocked(arch_spinlock_t lock) +{ + return !ticket_is_locked(&lock); +} + +#define arch_spin_lock(l) ticket_lock(l) +#define arch_spin_trylock(l) ticket_trylock(l) +#define arch_spin_unlock(l) ticket_unlock(l) +#define arch_spin_is_locked(l) ticket_is_locked(l) +#define arch_spin_is_contended(l) ticket_is_contended(l) +#define arch_spin_value_unlocked(l) ticket_value_unlocked(l) + +#endif /* __ASM_GENERIC_TICKET_LOCK_H */ --=20 2.34.1 From nobody Mon Jun 22 21:20:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3272C433F5 for ; Wed, 16 Mar 2022 23:28:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239437AbiCPX35 (ORCPT ); Wed, 16 Mar 2022 19:29:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238816AbiCPX3t (ORCPT ); Wed, 16 Mar 2022 19:29:49 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B9FE167D3 for ; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id q19so1311674pgm.6 for ; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=TemgJXj/dxR4o9xjuRPFVAtvCpC+RuCkyAl1o75jaTw=; b=5NAASYUylgaEPqqdOEpfJYNUGMgV0gFaeMRax5C5dSSZLOGAdqEJZnhq0wtD7zPoW0 G5ob4+N2yFpwbpch0DWhTknq6SOGVQuAlyx5O3Q4z+CabPYf0Q0nUxJYIRUm+1UXldia CUfn33UFTj/UQ6niZ6ytLeUqcODuQ8vUjiiE5/y4Cga4QKby77CDm6r7N0bPNRi39KC4 O/zvHvMm0khtUBVhRQ1vvm59igRk3I3+zGiCD9QZkEZ4zSYXLICNZRMaGnxXlOOI5pc8 96R6MSsMUafEA9jSOCcTsTp6f+fYDXiBsUqlxz1cBR4CM7sz772W9JvY9llYBqeaxTjL 2xnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=TemgJXj/dxR4o9xjuRPFVAtvCpC+RuCkyAl1o75jaTw=; b=Yj3W0MsKldy7WJXG/g5Bqf6h1B8I9RPtZmYBZEol1pn3HfsnK3Dk7HizS9cT5foSNM ZPDOiNI75OO7mhJ94q4StlW7EGCeTbsxAcQLcZPdErmR/BDLppaAM/1aOoiXKNVjz7tw jhebh8wibsEWzterezcG9I131yw1GcMitmQfJp0Iy+LOibMvoAZURSVPu/wMLlbTP8AL dO13P4GS5D85xmtEs+GQS9Y2KVLrNWQq1KD8oU4Z1Df3eX3RwJ0RCu1TdyzfcSACPjv4 99w8Vgl/+/cKCCRtUsThtTBNHKlSne68VnQszVoNPErUMbZlz2uM7fsQj994VMfF5lo1 l/Vg== X-Gm-Message-State: AOAM531+LtMFmemuzGEiPZFyUxFWK7uIJEsfJMgiEAumBxoO1EXVUcp+ Q9E2ZchpvUHrULUVlGs0mLZSL30oOfcG5w== X-Google-Smtp-Source: ABdhPJzu2zL3dDL4hiAgSK4z9Dcw0idEICTak8foXBS3lSnxmO9pbBmuyGxzNgKWSyVr5slYVp3eQg== X-Received: by 2002:a05:6a00:1a89:b0:4f7:b90b:17ee with SMTP id e9-20020a056a001a8900b004f7b90b17eemr2233942pfv.46.1647473308139; Wed, 16 Mar 2022 16:28:28 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id d16-20020a056a00245000b004f7728a4346sm4436763pfj.79.2022.03.16.16.28.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:27 -0700 (PDT) Subject: [PATCH 3/5] openrisc: Move to ticket-spinlock Date: Wed, 16 Mar 2022 16:25:58 -0700 Message-Id: <20220316232600.20419-4-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra We have no indications that openrisc meets the qspinlock requirements, so move to ticket-spinlock as that is more likey to be correct. Signed-off-by: Palmer Dabbelt Tested-by: Guo Ren Tested-by: Heiko Stuebner --- I have specifically not included Peter's SOB on this, as he sent his original patch without one. --- arch/openrisc/Kconfig | 1 - arch/openrisc/include/asm/Kbuild | 5 ++--- arch/openrisc/include/asm/spinlock.h | 3 +-- arch/openrisc/include/asm/spinlock_types.h | 2 +- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig index f724b3f1aeed..f5fa226362f6 100644 --- a/arch/openrisc/Kconfig +++ b/arch/openrisc/Kconfig @@ -30,7 +30,6 @@ config OPENRISC select HAVE_DEBUG_STACKOVERFLOW select OR1K_PIC select CPU_NO_EFFICIENT_FFS if !OPENRISC_HAVE_INST_FF1 - select ARCH_USE_QUEUED_SPINLOCKS select ARCH_USE_QUEUED_RWLOCKS select OMPIC if SMP select ARCH_WANT_FRAME_POINTERS diff --git a/arch/openrisc/include/asm/Kbuild b/arch/openrisc/include/asm/K= build index ca5987e11053..cb260e7d73db 100644 --- a/arch/openrisc/include/asm/Kbuild +++ b/arch/openrisc/include/asm/Kbuild @@ -1,9 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 generic-y +=3D extable.h generic-y +=3D kvm_para.h -generic-y +=3D mcs_spinlock.h -generic-y +=3D qspinlock_types.h -generic-y +=3D qspinlock.h +generic-y +=3D ticket-lock.h +generic-y +=3D ticket-lock-types.h generic-y +=3D qrwlock_types.h generic-y +=3D qrwlock.h generic-y +=3D user.h diff --git a/arch/openrisc/include/asm/spinlock.h b/arch/openrisc/include/a= sm/spinlock.h index 264944a71535..40e4c9fdc349 100644 --- a/arch/openrisc/include/asm/spinlock.h +++ b/arch/openrisc/include/asm/spinlock.h @@ -15,8 +15,7 @@ #ifndef __ASM_OPENRISC_SPINLOCK_H #define __ASM_OPENRISC_SPINLOCK_H =20 -#include - +#include #include =20 #define arch_spin_relax(lock) cpu_relax() diff --git a/arch/openrisc/include/asm/spinlock_types.h b/arch/openrisc/inc= lude/asm/spinlock_types.h index 7c6fb1208c88..58ea31fa65ce 100644 --- a/arch/openrisc/include/asm/spinlock_types.h +++ b/arch/openrisc/include/asm/spinlock_types.h @@ -1,7 +1,7 @@ #ifndef _ASM_OPENRISC_SPINLOCK_TYPES_H #define _ASM_OPENRISC_SPINLOCK_TYPES_H =20 -#include +#include #include =20 #endif /* _ASM_OPENRISC_SPINLOCK_TYPES_H */ --=20 2.34.1 From nobody Mon Jun 22 21:20:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 742EEC433EF for ; Wed, 16 Mar 2022 23:28:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348557AbiCPXaB (ORCPT ); Wed, 16 Mar 2022 19:30:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346844AbiCPX3t (ORCPT ); Wed, 16 Mar 2022 19:29:49 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8DD4167D5 for ; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id m11-20020a17090a7f8b00b001beef6143a8so3921734pjl.4 for ; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=QjUbWpYW3rVnM/AEVOfQY7Hb1962iPWLkT7GwNacZrs=; b=OPswRH2lRihBIfhRfn+/3FQDyS4CIDrFb0a6W/SFptnXwSb197U4E15ipwXAHAI0Ta TNyvuSpdp0Dl7Udy9D2GfQMyBJv1EAU0uSev1KuzOjG/5dK0WBhl8Mbqd7ahL1zjtniM Y1SqPUOdGayiPKAU6YQE0nUK5pJvsw2r8oicK3P3V5ZhK6gt2WLY4clS8KYMvMnsh/qH t1+CgGbeMH5nQ/NzMh338/CKlZC6QrLpB6Ol4xXtu3THVc5XuN017XXitVKtMXD9lQqc x1VUjFX+iePIisj0C7WmE0/fu1p8BI2Bk3rrSnzxy2JNeg8GQP2Fx2KEuKzIbkX7wL9d r1iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=QjUbWpYW3rVnM/AEVOfQY7Hb1962iPWLkT7GwNacZrs=; b=iNY09XbLEdYuMceKfyx1Jo+RtYcumk0vqA/jqypkG8DZThzk2NfnTpORxIaUvxeKlr P5hoTnp7Me9/JqrMo2LgPOkXgWK7TeSF0Fkq8K+2y/JtXPcy161M0led8rj0RXdpiqCZ NWvAsiEX9K0dqjvBjylIx/HuL1m2B71yB3p33C6BMJK35luuIsO0A+Xefa3EMItACZpr /jD97cCQrXeJzkR/D2/8FG/NODfF/ZWP7hynfLaj6qkuqIuHOPpylKnSg8ppZTZ4P2s5 C+H9AfdS5s9RD71gQbeChV/ALIKJEPjZVIf+91uiYM2ztQ0j+dtwFPPObKEfoHks5AJH UIuA== X-Gm-Message-State: AOAM531fbJd+4ZGzD5k3AXQFkklhNkiHiAFL495VUtSMRDeNpvup7duy prRaobRSNzYXPsJ5RfbWtYLjng== X-Google-Smtp-Source: ABdhPJx15zZafntzrXc+qFtN7TCKqNirxAH51x/OX1Z7DAEkhf7MHHxu+PKBTGIRQSByMNHXy1V3iA== X-Received: by 2002:a17:903:1206:b0:151:7d67:2924 with SMTP id l6-20020a170903120600b001517d672924mr1783497plh.45.1647473309449; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id pj13-20020a17090b4f4d00b001bf2ff56430sm8078615pjb.30.2022.03.16.16.28.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:29 -0700 (PDT) Subject: [PATCH 4/5] RISC-V: Move to ticket-spinlocks Date: Wed, 16 Mar 2022 16:25:59 -0700 Message-Id: <20220316232600.20419-5-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt Our existing spinlocks aren't fair and replacing them has been on the TODO list for a long time. This moves to the recently-introduced ticket spinlocks, which are simple enough that they are likely to be correct and fast on the vast majority of extant implementations. Signed-off-by: Palmer Dabbelt Tested-by: Guo Ren Tested-by: Heiko Stuebner --- arch/riscv/include/asm/Kbuild | 2 ++ arch/riscv/include/asm/spinlock.h | 41 +------------------------ arch/riscv/include/asm/spinlock_types.h | 6 +--- 3 files changed, 4 insertions(+), 45 deletions(-) diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 57b86fd9916c..42b1961af1a6 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -2,5 +2,7 @@ generic-y +=3D early_ioremap.h generic-y +=3D flat.h generic-y +=3D kvm_para.h +generic-y +=3D ticket-lock.h +generic-y +=3D ticket-lock-types.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spi= nlock.h index f4f7fa1b7ca8..38089cbdea92 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -10,46 +10,7 @@ #include #include #include - -/* - * Simple spin lock operations. These provide no fairness guarantees. - */ - -/* FIXME: Replace this with a ticket lock, like MIPS. */ - -#define arch_spin_is_locked(x) (READ_ONCE((x)->lock) !=3D 0) - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - int tmp =3D 1, busy; - - __asm__ __volatile__ ( - " amoswap.w %0, %2, %1\n" - RISCV_ACQUIRE_BARRIER - : "=3Dr" (busy), "+A" (lock->lock) - : "r" (tmp) - : "memory"); - - return !busy; -} - -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - while (1) { - if (arch_spin_is_locked(lock)) - continue; - - if (arch_spin_trylock(lock)) - break; - } -} - -/***********************************************************/ +#include =20 static inline void arch_read_lock(arch_rwlock_t *lock) { diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/a= sm/spinlock_types.h index 5a35a49505da..431ee08e26c4 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -10,11 +10,7 @@ # error "please don't include this file directly" #endif =20 -typedef struct { - volatile unsigned int lock; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { 0 } +#include =20 typedef struct { volatile unsigned int lock; --=20 2.34.1 From nobody Mon Jun 22 21:20:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9D3EC433EF for ; Wed, 16 Mar 2022 23:28:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347616AbiCPXaF (ORCPT ); Wed, 16 Mar 2022 19:30:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347074AbiCPX3t (ORCPT ); Wed, 16 Mar 2022 19:29:49 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EC0B167EC for ; Wed, 16 Mar 2022 16:28:31 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id e3so3443987pjm.5 for ; Wed, 16 Mar 2022 16:28:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=llt+unTQxxB47mMK9cTVf/sVk7YwdP8ahVPZ8vUzo+E=; b=P13QgI0zd+4RAOt3pbwZknViRr2CVhiVAFIRqM5bOGjAu9nzvjSnZusFhYRC5h9qSQ 9JCxra+cscVH+cq1tVm8CjiuMJlxW877K2j6AlCHNuh0DRBNi9EE4AcotfiNoC3XOOLK dhnYjAvGq4qBYfG/ws5BM4JI/1zMJoKWHl+gPx8IkFDWC/CuJ+vSOxvmg3M8IpV/BOKi Oz6nK2qVkXNZ02AyIkvsE3D3PHdimNMIXmdl8ruKlmv4JHoGQeGhFIjnxn/Mq7G4QX+y 2JDewQpoUe01b9wf6IjFRTdrHwJVTBH6ES8hoo446YGPwdfpUHyka6ajuquwNdtxfM3P F7kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=llt+unTQxxB47mMK9cTVf/sVk7YwdP8ahVPZ8vUzo+E=; b=6Bq+TyJWQqDevq4YeeVQngyKLFR1vym7WE8s78CkDSorK0Xetj4wC0MWuQanm8OD2z E4W46WAOvTEqK5zZE/IMYcworSJp9Mfgztw7717COb+j/+Tgm1GV5Eizv8b1Hq0iNwPb EJ3DrbhvZhKO2qlnD+UV5fpu2pXXvroy2AMBJjrETQtNK+k4sX4DN6v2zU5MYr1WdNHd oYyDYNmbxJ2KD6i4Aht9kexDQnXxEuHdPeGdQH8oV571xAWNp8xlZa9cQCRXpgEZX759 5BnejUHErvGalY0TeFf9XAP7cYtbDufDvB9mA9OyfBaXIsmWT90AGZCfi3y8EZiCN4lR T+1Q== X-Gm-Message-State: AOAM532IQZbJqoSqelUV9bXG55Oy9jZFxmgxbomVsWOPQhrD3wlmt0dI zwdZHSf4pXT/qhALN02pVHa3Dg== X-Google-Smtp-Source: ABdhPJzyPV7prsqRJmPgkY9DrfVu2bdBaNJER6RIDfsjbbPdF+/QlL1mJF+2to5VA22BRA7tJfuvxA== X-Received: by 2002:a17:903:11c9:b0:151:9521:d5c7 with SMTP id q9-20020a17090311c900b001519521d5c7mr1923340plh.73.1647473310711; Wed, 16 Mar 2022 16:28:30 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id o5-20020a056a0015c500b004f7988f16c3sm4696034pfu.30.2022.03.16.16.28.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:30 -0700 (PDT) Subject: [PATCH 5/5] RISC-V: Move to queued RW locks Date: Wed, 16 Mar 2022 16:26:00 -0700 Message-Id: <20220316232600.20419-6-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt With the move to fair spinlocks, we might as well move to fair rwlocks. Signed-off-by: Palmer Dabbelt Tested-by: Guo Ren Tested-by: Heiko Stuebner --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/Kbuild | 2 + arch/riscv/include/asm/spinlock.h | 82 +------------------------ arch/riscv/include/asm/spinlock_types.h | 7 +-- 4 files changed, 5 insertions(+), 87 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 5adcbd9b5e88..feb7030cfb6d 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -38,6 +38,7 @@ config RISCV select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU select ARCH_SUPPORTS_HUGETLBFS if MMU select ARCH_USE_MEMTEST + select ARCH_USE_QUEUED_RWLOCKS select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_HUGE_PMD_SHARE if 64BIT diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 42b1961af1a6..e8714070cbb9 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -4,5 +4,7 @@ generic-y +=3D flat.h generic-y +=3D kvm_para.h generic-y +=3D ticket-lock.h generic-y +=3D ticket-lock-types.h +generic-y +=3D qrwlock.h +generic-y +=3D qrwlock_types.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spi= nlock.h index 38089cbdea92..97dfb150d18c 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -11,86 +11,6 @@ #include #include #include - -static inline void arch_read_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1b\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=3D&r" (tmp) - :: "memory"); -} - -static inline void arch_write_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1b\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=3D&r" (tmp) - :: "memory"); -} - -static inline int arch_read_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1f\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=3D&r" (busy) - :: "memory"); - - return !busy; -} - -static inline int arch_write_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1f\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=3D&r" (busy) - :: "memory"); - - return !busy; -} - -static inline void arch_read_unlock(arch_rwlock_t *lock) -{ - __asm__ __volatile__( - RISCV_RELEASE_BARRIER - " amoadd.w x0, %1, %0\n" - : "+A" (lock->lock) - : "r" (-1) - : "memory"); -} - -static inline void arch_write_unlock(arch_rwlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} +#include =20 #endif /* _ASM_RISCV_SPINLOCK_H */ diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/a= sm/spinlock_types.h index 431ee08e26c4..3779f13706fa 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -11,11 +11,6 @@ #endif =20 #include - -typedef struct { - volatile unsigned int lock; -} arch_rwlock_t; - -#define __ARCH_RW_LOCK_UNLOCKED { 0 } +#include =20 #endif /* _ASM_RISCV_SPINLOCK_TYPES_H */ --=20 2.34.1