From nobody Mon May 11 04:12:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75BA9C433F5 for ; Thu, 14 Apr 2022 22:05:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343900AbiDNWHw (ORCPT ); Thu, 14 Apr 2022 18:07:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240022AbiDNWHn (ORCPT ); Thu, 14 Apr 2022 18:07:43 -0400 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67231AD13E for ; Thu, 14 Apr 2022 15:05:16 -0700 (PDT) Received: by mail-pf1-x42a.google.com with SMTP id bo5so6008453pfb.4 for ; Thu, 14 Apr 2022 15:05:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=0jyU6ZA416nNh7Gcc3z3dq8e/E2mHcoMg9kDd8TTUb4=; b=Omlb8jpF+D+nYKUWOpFN7C5wUnbMOSt06NY1KUKR8+1tqHzn+FDuA14XkvhFM1cUgp PhPzKk3i3BsX4GV0fiLhjbLyxSM9gCaY2YJLUL8AskEqVjUYvFeIrN+FagEVaQS0l7Oa RpX6mLWXN5CJtI8LdG96SjsjhBOnaVEo7IqjB4qW/YhG5MSDzXNGMa/GWxqs51LwN4Sj W5tOD/+TO3pe99NqqPLRz7SNv+BZCpXZf4TgWpnuZV/QCN3t9hxq5TApDT76NnIj/brZ gBHYFdFMw21OoclH7g/YRcVa5rfgW1DfVMvGIltgRPD4Z3XY5Gz+2W+PTn6/T53ZIVre b9iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=0jyU6ZA416nNh7Gcc3z3dq8e/E2mHcoMg9kDd8TTUb4=; b=5rkzuwGGVjh4pJfzESlKDOqazvH00BVWrC7cM9gT01CCUirk+nVUbDPK8lev+SrwWo OoX4JmRq2okA7AMFoYvbPYY5gzeOA3TGN1PH1aOt8q+fjPTrBTZD1pufXtDvM6rwjQjD yfqB6mO2PbDTFp124UsUkGPjXqmrWo98ty6qI/z4fVK0SomqB76iOEeEmPP9qYftnqIQ DfEInyArbH1n1ULolXisphkRcEw+hXXMUcajIwNcjRcq/UmV+fOvNIms1b3TfWjYO+8r iLBcvHj2dBgI7/Z21oTS4hCuHa5CLI7kRA0Y3PsxjjM4w4Cd5zBZ5vC6qYfS9wO7pw/O TesA== X-Gm-Message-State: AOAM532u5G9FW0tbKJ5ViTfbTscYW6Hf7nTcQQBOc/FrCkE57FlJYg19 GnI+OqJt6o7NigW5LScRmu1nXw== X-Google-Smtp-Source: ABdhPJxPSC2qv+JNqkY3kBHG9Zclmo46Dc4kjny0507USILvT/NU5jR2hhF5i++i3cLn2Bu23eMbig== X-Received: by 2002:a65:41cc:0:b0:380:6f53:a550 with SMTP id b12-20020a6541cc000000b003806f53a550mr3873419pgq.471.1649973915618; Thu, 14 Apr 2022 15:05:15 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id e6-20020a17090a728600b001cb646a4adfsm6514424pjg.52.2022.04.14.15.05.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 15:05:15 -0700 (PDT) Subject: [PATCH v3 1/7] asm-generic: ticket-lock: New generic ticket-based spinlock Date: Thu, 14 Apr 2022 15:02:08 -0700 Message-Id: <20220414220214.24556-2-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220414220214.24556-1-palmer@rivosinc.com> References: <20220414220214.24556-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , macro@orcam.me.uk, Greg KH , sudipm.mukherjee@gmail.com, wangkefeng.wang@huawei.com, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann , heiko@sntech.de, guoren@kernel.org, shorne@gmail.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra This is a simple, fair spinlock. Specifically it doesn't have all the subtle memory model dependencies that qspinlock has, which makes it more suitable for simple systems as it is more likely to be correct. It is implemented entirely in terms of standard atomics and thus works fine without any arch-specific code. This replaces the existing asm-generic/spinlock.h, which just errored out on SMP systems. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Palmer Dabbelt Tested-bys, as this is all based on 5.18-rc1 now and there's been a --- include/asm-generic/spinlock.h | 85 +++++++++++++++++++++++++--- include/asm-generic/spinlock_types.h | 17 ++++++ 2 files changed, 94 insertions(+), 8 deletions(-) create mode 100644 include/asm-generic/spinlock_types.h diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h index adaf6acab172..ca829fcb9672 100644 --- a/include/asm-generic/spinlock.h +++ b/include/asm-generic/spinlock.h @@ -1,12 +1,81 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __ASM_GENERIC_SPINLOCK_H -#define __ASM_GENERIC_SPINLOCK_H + /* - * You need to implement asm/spinlock.h for SMP support. The generic - * version does not handle SMP. + * 'Generic' ticket-lock implementation. + * + * It relies on atomic_fetch_add() having well defined forward progress + * guarantees under contention. If your architecture cannot provide this, = stick + * to a test-and-set lock. + * + * It also relies on atomic_fetch_add() being safe vs smp_store_release() = on a + * sub-word of the value. This is generally true for anything LL/SC althou= gh + * you'd be hard pressed to find anything useful in architecture specifica= tions + * about this. If your architecture cannot do this you might be better off= with + * a test-and-set. + * + * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and = hence + * uses atomic_fetch_add() which is SC to create an RCsc lock. + * + * The implementation uses smp_cond_load_acquire() to spin, so if the + * architecture has WFE like instructions to sleep instead of poll for word + * modifications be sure to implement that (see ARM64 for example). + * */ -#ifdef CONFIG_SMP -#error need an architecture specific asm/spinlock.h -#endif =20 -#endif /* __ASM_GENERIC_SPINLOCK_H */ +#ifndef __ASM_GENERIC_TICKET_LOCK_H +#define __ASM_GENERIC_TICKET_LOCK_H + +#include +#include + +static __always_inline void arch_spin_lock(arch_spinlock_t *lock) +{ + u32 val =3D atomic_fetch_add(1<<16, lock); /* SC, gives us RCsc */ + u16 ticket =3D val >> 16; + + if (ticket =3D=3D (u16)val) + return; + + atomic_cond_read_acquire(lock, ticket =3D=3D (u16)VAL); +} + +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock) +{ + u32 old =3D atomic_read(lock); + + if ((old >> 16) !=3D (old & 0xffff)) + return false; + + return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */ +} + +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock) +{ + u16 *ptr =3D (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN); + u32 val =3D atomic_read(lock); + + smp_store_release(ptr, (u16)val + 1); +} + +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock) +{ + u32 val =3D atomic_read(lock); + + return ((val >> 16) !=3D (val & 0xffff)); +} + +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock) +{ + u32 val =3D atomic_read(lock); + + return (s16)((val >> 16) - (val & 0xffff)) > 1; +} + +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock) +{ + return !arch_spin_is_locked(&lock); +} + +#include + +#endif /* __ASM_GENERIC_TICKET_LOCK_H */ diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spi= nlock_types.h new file mode 100644 index 000000000000..e56ddb84d030 --- /dev/null +++ b/include/asm-generic/spinlock_types.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_GENERIC_TICKET_LOCK_TYPES_H +#define __ASM_GENERIC_TICKET_LOCK_TYPES_H + +#include +typedef atomic_t arch_spinlock_t; + +/* + * qrwlock_types depends on arch_spinlock_t, so we must typedef that befor= e the + * include. + */ +#include + +#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0) + +#endif /* __ASM_GENERIC_TICKET_LOCK_TYPES_H */ --=20 2.34.1 From nobody Mon May 11 04:12:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8805C4332F for ; Thu, 14 Apr 2022 22:05:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344051AbiDNWH5 (ORCPT ); Thu, 14 Apr 2022 18:07:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244210AbiDNWHp (ORCPT ); Thu, 14 Apr 2022 18:07:45 -0400 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 893E1AD13B for ; Thu, 14 Apr 2022 15:05:17 -0700 (PDT) Received: by mail-pg1-x536.google.com with SMTP id t13so5926818pgn.8 for ; Thu, 14 Apr 2022 15:05:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=z18FUB9s7KBEskqVTf7aP0q/WSmapl+AGV7tm/Rzx+w=; b=4yj8GNYwMFBt5S0TCYDexDsiDgZ+CphToZcr69BGb+YO1dX8XzN0DW+MGc/xCJDeXb Hcz0uIdoJnuNbkWJiV2+Fo3CA8vLW0C1IgS1CVVzH9aUg5j68NMCEbjwNgdXnkv7fEqz F6CZHlw5j34X9Ls84Zrx86UjvsRc7ZpFw0TYHUhHn1yyKX3+gl4msFSRvBIaTAhSTZdL 2YApeozR+Yfgpu19kZ4rinoHRC5MkmRhG0bqlbbT5bTMnVca/5C3iwU5CORogN7li9gJ eSftO1vHZa891P1ksAqxpAa0QJfvz+pd7dN+ERBzaFvvo1jYWPruym4OXs80BdrcFNBg 2lPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=z18FUB9s7KBEskqVTf7aP0q/WSmapl+AGV7tm/Rzx+w=; b=LvNfCYN1qJJavqtCGYfawlDrApdnC8ZQQdtI5do4MDqVPBGMILwt2gaaYIrKOO7JUs LGk2aSgBJieKzVn5RDPBy4PSvZjXIDjx0EkZ5LcnhLM2jvx6ENzk8KuO9+5w95+tjDi4 AI1Myyj0As3rV8tC7IznmrFNp5Fj2JMBF9g49JVEkjFo4k31aREhx/EJ5J1lpJ18f8Hn GZ7z2n494odGSyz2EDb83QG77IGYeI4iy8aIygaQ1f04bqY5ZSeYiKiE1wqRF3TqduLX sspjfjSnAlrKea51D29Y85ThaR0+xQuv+MMgDJHrOsfaOs/ydHH+3rrUusXOWF1f4vNA 5O1A== X-Gm-Message-State: AOAM531Or9f2OjQ9T+1nY+FFtzDPOLCZo1vBy2SFouzvOvJZxcmr/eiS diQVqCee7T3YpwFgFidsfXf0Cg== X-Google-Smtp-Source: ABdhPJy61GZpC7zZcOHBW9CQYGCxUgM5MW1Ir+LhmYjjwp3P+2wGDB1OrlzNEu7uootqzIZd255ezw== X-Received: by 2002:a05:6a00:1a06:b0:4fc:d6c5:f3d7 with SMTP id g6-20020a056a001a0600b004fcd6c5f3d7mr5841959pfv.53.1649973916890; Thu, 14 Apr 2022 15:05:16 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id n19-20020a635c53000000b0039dc2ea9876sm2697213pgm.49.2022.04.14.15.05.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 15:05:16 -0700 (PDT) Subject: [PATCH v3 2/7] asm-generic: qspinlock: Indicate the use of mixed-size atomics Date: Thu, 14 Apr 2022 15:02:09 -0700 Message-Id: <20220414220214.24556-3-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220414220214.24556-1-palmer@rivosinc.com> References: <20220414220214.24556-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , macro@orcam.me.uk, Greg KH , sudipm.mukherjee@gmail.com, wangkefeng.wang@huawei.com, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann , heiko@sntech.de, guoren@kernel.org, shorne@gmail.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra The qspinlock implementation depends on having well behaved mixed-size atomics. This is true on the more widely-used platforms, but these requirements are somewhat subtle and may not be satisfied by all the platforms that qspinlock is used on. Document these requirements, so ports that use qspinlock can more easily determine if they meet these requirements. Signed-off-by: Peter Zijlstra (Intel) Acked-by: Waiman Long Signed-off-by: Palmer Dabbelt Tested-bys, as this is all based on 5.18-rc1 now and there's been a --- include/asm-generic/qspinlock.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinloc= k.h index d74b13825501..95be3f3c28b5 100644 --- a/include/asm-generic/qspinlock.h +++ b/include/asm-generic/qspinlock.h @@ -2,6 +2,37 @@ /* * Queued spinlock * + * A 'generic' spinlock implementation that is based on MCS locks. An + * architecture that's looking for a 'generic' spinlock, please first cons= ider + * ticket-lock.h and only come looking here when you've considered all the + * constraints below and can show your hardware does actually perform bett= er + * with qspinlock. + * + * + * It relies on atomic_*_release()/atomic_*_acquire() to be RCsc (or no we= aker + * than RCtso if you're power), where regular code only expects atomic_t t= o be + * RCpc. + * + * It relies on a far greater (compared to asm-generic/spinlock.h) set of + * atomic operations to behave well together, please audit them carefully = to + * ensure they all have forward progress. Many atomic operations may defau= lt to + * cmpxchg() loops which will not have good forward progress properties on + * LL/SC architectures. + * + * One notable example is atomic_fetch_or_acquire(), which x86 cannot (che= aply) + * do. Carefully read the patches that introduced + * queued_fetch_set_pending_acquire(). + * + * It also heavily relies on mixed size atomic operations, in specific it + * requires architectures to have xchg16; something which many LL/SC + * architectures need to implement as a 32bit and+or in order to satisfy t= he + * forward progress guarantees mentioned above. + * + * Further reading on mixed size atomics that might be relevant: + * + * http://www.cl.cam.ac.uk/~pes20/popl17/mixed-size.pdf + * + * * (C) Copyright 2013-2015 Hewlett-Packard Development Company, L.P. * (C) Copyright 2015 Hewlett-Packard Enterprise Development LP * --=20 2.34.1 From nobody Mon May 11 04:12:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C4E8C433F5 for ; Thu, 14 Apr 2022 22:05:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343707AbiDNWIC (ORCPT ); Thu, 14 Apr 2022 18:08:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245017AbiDNWHq (ORCPT ); Thu, 14 Apr 2022 18:07:46 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB428AD134 for ; Thu, 14 Apr 2022 15:05:18 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id 2so6301783pjw.2 for ; Thu, 14 Apr 2022 15:05:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=GsO1g7Ro+rsf0io9vopg1aQfP2P2g91oLlrkEyQDXiM=; b=CnWghtB/dN2XPpzq+zptNtM2MAL+NMPVHYsMQgwsTW7iLpCdsO92al51pP1s30INYy rwF44VxtoSsNlb8TLR6+PvivGyCZ5apI54By81gsXp8b4zJytpT14My0wRS1/XqlEqjn 9c60+wgJ448Ym18jElwrZxwugKHQ2CLL8h38oEC7pO1P/00c1v34g1qQ6JgcV2POF6XJ qT35OQ8QyKY5rNXOS1abuXba/eh8yrBZrITli2mG/rjIegR5xxQLu6j+6zg8d/wJOC96 t/JZMK7hUZ/uW4d5apxoZB276EgBxBgsh8TDMlVDZJzUdBERoLCAWxBKRYX15Cr0EDe1 B80g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=GsO1g7Ro+rsf0io9vopg1aQfP2P2g91oLlrkEyQDXiM=; b=gumRlgpxksQT8l+tpTcGHqRlSOEBDUPk8mK9oYU4C/v1Zdx8z0DgFt3yRwPSpl35QW MflzRfA22aSSKIxMY8DjfhnoH/heR5PGuhxEtqRyqESSKN7u9iFen9PXFO7Jl1wOdrOs EmJgtm8vHzbvA+u+Ke/q8z9Cdzww0vDYRaxndY/UbWg3M9wXsG/hCThSWBNGVLElqdR1 LY4hO+paTGWTz3w58H4dJT4k2GdBYr5qkxKB2Kj+Z31YZXu5Ff71EByVbhR7FrssJtcm A0iYb6NiiSQOBb2/+uxwSEThYSt+lknc3CUsSL5IofR0GRHIACURSfcp40LklGXV4ET+ H+Cg== X-Gm-Message-State: AOAM531w7CGVpJWNrmnyxfVnlY7RRQ9jaV7WCo+7tZVw2Dg1LhJFDviq CdEQNCe0eM1UDOxdBshMK/DFPw== X-Google-Smtp-Source: ABdhPJyLfdcVDkvOFuUrfAfCBiI+o5Yu4ll2mvpNoDdi2mqGNotIaUVXICITBKPJFNtsKr9Rm/qRqw== X-Received: by 2002:a17:902:eb82:b0:158:8feb:86d6 with SMTP id q2-20020a170902eb8200b001588feb86d6mr13587063plg.26.1649973918225; Thu, 14 Apr 2022 15:05:18 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id e10-20020a17090a630a00b001c685cfd9d1sm2715141pjj.20.2022.04.14.15.05.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 15:05:17 -0700 (PDT) Subject: [PATCH v3 3/7] asm-generic: qrwlock: Document the spinlock fairness requirements Date: Thu, 14 Apr 2022 15:02:10 -0700 Message-Id: <20220414220214.24556-4-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220414220214.24556-1-palmer@rivosinc.com> References: <20220414220214.24556-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , macro@orcam.me.uk, Greg KH , sudipm.mukherjee@gmail.com, wangkefeng.wang@huawei.com, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann , heiko@sntech.de, guoren@kernel.org, shorne@gmail.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt I could only find the fairness requirements documented as the C code, this calls them out in a comment just to be a bit more explicit. Signed-off-by: Palmer Dabbelt Tested-bys, as this is all based on 5.18-rc1 now and there's been a --- include/asm-generic/qrwlock.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/asm-generic/qrwlock.h b/include/asm-generic/qrwlock.h index 7ae0ece07b4e..24ae09c1db9f 100644 --- a/include/asm-generic/qrwlock.h +++ b/include/asm-generic/qrwlock.h @@ -2,6 +2,10 @@ /* * Queue read/write lock * + * These use generic atomic and locking routines, but depend on a fair spi= nlock + * implementation in order to be fair themselves. The implementation in + * asm-generic/spinlock.h meets these requirements. + * * (C) Copyright 2013-2014 Hewlett-Packard Development Company, L.P. * * Authors: Waiman Long --=20 2.34.1 From nobody Mon May 11 04:12:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A5A2C433FE for ; Thu, 14 Apr 2022 22:05:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344659AbiDNWII (ORCPT ); Thu, 14 Apr 2022 18:08:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55952 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245265AbiDNWHq (ORCPT ); Thu, 14 Apr 2022 18:07:46 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8245ADD5E for ; Thu, 14 Apr 2022 15:05:19 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id d15so5795397pll.10 for ; Thu, 14 Apr 2022 15:05:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=fGuiIocTsGNeGfljFQBqIaq8z11faA06QmkOS1ZNMsM=; b=R6qB8EHjaJgSIdCtMKhdYuPkgsZWKZJvItsuxrvYJe66b0kWr5qEcaypbW8fVsbM0X FhUqRtspmHufGwN2FALekT8CyXJm5S7kjyMcktlGU1XrpzbtotUCXZuO87tkl8dzpRsU rD7GHFid/eI0LZmwYI/csFXmn3CvJN7DaDKvJaMLpPJ/3/P4dIfleHYvn2zvLdnkO4tP 7bShSmxTBQk2haxc2USQ9dtIjlVAIPg3bDRc+xVXDklpfpHWqfjPqU6sgkCSldEnbFhC TWJSMeobIJFf8xFLlsze9XxDtvdwjGCazEbqvvXoqURGpeC9W8m/Vt3E4JkUOM5Z3ckq eeBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=fGuiIocTsGNeGfljFQBqIaq8z11faA06QmkOS1ZNMsM=; b=pdnlRWH1a04rcWAv9Lx+D2IZcQ6XkLaERU5KtCbVnBdiGo58ZvuEIP1yMDEGZ/0VIF zqduCv002V4vYdGYl/saHQS9P+7ZI1OzNImWrtjzlaY+pXDrwhWW3RJ05A8bytlwI+GF 4Bx/aEUVE8T5KggxsXbkHUjCBIdc5RncvyAg5Haovk6tcwfQETyohlYz543DNpwq1ic5 mFvsOhnJej8A3x4BBNXseoQKo7KPBC8XNi7/gLXcN8CF1qpyy2hc6/kQby0hp08oyPaB ki1yJvoOef470/205hyrC3GQYplr+Fu5Ng/mBXTT3D6fLSQ83zMEXj/VFD5JcQXe1gLB fIKw== X-Gm-Message-State: AOAM530lW8dsEFZpGruy/VdnQUEjUOM0aOM4H7sbqSEk7ZPQm1yUaVkt yEkUsvgqpBYvnNdduclnCxuz7A== X-Google-Smtp-Source: ABdhPJyAzBb8YxpknFCk5JeSfiuwavmwv/MB+OUr9ESYgO7bJCkViBS7KMaonnUZ91p8m5wnAMzoYg== X-Received: by 2002:a17:90a:4294:b0:1cd:5524:cd6a with SMTP id p20-20020a17090a429400b001cd5524cd6amr716857pjg.212.1649973919487; Thu, 14 Apr 2022 15:05:19 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id d5-20020a056a0024c500b004fae56b2921sm805319pfv.167.2022.04.14.15.05.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 15:05:19 -0700 (PDT) Subject: [PATCH v3 4/7] openrisc: Move to ticket-spinlock Date: Thu, 14 Apr 2022 15:02:11 -0700 Message-Id: <20220414220214.24556-5-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220414220214.24556-1-palmer@rivosinc.com> References: <20220414220214.24556-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , macro@orcam.me.uk, Greg KH , sudipm.mukherjee@gmail.com, wangkefeng.wang@huawei.com, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann , heiko@sntech.de, guoren@kernel.org, shorne@gmail.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra We have no indications that openrisc meets the qspinlock requirements, so move to ticket-spinlock as that is more likey to be correct. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Palmer Dabbelt Acked-by: Stafford Horne Tested-bys, as this is all based on 5.18-rc1 now and there's been a --- arch/openrisc/Kconfig | 1 - arch/openrisc/include/asm/Kbuild | 5 ++-- arch/openrisc/include/asm/spinlock.h | 27 ---------------------- arch/openrisc/include/asm/spinlock_types.h | 7 ------ 4 files changed, 2 insertions(+), 38 deletions(-) delete mode 100644 arch/openrisc/include/asm/spinlock.h delete mode 100644 arch/openrisc/include/asm/spinlock_types.h diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig index 0d68adf6e02b..99f0e4a4cbbd 100644 --- a/arch/openrisc/Kconfig +++ b/arch/openrisc/Kconfig @@ -30,7 +30,6 @@ config OPENRISC select HAVE_DEBUG_STACKOVERFLOW select OR1K_PIC select CPU_NO_EFFICIENT_FFS if !OPENRISC_HAVE_INST_FF1 - select ARCH_USE_QUEUED_SPINLOCKS select ARCH_USE_QUEUED_RWLOCKS select OMPIC if SMP select ARCH_WANT_FRAME_POINTERS diff --git a/arch/openrisc/include/asm/Kbuild b/arch/openrisc/include/asm/K= build index ca5987e11053..3386b9c1c073 100644 --- a/arch/openrisc/include/asm/Kbuild +++ b/arch/openrisc/include/asm/Kbuild @@ -1,9 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 generic-y +=3D extable.h generic-y +=3D kvm_para.h -generic-y +=3D mcs_spinlock.h -generic-y +=3D qspinlock_types.h -generic-y +=3D qspinlock.h +generic-y +=3D spinlock_types.h +generic-y +=3D spinlock.h generic-y +=3D qrwlock_types.h generic-y +=3D qrwlock.h generic-y +=3D user.h diff --git a/arch/openrisc/include/asm/spinlock.h b/arch/openrisc/include/a= sm/spinlock.h deleted file mode 100644 index 264944a71535..000000000000 --- a/arch/openrisc/include/asm/spinlock.h +++ /dev/null @@ -1,27 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-or-later */ -/* - * OpenRISC Linux - * - * Linux architectural port borrowing liberally from similar works of - * others. All original copyrights apply as per the original source - * declaration. - * - * OpenRISC implementation: - * Copyright (C) 2003 Matjaz Breskvar - * Copyright (C) 2010-2011 Jonas Bonn - * et al. - */ - -#ifndef __ASM_OPENRISC_SPINLOCK_H -#define __ASM_OPENRISC_SPINLOCK_H - -#include - -#include - -#define arch_spin_relax(lock) cpu_relax() -#define arch_read_relax(lock) cpu_relax() -#define arch_write_relax(lock) cpu_relax() - - -#endif diff --git a/arch/openrisc/include/asm/spinlock_types.h b/arch/openrisc/inc= lude/asm/spinlock_types.h deleted file mode 100644 index 7c6fb1208c88..000000000000 --- a/arch/openrisc/include/asm/spinlock_types.h +++ /dev/null @@ -1,7 +0,0 @@ -#ifndef _ASM_OPENRISC_SPINLOCK_TYPES_H -#define _ASM_OPENRISC_SPINLOCK_TYPES_H - -#include -#include - -#endif /* _ASM_OPENRISC_SPINLOCK_TYPES_H */ --=20 2.34.1 From nobody Mon May 11 04:12:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EEDF9C433FE for ; Thu, 14 Apr 2022 22:05:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241123AbiDNWIP (ORCPT ); Thu, 14 Apr 2022 18:08:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232664AbiDNWHs (ORCPT ); Thu, 14 Apr 2022 18:07:48 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50035AD135 for ; Thu, 14 Apr 2022 15:05:21 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id mm4-20020a17090b358400b001cb93d8b137so10448900pjb.2 for ; Thu, 14 Apr 2022 15:05:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=vsFDZUB/84NzhX1Z1IP0W5vQn6O0PaUxSIdB0QjeUDc=; b=IP4oGKTy+Wg0eeVn1tkZNqxzgquzqic9WWTzGoLpN8dXjfyAqSQGgu8PRr9T6udYTF 5fZKPv8j17r+U9Qaqhjpf3K0KdBkCeFJL4ANLfBJ4wxNzdkhiw+FEIEtyWged3OrkEeK O/ue1T+m3vgfwM1nGrP9iqTMKyWK0ZRIe6v+uXZ63Ecsv019nXDcM7EuupsWQDvf1udn wYtkzLqiCDPUVXc72nh9EMaLmRXwnwJPlV+3zrYp1Gkv2+8S4nq+fv7DJkmK+4Vvfjh8 /n045mQmP7nJ/ZUZPwg6Ivd1ywaBswAIDEDEYgKqEuAAtAgCkpvQEfy9jRfFuuMJAyOs Dt4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=vsFDZUB/84NzhX1Z1IP0W5vQn6O0PaUxSIdB0QjeUDc=; b=f7nEfydE/iqn1slbpZOLqtUtcXaXuy9yiWR/d3xsN5Ec7tlxbc1iIbdLVMl9htfYOD P99rJ0hSo7EvrhB8RKP+8HGjghlqVcB1tCTT/4XN0M5L+vJCWGd2x6XqK4PLjshtIXYJ RzlTAFmLmiZUkp+w5atq68BaLQ+rrHYl04PhddpHQDtKMXUmcaRoLk3PAYE10s5R42vx GKut2JTqpNhDSwnQ0o9xrQkd6CXN6ojk/+a7/8vopvug33SusG3DcbvZGQtl5jlOPKd7 krMMxIe7oZdNBsKGJ6GXpCbkrzG0VxuINquTEReYWoEu2tLjDOMnqT09mdqaz+GnvtpN 9tYA== X-Gm-Message-State: AOAM533hSZI6Fwe5VAwd0dtSxgByVoTHK0eKpMFhMLygsx4XdprkI33E SEhE/DmWXaLaIsyDBR992o+N7A== X-Google-Smtp-Source: ABdhPJy4A8/pox4dhxDr4yvuIr0aGfvJn7yO4mLZw9IKFjjNFu22lsbncs+sg8MOsAOHYZqRkccWhQ== X-Received: by 2002:a17:902:b582:b0:14c:a63d:3df6 with SMTP id a2-20020a170902b58200b0014ca63d3df6mr49682729pls.51.1649973920782; Thu, 14 Apr 2022 15:05:20 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b004fb157f136asm778947pfe.153.2022.04.14.15.05.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 15:05:20 -0700 (PDT) Subject: [PATCH v3 5/7] RISC-V: Move to generic spinlocks Date: Thu, 14 Apr 2022 15:02:12 -0700 Message-Id: <20220414220214.24556-6-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220414220214.24556-1-palmer@rivosinc.com> References: <20220414220214.24556-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , macro@orcam.me.uk, Greg KH , sudipm.mukherjee@gmail.com, wangkefeng.wang@huawei.com, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann , heiko@sntech.de, guoren@kernel.org, shorne@gmail.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt Our existing spinlocks aren't fair and replacing them has been on the TODO list for a long time. This moves to the recently-introduced ticket spinlocks, which are simple enough that they are likely to be correct and fast on the vast majority of extant implementations. This introduces a horrible hack that allows us to split out the spinlock conversion from the rwlock conversion. We have to do the spinlocks first because qrwlock needs fair spinlocks, but we don't want to pollute the asm-generic code to support the generic spinlocks without qrwlocks. Thus we pollute the RISC-V code, but just until the next commit as it's all going away. Signed-off-by: Palmer Dabbelt Tested-bys, as this is all based on 5.18-rc1 now and there's been a --- arch/riscv/include/asm/Kbuild | 2 ++ arch/riscv/include/asm/spinlock.h | 44 +++---------------------- arch/riscv/include/asm/spinlock_types.h | 9 +++-- 3 files changed, 10 insertions(+), 45 deletions(-) diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 5edf5b8587e7..c3f229ae8033 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -3,5 +3,7 @@ generic-y +=3D early_ioremap.h generic-y +=3D flat.h generic-y +=3D kvm_para.h generic-y +=3D parport.h +generic-y +=3D qrwlock.h +generic-y +=3D qrwlock_types.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spi= nlock.h index f4f7fa1b7ca8..88a4d5d0d98a 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -7,49 +7,13 @@ #ifndef _ASM_RISCV_SPINLOCK_H #define _ASM_RISCV_SPINLOCK_H =20 +/* This is horible, but the whole file is going away in the next commit. */ +#define __ASM_GENERIC_QRWLOCK_H + #include #include #include - -/* - * Simple spin lock operations. These provide no fairness guarantees. - */ - -/* FIXME: Replace this with a ticket lock, like MIPS. */ - -#define arch_spin_is_locked(x) (READ_ONCE((x)->lock) !=3D 0) - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - int tmp =3D 1, busy; - - __asm__ __volatile__ ( - " amoswap.w %0, %2, %1\n" - RISCV_ACQUIRE_BARRIER - : "=3Dr" (busy), "+A" (lock->lock) - : "r" (tmp) - : "memory"); - - return !busy; -} - -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - while (1) { - if (arch_spin_is_locked(lock)) - continue; - - if (arch_spin_trylock(lock)) - break; - } -} - -/***********************************************************/ +#include =20 static inline void arch_read_lock(arch_rwlock_t *lock) { diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/a= sm/spinlock_types.h index 5a35a49505da..f2f9b5d7120d 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -6,15 +6,14 @@ #ifndef _ASM_RISCV_SPINLOCK_TYPES_H #define _ASM_RISCV_SPINLOCK_TYPES_H =20 +/* This is horible, but the whole file is going away in the next commit. */ +#define __ASM_GENERIC_QRWLOCK_TYPES_H + #ifndef __LINUX_SPINLOCK_TYPES_RAW_H # error "please don't include this file directly" #endif =20 -typedef struct { - volatile unsigned int lock; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { 0 } +#include =20 typedef struct { volatile unsigned int lock; --=20 2.34.1 From nobody Mon May 11 04:12:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00F6FC433F5 for ; Thu, 14 Apr 2022 22:05:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345339AbiDNWIM (ORCPT ); Thu, 14 Apr 2022 18:08:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343835AbiDNWHs (ORCPT ); Thu, 14 Apr 2022 18:07:48 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7605BAD13C for ; Thu, 14 Apr 2022 15:05:22 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d15so5795455pll.10 for ; Thu, 14 Apr 2022 15:05:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=PhfY9SC+wYhs3i/pCnS1F0qe7XGea8rOjY/kEhC+LrE=; b=omTQGGg/c04ZbdiA8O9RZimwch5kIdoXKVihnkGxc8A6Am7IWif9ma1/iq8ndcyx3c ktAy8M70SL6Q2FGobQAxjrjX6GZ0Z/skqooakVw3Ao901XJVKxc2hfShN/tTwIKs5mjc Af/cNqn4W2g9cmI0u0IEYyyqhy7XRBV5UA5ndzazDCAnefmBxViy4EoCw+XQzLA/JrVT YVCu8yYSxQeVP0v6m7Hhj6siDRS8blg3JG7AKdNCsq3A3L6CVUhsFDJviMZWZWt+PRcp PHNs93ZkkT/m79Chhv8pfsYgO63f0ut65YGfLR2jVIE5NEhAIMwUAZ6xHWa+vewhrqVc nIVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=PhfY9SC+wYhs3i/pCnS1F0qe7XGea8rOjY/kEhC+LrE=; b=f5lCkKkGXJX7Ya6Ga80blhVadeHCO5ukX8r38meik16pTQWCReO5UyujgLVuaUGcJ0 t4zfwCwM0jMEen2aSBptupspfX43eWk9Z92Tiu2QIkc9OGyETKopDZ8557cs3RrGxk9b ROnB4jd6z8sgqzNxCQ4dDMREsT0vQqNPkr5U8/ggv2BPseM/xB22SjR+1Dz7XmLc1ieZ 2L/w2onST0kP6mvYj4x+yRJpTI7F/NMDobWZTqyVEPSlltL+llmMdvW807155GXHArnE /9cRDq7fTE+gN/sm/7/QKAV0IIsEOzXV3LTNDR0/KCiM8vryH18AQLRcGhb2+qdYKpHO RbqQ== X-Gm-Message-State: AOAM531hK6BkCGBSB69vCoeUFsRNmQZe680T4kcqivaUSM3BANtwZBL4 a6yoKYO3acuMgYReq7+7fr0Iug== X-Google-Smtp-Source: ABdhPJz4GEKyjjtdCqonV52xYL93iFUN6nNC0yVHzuZZyR1GP4XXMquZM3VabXZqrc8MPIpr3NjeVA== X-Received: by 2002:a17:902:db10:b0:158:657e:10f0 with SMTP id m16-20020a170902db1000b00158657e10f0mr22409599plx.125.1649973921997; Thu, 14 Apr 2022 15:05:21 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id d8-20020a17090a114800b001cb95a92bd7sm6604374pje.13.2022.04.14.15.05.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 15:05:21 -0700 (PDT) Subject: [PATCH v3 6/7] RISC-V: Move to queued RW locks Date: Thu, 14 Apr 2022 15:02:13 -0700 Message-Id: <20220414220214.24556-7-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220414220214.24556-1-palmer@rivosinc.com> References: <20220414220214.24556-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , macro@orcam.me.uk, Greg KH , sudipm.mukherjee@gmail.com, wangkefeng.wang@huawei.com, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann , heiko@sntech.de, guoren@kernel.org, shorne@gmail.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Palmer Dabbelt Now that we have fair spinlocks we can use the generic queued rwlocks, so we might as well do so. Signed-off-by: Palmer Dabbelt Tested-bys, as this is all based on 5.18-rc1 now and there's been a --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/Kbuild | 2 + arch/riscv/include/asm/spinlock.h | 99 ------------------------- arch/riscv/include/asm/spinlock_types.h | 24 ------ 4 files changed, 3 insertions(+), 123 deletions(-) delete mode 100644 arch/riscv/include/asm/spinlock.h delete mode 100644 arch/riscv/include/asm/spinlock_types.h diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 00fd9c548f26..f8a55d94016d 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -39,6 +39,7 @@ config RISCV select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU select ARCH_SUPPORTS_HUGETLBFS if MMU select ARCH_USE_MEMTEST + select ARCH_USE_QUEUED_RWLOCKS select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index c3f229ae8033..504f8b7e72d4 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -3,6 +3,8 @@ generic-y +=3D early_ioremap.h generic-y +=3D flat.h generic-y +=3D kvm_para.h generic-y +=3D parport.h +generic-y +=3D spinlock.h +generic-y +=3D spinlock_types.h generic-y +=3D qrwlock.h generic-y +=3D qrwlock_types.h generic-y +=3D user.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spi= nlock.h deleted file mode 100644 index 88a4d5d0d98a..000000000000 --- a/arch/riscv/include/asm/spinlock.h +++ /dev/null @@ -1,99 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (C) 2015 Regents of the University of California - * Copyright (C) 2017 SiFive - */ - -#ifndef _ASM_RISCV_SPINLOCK_H -#define _ASM_RISCV_SPINLOCK_H - -/* This is horible, but the whole file is going away in the next commit. */ -#define __ASM_GENERIC_QRWLOCK_H - -#include -#include -#include -#include - -static inline void arch_read_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1b\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=3D&r" (tmp) - :: "memory"); -} - -static inline void arch_write_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1b\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=3D&r" (tmp) - :: "memory"); -} - -static inline int arch_read_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1f\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=3D&r" (busy) - :: "memory"); - - return !busy; -} - -static inline int arch_write_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1f\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=3D&r" (busy) - :: "memory"); - - return !busy; -} - -static inline void arch_read_unlock(arch_rwlock_t *lock) -{ - __asm__ __volatile__( - RISCV_RELEASE_BARRIER - " amoadd.w x0, %1, %0\n" - : "+A" (lock->lock) - : "r" (-1) - : "memory"); -} - -static inline void arch_write_unlock(arch_rwlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -#endif /* _ASM_RISCV_SPINLOCK_H */ diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/a= sm/spinlock_types.h deleted file mode 100644 index f2f9b5d7120d..000000000000 --- a/arch/riscv/include/asm/spinlock_types.h +++ /dev/null @@ -1,24 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (C) 2015 Regents of the University of California - */ - -#ifndef _ASM_RISCV_SPINLOCK_TYPES_H -#define _ASM_RISCV_SPINLOCK_TYPES_H - -/* This is horible, but the whole file is going away in the next commit. */ -#define __ASM_GENERIC_QRWLOCK_TYPES_H - -#ifndef __LINUX_SPINLOCK_TYPES_RAW_H -# error "please don't include this file directly" -#endif - -#include - -typedef struct { - volatile unsigned int lock; -} arch_rwlock_t; - -#define __ARCH_RW_LOCK_UNLOCKED { 0 } - -#endif /* _ASM_RISCV_SPINLOCK_TYPES_H */ --=20 2.34.1 From nobody Mon May 11 04:12:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68059C433F5 for ; Thu, 14 Apr 2022 22:06:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245175AbiDNWIX (ORCPT ); Thu, 14 Apr 2022 18:08:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343871AbiDNWHu (ORCPT ); Thu, 14 Apr 2022 18:07:50 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC501ADD61 for ; Thu, 14 Apr 2022 15:05:23 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id mm4-20020a17090b358400b001cb93d8b137so10448961pjb.2 for ; Thu, 14 Apr 2022 15:05:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=uLn9nEAVfqfhF99bZmCoDyMs8/O9emD7jJiMXxYFAyY=; b=aCpk+hiR1i6NQteJMGoSFJLdn5Rgm8st+ic8YTNlQHLfaUlgHtMcfuGAfhQl3K+6g4 UmI+PZBc6djUKQNDbQ28uCRN+kmDtOXIVTEKtwMumAnydxR3HnsqPH3DA0sbMlbQAwrU jpy7qcIA5R2RjbhZPPRb0MbiE3lAy7OS3Iwj1cDXaC4TvcQoItO9lj4SSCoUz8N2T03c am9CxRoIA8kWRn5kdkC7F60d0pSGRFKNubmWK4iNXouTAsvDUaX7fSzOK0gKgvqKwaub Qi9xqMYtyCJ43ckr4k2sI2wGmYCS0qBvDvWN1moO+kJfRhoTr4apwv5ff6foANjKKm0d vijA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=uLn9nEAVfqfhF99bZmCoDyMs8/O9emD7jJiMXxYFAyY=; b=X+JfOH6xIBnC1z5BWlzvf9CkfWn/zrdM9cjZ2JPshBxok9y0Q0uxnuD6ZJdol8rpdl T2Nfse+jNeclMEsc6TyZ9f9cQLjh/YQeVeA9jBmZDVeCy7rR25qqNfVEBShnpw1swvWV FTKehBpYTzh3hZB1LLsvnRdUWcx3qCOBcZ2lZnJaMhptMV5iheQ7c8a68YkWmNwAY0Xc lhuvC/j/FqfX8XX1dgjyuwj8yaH0+NbN7PNERmTed1iBOc4lDrctollUFGSH7fsO9CAN 7ulUwegLfQX8fo40DjoKvPvIWVbSPQScLylNEVAeUgSBXsw5wTrEGAzcjwM5zXha81EH B58A== X-Gm-Message-State: AOAM532/WndaaR4P/PL4SxSJoIfuwnfFJH5IPdV3ZlOTXnZPl0bAbO2D sDgsaO9Yg/RHQG1d5n9LAljAvQ== X-Google-Smtp-Source: ABdhPJzhfrbMDiR890Mvpaf1KfA0KcEG1o/qsHftMGK/3W28SItjuhSFgVLfw1qtTLZgABu0T8DOAQ== X-Received: by 2002:a17:90a:a78f:b0:1bc:8042:9330 with SMTP id f15-20020a17090aa78f00b001bc80429330mr685887pjq.229.1649973923305; Thu, 14 Apr 2022 15:05:23 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id r8-20020a17090a0ac800b001c9e35d3a3asm2852568pje.24.2022.04.14.15.05.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 15:05:22 -0700 (PDT) Subject: [PATCH v3 7/7] csky: Move to generic ticket-spinlock Date: Thu, 14 Apr 2022 15:02:14 -0700 Message-Id: <20220414220214.24556-8-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220414220214.24556-1-palmer@rivosinc.com> References: <20220414220214.24556-1-palmer@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Cc: peterz@infradead.org, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , macro@orcam.me.uk, Greg KH , sudipm.mukherjee@gmail.com, wangkefeng.wang@huawei.com, jszhang@kernel.org, linux-csky@vger.kernel.org, linux-kernel@vger.kernel.org, openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Guo Ren , Palmer Dabbelt From: Palmer Dabbelt To: Arnd Bergmann , heiko@sntech.de, guoren@kernel.org, shorne@gmail.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Guo Ren There is no benefit from custom implementation for ticket-spinlock, so move to generic ticket-spinlock for easy maintenance. Signed-off-by: Guo Ren Signed-off-by: Palmer Dabbelt Tested-bys, as this is all based on 5.18-rc1 now and there's been a --- arch/csky/include/asm/Kbuild | 3 + arch/csky/include/asm/spinlock.h | 89 -------------------------- arch/csky/include/asm/spinlock_types.h | 27 -------- 3 files changed, 3 insertions(+), 116 deletions(-) delete mode 100644 arch/csky/include/asm/spinlock.h delete mode 100644 arch/csky/include/asm/spinlock_types.h diff --git a/arch/csky/include/asm/Kbuild b/arch/csky/include/asm/Kbuild index 888248235c23..103207a58f97 100644 --- a/arch/csky/include/asm/Kbuild +++ b/arch/csky/include/asm/Kbuild @@ -3,7 +3,10 @@ generic-y +=3D asm-offsets.h generic-y +=3D extable.h generic-y +=3D gpio.h generic-y +=3D kvm_para.h +generic-y +=3D spinlock.h +generic-y +=3D spinlock_types.h generic-y +=3D qrwlock.h +generic-y +=3D qrwlock_types.h generic-y +=3D parport.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/csky/include/asm/spinlock.h b/arch/csky/include/asm/spinl= ock.h deleted file mode 100644 index 69f5aa249c5f..000000000000 --- a/arch/csky/include/asm/spinlock.h +++ /dev/null @@ -1,89 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ - -#ifndef __ASM_CSKY_SPINLOCK_H -#define __ASM_CSKY_SPINLOCK_H - -#include -#include - -/* - * Ticket-based spin-locking. - */ -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - arch_spinlock_t lockval; - u32 ticket_next =3D 1 << TICKET_NEXT; - u32 *p =3D &lock->lock; - u32 tmp; - - asm volatile ( - "1: ldex.w %0, (%2) \n" - " mov %1, %0 \n" - " add %0, %3 \n" - " stex.w %0, (%2) \n" - " bez %0, 1b \n" - : "=3D&r" (tmp), "=3D&r" (lockval) - : "r"(p), "r"(ticket_next) - : "cc"); - - while (lockval.tickets.next !=3D lockval.tickets.owner) - lockval.tickets.owner =3D READ_ONCE(lock->tickets.owner); - - smp_mb(); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - u32 tmp, contended, res; - u32 ticket_next =3D 1 << TICKET_NEXT; - u32 *p =3D &lock->lock; - - do { - asm volatile ( - " ldex.w %0, (%3) \n" - " movi %2, 1 \n" - " rotli %1, %0, 16 \n" - " cmpne %1, %0 \n" - " bt 1f \n" - " movi %2, 0 \n" - " add %0, %0, %4 \n" - " stex.w %0, (%3) \n" - "1: \n" - : "=3D&r" (res), "=3D&r" (tmp), "=3D&r" (contended) - : "r"(p), "r"(ticket_next) - : "cc"); - } while (!res); - - if (!contended) - smp_mb(); - - return !contended; -} - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_mb(); - WRITE_ONCE(lock->tickets.owner, lock->tickets.owner + 1); -} - -static inline int arch_spin_value_unlocked(arch_spinlock_t lock) -{ - return lock.tickets.owner =3D=3D lock.tickets.next; -} - -static inline int arch_spin_is_locked(arch_spinlock_t *lock) -{ - return !arch_spin_value_unlocked(READ_ONCE(*lock)); -} - -static inline int arch_spin_is_contended(arch_spinlock_t *lock) -{ - struct __raw_tickets tickets =3D READ_ONCE(lock->tickets); - - return (tickets.next - tickets.owner) > 1; -} -#define arch_spin_is_contended arch_spin_is_contended - -#include - -#endif /* __ASM_CSKY_SPINLOCK_H */ diff --git a/arch/csky/include/asm/spinlock_types.h b/arch/csky/include/asm= /spinlock_types.h deleted file mode 100644 index db87a12c3827..000000000000 --- a/arch/csky/include/asm/spinlock_types.h +++ /dev/null @@ -1,27 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ - -#ifndef __ASM_CSKY_SPINLOCK_TYPES_H -#define __ASM_CSKY_SPINLOCK_TYPES_H - -#ifndef __LINUX_SPINLOCK_TYPES_RAW_H -# error "please don't include this file directly" -#endif - -#define TICKET_NEXT 16 - -typedef struct { - union { - u32 lock; - struct __raw_tickets { - /* little endian */ - u16 owner; - u16 next; - } tickets; - }; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { { 0 } } - -#include - -#endif /* __ASM_CSKY_SPINLOCK_TYPES_H */ --=20 2.34.1