From nobody Sat Feb 7 17:19:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A34C4C001DB for ; Wed, 9 Aug 2023 02:14:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230196AbjHICOp (ORCPT ); Tue, 8 Aug 2023 22:14:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229450AbjHICOl (ORCPT ); Tue, 8 Aug 2023 22:14:41 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93954FE for ; Tue, 8 Aug 2023 19:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547239; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cG8axZDwBuWJQ+k2FsV81xz+WKLWi2QNSuhJ6eUeaqw=; b=XRqnmWPyTZYG31ea3SCYHl3lm2SMlU82Zqp23COwH9n6Ot0i1EYPjLdvcZcbjSjjxcNFkU MWfQ25+nVqNa5hzksLlQjSEMPBkUk68AybqxFPQujSTSsOXsJxOPUvCh0IJIjaxuBM/VWF ndBbvaPhP294EiqjVupEGHNrvpZumyE= Received: from mail-ot1-f70.google.com (mail-ot1-f70.google.com [209.85.210.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-278-WdnLxjy4MlijCfrkKkqyEg-1; Tue, 08 Aug 2023 22:13:58 -0400 X-MC-Unique: WdnLxjy4MlijCfrkKkqyEg-1 Received: by mail-ot1-f70.google.com with SMTP id 46e09a7af769-6b9cf208fb5so11856780a34.3 for ; Tue, 08 Aug 2023 19:13:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547238; x=1692152038; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cG8axZDwBuWJQ+k2FsV81xz+WKLWi2QNSuhJ6eUeaqw=; b=hxbStUsIy3XFvyjQ23gqWiVmpxGsN7VuiCPA3O7voJmbbITR7gRTSt3TFkJh9PTjxZ R7oHdXQ5aklQFEdQshksQKMgUJhDp4HLTjtOutS7w0Ua1dAS9/06wOHL+C2YyJ2IE4g9 ZiHHsMvWWaxyUZ0XNH+2j8NB0C00LUBYMbHhKHlYKL9NTIawhNeay353JDktoISGjmO9 SKnEe6MgsJCYep9CzEH2oh5yFbY7ZVt3Se+NKepRyoD6Q09ZjOSdwD531gPeLb/nz+Hz aHuLZZwTt8T6atZXlw7lg3yOQIxa1faFUBnyn0VjcRuP+ued3QVItKd4TXJLx0dKjax5 pgRw== X-Gm-Message-State: AOJu0Yzd9Pw841GNPyovIeApTBNI67orY/FRClY+V5Q/lLEZSPUz2FRu zIH+kAKz4gEzEoFAd5xESAfcKYF+H5NRatTPI3GKFxAu2WKQwYb6PfbpeNCQCbMOd5HyFj8MjYh ANKvGw1MNUXoWUSLLgyKkOT6/ X-Received: by 2002:a05:6870:c084:b0:1bf:4f5e:55e2 with SMTP id c4-20020a056870c08400b001bf4f5e55e2mr1543844oad.53.1691547237749; Tue, 08 Aug 2023 19:13:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHukU42SPYGuHy2oEtAblgnpaB5UkpskCwb9snxi5LMKs5NyCZyz7VTISypO0xyDpTH5uoIvA== X-Received: by 2002:a05:6870:c084:b0:1bf:4f5e:55e2 with SMTP id c4-20020a056870c08400b001bf4f5e55e2mr1543823oad.53.1691547237498; Tue, 08 Aug 2023 19:13:57 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.13.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:13:57 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Date: Tue, 8 Aug 2023 23:13:06 -0300 Message-ID: <20230809021311.1390578-3-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_xchg* generation. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 136 +++++-------------------------- 1 file changed, 22 insertions(+), 114 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index 2f4726d3cfcc..ec4ea4f3f908 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,60 +11,30 @@ #include #include =20 -#define __xchg_relaxed(ptr, new, size) \ +#define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_relaxed(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_relaxed((ptr), \ - _x_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + " amoswap" sfx " %0, %2, %1\n" \ + append \ + : "=3Dr" (r), "+A" (*(p)) \ + : "r" (n) \ + : "memory"); \ }) =20 -#define __xchg_acquire(ptr, new, size) \ +#define _arch_xchg(ptr, new, sfx, prepend, append) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ + __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ - switch (size) { \ + __typeof__(ptr) __ptr =3D (ptr); \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".w" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".d" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -72,79 +42,17 @@ __ret; \ }) =20 -#define arch_xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) +#define arch_xchg_relaxed(ptr, x) \ + _arch_xchg(ptr, x, "", "", "") =20 -#define __xchg_release(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_xchg_acquire(ptr, x) \ + _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER) =20 #define arch_xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __arch_xchg(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "") =20 #define arch_xchg(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ -}) + _arch_xchg(ptr, x, ".aqrl", "", "") =20 #define xchg32(ptr, x) \ ({ \ --=20 2.41.0 From nobody Sat Feb 7 17:19:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C130C001DB for ; Wed, 9 Aug 2023 02:14:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230241AbjHICO4 (ORCPT ); Tue, 8 Aug 2023 22:14:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230404AbjHICOw (ORCPT ); Tue, 8 Aug 2023 22:14:52 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4B0E1BCF for ; Tue, 8 Aug 2023 19:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547245; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=68droSgtD1yUeWujN9KsDS4jg7ssA0NCkb8PZ8Tb+SA=; b=Hlj7cYC62XGhxBJoJ0YtfPFKlAuiW63TigiC/kyWTFkPKRvL6+ustHNXYvnb8auYlqBoPh siP/j9Wjb4s+9LgK5bD01j+qm/SYF/yRr3DsZkuGBqVRXK5dpagmRavJoBavZVFKM1BNw3 EK50kPoBnv4Pl1qjxbLwQeKZ0gosoeI= Received: from mail-oo1-f72.google.com (mail-oo1-f72.google.com [209.85.161.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-635-81yYtrFvMh22bvAcvpLuhA-1; Tue, 08 Aug 2023 22:14:03 -0400 X-MC-Unique: 81yYtrFvMh22bvAcvpLuhA-1 Received: by mail-oo1-f72.google.com with SMTP id 006d021491bc7-56c7404fafdso10497303eaf.1 for ; Tue, 08 Aug 2023 19:14:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547243; x=1692152043; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=68droSgtD1yUeWujN9KsDS4jg7ssA0NCkb8PZ8Tb+SA=; b=bhOny3s7e/3/RvwBMR7SgD2WFHZLgIuTsSv0UltWQdBukNDP39SHc80adaL8HwsTHy YJFjUti1RswKto6DUAi2bP9kJx1z+VMpQLI0bmTNESW97/qBDmKoX6GvYR6yEOITJnW0 CUQ2E2y0PQYeFIuSYA6vCmTzZXByVhaowrJjl1XmTEmoLao/NWZUlhFm1DbaBs6uhYY+ eX+uCLQFdRYOX5GdR1azcAdllfU+trwNq3wWVgdBHP/40KzUwUFuUny//HDUJtbaRoQ6 Zdq3qfb1cOgHNzczhMYwdqYvyGrDDWxz/1ZGEh/zOULZye4eteSYc62fXVX+ok9HDypp Glrg== X-Gm-Message-State: AOJu0YwYNDaZsKLVj9sWlSlbxZK9D8zeyuJvflRXXgJZfKfSkvG7GEeG +eWCHMqMN96nIR4Hzo9O7tWu4ujcw0O8AfyO7xvPPYRzMG9+RNoiAKLpAYd2xk4+A/3zeS/+jS3 weDz1MCYpyQwv7oRwvBdy7pKf X-Received: by 2002:a4a:3005:0:b0:56c:aa7a:eaeb with SMTP id q5-20020a4a3005000000b0056caa7aeaebmr1376286oof.4.1691547243031; Tue, 08 Aug 2023 19:14:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFRBi7ZrL7X8EpzKEKv574j/yN24o/CUG+jxNQTD3De3Bdux9OGGevu77kaKC1CyIqeEg4tkw== X-Received: by 2002:a4a:3005:0:b0:56c:aa7a:eaeb with SMTP id q5-20020a4a3005000000b0056caa7aeaebmr1376278oof.4.1691547242715; Tue, 08 Aug 2023 19:14:02 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.13.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:02 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Date: Tue, 8 Aug 2023 23:13:07 -0300 Message-ID: <20230809021311.1390578-4-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_cmpxchg* generation (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 187 +++++-------------------------- 1 file changed, 29 insertions(+), 158 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index ec4ea4f3f908..5a07646fae65 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -71,127 +71,40 @@ * store NEW in MEM. Return the initial value in MEM. Success is * indicated by comparing RETURN with OLD. */ -#define __cmpxchg_relaxed(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) =20 -#define arch_cmpxchg_relaxed(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_relaxed((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) =20 -#define __cmpxchg_acquire(ptr, old, new, size) \ +#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_cmpxchg_acquire(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=3D&r" (r), "=3D&r" (__rc), "+A" (*(p)) \ + : "rJ" (co o), "rJ" (n) \ + : "memory"); \ }) =20 -#define __cmpxchg_release(ptr, old, new, size) \ +#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr =3D (ptr); \ __typeof__(*(ptr)) __old =3D (old); \ __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ + \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ + __ret, __ptr, (long), __old, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".d", ".d" sc_sfx, prepend, append, \ + __ret, __ptr, /**/, __old, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -199,62 +112,20 @@ __ret; \ }) =20 -#define arch_cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) +#define arch_cmpxchg_relaxed(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", "") =20 -#define __cmpxchg(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_cmpxchg_acquire(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER) + +#define arch_cmpxchg_release(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "") =20 #define arch_cmpxchg(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) + _arch_cmpxchg((ptr), (o), (n), ".rl", "", " fence rw, rw\n") =20 #define arch_cmpxchg_local(ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) + arch_cmpxchg_relaxed((ptr), (o), (n)) =20 #define arch_cmpxchg64(ptr, o, n) \ ({ \ --=20 2.41.0 From nobody Sat Feb 7 17:19:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9770CC04A94 for ; Wed, 9 Aug 2023 02:14:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229460AbjHICO6 (ORCPT ); Tue, 8 Aug 2023 22:14:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230359AbjHICOy (ORCPT ); Tue, 8 Aug 2023 22:14:54 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA7C01BCD for ; Tue, 8 Aug 2023 19:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547251; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=VTf0kHbf7syq69H4/SQEYMO9GkiNYyaTPxkSTZu+98gfHMc7u8FW4BPIxfYo1tKRp5P3Ah MoH0q9QGqh0dhWyduypETmfPdlS0ewAek7a22/afqNGywudfH0TC/QFyZPwZsZZbL2Bqcj gM8yJDNLy2kI7XZCOUm6LZGOsy8WhiM= Received: from mail-oi1-f197.google.com (mail-oi1-f197.google.com [209.85.167.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-59-CDIct0agM8-iaz0Gjnwp2g-1; Tue, 08 Aug 2023 22:14:09 -0400 X-MC-Unique: CDIct0agM8-iaz0Gjnwp2g-1 Received: by mail-oi1-f197.google.com with SMTP id 5614622812f47-3a73fbef692so12356399b6e.2 for ; Tue, 08 Aug 2023 19:14:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547249; x=1692152049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=LHXAc9nM6zHt59Gqpc09SlFcQ6zr4302QgGsqUh+NHYPLqec5IW3Vung9dR3yyiPZN Wa2/0XDCI+p/pzPy65mFjN49uKhls8vFRJ24HVH0DQJhJJf8B3AD6UMScN8z5psXlTbX +p6JsQDcDhkOaT3c6WTWwX/GGGDAyI3+HVzdQkbw9JuVDEo/SXAb44m7otbbBTvhAxSk 05Ik0BDqE7hR2MGJfzMKOvwD51JyLqOeKDwHpnukYIB4kLM76AtAEQ3yJE+TtmYnkTHT qbSYLftnTi1M4bOWmydnke4+mXR3n2Vi/r6gly2QG8W+wwiPa97jTk5fFXXb73fjFwXd QWjQ== X-Gm-Message-State: AOJu0YzjMoxFMm4U/HnbOLATkI5lCSRVDsAywgXj2xYFTsRN65huRs9Q JsRXwKCvCNq/tqE3dKHC0c0364/sfFOHIAx//Tb/9o37DNbwE0jndow/keDlabPfqUoe7p2zd81 wqRfRdOjqfEEgWEA/3bztVuax X-Received: by 2002:a05:6870:c082:b0:1bf:74cc:c815 with SMTP id c2-20020a056870c08200b001bf74ccc815mr1491705oad.19.1691547248946; Tue, 08 Aug 2023 19:14:08 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGzmTXNkLqFmS1q1/FR/HeyFRU6fl+VZuoTT+xk0Lf/Zu30HJE0YhZ23Lp17N5hlJIuru2Olg== X-Received: by 2002:a05:6870:c082:b0:1bf:74cc:c815 with SMTP id c2-20020a056870c08200b001bf74ccc815mr1491689oad.19.1691547248647; Tue, 08 Aug 2023 19:14:08 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.14.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:08 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Date: Tue, 8 Aug 2023 23:13:08 -0300 Message-ID: <20230809021311.1390578-5-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Some functions use mostly the same asm for 32-bit and 64-bit versions. Make a macro that is generic enough and avoid code duplication. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Guo Ren Reviewed-by: Andrea Parri --- arch/riscv/include/asm/atomic.h | 164 +++++++++++++++----------------- 1 file changed, 76 insertions(+), 88 deletions(-) diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomi= c.h index f5dfef6c2153..80cca7ac16fd 100644 --- a/arch/riscv/include/asm/atomic.h +++ b/arch/riscv/include/asm/atomic.h @@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i) #undef ATOMIC_FETCH_OP #undef ATOMIC_OP_RETURN =20 +#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " beq %[p], %[u], 1f\n" \ + " add %[rc], %[p], %[a]\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : [a]"r" (_a), [u]"r" (_u) \ + : "memory"); \ +}) + /* This is required to provide a full barrier on success. */ static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a= , int u) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w"); + return prev; } #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless @@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_un= less(atomic64_t *v, s64 a, s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d"); + return prev; } #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless #endif =20 +#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bltz %[p], 1f\n" \ + " addi %[rc], %[p], 1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "w"); + return !(prev < 0); } =20 #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative =20 +#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bgtz %[p], 1f\n" \ + " addi %[rc], %[p], -1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "w"); + return !(prev > 0); } =20 #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive =20 +#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " addi %[rc], %[p], -1\n" \ + " bltz %[rc], 1f\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline int arch_atomic_dec_if_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "w"); + return prev - 1; } =20 @@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_n= egative(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "d"); + return !(prev < 0); } =20 @@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_p= ositive(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "d"); + return !(prev > 0); } =20 @@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positi= ve(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "d"); + return prev - 1; } =20 --=20 2.41.0 From nobody Sat Feb 7 17:19:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8914C001DE for ; Wed, 9 Aug 2023 02:15:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229819AbjHICPF (ORCPT ); Tue, 8 Aug 2023 22:15:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230455AbjHICPB (ORCPT ); Tue, 8 Aug 2023 22:15:01 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3560B1BD3 for ; Tue, 8 Aug 2023 19:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yR44ogH6UDfncEstCVw9mSlc1d7ANRRQAVF648wbdjc=; b=ccKJCTSeZLh+a3usv3PTHRAW1GYDOfgwiEoyYbrIag47rmQpz9f6ANzaB4shd4GJVSk63Q fmR/wiZbRxMAFeMFdtTJGB1/SQD+Ozzzbk7W7ssn3ArZmkExC3xYSOE9tsUhMNroCSaTHr 1hCJFm+8Oy6W81cOxw3kTSUoFd8Mzqs= Received: from mail-ot1-f72.google.com (mail-ot1-f72.google.com [209.85.210.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-161-xq9A6TIuN7mbAS6ZXW-eYQ-1; Tue, 08 Aug 2023 22:14:15 -0400 X-MC-Unique: xq9A6TIuN7mbAS6ZXW-eYQ-1 Received: by mail-ot1-f72.google.com with SMTP id 46e09a7af769-6bb31e13a13so11085595a34.1 for ; Tue, 08 Aug 2023 19:14:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547254; x=1692152054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yR44ogH6UDfncEstCVw9mSlc1d7ANRRQAVF648wbdjc=; b=WFiIawWKMOs16siPhz4KO0owLIAclMJBxA5zlm54/ilVeFGJc/Urs2i1SjFuNKaWZx KocSGB7NTdTlQkZd9yGLY62da/PNDVWKp8ty867b/K3nZEvCggE8j2XkNT3d63qcx9Ag MC5zlRInJJpKF1+N8lGtOPAkLm96bNcAZKCm2Ob6PGbVFFKi131ZDoR90ptaXPCWVzGh +J6brggIxd5u+lKFEqVYOncnZzPATFJtHTV02essNB+aRByctGyxe2v9iLY9YID8s9v+ 09Liv0B4ObRUmIijIPCQEV830+quotBmPafqBOkofu5a61Ve7HigCh98sv8z9B1mmUFk iEJQ== X-Gm-Message-State: AOJu0Yw3lTOkMRgf1pNw1cX1KvEfChaXGuCTqFE/i7yZ7VsGQlmAmhh8 HnZcPwG1oMrjj7eOordVgq0YZFinJ0zj4NplNsbUj14Aek4lgEKr3gaJoHiOBg7sKYIt1Sp+Q2p UclwjycqICpAAMqJK/1JLqx+K X-Received: by 2002:a05:6870:40d1:b0:1be:d3a1:fd9 with SMTP id l17-20020a05687040d100b001bed3a10fd9mr1815530oal.9.1691547254601; Tue, 08 Aug 2023 19:14:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHviWJ7FwqHHQS/IFWOJZUUEtSZUVOo6M1kD/UsKsoQJL4BRummKJuVjUiXc3Pder8uF/0zRw== X-Received: by 2002:a05:6870:40d1:b0:1be:d3a1:fd9 with SMTP id l17-20020a05687040d100b001bed3a10fd9mr1815505oal.9.1691547254170; Tue, 08 Aug 2023 19:14:14 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.14.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:13 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Date: Tue, 8 Aug 2023 23:13:09 -0300 Message-ID: <20230809021311.1390578-6-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" cmpxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 34 ++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index 5a07646fae65..cfada8a7cfd2 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -72,6 +72,35 @@ * indicated by comparing RETURN with OLD. */ =20 +#define __arch_cmpxchg_masked(sc_sfx, prepend, append, r, p, o, n) \ +({ \ + u32 *__ptr32b =3D (u32 *)((ulong)(p) & ~0x3); \ + ulong __s =3D ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask =3D GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx =3D (ulong)(n) << __s; \ + ulong __oldx =3D (ulong)(o) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z5\n" \ + " bne %1, %z3, 1f\n" \ + " and %1, %0, %z6\n" \ + " or %1, %1, %z4\n" \ + " sc.w" sc_sfx " %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=3D&r" (__retx), "=3D&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" ((long)__oldx), "rJ" (__newx), \ + "rJ" (__mask), "rJ" (~__mask) \ + : "memory"); \ + \ + r =3D (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) =20 #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ @@ -98,6 +127,11 @@ __typeof__(*(ptr)) __ret; \ \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_cmpxchg_masked(sc_sfx, prepend, append, \ + __ret, __ptr, __old, __new); \ + break; \ case 4: \ __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ __ret, __ptr, (long), __old, __new); \ --=20 2.41.0 From nobody Sat Feb 7 17:19:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83F1EC001DB for ; Wed, 9 Aug 2023 02:15:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229746AbjHICPN (ORCPT ); Tue, 8 Aug 2023 22:15:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229540AbjHICPI (ORCPT ); Tue, 8 Aug 2023 22:15:08 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA68E1BDA for ; Tue, 8 Aug 2023 19:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QSwkRG8bcSleBdtSsjCnyBmp/8Y+sHuiXOKQQOlQtH4=; b=WnzLQHsd7uNV/mInjNmcgWwQEaaqMqkhL+Q107HLGkQxwVk+E14r9nXvBDSaoKZqojPxDe bBNsRJqoffpb6Jizzm1e8sJSbD5PzZGvcLIfGXyRBI/JhxsbdJP8Klkiw4P+vk0kjVQl0r +dkCqWxK7VGWw0LjZUJ6ab14NaHa8/k= Received: from mail-oa1-f69.google.com (mail-oa1-f69.google.com [209.85.160.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-371-5fwvUZTtMJeHDZxbD_W5lg-1; Tue, 08 Aug 2023 22:14:21 -0400 X-MC-Unique: 5fwvUZTtMJeHDZxbD_W5lg-1 Received: by mail-oa1-f69.google.com with SMTP id 586e51a60fabf-1b0812d43a0so597717fac.0 for ; Tue, 08 Aug 2023 19:14:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547261; x=1692152061; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QSwkRG8bcSleBdtSsjCnyBmp/8Y+sHuiXOKQQOlQtH4=; b=dFc0LrQprMs9CqGnkc3Wbbm2DVSObRM1LEPTfAs7XL03ee32d3gBqsivnQdLqWe26R qdB2KWtFZMTOYsbcyLtH9Flpz88ELwpi63B95tr6E3n+YkAat9lLPWBw9AsQOB0Jm2Yc 9NgVctX+dNVJD3zMUjmG6LdQ0tP1ziWazSpt7bOoczSuCz93aPqV63H+xSW02zkFm4Mg t50d7I+nutIAp03kSVOlCZHGCNDLlXqH0o0miTVJn+Y19AuhFnFwtaY0+lNa4zhaf2Us tfeyCWF0/8GoL2VW9p5fHGrEggJg06X9L8LFW1kkCGOfwTujn8Y/SzJPhf0T3EI5pzkc NP2Q== X-Gm-Message-State: AOJu0YyEgv2SRvyg14A8BBynm4qyKFGSkdZ33GKCEC5njkl/7pumrmTY BzrZFMbFmo6ngyc6tf6fS8fs1W15HqPjRftyQ8c1TJINpGSdeo2+2meIRlGD7Tjjj1s/sY46D4y qN5H2/W8MjztJxxRl/bBY05R1 X-Received: by 2002:a05:6870:b6a9:b0:1bf:1c49:749c with SMTP id cy41-20020a056870b6a900b001bf1c49749cmr8730778oab.13.1691547260770; Tue, 08 Aug 2023 19:14:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEHlHClRw7hdQrrpejz4vuDyEoE85EOUKHc1btv2W4ZMe793U+R5Uo/+moePgCrObCJ/o3ANw== X-Received: by 2002:a05:6870:b6a9:b0:1bf:1c49:749c with SMTP id cy41-20020a056870b6a900b001bf1c49749cmr8730768oab.13.1691547260529; Tue, 08 Aug 2023 19:14:20 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.14.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:20 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2 Date: Tue, 8 Aug 2023 23:13:10 -0300 Message-ID: <20230809021311.1390578-7-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index cfada8a7cfd2..1640308f3c4d 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,6 +11,31 @@ #include #include =20 +#define __arch_xchg_masked(prepend, append, r, p, n) \ +({ \ + u32 *__ptr32b =3D (u32 *)((ulong)(p) & ~0x3); \ + ulong __s =3D ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask =3D GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx =3D (ulong)(n) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z4\n" \ + " or %1, %1, %z3\n" \ + " sc.w %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + : "=3D&r" (__retx), "=3D&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" (__newx), "rJ" (~__mask) \ + : "memory"); \ + \ + r =3D (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) + #define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ __asm__ __volatile__ ( \ @@ -27,7 +52,13 @@ __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ __typeof__(ptr) __ptr =3D (ptr); \ + \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_xchg_masked(prepend, append, \ + __ret, __ptr, __new); \ + break; \ case 4: \ __arch_xchg(".w" sfx, prepend, append, \ __ret, __ptr, __new); \ --=20 2.41.0