From nobody Sun Feb 8 21:27:23 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E993CEB64DD for ; Thu, 3 Aug 2023 05:15:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232274AbjHCFP1 (ORCPT ); Thu, 3 Aug 2023 01:15:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229848AbjHCFPS (ORCPT ); Thu, 3 Aug 2023 01:15:18 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DE8630F3 for ; Wed, 2 Aug 2023 22:14:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691039676; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wZcGwtCjzhW0OBBycEUaxWbT2VDTB4yqfuLafpABV3I=; b=BXZ0FV9j6TjX2EYvOwAwdRSW7R618ud/eBxwJNt20dZkzlXWx630dXa5ap8FjkKZLpEwSe QwpOS0hG+AhS2NY8W8h/HAkpe1N9MHLpc5HXyc8rtL6NdGvbhHyERfISuwPDAw5D7hhnlu bDbyn6KW+ia4jX8bQRKM56Ox7am1xnc= Received: from mail-oo1-f71.google.com (mail-oo1-f71.google.com [209.85.161.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-561-fG-lvLzbPhaETRbcPBGo8Q-1; Thu, 03 Aug 2023 01:14:35 -0400 X-MC-Unique: fG-lvLzbPhaETRbcPBGo8Q-1 Received: by mail-oo1-f71.google.com with SMTP id 006d021491bc7-56c7be22ca8so596318eaf.2 for ; Wed, 02 Aug 2023 22:14:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691039674; x=1691644474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wZcGwtCjzhW0OBBycEUaxWbT2VDTB4yqfuLafpABV3I=; b=U1PcKAT7hEGgrTTOI90vlzTHIlbbu7MVr8WnVvizyypOKB6GPu+xb59W+PIGxIj0m8 Tv3h/ofWpivQ22FaLt2qdmSUtxsqxCsKrPBV8FRTq69gaotHZy4xwMi6a5fcjc7LuHPq NWfJjYsox0OCFvUn2DIcQIZbpJyDTBNGMVYjNTuy3yh9TDdUkiOmFu5H5gZGJ4tF6yXc kcK/2HAAhk/C2Y/FdqnaCkuUfAHN7s3ny1LDQq4eLOYYpP8bFsfCESclEGoVq5CxYJ8s yZ60zWhs0SNoOGUPUMuxrDHA6FQP5rzfPK2OAnr5HewwKanJAoYo+4i+30gAdXIExON4 AsXw== X-Gm-Message-State: ABy/qLYKNzf4ux3oRlEf3E/4Quk4b80RnmIvoex9QeLcQDoTtRnSiVUv 668kcB5UVU6Wx9uDy9QuL3lH4AhNvkNi/xsS+/ahWS+CMr3SRTDb7efAox4Cfrpcq6Bm/ROSqo3 EvmzAjMV+Ux503fLEmAnvTHdd X-Received: by 2002:a05:6808:b08:b0:3a7:5314:e572 with SMTP id s8-20020a0568080b0800b003a75314e572mr5934128oij.53.1691039674431; Wed, 02 Aug 2023 22:14:34 -0700 (PDT) X-Google-Smtp-Source: APBJJlEfiz2uOFuhiqR6748NuQ/vIPQxwC/CwbLPbjx+5B22xsQA9uZm1XAzQNvPTBTAb3fsMZ+03g== X-Received: by 2002:a05:6808:b08:b0:3a7:5314:e572 with SMTP id s8-20020a0568080b0800b003a75314e572mr5934111oij.53.1691039674128; Wed, 02 Aug 2023 22:14:34 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id g51-20020a0568080df300b003a75593746asm1675473oic.57.2023.08.02.22.14.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Aug 2023 22:14:33 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrzej Hajda , Arnd Bergmann , Ingo Molnar , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v2 1/3] riscv/cmpxchg: Deduplicate xchg() asm functions Date: Thu, 3 Aug 2023 02:13:58 -0300 Message-ID: <20230803051401.710236-3-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230803051401.710236-2-leobras@redhat.com> References: <20230803051401.710236-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_xchg* generation. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri Tested-by. --- arch/riscv/include/asm/cmpxchg.h | 136 +++++-------------------------- 1 file changed, 22 insertions(+), 114 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index 2f4726d3cfcc..ec4ea4f3f908 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,60 +11,30 @@ #include #include =20 -#define __xchg_relaxed(ptr, new, size) \ +#define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_relaxed(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_relaxed((ptr), \ - _x_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + " amoswap" sfx " %0, %2, %1\n" \ + append \ + : "=3Dr" (r), "+A" (*(p)) \ + : "r" (n) \ + : "memory"); \ }) =20 -#define __xchg_acquire(ptr, new, size) \ +#define _arch_xchg(ptr, new, sfx, prepend, append) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ + __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ - switch (size) { \ + __typeof__(ptr) __ptr =3D (ptr); \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".w" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".d" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -72,79 +42,17 @@ __ret; \ }) =20 -#define arch_xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) +#define arch_xchg_relaxed(ptr, x) \ + _arch_xchg(ptr, x, "", "", "") =20 -#define __xchg_release(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_xchg_acquire(ptr, x) \ + _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER) =20 #define arch_xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __arch_xchg(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "") =20 #define arch_xchg(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ -}) + _arch_xchg(ptr, x, ".aqrl", "", "") =20 #define xchg32(ptr, x) \ ({ \ --=20 2.41.0 From nobody Sun Feb 8 21:27:23 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16BBDC001DF for ; Thu, 3 Aug 2023 05:15:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232504AbjHCFPm (ORCPT ); Thu, 3 Aug 2023 01:15:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232240AbjHCFP0 (ORCPT ); Thu, 3 Aug 2023 01:15:26 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 621E3E43 for ; Wed, 2 Aug 2023 22:14:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691039680; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W1KA0kD/6MDaOI4zfIG5REanwKgkBx0iY6arLWoliks=; b=Q0BsO6puKz1mftxMBynDhq8ajxIVo+TlNqK88xvAtgyAAkvNIOFQxiQhKh11l7wgX8XplP RdOQcT1vSxpipAxlY3R5l3S2MVKlHYyEhSv2qwGiKzaZrZuEz1TrMTtduH4XQx5tMC3HxK GnpijAup+RBenZEWJ7kL3OWNC/VQMjk= Received: from mail-oi1-f197.google.com (mail-oi1-f197.google.com [209.85.167.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-626-TcYUt8eeOW60GUcoO0oxxA-1; Thu, 03 Aug 2023 01:14:38 -0400 X-MC-Unique: TcYUt8eeOW60GUcoO0oxxA-1 Received: by mail-oi1-f197.google.com with SMTP id 5614622812f47-3a5b891b4e8so898346b6e.1 for ; Wed, 02 Aug 2023 22:14:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691039678; x=1691644478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W1KA0kD/6MDaOI4zfIG5REanwKgkBx0iY6arLWoliks=; b=hs8n8kZxQE3eW/eX0rAX0+aoqhaubgKM86RYACh50RfBXR3WIR9nD0XRp5KkTKYhSG 7n33cvKEn2z+bbCgPQefUkocKbx/ec9sZ1zZc8QgEkdaYyx1SnTAqNk02gEi86QYyB7l euItu6bdwGbkCGUK7jiNsMPupGVIsug5NP8DhtXD/MUVWolVWaXfreUKK62MbCQTSumJ 36Cakj5bOsvg02qvsuuWM09nl12ZwKL9aNp/bocyX5brgHxwikev/69PQqwcXXl8i68Y KICel6KwGHldfoGwt6yQ1msQ4jiGfAJaKfJeFOmgvApk89sr4G6Kwm2IEknrTx33oXRi CzHw== X-Gm-Message-State: ABy/qLbUSr9YzQFXaS8Y4vmr2zc4zfL4QF17P8GxwR/uVfOpU4QsspAa m5TCYxDjZmk3ZiBJx+/HzXuLeq/VweOB7NdKMgKr7aOT5jZ80rBtSpcu8n0ZKaaChrQ5MySQoJE kSTJUTvFUPK45/KrysKqwWa2Q X-Received: by 2002:aca:1e13:0:b0:3a3:60f0:5e0c with SMTP id m19-20020aca1e13000000b003a360f05e0cmr17682089oic.17.1691039678070; Wed, 02 Aug 2023 22:14:38 -0700 (PDT) X-Google-Smtp-Source: APBJJlFW66yJ6nJQdGaDS+gONtU0r4J6UZ2vOX1WZnWf69XOvBAHQUNuyc5AOVoWEZG4ifYlR/MX0g== X-Received: by 2002:aca:1e13:0:b0:3a3:60f0:5e0c with SMTP id m19-20020aca1e13000000b003a360f05e0cmr17682067oic.17.1691039677802; Wed, 02 Aug 2023 22:14:37 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id g51-20020a0568080df300b003a75593746asm1675473oic.57.2023.08.02.22.14.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Aug 2023 22:14:37 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrzej Hajda , Arnd Bergmann , Ingo Molnar , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v2 2/3] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Date: Thu, 3 Aug 2023 02:13:59 -0300 Message-ID: <20230803051401.710236-4-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230803051401.710236-2-leobras@redhat.com> References: <20230803051401.710236-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_cmpxchg* generation (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri Tested-by. --- arch/riscv/include/asm/cmpxchg.h | 182 +++++-------------------------- 1 file changed, 25 insertions(+), 157 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index ec4ea4f3f908..bb45ef83592f 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -71,81 +71,37 @@ * store NEW in MEM. Return the initial value in MEM. Success is * indicated by comparing RETURN with OLD. */ -#define __cmpxchg_relaxed(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) =20 -#define arch_cmpxchg_relaxed(ptr, o, n) \ +#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, rc, p, co, o, n= )\ ({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_relaxed((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=3D&r" (r), "=3D&r" (rc), "+A" (*(p)) \ + : "rJ" (co o), "rJ" (n) \ + : "memory"); \ }) =20 -#define __cmpxchg_acquire(ptr, old, new, size) \ +#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr =3D (ptr); \ __typeof__(*(ptr)) __old =3D (old); \ __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ register unsigned int __rc; \ - switch (size) { \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ + __ret, __rc, __ptr, (long), __old, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".d", ".d" sc_sfx, prepend, append, \ + __ret, __rc, __ptr, /**/, __old, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -153,108 +109,20 @@ __ret; \ }) =20 -#define arch_cmpxchg_acquire(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) +#define arch_cmpxchg_relaxed(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", "") =20 -#define __cmpxchg_release(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_cmpxchg_acquire(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER) =20 #define arch_cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) - -#define __cmpxchg(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "") =20 #define arch_cmpxchg(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) + _arch_cmpxchg((ptr), (o), (n), ".rl", "", " fence rw, rw\n") =20 #define arch_cmpxchg_local(ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) + arch_cmpxchg_relaxed((ptr), (o), (n)) =20 #define arch_cmpxchg64(ptr, o, n) \ ({ \ --=20 2.41.0 From nobody Sun Feb 8 21:27:23 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2F4CEB64DD for ; Thu, 3 Aug 2023 05:15:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232619AbjHCFPx (ORCPT ); Thu, 3 Aug 2023 01:15:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232332AbjHCFPb (ORCPT ); Thu, 3 Aug 2023 01:15:31 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AD5B1702 for ; Wed, 2 Aug 2023 22:14:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691039683; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h1h18rhaSf68S755mU5NBDbODv+jJVDksEUt3f2j0qk=; b=RYrSvqmdzcJv3kihRXrlVl9UXIyGb6Po7w2wZAn6uwOmL8WXTObXqoF3HNTD7SsphrHxvO qWK6aQ//xW1g5xFsyWxGyP2YDLEvx5jarF+dQoa8rahov3cXTzSO5vumu+4KJ2zfSSNhhI jA7A9Q1jjfVrxLzC3+MmeR2gC8d/Epc= Received: from mail-oi1-f198.google.com (mail-oi1-f198.google.com [209.85.167.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-gu5GiXLENFOLpqbEo8bsNg-1; Thu, 03 Aug 2023 01:14:42 -0400 X-MC-Unique: gu5GiXLENFOLpqbEo8bsNg-1 Received: by mail-oi1-f198.google.com with SMTP id 5614622812f47-3a6e180e49aso836552b6e.0 for ; Wed, 02 Aug 2023 22:14:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691039681; x=1691644481; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=h1h18rhaSf68S755mU5NBDbODv+jJVDksEUt3f2j0qk=; b=F/FRBNd6DqmbFd4gZi4nH5zQHR7dPcIT1IZIt/AvhG8OBXzT6uomsAof/vPJ/ldhje pD4A1sENCNg/SbxXuBygJ9GbmGS2OaAdKXhkRbeXlllIKDMGIExJLHidGrIqEzjMtKmY rzH7PkCjpEaW/yN1+HWCMb/c4A43rhXcvJ6lO4VhZPOjLbcucYrtC8+T8aXdU41uxqh/ ZUEc9HziVEbAHrRGxtMnoU11jU1kGO9pipdzyoIxYD+NpOVetcm4GZ8cvF9KV4nbZupG UTF4d3ujQUGGF/Zvi3aH9PGdXgN5XwrBSjzbM07yxdNZB4ZLr72Qk/U2GoN6lwFhPPeB q6MQ== X-Gm-Message-State: ABy/qLYsUMqklVjnzeOhc/OlE3wlTJLvOXvAfSS/eBmIPwPdI7+qzZ3t iHgWIkC7dLNVnxVAJgVGPcBcIgrQgCqUi1BbBeY29nO5OgVmTJd0blGMsMUW41zg0vc3+WXoIML 8LmQm4rRyV45ucJ+xzaCjKXnw X-Received: by 2002:a05:6808:23d4:b0:3a7:372f:c19a with SMTP id bq20-20020a05680823d400b003a7372fc19amr14491302oib.38.1691039681656; Wed, 02 Aug 2023 22:14:41 -0700 (PDT) X-Google-Smtp-Source: APBJJlENwadVi5uDsyPPKkHVKS9MXvZ6hztr6bcflbRSPmNVsi3Dh4uL43ln1uE8LA6um9fwL2P0ug== X-Received: by 2002:a05:6808:23d4:b0:3a7:372f:c19a with SMTP id bq20-20020a05680823d400b003a7372fc19amr14491285oib.38.1691039681430; Wed, 02 Aug 2023 22:14:41 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id g51-20020a0568080df300b003a75593746asm1675473oic.57.2023.08.02.22.14.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Aug 2023 22:14:41 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrzej Hajda , Arnd Bergmann , Ingo Molnar , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v2 3/3] riscv/atomic.h : Deduplicate arch_atomic.* Date: Thu, 3 Aug 2023 02:14:00 -0300 Message-ID: <20230803051401.710236-5-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230803051401.710236-2-leobras@redhat.com> References: <20230803051401.710236-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Some functions use mostly the same asm for 32-bit and 64-bit versions. Make a macro that is generic enough and avoid code duplication. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri Reviewed-by: Guo Ren =20 Tested-by. --- arch/riscv/include/asm/atomic.h | 164 +++++++++++++++----------------- 1 file changed, 76 insertions(+), 88 deletions(-) diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomi= c.h index f5dfef6c2153..80cca7ac16fd 100644 --- a/arch/riscv/include/asm/atomic.h +++ b/arch/riscv/include/asm/atomic.h @@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i) #undef ATOMIC_FETCH_OP #undef ATOMIC_OP_RETURN =20 +#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " beq %[p], %[u], 1f\n" \ + " add %[rc], %[p], %[a]\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : [a]"r" (_a), [u]"r" (_u) \ + : "memory"); \ +}) + /* This is required to provide a full barrier on success. */ static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a= , int u) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w"); + return prev; } #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless @@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_un= less(atomic64_t *v, s64 a, s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d"); + return prev; } #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless #endif =20 +#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bltz %[p], 1f\n" \ + " addi %[rc], %[p], 1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "w"); + return !(prev < 0); } =20 #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative =20 +#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bgtz %[p], 1f\n" \ + " addi %[rc], %[p], -1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "w"); + return !(prev > 0); } =20 #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive =20 +#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " addi %[rc], %[p], -1\n" \ + " bltz %[rc], 1f\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline int arch_atomic_dec_if_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "w"); + return prev - 1; } =20 @@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_n= egative(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "d"); + return !(prev < 0); } =20 @@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_p= ositive(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "d"); + return !(prev > 0); } =20 @@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positi= ve(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "d"); + return prev - 1; } =20 --=20 2.41.0