From nobody Fri Sep 12 05:23:41 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6A71C001B0 for ; Thu, 10 Aug 2023 04:05:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231771AbjHJEFC (ORCPT ); Thu, 10 Aug 2023 00:05:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229518AbjHJEE5 (ORCPT ); Thu, 10 Aug 2023 00:04:57 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF3E311F for ; Wed, 9 Aug 2023 21:04:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640255; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OScrv20Vk5PZ28CoHB13qKk09OrWPyCgb0XsKA59wTM=; b=AEL+gLRIL2frkBTT92O94PYXeOZ+cSM5Lck2r7LtOXD2fya+DRNYvrSMoUjWt+fGX2Dpfy DwsihkBtRh6K9B0SoHY4Ov0Od4dCZ5furricmebcca/oLV53jNI3iYQs8lAv6ViObm43m/ psr1+Ouu4IxoY+yoPtxE6krypvZBXwg= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-422-VSBmhtDHOgqGYPWzzHMhkg-1; Thu, 10 Aug 2023 00:04:13 -0400 X-MC-Unique: VSBmhtDHOgqGYPWzzHMhkg-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a78c2cc1adso549844b6e.3 for ; Wed, 09 Aug 2023 21:04:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640252; x=1692245052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OScrv20Vk5PZ28CoHB13qKk09OrWPyCgb0XsKA59wTM=; b=J0PTUww3Yiuw45xQWAkE7kITleVtL4WMXPhiaBC4GMmo0Z21q7FkzEeAHDXIaAEvXB TrV6IGijhq3l5sb4pMBZDyA27r1Ud07s+60xhH/PGUhWfL/bb1D78Imza9a8piYcNBcb zqBB9aCJ7awxZ9bCANtfx9PyYYNckUrYWMfM+fNXifvnShjgrhvNRopYm36mHxNn3ERI pKq70mlEep2U5vy4M30usqHWCzKUwcTH3oWFROjvL+BZELFDZ04CnlfU5HMDWuR5UNmY QKV9OiMiIJiXvW0Wp+gPTSBqrKzZJV9eHAO9Jb7wNf+WcELvYW5MIbaUb3mfqsidJLrt Pq3w== X-Gm-Message-State: AOJu0YxR7ghQwjea+ZF26s7nFs5GCsRw/UUYSvZ8uwn0uxL+6kVh49LB bhzYIWq+7XroQ33Mqw4h5n4QSMS0iRjBeZ9JlT9afkiaovw/Oslw4k3Vx5n6Ma/iT5LcCbSQVX3 wNKGVygGSr1vQP5MtZfOlcqIL X-Received: by 2002:a05:6808:16a8:b0:3a7:65ae:9ccf with SMTP id bb40-20020a05680816a800b003a765ae9ccfmr1751538oib.22.1691640252583; Wed, 09 Aug 2023 21:04:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGXuC0j5A0JXiqF9jKMAp5qoF1YnSFxPZWZ79s3qRiKxxZaWZdWWqxSD/vcnxGSyUwfd9n7/g== X-Received: by 2002:a05:6808:16a8:b0:3a7:65ae:9ccf with SMTP id bb40-20020a05680816a800b003a765ae9ccfmr1751518oib.22.1691640252341; Wed, 09 Aug 2023 21:04:12 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:11 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Date: Thu, 10 Aug 2023 01:03:43 -0300 Message-ID: <20230810040349.92279-3-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_xchg* generation. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri Tested-by: Guo Ren --- arch/riscv/include/asm/cmpxchg.h | 138 ++++++------------------------- 1 file changed, 23 insertions(+), 115 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index 2f4726d3cfcc..48478a8eecee 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,140 +11,48 @@ #include #include =20 -#define __xchg_relaxed(ptr, new, size) \ +#define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_relaxed(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_relaxed((ptr), \ - _x_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + " amoswap" sfx " %0, %2, %1\n" \ + append \ + : "=3Dr" (r), "+A" (*(p)) \ + : "r" (n) \ + : "memory"); \ }) =20 -#define __xchg_acquire(ptr, new, size) \ +#define _arch_xchg(ptr, new, sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ + __typeof__(*(__ptr)) __new =3D (new); \ + __typeof__(*(__ptr)) __ret; \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".w" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".d" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ default: \ BUILD_BUG(); \ } \ - __ret; \ + (__typeof__(*(__ptr)))__ret; \ }) =20 -#define arch_xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) +#define arch_xchg_relaxed(ptr, x) \ + _arch_xchg(ptr, x, "", "", "") =20 -#define __xchg_release(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_xchg_acquire(ptr, x) \ + _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER) =20 #define arch_xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __arch_xchg(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "") =20 #define arch_xchg(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ -}) + _arch_xchg(ptr, x, ".aqrl", "", "") =20 #define xchg32(ptr, x) \ ({ \ --=20 2.41.0 From nobody Fri Sep 12 05:23:41 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1CEAC001B0 for ; Thu, 10 Aug 2023 04:05:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232623AbjHJEFH (ORCPT ); Thu, 10 Aug 2023 00:05:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232363AbjHJEE7 (ORCPT ); Thu, 10 Aug 2023 00:04:59 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2180125 for ; Wed, 9 Aug 2023 21:04:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640260; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WcBubYngvVAQlkkTX8jFLD9nX24FtmcCz/1wvQAzhyI=; b=cTZvnkiYYwwOpSbpPxaZmn29lguBQ7qfGV3cONZr8skO6qpNC/8WDBtn101ckVda9A7YdI lZJ6kEDHzpP1McDsHyKIAAkNhn3pZcWowAk5xlO6U33rN3yWMZSwJxHPL5HtFg1/jSGi3L lMBGgY/m6jvDcFCETue+rfCVtkyf3/A= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-29-oUqOPZcyM-KfO_CCY-vZPA-1; Thu, 10 Aug 2023 00:04:18 -0400 X-MC-Unique: oUqOPZcyM-KfO_CCY-vZPA-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a7d4302c71so47670b6e.0 for ; Wed, 09 Aug 2023 21:04:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640258; x=1692245058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WcBubYngvVAQlkkTX8jFLD9nX24FtmcCz/1wvQAzhyI=; b=T4Mq/buAdRDFmBzjnnsaD5KUr0IfrBCR2cWo1VwiAyiVbq6CP//RhaFM41R4K01Mje 8qtppI+5664kUtFIMMhju2ozshQhlyewiIAV0+ORWy1+TsaYlTVKpb1M0fwTgSr3UzZ8 pSYDQZkw8FkrKptswJXQGGQMEWWye36jyqC9eXoxuKTkgE+oTfaTaMQchEG/nyM44Ixc w2/GzMWAKkcJv2WZ+Ifm/yPSgXThOGg+dP34lScjFeM/IJNNeEBFttc+ACFOUW27k2B4 wkxys20403B0E8hdZ03I5VgJVtcNL4cfxEaELhqT3JUAQB6XdZuSNyUc7c6lr9CA0oLs nb1A== X-Gm-Message-State: AOJu0Yx3MY1/YLnhXb6RTtRA/ioFGoXKg2cPocTu80WbGzfxKgLjsDMT qdSemwdt6/TdZZg+ANOUZMcU73OoWikRL6CUc/VoxOYSAlzcBblJUNJFRMYN17EAwd4oLSadw0w rc248I5o4eQ/zcZeL6WmSwVac X-Received: by 2002:a05:6808:1514:b0:3a2:5d31:b299 with SMTP id u20-20020a056808151400b003a25d31b299mr594885oiw.28.1691640257834; Wed, 09 Aug 2023 21:04:17 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHyCNfWl+g7sjPkEBxDgZwisNNLc1LhGyzcSUUej8NIuwUa4eFN8ZtD/RMr0iEkJ8Ba0KVwgw== X-Received: by 2002:a05:6808:1514:b0:3a2:5d31:b299 with SMTP id u20-20020a056808151400b003a25d31b299mr594877oiw.28.1691640257554; Wed, 09 Aug 2023 21:04:17 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:17 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Date: Thu, 10 Aug 2023 01:03:44 -0300 Message-ID: <20230810040349.92279-4-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_cmpxchg* generation (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri Tested-by: Guo Ren --- arch/riscv/include/asm/cmpxchg.h | 195 ++++++------------------------- 1 file changed, 33 insertions(+), 162 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index 48478a8eecee..e3e0ac7ba061 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -71,190 +71,61 @@ * store NEW in MEM. Return the initial value in MEM. Success is * indicated by comparing RETURN with OLD. */ -#define __cmpxchg_relaxed(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) =20 -#define arch_cmpxchg_relaxed(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_relaxed((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) =20 -#define __cmpxchg_acquire(ptr, old, new, size) \ +#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_cmpxchg_acquire(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=3D&r" (r), "=3D&r" (__rc), "+A" (*(p)) \ + : "rJ" (co o), "rJ" (n) \ + : "memory"); \ }) =20 -#define __cmpxchg_release(ptr, old, new, size) \ +#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ + __typeof__(*(__ptr)) __old =3D (old); \ + __typeof__(*(__ptr)) __new =3D (new); \ + __typeof__(*(__ptr)) __ret; \ + \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ + __ret, __ptr, (long), __old, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".d", ".d" sc_sfx, prepend, append, \ + __ret, __ptr, /**/, __old, __new); \ break; \ default: \ BUILD_BUG(); \ } \ - __ret; \ + (__typeof__(*(__ptr)))__ret; \ }) =20 -#define arch_cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) +#define arch_cmpxchg_relaxed(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", "") =20 -#define __cmpxchg(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_cmpxchg_acquire(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER) + +#define arch_cmpxchg_release(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "") =20 #define arch_cmpxchg(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) + _arch_cmpxchg((ptr), (o), (n), ".rl", "", " fence rw, rw\n") =20 #define arch_cmpxchg_local(ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) + arch_cmpxchg_relaxed((ptr), (o), (n)) =20 #define arch_cmpxchg64(ptr, o, n) \ ({ \ --=20 2.41.0 From nobody Fri Sep 12 05:23:41 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08DFCC001B0 for ; Thu, 10 Aug 2023 04:05:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232658AbjHJEFP (ORCPT ); Thu, 10 Aug 2023 00:05:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232583AbjHJEFF (ORCPT ); Thu, 10 Aug 2023 00:05:05 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E413120 for ; Wed, 9 Aug 2023 21:04:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=J9bu9AmFw8sltFqVpctH3UpjTbiNn38xvm5YfyPzVj0NAPZF94PD4yai4uMVrAkx+SgvE0 mtztcBZ1Vt30AURyOgf2v2XIvsDcV6FHp1xYRt6ybra+UEuorEUacbOxP0nvpxIn/kvmLu zW6Rtua2lrppq8isvjhKcIiRptVlh98= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-263-Z3_cq6pZP9W6V-MrCu7dNg-1; Thu, 10 Aug 2023 00:04:23 -0400 X-MC-Unique: Z3_cq6pZP9W6V-MrCu7dNg-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a7d402fc85so77409b6e.3 for ; Wed, 09 Aug 2023 21:04:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640263; x=1692245063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=DYteQqfF1lXhZZ27UZGPGl+y6u8CWctTL8ocNYq+jNk1O0mkVkkMfPvGVDZPUcpYhl djcmC97Jt8jBTZfwwrm4IRQrVahsc62NL/jzGRfTMeTkSPcfqX09RABH6vUUVDoZFeVV QeDK3HXpxYycxFPoJNYJrGOZwbpm4hKWwM3k9yHoJ2ryo/nbeuBD52ckMGGI5MOiq47A Yj3luTsw+1mVWk6b4XcdQuO9dtO835JiWTravi4YOAKPad+Wc1Dj6Aauqs4PIugVpx3J XvemBpye4wHcLt/l5wdXFspJOXGpExw0bO9trVExF7Xuyr5cciW2130wJwX2EIHMC2PC +KyA== X-Gm-Message-State: AOJu0YySb1a5FRCXEm6sC7/P5uTs0ViVaexzlContkfSFbf07F5Y9pl+ 9Mz157A95PVptOUrvM3AvDbrhaXks8mzgUEsbB/btsgXRN30IpyW0RKSOGp8cfSRKYVqG0G8Tkn RjZjL0sPesSx21+7jIZAYNAvt X-Received: by 2002:a05:6808:998:b0:3a7:540f:ca71 with SMTP id a24-20020a056808099800b003a7540fca71mr1397765oic.53.1691640262935; Wed, 09 Aug 2023 21:04:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEoKJIRWaoDwasWuxkhBvZiGi90hdZ5+o2Vt5j+J0Py97VRwdk+5mjOtZ8GjfL51BjhINmyUg== X-Received: by 2002:a05:6808:998:b0:3a7:540f:ca71 with SMTP id a24-20020a056808099800b003a7540fca71mr1397752oic.53.1691640262690; Wed, 09 Aug 2023 21:04:22 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:22 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Date: Thu, 10 Aug 2023 01:03:45 -0300 Message-ID: <20230810040349.92279-5-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Some functions use mostly the same asm for 32-bit and 64-bit versions. Make a macro that is generic enough and avoid code duplication. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Guo Ren Reviewed-by: Andrea Parri Tested-by: Guo Ren --- arch/riscv/include/asm/atomic.h | 164 +++++++++++++++----------------- 1 file changed, 76 insertions(+), 88 deletions(-) diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomi= c.h index f5dfef6c2153..80cca7ac16fd 100644 --- a/arch/riscv/include/asm/atomic.h +++ b/arch/riscv/include/asm/atomic.h @@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i) #undef ATOMIC_FETCH_OP #undef ATOMIC_OP_RETURN =20 +#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " beq %[p], %[u], 1f\n" \ + " add %[rc], %[p], %[a]\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : [a]"r" (_a), [u]"r" (_u) \ + : "memory"); \ +}) + /* This is required to provide a full barrier on success. */ static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a= , int u) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w"); + return prev; } #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless @@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_un= less(atomic64_t *v, s64 a, s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d"); + return prev; } #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless #endif =20 +#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bltz %[p], 1f\n" \ + " addi %[rc], %[p], 1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "w"); + return !(prev < 0); } =20 #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative =20 +#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bgtz %[p], 1f\n" \ + " addi %[rc], %[p], -1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "w"); + return !(prev > 0); } =20 #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive =20 +#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " addi %[rc], %[p], -1\n" \ + " bltz %[rc], 1f\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline int arch_atomic_dec_if_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "w"); + return prev - 1; } =20 @@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_n= egative(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "d"); + return !(prev < 0); } =20 @@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_p= ositive(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "d"); + return !(prev > 0); } =20 @@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positi= ve(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "d"); + return prev - 1; } =20 --=20 2.41.0 From nobody Fri Sep 12 05:23:41 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89991C001B0 for ; Thu, 10 Aug 2023 04:05:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231936AbjHJEFZ (ORCPT ); Thu, 10 Aug 2023 00:05:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232675AbjHJEFQ (ORCPT ); Thu, 10 Aug 2023 00:05:16 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34328E5B for ; Wed, 9 Aug 2023 21:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VLwtkHrR4Hnv0SG2htyTx6O/VUdZ3YGDzB9YQqYciKw=; b=a7fZc30nn5A0fu6C+UEEAnHvDKtTYklJnVxbW8MFMub1FRC78+JKbkGFm9ZrM3oS5ZS/af 3ffBd5L+OGY+swKWFLcR8GCWjeXI2gIypR9UNHiMnwKYAcxXeiTeSGj6ptZZrUe25nEVnK YO8XjPNqGbWmLLvrQ4eEEcKzs7Iwp+E= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-436-Euk6dC1sOp2ePg5o-fUyVQ-1; Thu, 10 Aug 2023 00:04:29 -0400 X-MC-Unique: Euk6dC1sOp2ePg5o-fUyVQ-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a7697e580fso597222b6e.1 for ; Wed, 09 Aug 2023 21:04:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640268; x=1692245068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VLwtkHrR4Hnv0SG2htyTx6O/VUdZ3YGDzB9YQqYciKw=; b=FWEQ9nZdtJXGNUh6AM/klGNeSQpsDYcv510oRhXpOVtOn0m/4ULAZQi4uXkrYugk7O NfiQ17tR5YIYwNJV+gsR3p/Q9iYpoGMoujF60BMOle7uQgb4EalWhm+UYqo5x46//dIk oRrK8Ra/ZgiNbtqNLXjV5FnszjENBonXVNQ0N7vxVHplLCFfKMqIBgZVbN6dhOlGv1Gk pbMc+eong9++mLNF2Sj+0DS70Ti3PXzCbveqgLUkKUeb8wRxqq0maAPT88qmgggMZH/7 +OlaRV22eikSsdaCNnODt068dvkUngPeBkPDb5DTU1J53AzgOLCMeBDLYQumJTTCoep/ iRKQ== X-Gm-Message-State: AOJu0Yw1OGdDwXxohznE3nVi9BWvTCtxPDO4UclenRsX3rSNwEwCS0k7 8dvomI62iBq7vSfQfZmXtWvZTyqy1OTrsYRtCHZ7ZcyHyQYxFBs0BdXe1AsH3LW1+vEzdiQVaVo rxHCaW/VB2RMrA0E0ARHTCR9X X-Received: by 2002:a05:6808:10cc:b0:3a3:d7de:5cfa with SMTP id s12-20020a05680810cc00b003a3d7de5cfamr1819128ois.11.1691640268304; Wed, 09 Aug 2023 21:04:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEAdIAbs7Buum1URpEJTh5T57eb0r1XDUCa+aKbZcc1fbEPGJr7vKxHbONmynKMK+8Fsxl9cg== X-Received: by 2002:a05:6808:10cc:b0:3a3:d7de:5cfa with SMTP id s12-20020a05680810cc00b003a3d7de5cfamr1819092ois.11.1691640268064; Wed, 09 Aug 2023 21:04:28 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:27 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Date: Thu, 10 Aug 2023 01:03:46 -0300 Message-ID: <20230810040349.92279-6-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" cmpxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras Tested-by: Guo Ren --- arch/riscv/include/asm/cmpxchg.h | 34 ++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index e3e0ac7ba061..ac9d0eeb74e6 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -72,6 +72,35 @@ * indicated by comparing RETURN with OLD. */ =20 +#define __arch_cmpxchg_masked(sc_sfx, prepend, append, r, p, o, n) \ +({ \ + u32 *__ptr32b =3D (u32 *)((ulong)(p) & ~0x3); \ + ulong __s =3D ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask =3D GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx =3D (ulong)(n) << __s; \ + ulong __oldx =3D (ulong)(o) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z5\n" \ + " bne %1, %z3, 1f\n" \ + " and %1, %0, %z6\n" \ + " or %1, %1, %z4\n" \ + " sc.w" sc_sfx " %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=3D&r" (__retx), "=3D&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" ((long)__oldx), "rJ" (__newx), \ + "rJ" (__mask), "rJ" (~__mask) \ + : "memory"); \ + \ + r =3D (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) =20 #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ @@ -98,6 +127,11 @@ __typeof__(*(__ptr)) __ret; \ \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_cmpxchg_masked(sc_sfx, prepend, append, \ + __ret, __ptr, __old, __new); \ + break; \ case 4: \ __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ __ret, __ptr, (long), __old, __new); \ --=20 2.41.0 From nobody Fri Sep 12 05:23:41 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7A53C04A6A for ; Thu, 10 Aug 2023 04:05:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232558AbjHJEF3 (ORCPT ); Thu, 10 Aug 2023 00:05:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232641AbjHJEFU (ORCPT ); Thu, 10 Aug 2023 00:05:20 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43FD0E69 for ; Wed, 9 Aug 2023 21:04:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S9RDXJ5aWrysjU+rhwlqeMwMOsK56lpvgf+6LxPUCTg=; b=IRHbNtScy0/Nrm7nV9r/y/jDk0kJK6cOENHOsCDLgqyMBAi4IIpvX9ov2xnbrD/akhZIK8 mHqHZOooBlGmgdisaSzREFkzPQr1WrS1myiJ40xZSsvnlGdAVUJhvHP8/1wnd50b2ko17m BLREvk0Csn+s2rLNZXYk0NViE+DvLIQ= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-283-Pk3jcTnrM8y507xMyNPyBQ-1; Thu, 10 Aug 2023 00:04:34 -0400 X-MC-Unique: Pk3jcTnrM8y507xMyNPyBQ-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a5ab57036fso584576b6e.3 for ; Wed, 09 Aug 2023 21:04:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640273; x=1692245073; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S9RDXJ5aWrysjU+rhwlqeMwMOsK56lpvgf+6LxPUCTg=; b=AVHSK3m9lea5FMgXD2ccnwiFZZy4Z/kiHwvRVWDLZGzKwP8b/7bkEpCH/guNS+xikv Dh0C145D27s90BNR2srMaOU1d8i9b/r7O/FGmtQYn/FlI6T+jvgDHiNPNb6JB6pWdNxE DGIvmAFfDm383XezPuWfh2WIgTjFstkvY57FJG4Q0duahSAwh8kXzF67AFZ1mex+HAHV 1ltq0Esv5Qttq+CeBJvR42/EnJ5SLlfD1yproadMVzADKn9I3x3oRO0TCqMakfvchmhQ ++EiKymOSch2tVu6mZCzz69a5gds8AvtFB27s4AmZ4hLlkUqu292cGWbFppUR5YE2X/X phzQ== X-Gm-Message-State: AOJu0YyswhD9dX4SULSfce5BwzVDPYYbM1TQyMxxM+RAbD9txfu9A8zm dEMIYucBfRAFJrNZp5e56iao2sWlDi1cQyjEbytZY/O8A1VissYbI5CoEVz/Tn/8tGOl8ty4DW6 ZXyyAE6cFKOnZS+GPP+BxdHjz X-Received: by 2002:a05:6808:2124:b0:3a7:82e8:8fd1 with SMTP id r36-20020a056808212400b003a782e88fd1mr1804195oiw.20.1691640273706; Wed, 09 Aug 2023 21:04:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEJGYyTweAhht6teeaheQlpzQ7zjq+oFXSk4DIrLLMDF2ExtlYA68o0L4A2hjKnDp8s+BMmiQ== X-Received: by 2002:a05:6808:2124:b0:3a7:82e8:8fd1 with SMTP id r36-20020a056808212400b003a782e88fd1mr1804180oiw.20.1691640273540; Wed, 09 Aug 2023 21:04:33 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:33 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2 Date: Thu, 10 Aug 2023 01:03:47 -0300 Message-ID: <20230810040349.92279-7-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras Tested-by: Guo Ren --- arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index ac9d0eeb74e6..26cea2395aae 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,6 +11,31 @@ #include #include =20 +#define __arch_xchg_masked(prepend, append, r, p, n) \ +({ \ + u32 *__ptr32b =3D (u32 *)((ulong)(p) & ~0x3); \ + ulong __s =3D ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask =3D GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx =3D (ulong)(n) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z4\n" \ + " or %1, %1, %z3\n" \ + " sc.w %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + : "=3D&r" (__retx), "=3D&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" (__newx), "rJ" (~__mask) \ + : "memory"); \ + \ + r =3D (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) + #define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ __asm__ __volatile__ ( \ @@ -27,7 +52,13 @@ __typeof__(ptr) __ptr =3D (ptr); \ __typeof__(*(__ptr)) __new =3D (new); \ __typeof__(*(__ptr)) __ret; \ + \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_xchg_masked(prepend, append, \ + __ret, __ptr, __new); \ + break; \ case 4: \ __arch_xchg(".w" sfx, prepend, append, \ __ret, __ptr, __new); \ --=20 2.41.0