From nobody Sun Feb 8 00:49:24 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E336C001DB for ; Fri, 4 Aug 2023 08:51:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229968AbjHDIvr (ORCPT ); Fri, 4 Aug 2023 04:51:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229822AbjHDIuQ (ORCPT ); Fri, 4 Aug 2023 04:50:16 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75BD949C7 for ; Fri, 4 Aug 2023 01:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691138966; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cG8axZDwBuWJQ+k2FsV81xz+WKLWi2QNSuhJ6eUeaqw=; b=aTHJZVPDvXQU2ACiubqkDYqnlmbj5zYDAG/bmn+G8tS70DyVMre0LOF/ZDT3KUToLFJwBi 3WzGhEABpzH0AgttQNx9eAZ2rYuHjl4+JruBywDNzMKmmG7vZe8ystAwHVasUd7IeSQ6HD i6QhTenlWLyNFwXkSjRdJSIqmIwtk3E= Received: from mail-oa1-f72.google.com (mail-oa1-f72.google.com [209.85.160.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-626-UQBDIiOyNkmOJjCclkQuLQ-1; Fri, 04 Aug 2023 04:49:25 -0400 X-MC-Unique: UQBDIiOyNkmOJjCclkQuLQ-1 Received: by mail-oa1-f72.google.com with SMTP id 586e51a60fabf-1b44332e279so2762838fac.0 for ; Fri, 04 Aug 2023 01:49:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691138965; x=1691743765; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cG8axZDwBuWJQ+k2FsV81xz+WKLWi2QNSuhJ6eUeaqw=; b=H7xeGF6X6WpNgVzDtInrI5wnb/sahGG9gos4qoXJTjLYCTsjWj8IgNFaeA4TVck8+i csvIJsMmCGb0+w4hkIkO7BQCzaoXLSltCPtTnrcrqTjpmceHYYDcsZT5FsFip+T1q7Xt rR5VGqcys8viFd/ie0Hkjnfbx4nxCJ1TeSh7w/85mVrXSZsCB+u/Z6TmxD8RcZ7aYs7a G2ECwBTVutwHjv4cokXaIvM7MIQgiyH9LW3npIuSQp0OUW84MZQZeiB+QKdbmVUlwS2K T0mE2PYZRyBfzI8UtwxxMgK0dHAVGgG2rC+xzyACu/XYqH6wCZ8FfFWtUXg+OLe6N+Ar TnKA== X-Gm-Message-State: AOJu0Yz7r6NktoZhPY4XB6TLZkLw7+4ss1sXDozs+/G9ntdnHLc5PJx9 wyuBQPFIzRDsqnjdf2TYIiwiQbD4AeLPhhf2rNl2wtZOCkbTbTK0oxP46ozT3P/7+qGoNcFZ7yh GNfXXsZEh8AZxrqUaY7fBfIin X-Received: by 2002:a05:6870:5b98:b0:1be:c8e2:3ec3 with SMTP id em24-20020a0568705b9800b001bec8e23ec3mr1046104oab.14.1691138964800; Fri, 04 Aug 2023 01:49:24 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGfXG4yRYPIwmiiQVWTf4MQ+ktY1Xgv8O9HBUKLqIvtNOkPHWsNOPcnRegEr6JIF1zXzs2y/A== X-Received: by 2002:a05:6870:5b98:b0:1be:c8e2:3ec3 with SMTP id em24-20020a0568705b9800b001bec8e23ec3mr1046089oab.14.1691138964579; Fri, 04 Aug 2023 01:49:24 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id f8-20020a4ab008000000b0055516447257sm685679oon.29.2023.08.04.01.49.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 01:49:24 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v3 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Date: Fri, 4 Aug 2023 05:48:54 -0300 Message-ID: <20230804084900.1135660-3-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230804084900.1135660-2-leobras@redhat.com> References: <20230804084900.1135660-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_xchg* generation. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 136 +++++-------------------------- 1 file changed, 22 insertions(+), 114 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index 2f4726d3cfcc..ec4ea4f3f908 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,60 +11,30 @@ #include #include =20 -#define __xchg_relaxed(ptr, new, size) \ +#define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_relaxed(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_relaxed((ptr), \ - _x_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + " amoswap" sfx " %0, %2, %1\n" \ + append \ + : "=3Dr" (r), "+A" (*(p)) \ + : "r" (n) \ + : "memory"); \ }) =20 -#define __xchg_acquire(ptr, new, size) \ +#define _arch_xchg(ptr, new, sfx, prepend, append) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ + __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ - switch (size) { \ + __typeof__(ptr) __ptr =3D (ptr); \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".w" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".d" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -72,79 +42,17 @@ __ret; \ }) =20 -#define arch_xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) +#define arch_xchg_relaxed(ptr, x) \ + _arch_xchg(ptr, x, "", "", "") =20 -#define __xchg_release(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_xchg_acquire(ptr, x) \ + _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER) =20 #define arch_xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __arch_xchg(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(new) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d.aqrl %0, %2, %1\n" \ - : "=3Dr" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "") =20 #define arch_xchg(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ =3D (x); \ - (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ -}) + _arch_xchg(ptr, x, ".aqrl", "", "") =20 #define xchg32(ptr, x) \ ({ \ --=20 2.41.0 From nobody Sun Feb 8 00:49:24 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F15D7C001DB for ; Fri, 4 Aug 2023 08:51:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229975AbjHDIvu (ORCPT ); Fri, 4 Aug 2023 04:51:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229828AbjHDIuT (ORCPT ); Fri, 4 Aug 2023 04:50:19 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D60FD49D8 for ; Fri, 4 Aug 2023 01:49:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691138971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=68droSgtD1yUeWujN9KsDS4jg7ssA0NCkb8PZ8Tb+SA=; b=TCyBPw8pX42QL5pxp2syZzqAzwE7EcS1hA2bfqkT21Y29U9O5M3kAJIE1vz0zvWxTrpwlx VZjZ+uBHU8eE6UKw9o4P1892hdXYPTf/thri9K4OaNhJLxGDfW9OlDouvi9E0MkvRca7kc ATC3DnkmfAfk1GEhxfh3iV4LvWdmExs= Received: from mail-oo1-f72.google.com (mail-oo1-f72.google.com [209.85.161.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-577-xBmS6bPBOwiYAvCj9XfntQ-1; Fri, 04 Aug 2023 04:49:29 -0400 X-MC-Unique: xBmS6bPBOwiYAvCj9XfntQ-1 Received: by mail-oo1-f72.google.com with SMTP id 006d021491bc7-56c6c502822so2526696eaf.2 for ; Fri, 04 Aug 2023 01:49:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691138969; x=1691743769; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=68droSgtD1yUeWujN9KsDS4jg7ssA0NCkb8PZ8Tb+SA=; b=Fb2amZFFgZLz50uvU11uwXKPeW0d/naHFMSyUf59mQB442QTFQmwFH4kxuY9auZU1m asN60Q++aEkcljhE/V4ToHr+Ac1vU1Dj/AjS/gZhzSQ3vcsDY8zUPp7fXvcTUwkbgi7o ujZG003ZnqWrhgpuKQ2DrzmAllfbH6t8+nZAGbO3r+06EeBKF+qyGSCA9CGiQe7TD8zS 4CCtxXxodV/UnizOnJImh4HbM291K1843lLG7TQtFc30PT6fqnWzwW2yQVx8eoT946G1 74vuoYLELxVOOQNv52i/CK7byjD0tk+5zNXfuf0KHpna5KAPbfhZfxvx1rK7PYQdAFEb imfg== X-Gm-Message-State: AOJu0YxS3W60v5fFUliAf74AjN5lQEW5YvcJfADB6hg4iFjd8fjNuY9e vVQSUKFwv1++o2TulhSh/vMiH3V6xQqXgbFv2oMnVrtaD1jF9XFqCZ0LrmdKsawQ+V/svTLSKUC L4mLJfGmu8OnxXQ0MC7eR6kTy X-Received: by 2002:a4a:3018:0:b0:56c:7428:4a35 with SMTP id q24-20020a4a3018000000b0056c74284a35mr1050183oof.7.1691138969021; Fri, 04 Aug 2023 01:49:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFjkpXXm0/Ur8WNB8kbC0Thn+LObgXQCNBatmDg00BqWmkNhBFSyxWSvjCulJQefjTpcM1mRQ== X-Received: by 2002:a4a:3018:0:b0:56c:7428:4a35 with SMTP id q24-20020a4a3018000000b0056c74284a35mr1050170oof.7.1691138968735; Fri, 04 Aug 2023 01:49:28 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id f8-20020a4ab008000000b0055516447257sm685679oon.29.2023.08.04.01.49.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 01:49:28 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v3 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Date: Fri, 4 Aug 2023 05:48:55 -0300 Message-ID: <20230804084900.1135660-4-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230804084900.1135660-2-leobras@redhat.com> References: <20230804084900.1135660-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_cmpxchg* generation (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 187 +++++-------------------------- 1 file changed, 29 insertions(+), 158 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index ec4ea4f3f908..5a07646fae65 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -71,127 +71,40 @@ * store NEW in MEM. Return the initial value in MEM. Success is * indicated by comparing RETURN with OLD. */ -#define __cmpxchg_relaxed(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) =20 -#define arch_cmpxchg_relaxed(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_relaxed((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) =20 -#define __cmpxchg_acquire(ptr, old, new, size) \ +#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_cmpxchg_acquire(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=3D&r" (r), "=3D&r" (__rc), "+A" (*(p)) \ + : "rJ" (co o), "rJ" (n) \ + : "memory"); \ }) =20 -#define __cmpxchg_release(ptr, old, new, size) \ +#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr =3D (ptr); \ __typeof__(*(ptr)) __old =3D (old); \ __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ + \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ + __ret, __ptr, (long), __old, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".d", ".d" sc_sfx, prepend, append, \ + __ret, __ptr, /**/, __old, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -199,62 +112,20 @@ __ret; \ }) =20 -#define arch_cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) +#define arch_cmpxchg_relaxed(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", "") =20 -#define __cmpxchg(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr =3D (ptr); \ - __typeof__(*(ptr)) __old =3D (old); \ - __typeof__(*(ptr)) __new =3D (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=3D&r" (__ret), "=3D&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_cmpxchg_acquire(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER) + +#define arch_cmpxchg_release(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "") =20 #define arch_cmpxchg(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ =3D (o); \ - __typeof__(*(ptr)) _n_ =3D (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) + _arch_cmpxchg((ptr), (o), (n), ".rl", "", " fence rw, rw\n") =20 #define arch_cmpxchg_local(ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) + arch_cmpxchg_relaxed((ptr), (o), (n)) =20 #define arch_cmpxchg64(ptr, o, n) \ ({ \ --=20 2.41.0 From nobody Sun Feb 8 00:49:24 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4364C001DB for ; Fri, 4 Aug 2023 08:52:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229989AbjHDIwD (ORCPT ); Fri, 4 Aug 2023 04:52:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229832AbjHDIuX (ORCPT ); Fri, 4 Aug 2023 04:50:23 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA7FA4C1E for ; Fri, 4 Aug 2023 01:49:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691138975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=PU3b9IVOOTDEUT6h4viBuCZGKTj8vAYUbBjmfDBGkgkreEZc5Ua6j5OF9c4NsX9Omts650 YEH2q5ghwPnlY39cTsYXjoZVelqNTs3WxuvU85OuqONhTnCPBLUxo4DULYquQHHnakJ6f9 reNaU5xZ7ifNGPEHp7t5xyjn3ztfdAs= Received: from mail-oa1-f72.google.com (mail-oa1-f72.google.com [209.85.160.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-249-ZAavzb9jPQOmxH0jMtxq5w-1; Fri, 04 Aug 2023 04:49:34 -0400 X-MC-Unique: ZAavzb9jPQOmxH0jMtxq5w-1 Received: by mail-oa1-f72.google.com with SMTP id 586e51a60fabf-1bb91fb58f2so2640147fac.2 for ; Fri, 04 Aug 2023 01:49:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691138973; x=1691743773; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=eywclt7Kw3GjcO4Mx0yDJ8AlalQ3G0v5kWwuAorLi1tGlpJW1WKSzvQgTdSMKb0XZh 7ESJ03iLbiHd8pEzNkmAMEiIFGe3J81HSGkS8gC3hD3QvZp+gRe7i0a7dcuKeb6W4nlQ VMngkAncw5HOhrtJGuDrX0rCXI9uiNwxozy0nO13nAKtZ4/TUHHbw/nSg88ngryFzMzx 4kzZrjWsVclPkbxq+DZk/GWpBFy0nxxQq4DOns4XOFUiMgY/4A4sLyfGMKvW610dVWgd bf5TE0L7XCVaOcgs9UCkhD/4Ldv9xXYNFnJscS+h1D7wk+xQIyX7WweHZViCPEXNFvMD kfeQ== X-Gm-Message-State: AOJu0Yzn0GkEtxBHGdqfzysS151Ei6plC8gYvS2ZJjmWvYXJP/deRIaR 6Ue/NnQgt5PO6qjUqwDEGypPZ3iWweYsrrtgG32JE0DEGynPs7olAAy+IZOSEKm1JaAa37QLE7X kLbLNndmaekrNBRHkzJ47fGBp X-Received: by 2002:a05:6870:9693:b0:1bf:42b9:907f with SMTP id o19-20020a056870969300b001bf42b9907fmr1292588oaq.20.1691138973311; Fri, 04 Aug 2023 01:49:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHo7vyNoP4vF9peGfFA/17nq0Yzj8bINsHDzCtyGDvVS4NPtZ3F+JVXZP0nVXXJYhxCYez1Dw== X-Received: by 2002:a05:6870:9693:b0:1bf:42b9:907f with SMTP id o19-20020a056870969300b001bf42b9907fmr1292569oaq.20.1691138973053; Fri, 04 Aug 2023 01:49:33 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id f8-20020a4ab008000000b0055516447257sm685679oon.29.2023.08.04.01.49.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 01:49:32 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v3 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Date: Fri, 4 Aug 2023 05:48:56 -0300 Message-ID: <20230804084900.1135660-5-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230804084900.1135660-2-leobras@redhat.com> References: <20230804084900.1135660-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Some functions use mostly the same asm for 32-bit and 64-bit versions. Make a macro that is generic enough and avoid code duplication. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Guo Ren Reviewed-by: Andrea Parri --- arch/riscv/include/asm/atomic.h | 164 +++++++++++++++----------------- 1 file changed, 76 insertions(+), 88 deletions(-) diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomi= c.h index f5dfef6c2153..80cca7ac16fd 100644 --- a/arch/riscv/include/asm/atomic.h +++ b/arch/riscv/include/asm/atomic.h @@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i) #undef ATOMIC_FETCH_OP #undef ATOMIC_OP_RETURN =20 +#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " beq %[p], %[u], 1f\n" \ + " add %[rc], %[p], %[a]\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : [a]"r" (_a), [u]"r" (_u) \ + : "memory"); \ +}) + /* This is required to provide a full barrier on success. */ static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a= , int u) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w"); + return prev; } #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless @@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_un= less(atomic64_t *v, s64 a, s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d"); + return prev; } #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless #endif =20 +#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bltz %[p], 1f\n" \ + " addi %[rc], %[p], 1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "w"); + return !(prev < 0); } =20 #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative =20 +#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bgtz %[p], 1f\n" \ + " addi %[rc], %[p], -1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "w"); + return !(prev > 0); } =20 #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive =20 +#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " addi %[rc], %[p], -1\n" \ + " bltz %[rc], 1f\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=3D&r" (_prev), [rc]"=3D&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline int arch_atomic_dec_if_positive(atomic_t *v) { int prev, rc; =20 - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "w"); + return prev - 1; } =20 @@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_n= egative(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "d"); + return !(prev < 0); } =20 @@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_p= ositive(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "d"); + return !(prev > 0); } =20 @@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positi= ve(atomic64_t *v) s64 prev; long rc; =20 - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=3D&r" (prev), [rc]"=3D&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "d"); + return prev - 1; } =20 --=20 2.41.0 From nobody Sun Feb 8 00:49:24 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCF5BC001DB for ; Fri, 4 Aug 2023 08:52:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230020AbjHDIwN (ORCPT ); Fri, 4 Aug 2023 04:52:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229744AbjHDIua (ORCPT ); Fri, 4 Aug 2023 04:50:30 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A57A94C3D for ; Fri, 4 Aug 2023 01:49:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691138979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7qr15U85Ji6JaTSGdRUgCJIrXJMY4cTkIs1Ye7aZqqk=; b=Sb9o5KTux0eFuu7yZ+1zcbboNAeVrAgotBnJYlfnab2sb+EF31NGHdhtI6F4ZElBB4Jljs y6iwhg08EBNw/j8mfU6WPEYw9lh2XNuFpV2cGSE9g9ppi7dwqIakIgUFxSW57F2ZwqgCqX PaYGNPAXIOnwSzSSNxVaQghAeOBDHxo= Received: from mail-oo1-f69.google.com (mail-oo1-f69.google.com [209.85.161.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-369-FanNZGoTM3OJwwoHoaOIPA-1; Fri, 04 Aug 2023 04:49:38 -0400 X-MC-Unique: FanNZGoTM3OJwwoHoaOIPA-1 Received: by mail-oo1-f69.google.com with SMTP id 006d021491bc7-56d46f0ac4bso1099820eaf.1 for ; Fri, 04 Aug 2023 01:49:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691138977; x=1691743777; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7qr15U85Ji6JaTSGdRUgCJIrXJMY4cTkIs1Ye7aZqqk=; b=HHWIW2dNKY+y/lcvOS5p3EFI3O2MwBKiAY6ufKyA3uaOFw9K0hMypVnkYqFkFhWd70 uAjQJ5ZAETt5LZD2rau03TCncEl8YyZAUoBB2muvoZ22ZQLf1WabZIsOY2FEpdX8NbGS EM9g3fiy+4IaFRY9WSRXzXedtDvapNdcJZA2Eg+7E7X9vDqLVI/s4J/IoTYqWYuKrGML jGgh5R87nhoONQjG0hEg6qLC/0tthkJFcJX4YQAuoDsPWehPotTL3C7TT584t764RE5C Y7N2DodajPr6namD5tSyU+csAwNMLwLKN2ERc8kOWSnz+RdS5r/S/FEWuSLOWG58aIDr T/ug== X-Gm-Message-State: AOJu0YwdagciWe07dtQYOGW/wKzOXftt/XCxS6GigeQ1mvXNb0A2Q3RH eu8D7neZhqXKvYn0frVUM9uCo9FDq0kETMu23eSFR2FHGiUdkgzDCqbBP1OYgZwugyrPdXewuGF NNHbg8kaqhaOBlJV4C/2GE0T8 X-Received: by 2002:a4a:d2dc:0:b0:56c:43cd:7267 with SMTP id j28-20020a4ad2dc000000b0056c43cd7267mr1155030oos.8.1691138977547; Fri, 04 Aug 2023 01:49:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFGpSmCpCsEW2JBmJiIqK4JHv1sEJQk4u7ARhdPhe1GUzhItUEcAzOrA7h+bfUyOp/7X0uLgw== X-Received: by 2002:a4a:d2dc:0:b0:56c:43cd:7267 with SMTP id j28-20020a4ad2dc000000b0056c43cd7267mr1155011oos.8.1691138977273; Fri, 04 Aug 2023 01:49:37 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id f8-20020a4ab008000000b0055516447257sm685679oon.29.2023.08.04.01.49.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 01:49:36 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v3 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Date: Fri, 4 Aug 2023 05:48:57 -0300 Message-ID: <20230804084900.1135660-6-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230804084900.1135660-2-leobras@redhat.com> References: <20230804084900.1135660-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" cmpxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 35 ++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index 5a07646fae65..dfb433ac544f 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -72,6 +72,36 @@ * indicated by comparing RETURN with OLD. */ =20 +#define __arch_cmpxchg_mask(sc_sfx, prepend, append, r, p, o, n) \ +({ \ + /* Depends on 2-byte variables being 2-byte aligned */ \ + ulong __s =3D ((ulong)(p) & 0x3) * BITS_PER_BYTE; \ + ulong __mask =3D GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx =3D (ulong)(n) << __s; \ + ulong __oldx =3D (ulong)(o) << __s; \ + ulong __retx; \ + register unsigned int __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %0, %0, %z5\n" \ + " bne %0, %z3, 1f\n" \ + " and %1, %0, %z6\n" \ + " or %1, %1, %z4\n" \ + " sc.w" sc_sfx " %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=3D&r" (__retx), "=3D&r" (__rc), "+A" (*(p)) \ + : "rJ" ((long)__oldx), "rJ" (__newx), \ + "rJ" (__mask), "rJ" (~__mask) \ + : "memory"); \ + \ + r =3D (__typeof__(*(p)))(__retx >> __s); \ +}) + =20 #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ @@ -98,6 +128,11 @@ __typeof__(*(ptr)) __ret; \ \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_cmpxchg_mask(sc_sfx, prepend, append, \ + __ret, __ptr, __old, __new); \ + break; \ case 4: \ __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ __ret, __ptr, (long), __old, __new); \ --=20 2.41.0 From nobody Sun Feb 8 00:49:24 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E173C001DB for ; Fri, 4 Aug 2023 08:52:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230029AbjHDIwS (ORCPT ); Fri, 4 Aug 2023 04:52:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229746AbjHDIua (ORCPT ); Fri, 4 Aug 2023 04:50:30 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82F4349F8 for ; Fri, 4 Aug 2023 01:49:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691138983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3amwd/lpP8rvZ9IhpcoisHZeefZgWLzf3BsTgVcPLgw=; b=RUw9KxHl6bjjS75FGcDCm3R4QKkADOXkMAFdvD+SVovKRB6BVSjnpAxj4cVCjgejaz/c/e KszWSzqvWdjs9XlMhcv5ttVpGON7XM+0WfYF+T+O2E1B2c9q50Z+5rsGY+g+4t4nq4Yj8b ZpYl/KQM4PS2zbG+S3KS5JkPS/5cUR4= Received: from mail-oo1-f69.google.com (mail-oo1-f69.google.com [209.85.161.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-164-aAxCM3J-NIi8aXpUABlmLQ-1; Fri, 04 Aug 2023 04:49:42 -0400 X-MC-Unique: aAxCM3J-NIi8aXpUABlmLQ-1 Received: by mail-oo1-f69.google.com with SMTP id 006d021491bc7-56ca7604435so2579038eaf.3 for ; Fri, 04 Aug 2023 01:49:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691138981; x=1691743781; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3amwd/lpP8rvZ9IhpcoisHZeefZgWLzf3BsTgVcPLgw=; b=PjRa1wWZidsVzoHiN9JdINRSH53miCIOx2fK1GoIGXx7QJFKNt13A70YS++w4uHqvq EC5ga/DKxObEcv1dHH4H1bSAAxBXapwHfM3EbHHDiqzGbiBlTL4i4WUUuodyIclVSZxy meVBTSWvLdNaRmb77cpFf2QCUsSXVRxfTll2nlwLdCHWTvTxffVZv3KKsaVqzCUYW3JB iphbkqAR7v1k08AOGUzpvj67XHGpt2L8aY5bYSXOJDapera+xq8Km5SuhrDmnsBD45V6 KgqKPRdW3RCsRECNZ+DIjrHKyDYQtlfE2mWcH3Y5DGFlSr3mxmefBguVcVXXSU0vStDs HHqw== X-Gm-Message-State: AOJu0YycCZQ8KXwylcTGUxFa3JDEfJF57spkbiw6auQlq1ZkQkavFZES Jx8C2v3Qju0E/fvZ0zdjZmiExIanCVGOys4y1kdpqdezvOMKwBbbiMcir3JB/bCeAnJ1v65Yzsa w9VLdAJ4GgU/LivSc+eMKTThy X-Received: by 2002:a4a:6519:0:b0:56c:e856:8b2c with SMTP id y25-20020a4a6519000000b0056ce8568b2cmr1003967ooc.9.1691138981682; Fri, 04 Aug 2023 01:49:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFvsWBOYEPFQASeYA1dLIyHiNIkhde3nNiq2XUol3devppgxjgdj9GmPoeRlh/oNYHjlwpP4w== X-Received: by 2002:a4a:6519:0:b0:56c:e856:8b2c with SMTP id y25-20020a4a6519000000b0056ce8568b2cmr1003954ooc.9.1691138981435; Fri, 04 Aug 2023 01:49:41 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a801:d380:694f:4f52:764c:4b7f]) by smtp.gmail.com with ESMTPSA id f8-20020a4ab008000000b0055516447257sm685679oon.29.2023.08.04.01.49.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 01:49:41 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v3 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2 Date: Fri, 4 Aug 2023 05:48:58 -0300 Message-ID: <20230804084900.1135660-7-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230804084900.1135660-2-leobras@redhat.com> References: <20230804084900.1135660-2-leobras@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpx= chg.h index dfb433ac544f..943f094375c7 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,6 +11,31 @@ #include #include =20 +#define __arch_xchg_mask(prepend, append, r, p, n) \ +({ \ + /* Depends on 2-byte variables being 2-byte aligned */ \ + ulong __s =3D ((ulong)(p) & 0x3) * BITS_PER_BYTE; \ + ulong __mask =3D GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx =3D (ulong)(n) << __s; \ + ulong __retx; \ + register unsigned int __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z4\n" \ + " or %1, %1, %z3\n" \ + " sc.w %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + : "=3D&r" (__retx), "=3D&r" (__rc), "+A" (*(p)) \ + : "rJ" (__newx), "rJ" (~__mask) \ + : "memory"); \ + \ + r =3D (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) + #define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ __asm__ __volatile__ ( \ @@ -27,7 +52,13 @@ __typeof__(*(ptr)) __new =3D (new); \ __typeof__(*(ptr)) __ret; \ __typeof__(ptr) __ptr =3D (ptr); \ + \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_xchg_mask(prepend, append, \ + __ret, __ptr, __new); \ + break; \ case 4: \ __arch_xchg(".w" sfx, prepend, append, \ __ret, __ptr, __new); \ --=20 2.41.0