From nobody Sat Feb 7 20:36:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA546EDEC7B for ; Wed, 13 Sep 2023 15:51:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229914AbjIMPvm (ORCPT ); Wed, 13 Sep 2023 11:51:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229655AbjIMPvk (ORCPT ); Wed, 13 Sep 2023 11:51:40 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 61F3BE54 for ; Wed, 13 Sep 2023 08:50:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694620252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=mYBRu8ZisVOcVrXjmDw3lj2GZxU9S1CJfHT4uVPGf+o=; b=JyO2v3/BSjmcJsR4QKy19uVYfeojdyzXLBBJQ97JVwGfgEdacuKVJjOTNJk00BsB5JP8Bt vsQJjX9ktktZ6ITCw8QH+0D8R9TLjwQ+4MAkkpCgWJM+etRn0etT+yWbEzudOJqip2gqm1 +dMyriybIyw1KXoIXwqIEWpejPaYf/I= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-519-EJ7jLDN_NRCYgLxLCqtGtA-1; Wed, 13 Sep 2023 11:50:49 -0400 X-MC-Unique: EJ7jLDN_NRCYgLxLCqtGtA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A4D33816521; Wed, 13 Sep 2023 15:50:48 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.76]) by smtp.corp.redhat.com (Postfix) with SMTP id 2F9351008808; Wed, 13 Sep 2023 15:50:45 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Wed, 13 Sep 2023 17:49:56 +0200 (CEST) Date: Wed, 13 Sep 2023 17:49:53 +0200 From: Oleg Nesterov To: Boqun Feng , Ingo Molnar , Peter Zijlstra , Rik van Riel , Thomas Gleixner , Waiman Long , Will Deacon Cc: Alexey Gladkov , "Eric W. Biederman" , linux-kernel@vger.kernel.org Subject: [PATCH 1/5] seqlock: simplify SEQCOUNT_LOCKNAME() Message-ID: <20230913154953.GA26242@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230913154907.GA26210@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" 1. Kill the "lockmember" argument. It is always s->lock plus __seqprop_##lockname##_sequence() already uses s->lock and ignores "lockmember". 2. Kill the "lock_acquire" argument. __seqprop_##lockname##_sequence() can use the same "lockbase" prefix for _lock and _unlock. Apart from line numbers, gcc -E outputs the same code. Signed-off-by: Oleg Nesterov Reviewed-by: Alexey Gladkov --- include/linux/seqlock.h | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 987a59d977c5..ac6631bd5706 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -191,11 +191,9 @@ static inline void seqcount_lockdep_reader_access(cons= t seqcount_t *s) * @lockname: "LOCKNAME" part of seqcount_LOCKNAME_t * @locktype: LOCKNAME canonical C data type * @preemptible: preemptibility of above locktype - * @lockmember: argument for lockdep_assert_held() - * @lockbase: associated lock release function (prefix only) - * @lock_acquire: associated lock acquisition function (full call) + * @lockbase: prefix for associated lock/unlock */ -#define SEQCOUNT_LOCKNAME(lockname, locktype, preemptible, lockmember, loc= kbase, lock_acquire) \ +#define SEQCOUNT_LOCKNAME(lockname, locktype, preemptible, lockbase) \ typedef struct seqcount_##lockname { \ seqcount_t seqcount; \ __SEQ_LOCK(locktype *lock); \ @@ -216,7 +214,7 @@ __seqprop_##lockname##_sequence(const seqcount_##lockna= me##_t *s) \ return seq; \ \ if (preemptible && unlikely(seq & 1)) { \ - __SEQ_LOCK(lock_acquire); \ + __SEQ_LOCK(lockbase##_lock(s->lock)); \ __SEQ_LOCK(lockbase##_unlock(s->lock)); \ \ /* \ @@ -242,7 +240,7 @@ __seqprop_##lockname##_preemptible(const seqcount_##loc= kname##_t *s) \ static __always_inline void \ __seqprop_##lockname##_assert(const seqcount_##lockname##_t *s) \ { \ - __SEQ_LOCK(lockdep_assert_held(lockmember)); \ + __SEQ_LOCK(lockdep_assert_held(s->lock)); \ } =20 /* @@ -271,10 +269,10 @@ static inline void __seqprop_assert(const seqcount_t = *s) =20 #define __SEQ_RT IS_ENABLED(CONFIG_PREEMPT_RT) =20 -SEQCOUNT_LOCKNAME(raw_spinlock, raw_spinlock_t, false, s->lock, = raw_spin, raw_spin_lock(s->lock)) -SEQCOUNT_LOCKNAME(spinlock, spinlock_t, __SEQ_RT, s->lock, = spin, spin_lock(s->lock)) -SEQCOUNT_LOCKNAME(rwlock, rwlock_t, __SEQ_RT, s->lock, = read, read_lock(s->lock)) -SEQCOUNT_LOCKNAME(mutex, struct mutex, true, s->lock, = mutex, mutex_lock(s->lock)) +SEQCOUNT_LOCKNAME(raw_spinlock, raw_spinlock_t, false, raw_spin) +SEQCOUNT_LOCKNAME(spinlock, spinlock_t, __SEQ_RT, spin) +SEQCOUNT_LOCKNAME(rwlock, rwlock_t, __SEQ_RT, read) +SEQCOUNT_LOCKNAME(mutex, struct mutex, true, mutex) =20 /* * SEQCNT_LOCKNAME_ZERO - static initializer for seqcount_LOCKNAME_t --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 20:36:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 183BFEDEC7C for ; Wed, 13 Sep 2023 15:51:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230013AbjIMPvu (ORCPT ); Wed, 13 Sep 2023 11:51:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38462 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229977AbjIMPvt (ORCPT ); Wed, 13 Sep 2023 11:51:49 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id DDA0A1724 for ; Wed, 13 Sep 2023 08:50:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694620258; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=VnG66QyFsTzARJUHVO8Khm4VR1nzFPa/6kCMVpwGaPA=; b=Ne1CD3Dgx+v/N0NdG4UBEiEYODQubnpQFAGrUvYAiq8VdzqyivzrEeHlzm+u1V3xOd+2Xt C9jCqjVaIjnOopyVWC6kLA6SScG401QZoQTosMpxHi7sGVZ5lob7Lh0Dbva46xucHo1h25 af3gKAuUUNkzVZ9zFhw6gfraKGPQCDI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-424-8fcwpHowNhq13iRSFKUVLg-1; Wed, 13 Sep 2023 11:50:52 -0400 X-MC-Unique: 8fcwpHowNhq13iRSFKUVLg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3F876857A9A; Wed, 13 Sep 2023 15:50:52 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.76]) by smtp.corp.redhat.com (Postfix) with SMTP id BE2A940C6EBF; Wed, 13 Sep 2023 15:50:49 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Wed, 13 Sep 2023 17:49:59 +0200 (CEST) Date: Wed, 13 Sep 2023 17:49:56 +0200 From: Oleg Nesterov To: Boqun Feng , Ingo Molnar , Peter Zijlstra , Rik van Riel , Thomas Gleixner , Waiman Long , Will Deacon Cc: Alexey Gladkov , "Eric W. Biederman" , linux-kernel@vger.kernel.org Subject: [PATCH 2/5] seqlock: change __seqprop() to return the function pointer Message-ID: <20230913154956.GA26245@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230913154907.GA26210@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Preparation, see the next patch. This way it is trivial to add the new seqprop's with 2 or more args, and we do not loose the type info. Signed-off-by: Oleg Nesterov --- include/linux/seqlock.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index ac6631bd5706..41e36f8afad4 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -200,9 +200,9 @@ typedef struct seqcount_##lockname { \ } seqcount_##lockname##_t; \ \ static __always_inline seqcount_t * \ -__seqprop_##lockname##_ptr(seqcount_##lockname##_t *s) \ +__seqprop_##lockname##_ptr(const seqcount_##lockname##_t *s) \ { \ - return &s->seqcount; \ + return (void *)&s->seqcount; /* drop const */ \ } \ \ static __always_inline unsigned \ @@ -292,19 +292,19 @@ SEQCOUNT_LOCKNAME(mutex, struct mutex, true= , mutex) #define SEQCNT_WW_MUTEX_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, loc= k) =20 #define __seqprop_case(s, lockname, prop) \ - seqcount_##lockname##_t: __seqprop_##lockname##_##prop((void *)(s)) + seqcount_##lockname##_t: __seqprop_##lockname##_##prop =20 #define __seqprop(s, prop) _Generic(*(s), \ - seqcount_t: __seqprop_##prop((void *)(s)), \ + seqcount_t: __seqprop_##prop, \ __seqprop_case((s), raw_spinlock, prop), \ __seqprop_case((s), spinlock, prop), \ __seqprop_case((s), rwlock, prop), \ __seqprop_case((s), mutex, prop)) =20 -#define seqprop_ptr(s) __seqprop(s, ptr) -#define seqprop_sequence(s) __seqprop(s, sequence) -#define seqprop_preemptible(s) __seqprop(s, preemptible) -#define seqprop_assert(s) __seqprop(s, assert) +#define seqprop_ptr(s) __seqprop(s, ptr)(s) +#define seqprop_sequence(s) __seqprop(s, sequence)(s) +#define seqprop_preemptible(s) __seqprop(s, preemptible)(s) +#define seqprop_assert(s) __seqprop(s, assert)(s) =20 /** * __read_seqcount_begin() - begin a seqcount_t read section w/o barrier --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 20:36:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC4A3EDEC7C for ; Wed, 13 Sep 2023 15:51:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230044AbjIMPvy (ORCPT ); Wed, 13 Sep 2023 11:51:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229948AbjIMPvt (ORCPT ); Wed, 13 Sep 2023 11:51:49 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E0AFECCD for ; Wed, 13 Sep 2023 08:51:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694620261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=iABk60dwatWobxRjc3Fair6PpujlXQNgr6XGkMBYTOU=; b=dA22QlWSDsjKlU1e1MJ5CdNu1aTK12NJ/aAIyJy+ZcKpFUFYs29D0oWr+VHLP9guZST684 o3AeYann9ipLbmVOT0QOMzfM0zzs4I9j1NvOJ/m6U8xTK6UiZZ0J6mh/7rzFKXIWKSVeWF oyn1p9CzHQouGBzdbvzNlXwyLLg2zKs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-661-X13l6M4sMwm8QOibNTuw-A-1; Wed, 13 Sep 2023 11:50:57 -0400 X-MC-Unique: X13l6M4sMwm8QOibNTuw-A-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D423981652F; Wed, 13 Sep 2023 15:50:56 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.76]) by smtp.corp.redhat.com (Postfix) with SMTP id 5E22810F1BE7; Wed, 13 Sep 2023 15:50:54 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Wed, 13 Sep 2023 17:50:04 +0200 (CEST) Date: Wed, 13 Sep 2023 17:50:00 +0200 From: Oleg Nesterov To: Boqun Feng , Ingo Molnar , Peter Zijlstra , Rik van Riel , Thomas Gleixner , Waiman Long , Will Deacon Cc: Alexey Gladkov , "Eric W. Biederman" , linux-kernel@vger.kernel.org Subject: [PATCH 3/5] seqlock: introduce seqprop_lock/unlock Message-ID: <20230913155000.GA26248@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230913154907.GA26210@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" which can be used to take/release the corresponding lock. Thanks to the previous patch, it is trivial to pass 2 arguments to the new __seqprop_##lockname##_lock/unlock "methods", plus we do not loose the type info and thus the new seqprop's are "type safe". So for example void func(seqcount_rwlock_t *s, rwlock_t *l) { seqprop_lock(s, l); } happily compiles, but this one void func(seqcount_rwlock_t *s, spinlock_t *l) { seqprop_lock(s, l); } doesn't. Signed-off-by: Oleg Nesterov --- include/linux/seqlock.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 41e36f8afad4..9831683a0102 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -241,6 +241,21 @@ static __always_inline void \ __seqprop_##lockname##_assert(const seqcount_##lockname##_t *s) \ { \ __SEQ_LOCK(lockdep_assert_held(s->lock)); \ +} \ + \ +static __always_inline void \ +__seqprop_##lockname##_lock(seqcount_##lockname##_t *s, \ + locktype *lock) \ +{ \ + __SEQ_LOCK(WARN_ON_ONCE(s->lock !=3D lock)); \ + lockbase##_lock(lock); \ +} \ + \ +static __always_inline void \ +__seqprop_##lockname##_unlock(seqcount_##lockname##_t *s, \ + locktype *lock) \ +{ \ + lockbase##_unlock(lock); \ } =20 /* @@ -306,6 +321,12 @@ SEQCOUNT_LOCKNAME(mutex, struct mutex, true,= mutex) #define seqprop_preemptible(s) __seqprop(s, preemptible)(s) #define seqprop_assert(s) __seqprop(s, assert)(s) =20 +/* seqcount_t doesn't have these methods */ +static inline void __seqprop_lock (seqcount_t *s, void *l) { BUILD_BUG()= ; } +static inline void __seqprop_unlock (seqcount_t *s, void *l) { BUILD_BUG()= ; } +#define seqprop_lock(s, l) __seqprop(s, lock)(s, l) +#define seqprop_unlock(s, l) __seqprop(s, unlock)(s, l) + /** * __read_seqcount_begin() - begin a seqcount_t read section w/o barrier * @s: Pointer to seqcount_t or any of the seqcount_LOCKNAME_t variants --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 20:36:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01F3EEDEC7B for ; Wed, 13 Sep 2023 15:51:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230127AbjIMPwB (ORCPT ); Wed, 13 Sep 2023 11:52:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230020AbjIMPv5 (ORCPT ); Wed, 13 Sep 2023 11:51:57 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A2896E6D for ; Wed, 13 Sep 2023 08:51:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694620265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=Bq0tWUbQ6rNODWWmwzdrtdls65kdWejnGNjG71Arkk8=; b=ZeUrb2YxWS8aPXyMnDXhOkzonEskBguFx4h8kZQ11kiv7+Xw83DR+43Cyao9iaS/eVPaQg XFchiOEiYi5X4gxisMuojhxx9R6U2cU3L38JuJSoGjp0rfa0q0ty+TwGP19H1jeO6BHASP QaG1S7AR+OeI6Sh0TKxHFP1+OKvVRoA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-197-rS34Ja5yP_a0pZ4Q5k8knA-1; Wed, 13 Sep 2023 11:51:01 -0400 X-MC-Unique: rS34Ja5yP_a0pZ4Q5k8knA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B957481652A; Wed, 13 Sep 2023 15:51:00 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.76]) by smtp.corp.redhat.com (Postfix) with SMTP id 4356D10F1BE7; Wed, 13 Sep 2023 15:50:58 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Wed, 13 Sep 2023 17:50:08 +0200 (CEST) Date: Wed, 13 Sep 2023 17:50:05 +0200 From: Oleg Nesterov To: Boqun Feng , Ingo Molnar , Peter Zijlstra , Rik van Riel , Thomas Gleixner , Waiman Long , Will Deacon Cc: Alexey Gladkov , "Eric W. Biederman" , linux-kernel@vger.kernel.org Subject: [PATCH 4/5] seqlock: introduce read_seqcount_begin_or_lock() and friends Message-ID: <20230913155005.GA26252@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230913154907.GA26210@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" See the comment in the patch. NOTE: currently __seqprop_##lockname##_sequence() takes and drops s->lock if preemptible && CONFIG_PREEMPT_RT. With the previous changes it is simple to change this behaviour for the read_seqcount_begin_or_lock() case, iiuc it makes more sense to return with s->lock held and "(seq & 1) =3D=3D 1". Signed-off-by: Oleg Nesterov --- include/linux/seqlock.h | 50 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 9831683a0102..503813b3bab6 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -1239,4 +1239,54 @@ done_seqretry_irqrestore(seqlock_t *lock, int seq, u= nsigned long flags) if (seq & 1) read_sequnlock_excl_irqrestore(lock, flags); } + +/* + * Like read_seqbegin_or_lock/need_seqretry/done_seqretry above + * but for seqcount_LOCKNAME_t. + */ + +#define read_seqcount_begin_or_lock(s, lock, seq) \ +do { \ + if (!(*(seq) & 1)) \ + *(seq) =3D read_seqcount_begin(s); \ + else \ + seqprop_lock((s), (lock)); \ +} while (0) + +#define need_seqcount_retry(s, seq) \ +({ \ + !((seq) & 1) && read_seqcount_retry((s), (seq)); \ +}) + +#define done_seqcount_retry(s, lock, seq) \ +do { \ + if ((seq) & 1) \ + seqprop_unlock((s), (lock)); \ +} while (0) + + +#define read_seqcount_begin_or_lock_irqsave(s, lock, seq) \ +({ \ + unsigned long flags =3D 0; \ + \ + if (!(*(seq) & 1)) \ + *(seq) =3D read_seqcount_begin(s); \ + else { \ + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) \ + local_irq_save(flags); \ + seqprop_lock((s), (lock)); \ + } \ + \ + flags; \ +}) + +#define done_seqcount_retry_irqrestore(s, lock, seq, flags) \ +do { \ + if ((seq) & 1) { \ + seqprop_unlock((s), (lock)); \ + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) \ + local_irq_restore((flags)); \ + } \ +} while (0) + #endif /* __LINUX_SEQLOCK_H */ --=20 2.25.1.362.g51ebf55 From nobody Sat Feb 7 20:36:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3CEDEDEC7C for ; Wed, 13 Sep 2023 15:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230107AbjIMPv6 (ORCPT ); Wed, 13 Sep 2023 11:51:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230048AbjIMPvy (ORCPT ); Wed, 13 Sep 2023 11:51:54 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 72C0BE3 for ; Wed, 13 Sep 2023 08:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694620267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=YOIimXAa4BnacYFigSUrd1F+Xa9jb+T8PxsuF24Vw+o=; b=ResrXRHn3CQwn21zxVIuGsFVVqdV7H0dO9JvxAHswicaq2NyzB2cf5pNeoykcozvevbuVK rhc9qgqb5BVn0PSXGVrz40JXrWp4O21b/yLy9imCSTFYHHU0sj0/jASuKuVbyR+R1u+jKQ auFYY9liN/BgEXncTFylp3x8XspEl2w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-518--MmqiAbIOYmQToADvHS67Q-1; Wed, 13 Sep 2023 11:51:05 -0400 X-MC-Unique: -MmqiAbIOYmQToADvHS67Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5976F801779; Wed, 13 Sep 2023 15:51:05 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.76]) by smtp.corp.redhat.com (Postfix) with SMTP id D291D10085AA; Wed, 13 Sep 2023 15:51:01 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Wed, 13 Sep 2023 17:50:13 +0200 (CEST) Date: Wed, 13 Sep 2023 17:50:09 +0200 From: Oleg Nesterov To: Boqun Feng , Ingo Molnar , Peter Zijlstra , Rik van Riel , Thomas Gleixner , Waiman Long , Will Deacon Cc: Alexey Gladkov , "Eric W. Biederman" , linux-kernel@vger.kernel.org Subject: [PATCH 5/5] time,signal: turn signal_struct.stats_lock into seqcount_rwlock_t Message-ID: <20230913155009.GA26255@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230913154907.GA26210@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This way thread_group_cputime() doesn't exclude other readers on the 2nd pass. thread_group_cputime() still needs to disable irqs because stats_lock nests inside siglock. But once we change the getrusage()-like users to rely on stats_lock we can remove this dependency, and after that there will be no need for _irqsave. And IIUC, this is the bugfix for CONFIG_PREEMPT_RT? Before this patch read_seqbegin_or_lock() can spin in __read_seqcount_begin() while the write_seqlock(stats_lock) section was preempted. While at it, change the main loop to use __for_each_thread(sig, t). Signed-off-by: Oleg Nesterov --- include/linux/sched/signal.h | 4 +++- kernel/exit.c | 12 ++++++++---- kernel/fork.c | 3 ++- kernel/sched/cputime.c | 10 ++++++---- 4 files changed, 19 insertions(+), 10 deletions(-) diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h index d7fa3ca2fa53..c7c0928b877d 100644 --- a/include/linux/sched/signal.h +++ b/include/linux/sched/signal.h @@ -182,7 +182,9 @@ struct signal_struct { * Live threads maintain their own counters and add to these * in __exit_signal, except for the group leader. */ - seqlock_t stats_lock; + rwlock_t stats_lock; + seqcount_rwlock_t stats_seqc; + u64 utime, stime, cutime, cstime; u64 gtime; u64 cgtime; diff --git a/kernel/exit.c b/kernel/exit.c index f3ba4b97a7d9..8dedb7138f9c 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -182,7 +182,8 @@ static void __exit_signal(struct task_struct *tsk) * see the empty ->thread_head list. */ task_cputime(tsk, &utime, &stime); - write_seqlock(&sig->stats_lock); + write_lock(&sig->stats_lock); + write_seqcount_begin(&sig->stats_seqc); sig->utime +=3D utime; sig->stime +=3D stime; sig->gtime +=3D task_gtime(tsk); @@ -196,7 +197,8 @@ static void __exit_signal(struct task_struct *tsk) sig->sum_sched_runtime +=3D tsk->se.sum_exec_runtime; sig->nr_threads--; __unhash_process(tsk, group_dead); - write_sequnlock(&sig->stats_lock); + write_seqcount_end(&sig->stats_seqc); + write_unlock(&sig->stats_lock); =20 /* * Do this under ->siglock, we can race with another thread @@ -1160,7 +1162,8 @@ static int wait_task_zombie(struct wait_opts *wo, str= uct task_struct *p) */ thread_group_cputime_adjusted(p, &tgutime, &tgstime); spin_lock_irq(¤t->sighand->siglock); - write_seqlock(&psig->stats_lock); + write_lock(&psig->stats_lock); + write_seqcount_begin(&psig->stats_seqc); psig->cutime +=3D tgutime + sig->cutime; psig->cstime +=3D tgstime + sig->cstime; psig->cgtime +=3D task_gtime(p) + sig->gtime + sig->cgtime; @@ -1183,7 +1186,8 @@ static int wait_task_zombie(struct wait_opts *wo, str= uct task_struct *p) psig->cmaxrss =3D maxrss; task_io_accounting_add(&psig->ioac, &p->ioac); task_io_accounting_add(&psig->ioac, &sig->ioac); - write_sequnlock(&psig->stats_lock); + write_seqcount_end(&psig->stats_seqc); + write_unlock(&psig->stats_lock); spin_unlock_irq(¤t->sighand->siglock); } =20 diff --git a/kernel/fork.c b/kernel/fork.c index b9d3aa493bbd..bbd5604053f8 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1870,7 +1870,8 @@ static int copy_signal(unsigned long clone_flags, str= uct task_struct *tsk) sig->curr_target =3D tsk; init_sigpending(&sig->shared_pending); INIT_HLIST_HEAD(&sig->multiprocess); - seqlock_init(&sig->stats_lock); + rwlock_init(&sig->stats_lock); + seqcount_rwlock_init(&sig->stats_seqc, &sig->stats_lock); prev_cputime_init(&sig->prev_cputime); =20 #ifdef CONFIG_POSIX_TIMERS diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index af7952f12e6c..bd6a85bd2a49 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -333,12 +333,13 @@ void thread_group_cputime(struct task_struct *tsk, st= ruct task_cputime *times) nextseq =3D 0; do { seq =3D nextseq; - flags =3D read_seqbegin_or_lock_irqsave(&sig->stats_lock, &seq); + flags =3D read_seqcount_begin_or_lock_irqsave(&sig->stats_seqc, + &sig->stats_lock, &seq); times->utime =3D sig->utime; times->stime =3D sig->stime; times->sum_exec_runtime =3D sig->sum_sched_runtime; =20 - for_each_thread(tsk, t) { + __for_each_thread(sig, t) { task_cputime(t, &utime, &stime); times->utime +=3D utime; times->stime +=3D stime; @@ -346,8 +347,9 @@ void thread_group_cputime(struct task_struct *tsk, stru= ct task_cputime *times) } /* If lockless access failed, take the lock. */ nextseq =3D 1; - } while (need_seqretry(&sig->stats_lock, seq)); - done_seqretry_irqrestore(&sig->stats_lock, seq, flags); + } while (need_seqcount_retry(&sig->stats_seqc, seq)); + done_seqcount_retry_irqrestore(&sig->stats_seqc, &sig->stats_lock, + seq, flags); rcu_read_unlock(); } =20 --=20 2.25.1.362.g51ebf55