From nobody Mon Apr 27 03:23:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49934C43334 for ; Fri, 17 Jun 2022 09:10:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1380682AbiFQJKr (ORCPT ); Fri, 17 Jun 2022 05:10:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232153AbiFQJKp (ORCPT ); Fri, 17 Jun 2022 05:10:45 -0400 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3FFA2554BE for ; Fri, 17 Jun 2022 02:10:44 -0700 (PDT) Received: by mail-pf1-x435.google.com with SMTP id s37so3651338pfg.11 for ; Fri, 17 Jun 2022 02:10:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=PJGnbwi5MYQyjS4+9q5Y7gKs9wxJntkJMUiVbv/TYHI=; b=NyiA/gs6QG0Ta4mkYNXUvyn1qOn1K+ZO30n5HI3lmBVDQAJYvZBupBPVyEMe9wBTIU j2DtNgtCFk/g/jp/Dbd/W+fG2CEkl2p9H3YO+t/E//0sM8fxuGVp7BNquAyX2SeI/j1f zZ4/FIPIURLTAQQdgTsouj20s6Bk69Lc3R9rOrpK2a+iS28efzILEefHdCWThlLnK23K HctuUwrcwaqu6nzqI9jPl7xInYUZ1CqRCHbn1l2JKK8Eipgve2VegiOACvLhHZUxNx6X /TuZwGfE0a9cFIei7/TQzQtULShDeHujYG2QNKLgNiw1XDwl9aXFceDFN1NIThchpMH1 g6mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=PJGnbwi5MYQyjS4+9q5Y7gKs9wxJntkJMUiVbv/TYHI=; b=3b+sRejZX2k2JJfO8J7KtqDSo1xafgb/cnzhm6JhnBY6OX3iYr1YWxbSb0VZVLd6EE dvE0u+9LayTsO7215F8Fl/fdIOBBsyVvkwd0n2yKxeFwSMIEvX+Zm+axZjae4sPMCTXw 1cZL+Ny3V7epg2lnGpAFLSUkMtDZKxKgjpcO7PH/wHLSpUgpipsoVGV2XbXp4FsORFaC 18/desOtzyoTJq/Re/R7CK/TxRKzkHelt0H/Tu2LI5KgjJiGI/SQD+RXZiHeAHKU+Zcq oZpIeJKViIv4qGl5/U4wwTRhe1CpxXzka0EtjMH8PL+0wVLGblK1jwpMsKDj8CIV9+x5 nwcQ== X-Gm-Message-State: AJIora8KQfXTn5VrbrQNPEg97HI5ARNM3STJ55Ccv5PZh5z4BX/ilkiA 1BHxBQSbY7gGURyyhllqec0= X-Google-Smtp-Source: AGRyM1s/ndiCPHLLQcu2gvM1GW8ShZw9MwH6w3kTK1PxJbNF0lorQiBEKZklvOD/sgdqq4Q+zzboaw== X-Received: by 2002:a63:fc01:0:b0:403:af21:9729 with SMTP id j1-20020a63fc01000000b00403af219729mr8233226pgi.54.1655457043691; Fri, 17 Jun 2022 02:10:43 -0700 (PDT) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:d5d2:fc18:6baf:e16b]) by smtp.gmail.com with ESMTPSA id e11-20020a63744b000000b003c25dfd7372sm3272201pgn.26.2022.06.17.02.10.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jun 2022 02:10:42 -0700 (PDT) From: Eric Dumazet To: Peter Zijlstra , Linus Torvalds Cc: linux-kernel , Eric Dumazet , Eric Dumazet , Ingo Molnar , Waiman Long , Boqun Feng , Will Deacon , Roman Penyaev , Shakeel Butt Subject: [PATCH] locking/rwlocks: do not starve writers Date: Fri, 17 Jun 2022 02:10:39 -0700 Message-Id: <20220617091039.2257083-1-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Eric Dumazet Networking is still using rwlocks and read_lock() is called from softirq context, potentially from many cpus. In this (soft)irq context, rwlock code is unfair to writers and can cause soft lockups. We first noticed an issue with epoll after commit a218cc491420 ("epoll: use rwlock in order to reduce ep_poll_callback() contention"), but it is trivial to brick a host using this repro: for i in {1..48} do ping -f -n -q 127.0.0.1 & sleep 0.1 done If really an unfair version of rwlocks is needed, we should introduce a new read_lock_unfair(). [ 673.678717][ C34] watchdog: BUG: soft lockup - CPU#34 stuck for 82s! [= ping:17794] [ 673.700713][ C45] watchdog: BUG: soft lockup - CPU#45 stuck for 82s! [= ping:17796] [ 673.702712][ C46] watchdog: BUG: soft lockup - CPU#46 stuck for 78s! [= ping:17802] [ 673.704712][ C47] watchdog: BUG: soft lockup - CPU#47 stuck for 82s! [= ping:17798] [ 677.636023][ C13] watchdog: BUG: soft lockup - CPU#13 stuck for 82s! [= ping:17804] [ 677.638022][ C14] watchdog: BUG: soft lockup - CPU#14 stuck for 75s! [= ping:17825] [ 677.644021][ C17] watchdog: BUG: soft lockup - CPU#17 stuck for 75s! [= ping:17821] [ 677.650020][ C20] watchdog: BUG: soft lockup - CPU#20 stuck for 82s! [= ping:17800] [ 677.686014][ C38] watchdog: BUG: soft lockup - CPU#38 stuck for 75s! [= ping:17819] [ 681.691318][ C41] watchdog: BUG: soft lockup - CPU#41 stuck for 74s! [= ping:17823] [ 684.657807][ C46] rcu: INFO: rcu_sched self-detected stall on CPU [ 684.664075][ C46] rcu: 46-....: (1 GPs behind) idle=3D529/1/0x4000000= 000000000 softirq=3D22717/22717 fqs=3D20200 [ 705.633252][ C14] watchdog: BUG: soft lockup - CPU#14 stuck for 101s! = [ping:17825] [ 706.999058][ T309] rcu: INFO: rcu_sched detected expedited stalls on CP= Us/tasks: { 14-... 41-... } 88575 jiffies s: 2325 root: 0x5/. [ 706.999069][ T309] rcu: blocking rcu_node structures (internal RCU debu= g): l=3D1:0-15:0x4000/. l=3D1:32-47:0x200/. [ 709.686574][ C41] watchdog: BUG: soft lockup - CPU#41 stuck for 100s! = [ping:17823] [ 714.457782][ C41] rcu: INFO: rcu_sched self-detected stall on CPU [ 714.464047][ C41] rcu: 41-....: (1 GPs behind) idle=3D403/1/0x4000000= 000000000 softirq=3D24654/24655 fqs=3D4653 Fixes: 70af2f8a4f48 ("locking/rwlocks: Introduce 'qrwlocks' - fair, queued = rwlocks") Signed-off-by: Eric Dumazet Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Waiman Long Cc: Boqun Feng Cc: Will Deacon Cc: Boqun Feng Cc: Roman Penyaev Cc: Shakeel Butt --- kernel/locking/qrwlock.c | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c index 2e1600906c9f5cd868415d20e2d7024c5b1e0531..bf64d14f0fc88613363c3c007bc= a8c0918709123 100644 --- a/kernel/locking/qrwlock.c +++ b/kernel/locking/qrwlock.c @@ -23,16 +23,6 @@ void queued_read_lock_slowpath(struct qrwlock *lock) /* * Readers come here when they cannot get the lock without waiting */ - if (unlikely(in_interrupt())) { - /* - * Readers in interrupt context will get the lock immediately - * if the writer is just waiting (not holding the lock yet), - * so spin with ACQUIRE semantics until the lock is available - * without waiting in the queue. - */ - atomic_cond_read_acquire(&lock->cnts, !(VAL & _QW_LOCKED)); - return; - } atomic_sub(_QR_BIAS, &lock->cnts); =20 trace_contention_begin(lock, LCB_F_SPIN | LCB_F_READ); --=20 2.36.1.476.g0c4daa206d-goog