From nobody Wed Dec 31 08:30:08 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CF39C4332F for ; Mon, 6 Nov 2023 19:36:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233086AbjKFTg2 (ORCPT ); Mon, 6 Nov 2023 14:36:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232748AbjKFTf7 (ORCPT ); Mon, 6 Nov 2023 14:35:59 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68B5E10C4 for ; Mon, 6 Nov 2023 11:35:56 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-da03ef6fc30so5576977276.0 for ; Mon, 06 Nov 2023 11:35:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1699299355; x=1699904155; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SA9U3BYKpPLNMavyPlNHV0DBfe0X28VHPrgcDnlNWu4=; b=dtPbMDawfzBhwhQq3OO+tqSrn9Z5NXNBhaCYZ0iaiOKCRu5v+wr4ItugAqtwQwRmH+ uP0YW2zXibSwOHKrnn96JOSFiloFEbTMa5IYbVJ1QNWJbpq+ZUvN86/rQFa8hDCwHXtV Bffm/zFcrGUJSR6pNzTE7IvDneaj9JKB2fulmEv0JzTi06PUaBFDUof+mafYI1naHrPm GYsWi6iMQRQoqX+oLKixJlre6CVGg0Qi6paJCmLo0hVrm9pkP2rgBhWLs4q0zNH9c+xg 2tOT0E1Z6f2VWQUUoSxZxQ7TbmMJmUkKuIS2Uz2X9JdvzFSQv5TqRxaNg6YIwUn0bWUi 6YWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699299355; x=1699904155; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SA9U3BYKpPLNMavyPlNHV0DBfe0X28VHPrgcDnlNWu4=; b=c9z2kFU4GhWvTIHxYoDSTLvmIpNwpXC1mKfDODzIHg7yvcC8BP0G9FgQdZEsCAJxpC AXGPQf+HJgG2mff0BST+5wyingxk/PbTnyUu6g+yWyn0MivPWFFSnn8eW64D+Fa/ON8/ wYPkQdUhdXH2o/i5KL8aK8Gl07osWnssi3ADBTyv4eWulgqQMcYro4mD9zQ0n4+JFUuH Cn47fFdz1A7lqBAjqrXUEislUhZg2SkMvSwLJrjA/yjR4m30JRl9ygj2WFtIHitxuddq iq6i6gIPC1rH+SQWkUtpOhROrTclgclbYpo2B8jlvfPvlTC+YVyavNAhsSiOd0ppOeb0 PvVg== X-Gm-Message-State: AOJu0YzbuRBitWnpaBM5SDdrDgIeAFBFbkS/4sM6t5/u11nwzzgKEBWe sCWOwPapNHBJgCa2Ys/wTiUKgex/kJqEC6QkOeMZ6faW2QB8Oqu2cnTJXOQr7DUHBk1FqSTUlOC Y3wLnoEyuBOOXs8ieIhfKbMw8qUt55YaXOaY5BUKs35nhO+H7OfrJNawfMJB5vLBe0+UYl+Y= X-Google-Smtp-Source: AGHT+IF5RAlbcLYjtzUA5D89EB529oUqPyEcJ5pKnV8ZnWdz83UyUdJSJ9VqPvsLcCCnDWjLa0aBuLaZTMhp X-Received: from jstultz-noogler2.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:600]) (user=jstultz job=sendgmr) by 2002:a05:6902:1802:b0:da0:94e8:84d4 with SMTP id cf2-20020a056902180200b00da094e884d4mr554056ybb.12.1699299355240; Mon, 06 Nov 2023 11:35:55 -0800 (PST) Date: Mon, 6 Nov 2023 19:34:53 +0000 In-Reply-To: <20231106193524.866104-1-jstultz@google.com> Mime-Version: 1.0 References: <20231106193524.866104-1-jstultz@google.com> X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog Message-ID: <20231106193524.866104-11-jstultz@google.com> Subject: [PATCH v6 10/20] locking/mutex: Switch to mutex handoffs for CONFIG_PROXY_EXEC From: John Stultz To: LKML Cc: Peter Zijlstra , Joel Fernandes , Qais Yousef , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Valentin Schneider , Steven Rostedt , Ben Segall , Zimuzo Ezeozue , Youssef Esmat , Mel Gorman , Daniel Bristot de Oliveira , Will Deacon , Waiman Long , Boqun Feng , "Paul E . McKenney" , kernel-team@android.com, Valentin Schneider , "Connor O'Brien" , John Stultz Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra Since with PROXY_EXEC, we will want to hand off locks to the task's we are running on behalf of, switch to using mutex handoffs. Cc: Joel Fernandes Cc: Qais Yousef Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Cc: Dietmar Eggemann Cc: Valentin Schneider Cc: Steven Rostedt Cc: Ben Segall Cc: Zimuzo Ezeozue Cc: Youssef Esmat Cc: Mel Gorman Cc: Daniel Bristot de Oliveira Cc: Will Deacon Cc: Waiman Long Cc: Boqun Feng Cc: "Paul E . McKenney" Cc: kernel-team@android.com Signed-off-by: Peter Zijlstra (Intel) [rebased, added comments and changelog] Signed-off-by: Juri Lelli [Fixed rebase conflicts] [squashed sched: Ensure blocked_on is always guarded by blocked_lock] Signed-off-by: Valentin Schneider [fix rebase conflicts, various fixes & tweaks commented inline] [squashed sched: Use rq->curr vs rq->proxy checks] Signed-off-by: Connor O'Brien [jstultz: Split out only the very basic initial framework for proxy logic from a larger patch.] Signed-off-by: John Stultz --- v5: * Split out from core proxy patch v6: * Rework to use sched_proxy_exec() instead of #ifdef CONFIG_PROXY_EXEC --- kernel/Kconfig.locks | 2 +- kernel/locking/mutex.c | 39 ++++++++++++++++++++++----------------- 2 files changed, 23 insertions(+), 18 deletions(-) diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index 4198f0273ecd..791c98f1d329 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -226,7 +226,7 @@ config ARCH_SUPPORTS_ATOMIC_RMW =20 config MUTEX_SPIN_ON_OWNER def_bool y - depends on SMP && ARCH_SUPPORTS_ATOMIC_RMW + depends on SMP && ARCH_SUPPORTS_ATOMIC_RMW && !PROXY_EXEC =20 config RWSEM_SPIN_ON_OWNER def_bool y diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index f37b7afe8aa5..5394a3c4b5d9 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -914,26 +914,31 @@ static noinline void __sched __mutex_unlock_slowpath(= struct mutex *lock, unsigne =20 mutex_release(&lock->dep_map, ip); =20 - /* - * Release the lock before (potentially) taking the spinlock such that - * other contenders can get on with things ASAP. - * - * Except when HANDOFF, in that case we must not clear the owner field, - * but instead set it to the top waiter. - */ - owner =3D atomic_long_read(&lock->owner); - for (;;) { - MUTEX_WARN_ON(__owner_task(owner) !=3D current); - MUTEX_WARN_ON(owner & MUTEX_FLAG_PICKUP); - - if (owner & MUTEX_FLAG_HANDOFF) - break; + if (sched_proxy_exec()) { + /* Always force HANDOFF for Proxy Exec for now. Revisit. */ + owner =3D MUTEX_FLAG_HANDOFF; + } else { + /* + * Release the lock before (potentially) taking the spinlock + * such that other contenders can get on with things ASAP. + * + * Except when HANDOFF, in that case we must not clear the + * owner field, but instead set it to the top waiter. + */ + owner =3D atomic_long_read(&lock->owner); + for (;;) { + MUTEX_WARN_ON(__owner_task(owner) !=3D current); + MUTEX_WARN_ON(owner & MUTEX_FLAG_PICKUP); =20 - if (atomic_long_try_cmpxchg_release(&lock->owner, &owner, __owner_flags(= owner))) { - if (owner & MUTEX_FLAG_WAITERS) + if (owner & MUTEX_FLAG_HANDOFF) break; =20 - return; + if (atomic_long_try_cmpxchg_release(&lock->owner, &owner, + __owner_flags(owner))) { + if (owner & MUTEX_FLAG_WAITERS) + break; + return; + } } } =20 --=20 2.42.0.869.gea05f2083d-goog