From nobody Fri Dec 19 17:35:42 2025 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B71B71922E4 for ; Wed, 26 Jun 2024 19:47:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431233; cv=none; b=CejIVzO2HdsjYXf9ZRC5Ym2vVcvTAT+erhXj6EFFDJOFY1c7q0usjHmpVTyW1ipeTAEMOEQPy3cjce0PjK+JgySqKgx7SORyjJZqF2Nvy7TKPF5h8y2zEysjJoM2Kg05IGw2cvikAMZKWB4PdnOwJHg9XKbybOZPLk1ByyuXxWY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431233; c=relaxed/simple; bh=l3bUJ9KnFiZZ581aAAIDi02PIvqwbk9jreOFCeFI5mk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Fb1JmAi88gEITimaDzxXcfzWfWdqxsvbZ2+5luowshAzcU+MDTJrIB8b1f0Ym4XTOZpVvhFuCEVW1B50xFhA7cMsnYFVU5ijaD86kQsV8prQe2PUd7Pr741SKW1Nj0arTRaCEsj1kEHMNUmOIp8ROMRuEbt0HWoH46tsjLTUlUk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YjADkj13; arc=none smtp.client-ip=209.85.219.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YjADkj13" Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-6b52eb2ee2eso21128026d6.1 for ; Wed, 26 Jun 2024 12:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719431230; x=1720036030; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=quNMt21k5Tu+QFDqj3HStB/o/6SWdJctxGCAsXhdbxw=; b=YjADkj13Yov7Wqn207XbdoZb2aqJoYvjTBnC9zhEsqiMMfnmjWYbrXa9V7RY4tMVyx EwFzWaq0AmsJGYrwbH4Ev0VrYYc9Rw3mQPX3jyr+Fd98HXRo+ogcFbJsPGDj/nwjN89m CvZMJc0JR2e6nKbKhF81CnC108NvcJX/zJxF8rpDpAcG2wRD+pJHUpqz3lzxkMzuUEGC Z5IvNfAc5SWC71HqOnYoMlzatW2NhxtIpGAMrcYRPv2FIOHEH5wLV//gzU0dzd1UfWR7 XsXIaoy1QwgXcnsFVeCtn+XudPsxv3oTpQFoJm1aedsCM1Sl3sJTgbwggS1aNODGkrkV 4LKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719431230; x=1720036030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=quNMt21k5Tu+QFDqj3HStB/o/6SWdJctxGCAsXhdbxw=; b=aog3WflTw5254P0Mik/OFppSzmelFPEpYotDkautrCTdZu3d7PY6fako07NnxCnxAi vraI9cLU/FD+TZTPIplqQvimkHovBcoJBT7WvHriljmqoQylyvw/MzB7t00YIwevALw8 lHpFy+QQPOTko2afKK9ACm/p+aEev5dU0u3Zmq5Wv8zYa5rH8yDoQ4OoDL/U9i7Rz3+Z aQ8M55kd5cjJaK624IOjZzO2jX7AQKUP8d0FmWauOCSIMV9UiQIdH++G+SBUIUPopiAw /mWjtJM4l89Fh+j9y2J83N9Igtu9FrEhdNMXKflXUwFPpVvRjlgqgjU9TwLmdN39WT8W crAQ== X-Gm-Message-State: AOJu0Yw3DIqcKhXt/F+atGCWhR3YMXI9jAHiXH7dW2M5WPUOPWnTBPeV fMMv8gNefX+zNqxgyQVtJpP5ZTUH94iI029M+dg/OzbJyl178/CuLiX1tAWX X-Google-Smtp-Source: AGHT+IHM/Hxcy2etI4hhkfIvRl+8PUp0crGZyN/TKijgBq+jzHdzfOIXVu30DrVTR8nykcty6UyYRQ== X-Received: by 2002:a05:6214:1630:b0:6b2:c840:7a2 with SMTP id 6a1803df08f44-6b540aa8879mr114912716d6.45.1719431230584; Wed, 26 Jun 2024 12:47:10 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b52ac1028csm48489996d6.99.2024.06.26.12.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 12:47:10 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v7 1/4] driver core: don't always lock parent in shutdown Date: Wed, 26 Jun 2024 14:46:47 -0500 Message-Id: <20240626194650.3837-2-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240626194650.3837-1-stuart.w.hayes@gmail.com> References: <20240626194650.3837-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't lock a parent device unless it is needed in device_shutdown. This is in preparation for making device shutdown asynchronous, when it will be needed to allow children of a common parent to shut down simultaneously. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Christoph Hellwig Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg --- drivers/base/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index 2b4c0624b704..03edf7a7ec37 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4815,7 +4815,7 @@ void device_shutdown(void) spin_unlock(&devices_kset->list_lock); =20 /* hold lock to avoid race with probe/release */ - if (parent) + if (parent && dev->bus && dev->bus->need_parent_lock) device_lock(parent); device_lock(dev); =20 @@ -4839,7 +4839,7 @@ void device_shutdown(void) } =20 device_unlock(dev); - if (parent) + if (parent && dev->bus && dev->bus->need_parent_lock) device_unlock(parent); =20 put_device(dev); --=20 2.39.3 From nobody Fri Dec 19 17:35:42 2025 Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F819192B7A for ; Wed, 26 Jun 2024 19:47:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431235; cv=none; b=UgMx0r/qGISy1bjUlVU9EYyvK6DLAi98esKp5do0A54/4O870oX47oqSjJ4eq7I9kl/f7ufY1TGroZxgTj3yNbMeuCI0gfml0Z8gDhZAgzwanScQCao7uWEScvvrqWkXrazWwg+X+KcWNyTV0T0DOJOQaqtPISGJ+QylrURAGZQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431235; c=relaxed/simple; bh=whW4NEDtUAWvPFkHfGqM/eWmy8LMHrs8iCjhNrO7gzo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=V+X+jGPV0oMxfv/si/gp4f1JDX8YQVZ54Ygk4OnbNjWGqj8KJ2p5bwbOhDrEhK593V1IADc0k3e5loIOOoRDlj0TzUzFuNvqYfY+9srod+SBIbrI3Nny2wpKWvqrZGaFiusEFH75UP7HrFhNFA/uFvKl/5LIw4ji2PXQ09EAhYg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OWvcV22N; arc=none smtp.client-ip=209.85.219.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OWvcV22N" Received: by mail-qv1-f54.google.com with SMTP id 6a1803df08f44-6b4ff7cb429so30173196d6.2 for ; Wed, 26 Jun 2024 12:47:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719431232; x=1720036032; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/t53QVoqoslUPlZCx8x17h7LOVaKiKwVjI+I5ylRm50=; b=OWvcV22Nk8fG+0KkXqIANZZiqaDsigJlfeSjbLgQqG7Flxfv9zuLHMJuw7QvVXQzGb 3abQf4TSyFK7v7C2QxWx3E4dcfxwJ8dhf1afugKbwY88o7LguLvSOgNv5bhcpwo06Hny Odc8Ytfy79w069NapqnNrmXvvf+OwK6mbFsFl/Kiyz8u5Y/KgbS8OH3VlI9+gENjVea2 B3Xkw7e/N2PpwA+vnGvZumpODGNOvBwao4h/dFSItRh8knU2BFBYZfqyGtbjLhGFzQnP Z6MwG2Qz5kbL777FkJPu0S1qyw6DKrQXGYzQxhrQuQPN1gGdYm+kz8U5SG0qit4zXQaU QTww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719431232; x=1720036032; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/t53QVoqoslUPlZCx8x17h7LOVaKiKwVjI+I5ylRm50=; b=FPrEbIcjn0zB5R9cWAhmz5dTHzvBKv1+V6pkOzWbrzZxGCdnqvH8YzQWCcQJ/H3FKG wpaYzK/hsp1PU2/neTm/4mQwAkXDT4uPfZAdI/vhEyptJweNhrpmd23TNcwnWIMZ1d19 /nb91Oj1fkvvqoFs+ZoxIfzI+9qCsWubXLYFN5CvYr/nSfbtlccB9rxM3WbTKPVskVc8 M8ktxu1OofUVLyte3PFz3KYIUc3k/gEAd+tp7Us6nxlV/1V2PDJ0E86qiW4oRrjuatLn ApgMPS0KYxV85ndMGO6Jwap6UrX8LUkxfui2Z+RUHTPGFDbhugj8GeYMpHscA/guNe9N XqQA== X-Gm-Message-State: AOJu0YwGmz/7S9qMBMnwNiuZbtWR6Nsz5lp2z++r3T/hmFLN+m+OGz1N fLSCS8kdZZavbnJbL2bB9YNZ0TvjH/ddNp41ZpzcjKohk6VRagsFYOsT1nDP X-Google-Smtp-Source: AGHT+IFBvOA+iyHRobdSxdaxMTRLIM1evBlCzixHNIMYcGypuDRbefQ1cYpWHTm6lnaQXv9sHGFacA== X-Received: by 2002:a0c:e441:0:b0:6af:2344:5811 with SMTP id 6a1803df08f44-6b540afb170mr122808676d6.55.1719431232429; Wed, 26 Jun 2024 12:47:12 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b52ac1028csm48489996d6.99.2024.06.26.12.47.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 12:47:12 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v7 2/4] driver core: separate function to shutdown one device Date: Wed, 26 Jun 2024 14:46:48 -0500 Message-Id: <20240626194650.3837-3-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240626194650.3837-1-stuart.w.hayes@gmail.com> References: <20240626194650.3837-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make a separate function for the part of device_shutdown() that does the shutown for a single device. This is in preparation for making device shutdown asynchronous. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Christoph Hellwig Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg --- drivers/base/core.c | 66 ++++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 30 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index 03edf7a7ec37..4be6071c2175 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4778,6 +4778,41 @@ int device_change_owner(struct device *dev, kuid_t k= uid, kgid_t kgid) } EXPORT_SYMBOL_GPL(device_change_owner); =20 +static void shutdown_one_device(struct device *dev) +{ + /* hold lock to avoid race with probe/release */ + if (dev->parent && dev->bus && dev->bus->need_parent_lock) + device_lock(dev->parent); + device_lock(dev); + + /* Don't allow any more runtime suspends */ + pm_runtime_get_noresume(dev); + pm_runtime_barrier(dev); + + if (dev->class && dev->class->shutdown_pre) { + if (initcall_debug) + dev_info(dev, "shutdown_pre\n"); + dev->class->shutdown_pre(dev); + } + if (dev->bus && dev->bus->shutdown) { + if (initcall_debug) + dev_info(dev, "shutdown\n"); + dev->bus->shutdown(dev); + } else if (dev->driver && dev->driver->shutdown) { + if (initcall_debug) + dev_info(dev, "shutdown\n"); + dev->driver->shutdown(dev); + } + + device_unlock(dev); + if (dev->parent && dev->bus && dev->bus->need_parent_lock) + device_unlock(dev->parent); + + put_device(dev); + if (dev->parent) + put_device(dev->parent); +} + /** * device_shutdown - call ->shutdown() on each device to shutdown. */ @@ -4814,36 +4849,7 @@ void device_shutdown(void) list_del_init(&dev->kobj.entry); spin_unlock(&devices_kset->list_lock); =20 - /* hold lock to avoid race with probe/release */ - if (parent && dev->bus && dev->bus->need_parent_lock) - device_lock(parent); - device_lock(dev); - - /* Don't allow any more runtime suspends */ - pm_runtime_get_noresume(dev); - pm_runtime_barrier(dev); - - if (dev->class && dev->class->shutdown_pre) { - if (initcall_debug) - dev_info(dev, "shutdown_pre\n"); - dev->class->shutdown_pre(dev); - } - if (dev->bus && dev->bus->shutdown) { - if (initcall_debug) - dev_info(dev, "shutdown\n"); - dev->bus->shutdown(dev); - } else if (dev->driver && dev->driver->shutdown) { - if (initcall_debug) - dev_info(dev, "shutdown\n"); - dev->driver->shutdown(dev); - } - - device_unlock(dev); - if (parent && dev->bus && dev->bus->need_parent_lock) - device_unlock(parent); - - put_device(dev); - put_device(parent); + shutdown_one_device(dev); =20 spin_lock(&devices_kset->list_lock); } --=20 2.39.3 From nobody Fri Dec 19 17:35:42 2025 Received: from mail-yb1-f172.google.com (mail-yb1-f172.google.com [209.85.219.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A61119307D for ; Wed, 26 Jun 2024 19:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431237; cv=none; b=nyCoIW4/HHArD+r70RbX3WPPm6PVHC0LNh7aNcBTCxPKhN+2qPrck3JMk4XD1rLqYSSggKBH2y/z48T4IZ7/lGpPG+k9khNPoTVSUhc7TKJkjL+95DOdYCCOE32fdNRzkQ4nnY+24R7bvewRWV9eBt3PgTRYZE9Q32y4wzm/Yuo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431237; c=relaxed/simple; bh=i3WX7/RSaA4I81jbiK9gsiNoB5j4kNcBfACp8dfK87A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AzZzw/9Zb9loSUT+kmhD2dEYBkgLYOQO1w9V10wEoWaYRAsLXK7BKkGVUH9Kpw5pHZupzBmRMVwnXJK5p0m65ksb4gUUrluXfzp0ucaZOL9H1M0E4z3yQYbGtOT3L17+2bvwNFyjy5ygLtM9WalrOYvi/f1F4bpyBy32RgjcVB8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=D8ac2TrO; arc=none smtp.client-ip=209.85.219.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="D8ac2TrO" Received: by mail-yb1-f172.google.com with SMTP id 3f1490d57ef6-e02c4983bfaso7090795276.2 for ; Wed, 26 Jun 2024 12:47:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719431234; x=1720036034; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ng2WmGXQZql1jDXPW/vYyFeh8nYeJu3JtauFqQLagBk=; b=D8ac2TrOqfRqXExLXZhD1TZAlo/JV2UvSgYDTxbbZBtyGXaV1Cn6o0nWC91xNoQIDW oC1DrF+z9ByJKsWmjz6bT6JZEqTpTxLm8rJ80cUp5/UNaFbEjd/p/EVJDKWr8eDkT6Pz KH/j3FGJczEVQpUFB6F8tUpd1RGkI3eY6Pmu88B/5uAVg9PqxAotqfBS7GJNrRNTj+c0 93jM3DBH+yFMr3rKnVrfjjwA5Gjth2RVSw9VVEA+SN1dfXsYaTtV1md7Zs6hgZrsEHHL UVmy9XI7RI6vpbaQ0JS/O35+Dtr65VddJ6hvhUiK+hgLeh8eWDmPSfk7ejwZL2io7hWd fsoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719431234; x=1720036034; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ng2WmGXQZql1jDXPW/vYyFeh8nYeJu3JtauFqQLagBk=; b=prHwdMDLAKVeFmWO7JFOhUv80FyCNM/H2YPprFCRBRTI470RjSCXc8+qMm3hytuCt3 wGE3JsVOLS7JZobCaNPRqyw5qnsDA83AKSpzlxQAbzVausYz5nDTCPNrtg+A+epYBlML cHru5syc8raLRt24+CtGrmpYCa1TuIl3LaUM+AfcUZP8mffCr2TnNU5t1wGYZwhzH2Hr KC6VW69ge0FmKLFCW+VO5kLiEEZhyaNyTcKPeqSnLeZNys+QRplrHfLkzBEpxwCCMmnd +FRQpe8vYrqZEuYSGFiZbuV/ayCyjSyb4rTzbuiM4JD77G5fiA6ZfdrTnPhzXPguDABg cpQw== X-Gm-Message-State: AOJu0YxjOUg62qzQmz8jFM74gkFLjB42f3YB0FKkyTn80b7QNFDpZ1G1 rB5NHR/RzW9WSG4wN5WTRYs49+7No5WXl4I8QrhWmHUZcSsLTwSw/Wi/iAns X-Google-Smtp-Source: AGHT+IHl9hABe7xX2H7iF4FE+OsGRaWYvA0ZYF+mCybXMF3BftUU+kXejmxt9lePOISoscwqnHSTSw== X-Received: by 2002:a81:8904:0:b0:63b:ee94:1eb7 with SMTP id 00721157ae682-6433e906b0bmr109055207b3.25.1719431234276; Wed, 26 Jun 2024 12:47:14 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b52ac1028csm48489996d6.99.2024.06.26.12.47.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 12:47:13 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v7 3/4] driver core: shut down devices asynchronously Date: Wed, 26 Jun 2024 14:46:49 -0500 Message-Id: <20240626194650.3837-4-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240626194650.3837-1-stuart.w.hayes@gmail.com> References: <20240626194650.3837-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add code to allow asynchronous shutdown of devices, ensuring that each device is shut down before its parents & suppliers. Add async_shutdown_enable to struct device_driver, and expose it via sysfs. This can be used to view or change driver opt-in to asynchronous shutdown. Only devices with drivers that have async_shutdown_enable enabled will be shut down asynchronously. This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg --- drivers/base/base.h | 3 ++ drivers/base/bus.c | 26 +++++++++++++++++ drivers/base/core.c | 54 ++++++++++++++++++++++++++++++++++- include/linux/device/driver.h | 2 ++ 4 files changed, 84 insertions(+), 1 deletion(-) diff --git a/drivers/base/base.h b/drivers/base/base.h index db4f910e8e36..6f65f159d039 100644 --- a/drivers/base/base.h +++ b/drivers/base/base.h @@ -10,6 +10,7 @@ * shared outside of the drivers/base/ directory. * */ +#include #include =20 /** @@ -97,6 +98,7 @@ struct driver_private { * the device; typically because it depends on another driver getting * probed first. * @async_driver - pointer to device driver awaiting probe via async_probe + * @shutdown_after - used during device shutdown to ensure correct shutdow= n ordering. * @device - pointer back to the struct device that this structure is * associated with. * @dead - This device is currently either in the process of or has been @@ -114,6 +116,7 @@ struct device_private { struct list_head deferred_probe; struct device_driver *async_driver; char *deferred_probe_reason; + async_cookie_t shutdown_after; struct device *device; u8 dead:1; }; diff --git a/drivers/base/bus.c b/drivers/base/bus.c index ffea0728b8b2..97fd02cff888 100644 --- a/drivers/base/bus.c +++ b/drivers/base/bus.c @@ -10,6 +10,7 @@ */ =20 #include +#include #include #include #include @@ -635,6 +636,25 @@ static ssize_t uevent_store(struct device_driver *drv,= const char *buf, } static DRIVER_ATTR_WO(uevent); =20 +static ssize_t async_shutdown_enable_show(struct device_driver *drv, char = *buf) +{ + return sysfs_emit(buf, "%d\n", drv->async_shutdown_enable); +} + +static ssize_t async_shutdown_enable_store(struct device_driver *drv, cons= t char *buf, + size_t count) +{ + if (!capable(CAP_SYS_BOOT)) + return -EPERM; + + if (kstrtobool(buf, &drv->async_shutdown_enable) < 0) + return -EINVAL; + + return count; +} + +static DRIVER_ATTR_RW(async_shutdown_enable); + /** * bus_add_driver - Add a driver to the bus. * @drv: driver. @@ -702,6 +722,12 @@ int bus_add_driver(struct device_driver *drv) } } =20 + error =3D driver_create_file(drv, &driver_attr_async_shutdown_enable); + if (error) { + pr_err("%s: async_shutdown attr (%s) failed\n", + __func__, drv->name); + } + return 0; =20 out_detach: diff --git a/drivers/base/core.c b/drivers/base/core.c index 4be6071c2175..1eb5a7286c79 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -9,6 +9,7 @@ */ =20 #include +#include #include #include #include @@ -3528,6 +3529,7 @@ static int device_private_init(struct device *dev) klist_init(&dev->p->klist_children, klist_children_get, klist_children_put); INIT_LIST_HEAD(&dev->p->deferred_probe); + dev->p->shutdown_after =3D 0; return 0; } =20 @@ -4778,6 +4780,8 @@ int device_change_owner(struct device *dev, kuid_t ku= id, kgid_t kgid) } EXPORT_SYMBOL_GPL(device_change_owner); =20 +static ASYNC_DOMAIN(sd_domain); + static void shutdown_one_device(struct device *dev) { /* hold lock to avoid race with probe/release */ @@ -4813,12 +4817,34 @@ static void shutdown_one_device(struct device *dev) put_device(dev->parent); } =20 +/** + * shutdown_one_device_async + * @data: the pointer to the struct device to be shutdown + * @cookie: not used + * + * Shuts down one device, after waiting for shutdown_after to complete. + * shutdown_after should be set to the cookie of the last child or consumer + * of this device to be shutdown (if any), or to the cookie of the previous + * device to be shut down for devices that don't enable asynchronous shutd= own. + */ +static void shutdown_one_device_async(void *data, async_cookie_t cookie) +{ + struct device *dev =3D data; + + async_synchronize_cookie_domain(dev->p->shutdown_after + 1, &sd_domain); + + shutdown_one_device(dev); +} + /** * device_shutdown - call ->shutdown() on each device to shutdown. */ void device_shutdown(void) { struct device *dev, *parent; + async_cookie_t cookie =3D 0; + struct device_link *link; + int idx; =20 wait_for_device_probe(); device_block_probing(); @@ -4849,11 +4875,37 @@ void device_shutdown(void) list_del_init(&dev->kobj.entry); spin_unlock(&devices_kset->list_lock); =20 - shutdown_one_device(dev); + + /* + * Set cookie for devices that will be shut down synchronously + */ + if (!dev->driver || !dev->driver->async_shutdown_enable) + dev->p->shutdown_after =3D cookie; + + get_device(dev); + get_device(parent); + + cookie =3D async_schedule_domain(shutdown_one_device_async, + dev, &sd_domain); + /* + * Ensure parent & suppliers wait for this device to shut down + */ + if (parent) { + parent->p->shutdown_after =3D cookie; + put_device(parent); + } + + idx =3D device_links_read_lock(); + list_for_each_entry_rcu(link, &dev->links.suppliers, c_node, + device_links_read_lock_held()) + link->supplier->p->shutdown_after =3D cookie; + device_links_read_unlock(idx); + put_device(dev); =20 spin_lock(&devices_kset->list_lock); } spin_unlock(&devices_kset->list_lock); + async_synchronize_full_domain(&sd_domain); } =20 /* diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h index 7738f458995f..1e78f2ab1366 100644 --- a/include/linux/device/driver.h +++ b/include/linux/device/driver.h @@ -56,6 +56,7 @@ enum probe_type { * @mod_name: Used for built-in modules. * @suppress_bind_attrs: Disables bind/unbind via sysfs. * @probe_type: Type of the probe (synchronous or asynchronous) to use. + * @async_shutdown_enable: Enables devices to be shutdown asynchronously. * @of_match_table: The open firmware table. * @acpi_match_table: The ACPI match table. * @probe: Called to query the existence of a specific device, @@ -102,6 +103,7 @@ struct device_driver { =20 bool suppress_bind_attrs; /* disables bind/unbind via sysfs */ enum probe_type probe_type; + bool async_shutdown_enable; =20 const struct of_device_id *of_match_table; const struct acpi_device_id *acpi_match_table; --=20 2.39.3 From nobody Fri Dec 19 17:35:42 2025 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FA74192B88 for ; Wed, 26 Jun 2024 19:47:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431239; cv=none; b=IMDM7a9KAD8d5oxvozoDAcOUbyEfwxkBK84hmPxXwXwB30MLVZmjsaf0ayOmajCYOr6QkWNW6fvDhsImXIGtSybhrj3rXnkxQWsXQ5A8V7yCG9Lv+bP/dh2simtUeGcMJMU2VneFOyfKQeKTjbnmJSuGc6eNZRDu8tQ1lbveQ+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719431239; c=relaxed/simple; bh=sTdUENHSC3zhjJq53bao9mwxCnxqZvoZJKSCKbBoydc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IW7e0d9i68yBWV9jBMgxLcizFU47Cwt0YMYTvnndKbwSLFkfuQEmuENlMbLI+YOzuUd+LPsi0qEWVu0loYywymITWF0kIHyoAcW/HP443ZZ6O+LvY2QYnJuhERClLGaFqqJ24PuMByeBolBMZcTMMo3ED0CAL1lr/25qGvb1xS0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KY9dT8J2; arc=none smtp.client-ip=209.85.219.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KY9dT8J2" Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-6b5253ffd24so21213956d6.2 for ; Wed, 26 Jun 2024 12:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719431236; x=1720036036; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xXDZTLeuWuzR4JFBbSliTdNmKT1J60juYffwmH3Ya+w=; b=KY9dT8J2QKvqCMCAM0hleNlb+uR+WuE9c4j7ekLe7S/gn0pnHT5K3mLg6sBg5VJzq3 SNBvumMstKY7U62ujHldJWxLx5SyZS9WdTdp70BlzcpTHrsfwuaqTKRG16C3n8hyrZIt Lct6HGFiyr+9fX2cNjHh6HkvhsdXKc1xEECQL+AS084C6VnGBpxuP3z84TaVspSIYbjc pdP3qutsI+9cVUp9deT3q3GPf04lOOcmcM8SWRdzMXuzfq1Rj6yTPHxY5vBV4PQqICof ziRhu9XGub8y8Edk/RJnbObGY4FEOukf5vZR7F1hN+UwnRht2siVY4NLooppwDAecpiP 7ZFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719431236; x=1720036036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xXDZTLeuWuzR4JFBbSliTdNmKT1J60juYffwmH3Ya+w=; b=SjDYGawi39SdciqNkeThsTj9uII0cBVjIG9V+YXzy1DmHiQ/YwNdwPSo9+QqVpWwI7 BpIhnLCHcAuzC0kbJSOW6BWbkKGkkNFsNKH55k9QqhaH9CKDcJbj+3I5MLQ/onPYa9XP Zuxi4qjOWlB1P0Nx7/DmopqRGwdJyRHDS6TO9Sk4T6AvKQT0paowvMBM5JuIiAVTN1nF LaMlxdl7eMxmCgymYLjgUHdF+mUkNlF60JJMdIMty+5IMlKA7Ll6QZKD0H0q3ENmtTbe z4KYYovmGjgH+sD39MWyra32VP8Rd9ZOWW+jOO9jFZ1Us/vd2Xf5pw53AglZjrtAqJkN 1gFQ== X-Gm-Message-State: AOJu0YwoSQP+QAbX/HsLnx1snykln8TOlT399bnxtc8LbEZCJiYPVnjN r4No68ggWoGzNay0jt7pzBZSXHfwTVWqfDTl0uf6LMfSAO84psMM8Gy85ax0 X-Google-Smtp-Source: AGHT+IFEq2G29a9cZ3CtqTm8lu6M1ohFZCP2KJGvd+Bn8S2ZMURyxwzcnnFwPwWRhHdcNMdssoKpIA== X-Received: by 2002:a0c:c384:0:b0:6b5:1f3d:c810 with SMTP id 6a1803df08f44-6b5364f79efmr134299206d6.52.1719431236284; Wed, 26 Jun 2024 12:47:16 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b52ac1028csm48489996d6.99.2024.06.26.12.47.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jun 2024 12:47:15 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v7 4/4] nvme-pci: Make driver prefer asynchronous shutdown Date: Wed, 26 Jun 2024 14:46:50 -0500 Message-Id: <20240626194650.3837-5-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240626194650.3837-1-stuart.w.hayes@gmail.com> References: <20240626194650.3837-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the driver default to enable asynchronous shutdown. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg --- drivers/nvme/host/pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 102a9fb0c65f..8138e37547c9 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -3531,6 +3531,7 @@ static struct pci_driver nvme_driver =3D { .shutdown =3D nvme_shutdown, .driver =3D { .probe_type =3D PROBE_PREFER_ASYNCHRONOUS, + .async_shutdown_enable =3D true, #ifdef CONFIG_PM_SLEEP .pm =3D &nvme_dev_pm_ops, #endif --=20 2.39.3