From nobody Sat Feb 7 22:21:12 2026 Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D73531CF2BA for ; Thu, 22 Aug 2024 20:28:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358497; cv=none; b=nIHy7tHUoyFuxewTIxm8S8P8iRaZVrDsbccenVTM+raL4GSalsJqC9+Zpc0RSkqU62iP8FUztAgNb+R5HQXKcJlqh/9T4BCuaOoalU/1D0w6TlInQdulSOzeKepVTYwoaS/HSLHmPFryGbeZnLT4q52zoPSQxDzC76uNLuNSFaI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358497; c=relaxed/simple; bh=sihxqV4aIZQMIil8+9BUF5iovP+aVDiXz4bTbGf1B08=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=s8qrJyjJmQM8vvB5vqerd5FmJLUHebb7Lco9ixsj1txSTAvKp0wgtYPR0WcmJ4DOcfrcTdzrXJscSD/YMLsbigZEqGyRxDDa7Qe3J15nidkxl2PHzrbLXMEI7O8EGA16buEAFETFTbQGiNWWQA4pmIvIgRL+vs4BPcqGerD8X9I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Z4vG2I+j; arc=none smtp.client-ip=209.85.210.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Z4vG2I+j" Received: by mail-ot1-f41.google.com with SMTP id 46e09a7af769-70e00cb1ee7so848247a34.1 for ; Thu, 22 Aug 2024 13:28:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724358495; x=1724963295; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kN6DwnGZ7LnuM1YHlsixLSHjYDgpQQZNtiJ32vWBOs4=; b=Z4vG2I+jxXvDg5ou/SGmx8dhEWjySNg+4UTNFT9nwXtyI46yNR6llfca1IsSL6pvl2 fkxYm9zFZPd8dYvr1YVfNuZa9XdnckLnYmqJvwn/vP9fMWUgLOj5BOCoK0lUS5glrtRz 125aRiXQuvoNtUYBZ/hGwOuoJUtq71VbZ05EKlrtcemolfdbnlD8mRMVuipZ1f6OZdP9 Gqu67QUKrBkA2ZnFbkG8Y5HYQVIbhbdnqz0ZDNG82oksSRuidY5sm1wzLqwGY6uth8xp SdOvmoeiU49WHA0eNwky6745JKRaJZobjhiDFhNjHoHG38ag+AR7cXW1/1+46IJ9Cys/ eybw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724358495; x=1724963295; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kN6DwnGZ7LnuM1YHlsixLSHjYDgpQQZNtiJ32vWBOs4=; b=IJumhtVJemWahn3Yz97NrzCvYpUu3DzX2LKwBGWHTXKwU5fX7wEKvlL3dr1cKGjIqb gXxcWS458jlceMXSWL1BURo+AgQst4E5UpadODg9jI5p7UH0t2jKaWL6/FMiBeqEEaS1 ly8UJ1xtAbEAbMXLRBRCBi/ogdwV/BCEGJ312jNEGChgwrwDnx1HDgN+XuZLt1YDXyTu 6mI1uBgQ4829RGEyTOgyznmwMA7S+nMYrd9zZa4wMk82vMrKjJ+tMUOPhLWoY00blUAk I+8GsFcps3zcAlrK88pcccx3ADdSRAogk0DH8L4ABiYNi/YHeCHbjcbt19wMVDEnYnHt dUvQ== X-Gm-Message-State: AOJu0YziULC+RUOUqg6Dr1lw3PCNVU9W0Qnx/jiLcQfSXgOODBPpr0LD KmaM+G7xZ7P5i1cfN5WPXWmH3p8BcSvJucSmNLUWdnJ3RYNT6QTNoFR337w/ X-Google-Smtp-Source: AGHT+IGM9eaD6qRrORCTD3ILenlNYjzUDdYyP8kMrFM2XEXW2BxWn6SCb4xiJnKAbfa0J/SN7nuIiQ== X-Received: by 2002:a05:6808:3c4f:b0:3d9:2154:2ffd with SMTP id 5614622812f47-3de2a8416bdmr32232b6e.3.1724358494866; Thu, 22 Aug 2024 13:28:14 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3de2252d5fbsm421265b6e.1.2024.08.22.13.28.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 13:28:14 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v8 1/4] driver core: don't always lock parent in shutdown Date: Thu, 22 Aug 2024 15:28:02 -0500 Message-Id: <20240822202805.6379-2-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240822202805.6379-1-stuart.w.hayes@gmail.com> References: <20240822202805.6379-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't lock a parent device unless it is needed in device_shutdown. This is in preparation for making device shutdown asynchronous, when it will be needed to allow children of a common parent to shut down simultaneously. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Christoph Hellwig Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg Tested-by: Keith Busch --- drivers/base/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index 8c0733d3aad8..13253d105062 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4818,7 +4818,7 @@ void device_shutdown(void) spin_unlock(&devices_kset->list_lock); =20 /* hold lock to avoid race with probe/release */ - if (parent) + if (parent && dev->bus && dev->bus->need_parent_lock) device_lock(parent); device_lock(dev); =20 @@ -4842,7 +4842,7 @@ void device_shutdown(void) } =20 device_unlock(dev); - if (parent) + if (parent && dev->bus && dev->bus->need_parent_lock) device_unlock(parent); =20 put_device(dev); --=20 2.39.3 From nobody Sat Feb 7 22:21:12 2026 Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com [209.85.210.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47D621D0DD7 for ; Thu, 22 Aug 2024 20:28:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358498; cv=none; b=qrauIdUkXLgLJ9KYoeBuI9PYiAn+3LhLp4GF+L5ti7BWbpyyO8bvn0uYadR1j9Y143HvR1hI75vBZieuCO+W5gKGtmVJRQFW8btcsIHDBUR3MNhfGB61Y9fuS6EpFx+tRWtWTeeplxLRPQJk7MfKDHbg+hTM/EAYSEQmvuz//hU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358498; c=relaxed/simple; bh=PYlTbsEA2Gnf3V9BDLC5a93rkztOt1IWXxJrQlshLy8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HPzxZcSMcUml9iRaEbQURouQfo2lyPXWxK400yP7pUpfED+dFmG+kgvluD2Bq5kMWM29EbM4CJ6UvYUmMwXMUg6p9ucML0nPRBeLMYRrudrsrRIzg3UWfRGepPpjY3NoJz2KaOD+OcW8J+f7qwpYJbYMjtBJAQE1JIQsgy5TGQk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ORYe9iCZ; arc=none smtp.client-ip=209.85.210.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ORYe9iCZ" Received: by mail-ot1-f42.google.com with SMTP id 46e09a7af769-7093ba310b0so758811a34.2 for ; Thu, 22 Aug 2024 13:28:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724358496; x=1724963296; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=M1s37SmFSoutq4Cla0ROl94+jxjLFGaNMPkeu36ZGaw=; b=ORYe9iCZUTr7Dx55ALjhtS4WPgdA8iwpWKxL3445mBibIGBhyMwnfu57DhRCDfUSqR HCSaf2L/vOh9ZsBvR7r9TxLOCt+dBDKBtzoUnDv7DpuSM8oOgbWIwQiP7sW+ENh8Isc1 Wxnm+1QGsKobU9GEsMG5euDlSSfrtjFB3Tu34FK2OQszDoCTdsIS3u+1kLXJT82Stdsh uqb+ILSxtXh6ncw3Dit2wHd47D3LLMkUWmZhmwRYr2k8MoJaf5j1XiUHVPqViYsfpHNp 9V6jIW0FFYO+QaVxA0z5qpPZhXzjOwuG386lc2MncDd9muiwi3n8gGWzUC/NE4Lm24va Kuvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724358496; x=1724963296; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M1s37SmFSoutq4Cla0ROl94+jxjLFGaNMPkeu36ZGaw=; b=MJBEc2FTqLZ9cIE2RyQdfZFl4lo0SW0U1nEkibkUHR/qve7S4anANZOecUoKxnloTl imqtgwubgQlb7aAUmoJSpWzaemz1Fa0R8I/mqMI6eG8lyUM3YK9ESysi8/m41B31xleA CearRMG3V6R6A78+XJxa1z+P7QAhBIePcVOg3GNn7dJmM+T3OCbIMhbTyXe/vAiRn9Pj JYKmC7XsZV8gtySsnwcBYIrk691hAaLcBdhtGih3ayi3+z1RHDaLxLioC4q0bT+VD29Q 4Agu6CpA9EWjcuaKobYn8TqLLZMGJXlXF0pSak7mUKCOenq4+8LF/GTWJvcgOc/gAa3p lL+w== X-Gm-Message-State: AOJu0YwkBHyilbrtKeF49lyBRRC7oNwr6CYDNEa/6P4Jm3yUZvqp7jWT 6DX8aqAhCAV/D/OTFn09dGEEi2dZCYc6usFY0v3xkizCdHuFL2tQ9Jtp277R X-Google-Smtp-Source: AGHT+IESHeMuha44i7IqObiLapfOnL+cMBaJAbHcBE9nj4YdYiZRc6zMz0Cm5JYiCByoc907pOT7Yg== X-Received: by 2002:a05:6808:30a6:b0:3d9:e22a:97a4 with SMTP id 5614622812f47-3de19576b43mr8046514b6e.36.1724358496188; Thu, 22 Aug 2024 13:28:16 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3de2252d5fbsm421265b6e.1.2024.08.22.13.28.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 13:28:15 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v8 2/4] driver core: separate function to shutdown one device Date: Thu, 22 Aug 2024 15:28:03 -0500 Message-Id: <20240822202805.6379-3-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240822202805.6379-1-stuart.w.hayes@gmail.com> References: <20240822202805.6379-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make a separate function for the part of device_shutdown() that does the shutown for a single device. This is in preparation for making device shutdown asynchronous. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Christoph Hellwig Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg Tested-by: Keith Busch --- drivers/base/core.c | 66 ++++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 30 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index 13253d105062..7e50daa65ca0 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4781,6 +4781,41 @@ int device_change_owner(struct device *dev, kuid_t k= uid, kgid_t kgid) } EXPORT_SYMBOL_GPL(device_change_owner); =20 +static void shutdown_one_device(struct device *dev) +{ + /* hold lock to avoid race with probe/release */ + if (dev->parent && dev->bus && dev->bus->need_parent_lock) + device_lock(dev->parent); + device_lock(dev); + + /* Don't allow any more runtime suspends */ + pm_runtime_get_noresume(dev); + pm_runtime_barrier(dev); + + if (dev->class && dev->class->shutdown_pre) { + if (initcall_debug) + dev_info(dev, "shutdown_pre\n"); + dev->class->shutdown_pre(dev); + } + if (dev->bus && dev->bus->shutdown) { + if (initcall_debug) + dev_info(dev, "shutdown\n"); + dev->bus->shutdown(dev); + } else if (dev->driver && dev->driver->shutdown) { + if (initcall_debug) + dev_info(dev, "shutdown\n"); + dev->driver->shutdown(dev); + } + + device_unlock(dev); + if (dev->parent && dev->bus && dev->bus->need_parent_lock) + device_unlock(dev->parent); + + put_device(dev); + if (dev->parent) + put_device(dev->parent); +} + /** * device_shutdown - call ->shutdown() on each device to shutdown. */ @@ -4817,36 +4852,7 @@ void device_shutdown(void) list_del_init(&dev->kobj.entry); spin_unlock(&devices_kset->list_lock); =20 - /* hold lock to avoid race with probe/release */ - if (parent && dev->bus && dev->bus->need_parent_lock) - device_lock(parent); - device_lock(dev); - - /* Don't allow any more runtime suspends */ - pm_runtime_get_noresume(dev); - pm_runtime_barrier(dev); - - if (dev->class && dev->class->shutdown_pre) { - if (initcall_debug) - dev_info(dev, "shutdown_pre\n"); - dev->class->shutdown_pre(dev); - } - if (dev->bus && dev->bus->shutdown) { - if (initcall_debug) - dev_info(dev, "shutdown\n"); - dev->bus->shutdown(dev); - } else if (dev->driver && dev->driver->shutdown) { - if (initcall_debug) - dev_info(dev, "shutdown\n"); - dev->driver->shutdown(dev); - } - - device_unlock(dev); - if (parent && dev->bus && dev->bus->need_parent_lock) - device_unlock(parent); - - put_device(dev); - put_device(parent); + shutdown_one_device(dev); =20 spin_lock(&devices_kset->list_lock); } --=20 2.39.3 From nobody Sat Feb 7 22:21:12 2026 Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A62851D0DFF for ; Thu, 22 Aug 2024 20:28:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358500; cv=none; b=sI0kkQ9e/rIkc4pmf2dFm9BGn6jn6otipoqSISEj2oL5s/0+z1trgRdNXy9cFuLuk7hNJP4oQfagcd6I1KFXMAO43zqwhHLN/wSqAMr8AGtQ18u4B0e1Ad6H3yvsYOUCkaT/xJopLtEmtwoJrFlCFtec9Vku8U+DPWZNH/F5sks= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358500; c=relaxed/simple; bh=czV01+a0eIjwfx1grTpqwzCDJjgn31W9N6Mn8Nhx8oI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bUjS2xRJ5JfgbNHJxU3rn5e07uC6XWtgNz8QGFtuOT6lF+z6LFxrV49ojsYx8B2s3HaMUI8mgyxl7UuIP1CL8A85UvLe8huOnX6rhdEfxng5Gf4CeTaQ3F5yQ7GvyVl3j24u4ZK3epU4cN7j9gA6B8Dmq51cUYBBxmuWtOEW5aY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=S8EiFFgH; arc=none smtp.client-ip=209.85.210.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="S8EiFFgH" Received: by mail-ot1-f44.google.com with SMTP id 46e09a7af769-709485aca4bso870820a34.1 for ; Thu, 22 Aug 2024 13:28:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724358497; x=1724963297; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BN9Dqeft6D+8f3HlmvNpHrHmxwq4si7sL5spWhZXP/0=; b=S8EiFFgHneV7/CBh9PTl60+wpXUNJTF0ggyN3z555UXaXoNeAwVlBzbLbOASyVZ1Zo oECo611gc1XG6pv6Et+5L06GWmr3BXOlWdUd5eG6gxV8p9MqNCrzjaGtTqLxMZ9FVisH K8i8VQT1V1ULpF4ndFESnnPgYhXdN8T1Xo6pFzw+4owTFqh8fLRvrUVqJZsnFxWwXsHP a1eRCq2Q6l750eSNFuxLonXPEUglESdX23v6eIWmdvck9CMjtpeJprEiauPK3J2VNWoD 9tANHK1QxC6aWEqXN4yK98dwQNDjOUmuMh+nPeXZB7SR9oVwwAgNBZwwaElGIDVXRfl9 UqQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724358497; x=1724963297; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BN9Dqeft6D+8f3HlmvNpHrHmxwq4si7sL5spWhZXP/0=; b=Xib+MjeaIfEj+3P5CD+uvI0kxIeCQ4trg3KElCprGOIULOc2wh2R3zmOGHLFcS64l6 B3iwjyI2C5wUunNNOmG2rjIJmR4Ts7eqNdU6PPxb/++aIWP8SJbBd03ulk69gZpnMs52 UPxY7z5n1IhnRjBPOdSQUd6D5G5px8+txXdJCW0D3bhqFLONio20zN4bjrtTZxCeYudu 1uie6htA8GVkZI5zoChACblGOF8uO2M6FQ0U9BfcRQ9UPhI7Yo4/neaD9gDtFfniDflr nIrwLsJNpM92n8pHu41XDmhqES+kwrPJIa+DjAh5rihoN8q2vkjh8LeMVWbnf67giMH0 F/ww== X-Gm-Message-State: AOJu0Yy65d3kz3K2MErssmPgTtuZX4in6YEMcf4YZCZXWcgyQJkQ8tkc Ap/lq21DW5Cg0kY8qO4pAwICxoisUl0jO1vJidUP9h7rO4YFT35X1XTxZfNE X-Google-Smtp-Source: AGHT+IGVMGaJf5A17/+BmjJ2vW4XtZLorWv8OhPuJs/pRN8TEJbUI0Rv/UopCyIInmLEHWlKrzkavA== X-Received: by 2002:a54:4393:0:b0:3d9:245c:422e with SMTP id 5614622812f47-3de2a869a43mr21710b6e.21.1724358497424; Thu, 22 Aug 2024 13:28:17 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3de2252d5fbsm421265b6e.1.2024.08.22.13.28.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 13:28:17 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v8 3/4] driver core: shut down devices asynchronously Date: Thu, 22 Aug 2024 15:28:04 -0500 Message-Id: <20240822202805.6379-4-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240822202805.6379-1-stuart.w.hayes@gmail.com> References: <20240822202805.6379-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add code to allow asynchronous shutdown of devices, ensuring that each device is shut down before its parents & suppliers. Only devices with drivers that have async_shutdown_enable enabled will be shut down asynchronously. This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Christoph Hellwig Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg Tested-by: Keith Busch --- drivers/base/base.h | 4 +++ drivers/base/core.c | 54 ++++++++++++++++++++++++++++++++++- include/linux/device/driver.h | 2 ++ 3 files changed, 59 insertions(+), 1 deletion(-) diff --git a/drivers/base/base.h b/drivers/base/base.h index 0b53593372d7..aa5a2bd3f2b8 100644 --- a/drivers/base/base.h +++ b/drivers/base/base.h @@ -10,6 +10,7 @@ * shared outside of the drivers/base/ directory. * */ +#include #include =20 /** @@ -97,6 +98,8 @@ struct driver_private { * the device; typically because it depends on another driver getting * probed first. * @async_driver - pointer to device driver awaiting probe via async_probe + * @shutdown_after - used during device shutdown to ensure correct shutdown + * ordering. * @device - pointer back to the struct device that this structure is * associated with. * @dead - This device is currently either in the process of or has been @@ -114,6 +117,7 @@ struct device_private { struct list_head deferred_probe; const struct device_driver *async_driver; char *deferred_probe_reason; + async_cookie_t shutdown_after; struct device *device; u8 dead:1; }; diff --git a/drivers/base/core.c b/drivers/base/core.c index 7e50daa65ca0..dd3652ea56fe 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -9,6 +9,7 @@ */ =20 #include +#include #include #include #include @@ -3531,6 +3532,7 @@ static int device_private_init(struct device *dev) klist_init(&dev->p->klist_children, klist_children_get, klist_children_put); INIT_LIST_HEAD(&dev->p->deferred_probe); + dev->p->shutdown_after =3D 0; return 0; } =20 @@ -4781,6 +4783,8 @@ int device_change_owner(struct device *dev, kuid_t ku= id, kgid_t kgid) } EXPORT_SYMBOL_GPL(device_change_owner); =20 +static ASYNC_DOMAIN(sd_domain); + static void shutdown_one_device(struct device *dev) { /* hold lock to avoid race with probe/release */ @@ -4816,12 +4820,34 @@ static void shutdown_one_device(struct device *dev) put_device(dev->parent); } =20 +/** + * shutdown_one_device_async + * @data: the pointer to the struct device to be shutdown + * @cookie: not used + * + * Shuts down one device, after waiting for shutdown_after to complete. + * shutdown_after should be set to the cookie of the last child or consumer + * of this device to be shutdown (if any), or to the cookie of the previous + * device to be shut down for devices that don't enable asynchronous shutd= own. + */ +static void shutdown_one_device_async(void *data, async_cookie_t cookie) +{ + struct device *dev =3D data; + + async_synchronize_cookie_domain(dev->p->shutdown_after + 1, &sd_domain); + + shutdown_one_device(dev); +} + /** * device_shutdown - call ->shutdown() on each device to shutdown. */ void device_shutdown(void) { struct device *dev, *parent; + async_cookie_t cookie =3D 0; + struct device_link *link; + int idx; =20 wait_for_device_probe(); device_block_probing(); @@ -4852,11 +4878,37 @@ void device_shutdown(void) list_del_init(&dev->kobj.entry); spin_unlock(&devices_kset->list_lock); =20 - shutdown_one_device(dev); + + /* + * Set cookie for devices that will be shut down synchronously + */ + if (!dev->driver || !dev->driver->async_shutdown_enable) + dev->p->shutdown_after =3D cookie; + + get_device(dev); + get_device(parent); + + cookie =3D async_schedule_domain(shutdown_one_device_async, + dev, &sd_domain); + /* + * Ensure parent & suppliers wait for this device to shut down + */ + if (parent) { + parent->p->shutdown_after =3D cookie; + put_device(parent); + } + + idx =3D device_links_read_lock(); + list_for_each_entry_rcu(link, &dev->links.suppliers, c_node, + device_links_read_lock_held()) + link->supplier->p->shutdown_after =3D cookie; + device_links_read_unlock(idx); + put_device(dev); =20 spin_lock(&devices_kset->list_lock); } spin_unlock(&devices_kset->list_lock); + async_synchronize_full_domain(&sd_domain); } =20 /* diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h index 1fc8b68786de..2b6127faaa25 100644 --- a/include/linux/device/driver.h +++ b/include/linux/device/driver.h @@ -56,6 +56,7 @@ enum probe_type { * @mod_name: Used for built-in modules. * @suppress_bind_attrs: Disables bind/unbind via sysfs. * @probe_type: Type of the probe (synchronous or asynchronous) to use. + * @async_shutdown_enable: Enables devices to be shutdown asynchronously. * @of_match_table: The open firmware table. * @acpi_match_table: The ACPI match table. * @probe: Called to query the existence of a specific device, @@ -102,6 +103,7 @@ struct device_driver { =20 bool suppress_bind_attrs; /* disables bind/unbind via sysfs */ enum probe_type probe_type; + bool async_shutdown_enable; =20 const struct of_device_id *of_match_table; const struct acpi_device_id *acpi_match_table; --=20 2.39.3 From nobody Sat Feb 7 22:21:12 2026 Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9715E1D173E for ; Thu, 22 Aug 2024 20:28:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358501; cv=none; b=VXgxlPGTh2I5dal7lgz4iqrryaCQZ+ZrusRST+Adje9wS5HOXvU0b9ExjjODwhrBVUeVxxe5kt+NKmHn/7XwSP3EA/akams8Pap2hs1RSRLIA2W/IsIOs3AGqZQmzR0fN/vDXA4WvXpNTcO3HzAAddKNW2VVXQOahg0yTGrpcy4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724358501; c=relaxed/simple; bh=UOKVmw1hpJeUfeTwlRyjGxIeLeAGEcqAj99DGptPw0Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IusludXrjlt/VBdj1IVjURDsxiPM0+aOwPXdyv6/gX3IPd2COS+DUBP7z4xg8CI4SU7rXMLe4ywZvvDdpbNSPrsK8bdKpUKOmHKBDgT6QhxPaHMhJtBLR0PHm1bLo6+8J5a4qJuejqpVl4L14uCNwL+YF2yR6jQd2MPj2I52Z30= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JvyWhqho; arc=none smtp.client-ip=209.85.210.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JvyWhqho" Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-709428a9469so1057353a34.3 for ; Thu, 22 Aug 2024 13:28:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724358498; x=1724963298; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SFs61gq37YkVZz7WirFZIsP//0+vKUsSxKZKnV0KhqU=; b=JvyWhqhoHuAXXbBiWn+kIee2wQ84/ClHRv1Az7pfNxIzkF/nn8r+ZsCccgmRjcGzkd V3loiWmkpRaIbAs+DZJfjhDNGtftBz/k3yNRNWOg6RF2FbtfLs4IR+1urmJwvU2wo66E lV1LiykACVYx/B9jphm9XXQd5mFS5BIzLQiqok4v59bAKr2/vrCjHdTByF8JAzdKmAiQ 2M1Cbv/dNaATJ2LgMpsE8MtiUu5LX4IRuGWRKjWNS/jo7F6TA9Agpz0tz2/z8W/7nS65 EnjLri1wHy0hMVOiMv9dZNB+DceMplb+kWfHf2JxTvDDyBy3raNaOfQnEGGhHgs3EggO 5Idw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724358498; x=1724963298; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SFs61gq37YkVZz7WirFZIsP//0+vKUsSxKZKnV0KhqU=; b=ceFIXcquRpyZc8lwNi6HxBB81FQtVF1MAd/6ydS6wxGJ3Fc4sMpDLZQmm1tEnu8LxU hM7RY6q0l0GoGjDSLpGu0FsUADzWR35Up2pNycNhr78bOHX16OufFpI/Bo3Kb6RXa5nL G8pjaf5ccavZxDEk+CWDjDo6diddxm9wiXDcLQdg15t9FZblpzEfmMLld2+Gt32pG5C9 zQ3I1NISDyKqgEpdvGKM9id3CQK8UDgjMRbAR+fDmvzQEamTV2gasDnlqXL95Fv+l5jP e5zywAKBudNk/E4gnzZEKzi+fQaGpN/p4J6uTdu+gVYVcaiDlb2pRwl5peYrk4QsTRmH a/Ag== X-Gm-Message-State: AOJu0YyXhktHZHScbkAsRW9D5PU1iQiEpumOaJz586l2H18tTIN0W3Oh u8AUhiWBRjg3YqKAPCSTQC3d3Z9bE5Bg5bz3iNtmyGb/Qq478UQmV1kwsYyD X-Google-Smtp-Source: AGHT+IFrVRHrCiCi5cIp28BTyotzWR1lcgkXpE435Imqtgm8r2l4jvzH3zyAhCus82rF+WPtcyribA== X-Received: by 2002:a05:6808:1b22:b0:3dd:3651:d2eb with SMTP id 5614622812f47-3de2a8932bamr10861b6e.22.1724358498688; Thu, 22 Aug 2024 13:28:18 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3de2252d5fbsm421265b6e.1.2024.08.22.13.28.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 13:28:18 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v8 4/4] nvme-pci: Make driver prefer asynchronous shutdown Date: Thu, 22 Aug 2024 15:28:05 -0500 Message-Id: <20240822202805.6379-5-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240822202805.6379-1-stuart.w.hayes@gmail.com> References: <20240822202805.6379-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the driver default to enable asynchronous shutdown. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery Reviewed-by: Christoph Hellwig Reviewed-by: Keith Busch Reviewed-by: Sagi Grimberg Tested-by: Keith Busch --- drivers/nvme/host/pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 6cd9395ba9ec..58d0d517fead 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -3580,6 +3580,7 @@ static struct pci_driver nvme_driver =3D { .shutdown =3D nvme_shutdown, .driver =3D { .probe_type =3D PROBE_PREFER_ASYNCHRONOUS, + .async_shutdown_enable =3D true, #ifdef CONFIG_PM_SLEEP .pm =3D &nvme_dev_pm_ops, #endif --=20 2.39.3